ABET 2000 - Computer and Telecommunications Engineering

Misson Statement
Program Educational Objectives
Program Outcomes
Constituents

Assessment Plan

A high-level flow diagram outlining our assessment and evaluation processes is shown in Figure 1. The diagram shows the two loops for the assessment and evaluation of the CTE Program Educational Objectives (Loop 1) and CTE Program Outcomes (Loop 2), and shows how the CTE Program Educational Objectives feed into the CTE Program Outcomes. The CTE Program Educational Objectives support the missions of the University, School, and Department, and were determined in consultation with the CTE program constituencies (alumni, employers, and students).

Fig. 1

Figure 1: Flow diagram illustrating the processes for the assessment and evaluation of the CTE Program Educational Objectives (Loop 1) and the CTE Program Outcomes (Loop 2).

A.  CTE Program Educational Objectives

A detailed flow diagram for the assessment and evaluation of the CTE Program Educational Objectives is shown in Figure 2. The CTE Program Educational Objectives have been developed in consultation with the CTE Program core constituencies who also help us evaluate how well these objectives are being achieved.

An effective assessment requires both a well thought-out process and interpretable data that provides meaningful input to the process. As indicated in Figure 2, the ESE ABET Committee evaluates the assessed data and comes up with initial recommendations based upon whether objectives have been met or not.  The results of the evaluation and the ESE ABET Committee’s initial recommendations are then presented by the CTE ABET Coordinator (and members of the ESE ABET Committee) at the yearly ESE retreat and at periodic ESE Faculty Meetings. If appropriate, the faculty then authorizes the ESE Undergraduate Curriculum Committee (UCC) to develop a recommendation that could include specific changes to the curriculum, courses, student activities, or lab facilities to further facilitate the attainment of the objectives by our graduates. The recommendation is brought back to the faculty for discussion and their final approval. Major curricular changes require subsequent approval by the school’s Undergraduate Affairs Committee (UAC).

 

Fig. 2

Figure 2: Process diagram for the assessment and evaluation of CTE Program Educational Objectives.


The success of this assessment and evaluation process is dependent on the data that feeds into this process. As indicated in Fig. 2, four data collection instruments have been tabbed for this purpose. These consist of the following:

 

In order to aid with the evaluation of the CTE Program Educational Objectives we have developed a matrix that describes in more detail the methods for evaluating our defined measures of success and the ways in which feedback is provided to relevant parties about the level of achievement of the objectives and outcomes, respectively.  The matrix includes the following aspects for each CTE Program Educational Objective:

 

The matrices for all four CTE Program Educational Objectives are given in here.

B.  CTE Program Outcomes

A flow diagram for the assessment and evaluation of the CTE Program Outcomes is shown in Figure 3.  The process consists of comparing the results of the assessment with the desired CTE Program Outcomes. The ESE ABET Committee evaluate the assessed data and come up with initial recommendations based upon whether outcomes have been met or not.  The results are then reviewed by the ESE Undergraduate Curriculum Committee (UCC) and the CTE Program ABET Coordinator. After evaluation of the CTE Program Outcomes, the committee discusses the results with the ESE faculty and suggest changes, if needed, to the curriculum, courses, lab facilities, or student activities. This is done primarily at the yearly retreat where extensive discussion can be devoted to the evaluation of the assessment.

Fig. 3

Figure 3: Process for the Assessment and Evaluation of the CTE Program Outcomes.

Once again, the development of interpretable, relevant data is a critical aspect of an effective assessment process. Both direct and indirect measures are employed in the assessment and evaluation process as illustrated in Figure 3. The direct assessment instruments consist of the Course Outcomes assessment, lab and design project reports, and senior design project assessment. The indirect assessment instruments include a student survey, senior exit survey, and bi-annual student focus groups. The Course Outcomes Assessment is a key component of the direct assessment and includes the use of “Targeted Questions” on exams and other deliverables such as project and lab reports in core computer and telecommunications engineering courses to assess a specific outcome. The ESE ABET Committee, with input from the full ESE faculty, establishes the performance criteria for each of the CTE Program Outcomes.

Similarly as for the Educational Objectives, we have developed a matrix that describes the methods for assessing and evaluating the CTE Program Outcomes.  It gives the main key steps involved in the assessment and evaluation process.  These steps include:

The matrices for all CTE Program Outcomes are given in here.

Course Outcomes Assessment

One of the components of the CTE Program Outcomes Assessment is the Course Outcomes Assessment developed using targeted exam questions or other deliverables such as project and lab reports in core engineering courses and the senior design project. This direct method complements the indirect tools such as the student surveys, senior exit surveys, and the student focus groups.

The instructor writes a set of “Course Outcomes” and relate them to specific CTE Program Outcomes prior to the start of the semester when the course is to be offered. These Course Outcomes are written such that they can be measured directly and quantitatively using a set of specific course deliverables, such as components of assignments or specific questions on exams. The Course Outcomes are discussed with the students at the beginning of the semester and are posted on the course home page. Students are also informed about the relationship of the Course Outcomes to the overall EE Program Outcomes.

All Course Outcomes must correspond to one or more CTE Program Outcomes. The relationship between the Course Outcomes and CTE Program Outcomes is indicated on each of the “Course Outcomes Assessment Forms (COAF)”. Figure 4 shows a COAF template. The column with the heading “Course Outcomes” lists the Course Outcomes grouped under the respective “CTE Program Outcomes”. In this way a clear relationship between the CTE Program and Course Outcomes is established. The column “Data Source” lists the specific components of assignments, questions on exams, or projects that are to be used to assess the Course Outcome. These items are prepared specifically to assess the Course Outcome in question.

Fig. 4

Figure 4:  Template of the Course Outcomes Assessment Form (COAF).  [Click here for set of guidelines].

A Course Outcome is considered to be achieved when at least 85% of the students have achieved a “Pass” score for the outcome. The instructor indicates on the COAF the number of students who have received “Pass” and “Fail” scores on each course outcome. If there is a preponderance of Fails (> 15%) on a specific Course Outcome, the instructor indicates the specific subject area that needs improvement (in the Comments column). At the end of the COAF is a section on Data Analysis and Interpretation. This allows the instructor to describe the analysis of students’ work and any other collected data used in the assessment.  This section also identifies specific areas where the students fell short in achieving a Course Outcome, and areas of improvement, if any.  An important part of the COAF is the Improvement Plan which states what changes, if any, are needed in the course syllabus, text book, and assignments. This helps the ESE ABET Committee close the loop on the course assessment. Finally, there is a section for the ESE ABET Committee to make comments and recommendations to be fed back to the course instructor.

A detailed set of guidelines explaining the use of the Course Outcomes Assessment Form is given to the faculty and can be found here.

CLOSING THE LOOP

A.  CTE Program Educational Objectives

As indicated in Figure 2, input from various constituencies is evaluated first by the ESE ABET Committee and second by ESE faculty. The results of the evaluation are presented by the CTE Program ABET Coordinator at the yearly ESE retreat and at periodic ESE Faculty Meetings. If a CTE Program Educational Objective is not met, this finding together with supporting data and recommendations by the ESE ABET Committee is presented to the faculty. If needed, the ESE faculty authorizes the ESE Undergraduate Curriculum Committee to recommend further changes to the CTE curriculum, courses, student activities, and lab facilities to further facilitate the achievement of the objectives by our graduates.  The Undergraduate Curriculum Chair is charged with bringing the ESE Undergraduate Curriculum Committee’s recommendations to the faculty for further discussion and approval.

Substantial programmatic changes to the CTE curriculum or the introduction of new courses must also be discussed and approved at the School of Engineering and Applied Science Undergraduate Affairs Committee. Results and actions to remedy any shortcoming are shared with the constituencies and the current students.

B.  CTE Program Outcomes

In a process similar to that followed for the CTE Program Educational Objectives, the ESE ABET Committee will evaluate the input from the direct and indirect measures used to assess the CTE Program Outcomes (see Figure 3).  Of particular importance is the aggregation and integration of the Course and CTE Program Outcome assessment results to achieve a composite result (score and recommendations) for each CTE Program Outcome. In doing so the committee reviews the Course Outcome results on a per outcome basis and determines how the program as a whole is doing with respect to achieving each CTE Program Outcome. If any shortcomings are identified, the committee uses the CTE Program Outcomes assessment instruments to track down the source(s) of these shortcomings. This allows the committee to suggest specific changes to courses, improvement in instruction, lab facilities, or students’ activities.

The results of the assessment and evaluation are presented by the CTE Program ABET Coordinator to the ESE Faculty at the faculty meetings and the yearly retreat (the retreat is used to give a comprehensive review of the assessment/evaluation and to propose actions for change; the faculty meetings are used to follow up on the discussions at the retreat in case the ABET committee is charged to come up with further recommendations). Specific recommendations are proposed by the committee. If appropriate, the ESE faculty authorizes the ESE Undergraduate Curriculum Committee to further review the report presented by the ESE ABET Committee and to particularly focus on issues correctable by updating the EE curriculum. The ESE Undergraduate Curriculum Chair presents his or her committee’s recommendations to the ESE faculty for discussion and approval.

Major changes to a course or the curriculum must also be approved by the School of Engineering and Applied Science Undergraduate Affairs Committee (UAC). The Undergraduate Curriculum Chair is charged with bringing such changes to the UAC for approval.

The results of the assessment and proposed changes that arise from the evaluation process are discussed with the student body during the semi-annual student focus groups.

C.  Implementation and Dissemination of Changes

The changes to the CTE Program Educational Objectives and CTE Program Outcomes that were approved as a result of the assessment and evaluation processes are disseminated using the following instruments:

  1. The department’s website.
  2. The university academic bulletin and other relevant publications.
  3. The school’s on-line undergraduate student handbook.