# The Role of Mathmatics Courses in the Assessment Process of Engineering ABET-Outcomes

Authors: Salah-Addin B. Al-Omari; Qasem M. Al-Mdallal; Youssef El-Khatib; Basim Abu-Jdayil
Journal: Education Research International
 (2015)

---

## Abstract

This paper deals with the mathematics courses offered to the engineering students at the United Arab Emirates University (UAEU). The paper focuses essentially on the level of achievement by the students of the outcomes of the mathematics and how this reflects on the satisfaction of the engineering ABET-outcomes. Mapping between the course outcomes of the different math courses offered at UAEU to engineering students and the engineering ABET criteria ((a)–(k)) was made. It is found that most of the ABET-outcomes are to a great extent in-line with the outcomes of our mathematics courses. This encourages us to use the achievement of the courses outcomes as an assessment tool for the engineering ABET-outcomes. We considered both direct and indirect assessment tools to assess the level of satisfaction of the math courses outcomes. The performances of the students according to both methods are then used to assess the achievements of the ABET-outcomes. The results generally show very good level of achievement of the outcomes, although few ABET-outcomes were not achieved well according to our performance criteria. Accordingly, we provide some comments and recommendations aiming at the improvement of the program.


---

## Body

## 1. Introduction

In any engineering program mathematics comes as an essential component that every such program must fulfill adequately. Mathematics helps and represents a tool engineering graduates can use to understand and apply engineering concepts. Hence, the graduates of engineering programs must clearly and explicitly demonstrate good ability to apply knowledge of mathematics in the different phases of their engineering study. This is what well recognized accreditation bodies for engineering education strongly and clearly emphasize [1]. Different articles addressed the designing of different courses to satisfy ABET engineering criteria; see, for example, [2–4]. To achieve the above goal and to help having an engineering curriculum that achieves its purpose effectively, foundations of mathematics should be introduced in early stages and should be integrated well throughout the curriculum. Mathematics used to be taught and injected throughout engineering curricula in different ways. The way mathematics is delivered will affect the perception of the students towards it and this is expected to play a clear role regarding the success of students in using mathematics in the subsequent math-dependent courses [5]. The relation between the students’ performance in math courses and the other engineering courses has been studied by different researchers [6]. In addition, the relationship between achievements by students in math courses as related to the gender and other gender-based issues has been addressed in different investigations [6–9]. Some researchers considered case studies and research to highlight the importance of acquiring strong math basis and highlighted strong correlation between the critical-thinking abilities in math acquired by the engineering graduates and their critical-thinking abilities they will acquire in their disciplines, for example, civil engineering [10].The College of Engineering at the United Arab Emirates University (COE-UAEU) is keen to meet international standards in the delivery of their curricula for all departments in the college [3, 4, 11]. As such, the COE-UAEU was keen to maintain ABET accreditation over the years, since 1998. While so doing, continuing efforts are put to ensure that the doze of math and basic sciences contained in the curricula of the different departments in the College are in compliance with the updated international ABET standards. The present paper touches upon and addresses the math courses offered within all engineering programs delivered at the COE-UAEU and attempts to highlight the salient features pertaining to these courses and the performance of the students in these courses aiming at drawing conclusions that further contribute to the continuous improvements efforts that represent an essential part of the ABET criteria (ABET Criterion 4).In this paper we try to highlight the experience of math courses in the engineering curricula of the five different departments at the COE-UAEU, namely, architectural, chemical and petroleum, civil, electrical, and mechanical engineering, and the level of satisfaction of the outcomes of these courses by the engineering students of these five departments and how these outcomes are linked to the ABET criteria ((a) through (k)). Towards this end, data on the students’ performance in these math courses have been collected and analyzed for three consecutive semesters and discussions and conclusions have been drawn from them. The different assessment tools used to assess the performance of students in these courses and the results collected based on these are also presented and discussed in this paper. Some emphasis is also put on the “students opinion” as an indirect assessment tool and how this can be used as helpful feedback towards the goal of boosting the learning process in these math courses. Hereby the study draws conclusions not only from the direct data collected on students’ performance in the math courses we offer based on exams of different kinds and other course activities such as projects, homework assignments, and quizzes, but also based on the feedback collected from students through surveys conducted at the end of each semester, as part of the assessment tools used in the process. Hereby it is important to see how students judge their level of satisfaction for the courses outcomes, as they see it, aside from the learning assessments mentioned above. This kind of feedback obtained from the students is looked at as a means of improving the way the scientific material is delivered and a feedback to the instructor to ensure that students who at the end and foremost are partners and major constituent in the learning and teaching processes are active and really receiving the doze of material the courses target instilling in them.The results of this study and the conclusions and recommendations drawn from the analysis of the presented data aim at contributing to the continuing efforts that target keeping up the level of math courses delivery at UAEU at highest level possible, as well as meeting the worldwide well recognized ABET standards. Moreover, as part of our commitment towards the international community, we publicize and share hereby our math-teaching experience with the peers in the field internationally to add to the literature in this vital area and to invite hereby contributions from peers in that direction that are expected to enrich the educational experience at the global level.


### 1.1. Math Courses Offered at COE-UAEU

The math courses offered at the COE-UAEU, are(i)
MATH1110: Calculus I for engineers,
(ii)
MATH1120: Calculus II for engineers,
(iii)
MATH2210: differential equations and engineering applications,
(iv)
MATH2220: linear algebra and engineering applications.The above courses address course outcomes that have been mapped to the ABET criteria ((a) through (k)) and the satisfaction of these outcomes by the students are taken as one of the indicators of the satisfaction of these courses for the relevant ABET criteria they are mapped to.Details on the mapping of the course outcomes for the above courses with the ABET criteria ((a)–(k)) as well as details on the statements of these course outcomes, the outcome assessment tools used in these courses to assess level of satisfaction of the outcomes, data on the level of satisfaction of the course outcomes judged based on the assessment tools adopted are all presented and discussed in the subsequent sections.


## 1.1. Math Courses Offered at COE-UAEU

The math courses offered at the COE-UAEU, are(i)
MATH1110: Calculus I for engineers,
(ii)
MATH1120: Calculus II for engineers,
(iii)
MATH2210: differential equations and engineering applications,
(iv)
MATH2220: linear algebra and engineering applications.The above courses address course outcomes that have been mapped to the ABET criteria ((a) through (k)) and the satisfaction of these outcomes by the students are taken as one of the indicators of the satisfaction of these courses for the relevant ABET criteria they are mapped to.Details on the mapping of the course outcomes for the above courses with the ABET criteria ((a)–(k)) as well as details on the statements of these course outcomes, the outcome assessment tools used in these courses to assess level of satisfaction of the outcomes, data on the level of satisfaction of the course outcomes judged based on the assessment tools adopted are all presented and discussed in the subsequent sections.


## 2. Outcomes of Our Math Courses and Their Mapping with ABET Outcomes

The ABET identifies 11 outcomes for engineering programs, and they are listed below:(a)
an ability to apply knowledge of mathematics, science, and engineering,
(b)
an ability to design and conduct experiments, as well as to analyze and interpret data,
(c)
an ability to design a system, component, or process to meet desired needs,
(d)
an ability to function on multidisciplinary teams,
(e)
an ability to identify, formulate, and solve engineering problems,
(f)
an understanding of professional and ethical responsibility,
(g)
an ability to communicate effectively,
(h)
the broad education necessary to understand the impact of engineering solutions in a global and societal context,
(i)
a recognition of the need and an ability to engage in life-long learning,
(j)
a knowledge of contemporary issues,
(k)
an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.Tables 1 to 4 present the statement of the courses outcomes for the four math courses in place at the COE-UAEU. Their mapping with ABET criteria, as well as the direct assessment tools used in each one of these courses, are also presented in these tables.Table 1
Course outcomes, mapping with ABET and students direct assessment tools for the course MATH1110-Calculus I for engineers.
Relevant ABET outcomes
Course outcome number
MATH1110 course outcomes
Assessment   tools
Assignment
Quizzes
Labs
Misc
Exams
(a), (b), (e)
1
Sketch and analyze the graphs of functions and interpret the results
✓
✓
□
□
✓
(a), (e), (k)
2
Find derivatives of different functions
✓
✓
□
□
✓
(a), (e), (k)
3
Understand the conceptual foundations of rate of change, slope of tangent line, and their application to engineering problems
✓
✓
✓
□
✓
(a), (c), (e), (k)
4
Learn how the derivatives can be used to model engineering problems
✓
✓
✓
□
✓
(a), (b), (c), (i)
5
Demonstrate ability to think critically in analyzing engineering problems
✓
□
✓
□
□
(d), (f), (g)
6
Work effectively with others
✓
□
✓
□
□
(a), (e), (k)
7
Able to integrate (to find the antiderivative of) different functions
✓
□
□
□
✓
(a), (e), (k)
8
Able to use the integration to find the areas under or between curves, displacements given the accelerations, and work done by a particle or so
✓
□
□
□
✓As can be seen from Tables1–4, the courses outcomes versus ABET criteria mapping for our four math courses cover all ABET criteria ((a) through (k)) except ABET criterion (h) (impact of engineering solutions in a global and societal context), which is mainly addressed by the engineering courses that the students take at a later stage. The entity in the COE-UAEU that coordinates with the College of Science (namely, the Math Department) issues related to the offering of the above four math courses for our engineering students and at the same time is in charge of doing the mapping between the math course outcomes and the ABET criteria, as shown in Tables 1–4, is called the “Engineering Requirement Unit” (ERU). The above mapping had been suggested based on the content of each of these math courses as decided based on the course catalogue description and the course syllabus. Revisiting the above math courses/ABET mapping is normally a task that is being triggered in the context of the continuous quality review and monitoring process that is a major part in the assessment processes of each of the five departments at the COE-UAEU that receive the services of the ERU. The five departments, while being involved on-regular-basis in possible improvements and monitoring cycles of the quality of their own curricula (fulfilling thereby ABET criterion 4 related to continuous quality improvement) may give feedback to the ERU based on which changes in the math courses of the ERU in terms of the delivered material content as well as the outcomes/ABET mapping can be done. While the departments are doing this, they aim at improving math skills abilities in their engineering students to enable them to further fulfill the needs and requirements of the different components in their curricula for the sake of whose satisfaction the proposed improvement in the ERU course(s) is being suggested. For example, some courses in the different engineering curricula may require enhancing components related to linear algebra, statistical analysis, differential equations, and so forth, which would then require considering some modifications and changes in the math courses and the way of their delivery and perhaps their relation to ABET criteria and the assessment tools used to assess their level of satisfaction.Table 2
Course outcomes, mapping with ABET and students direct assessment tools for the course MATH1120-Calculus II for engineers.
Relevant ABET outcomes
Course outcome number
MATH1120 course outcomes
Assessment   tools
Assignment
Quizzes
Labs
Misc
Exams
(a), (b),  (c),  (i)
1
Show the  ability to think critically in analyzing engineering problems
✓
□
□
□
✓
(a),  (e), (k),  (b)
2
Parameterize a curve
✓
□
□
□
✓
(a),  (d), (g), (k)
3
Use computers to enhance visualization and solve calculus problems in several variables
✓
□
□
□
□
(a),  (e),  (k)
4
Understand the properties of vectors and know some of their applications
✓
□
□
✓
✓
(a),  (b),  (e)
5
Identify the equations of common surfaces such as sphere, cone, and paraboloid
□
□
□
✓
✓
(a),  (e), (k)
6
Calculate partial derivatives, the rate of change, and the extrema of function of several variables
✓
□
□
□
✓
(a), (c), (k)
7
Use multiple integrals to get the areas, volumes, and center of mass for different configurations
✓
□
□
✓
✓
(a),  (k)
8
Realize which system of coordinates is more convenient to describe a physical situation
□
□
□
✓
□
(a),  (e), (k)
9
Calculate line integrals
✓
□
□
□
✓
(a), (k), (e), (j)
10
Understand the notion of the curl and divergence of a vector field
□
□
□
□
✓Table 3
Course outcomes, mapping with ABET and students direct assessment tools for the course MATH2210-differential equations and engineering applications.
Relevant ABET outcomes
Course outcome number
MATH2210 course outcomes
Assessment   tools
Assignment
Quizzes
Labs
Misc
Exams
(a),   (e),  (j),  (k)
1
Analytically solving first-order differential equations
□
✓
□
□
✓
(a),   (e),  (j),  (k)
2
Analytically solving second order differential equations both homogenous and nonhomogenous
□
✓
□
□
✓
(a),   (e),  (j),  (k)
3
Analytically solving differential equations using the Laplace transform
□
✓
□
□
✓
(a),  (c),  (e),  (k)
4
Modeling thermal systems
□
□
□
□
✓
(a),  (c),  (e),  (k)
5
Modeling translational mechanical systems
□
✓
□
□
□
(a),  (c),  (e),  (k)
6
Modeling rotational mechanical systems
□
□
□
□
✓
(a),  (c),  (e),  (k)
7
Modeling electrical systems
□
✓
✓
□
✓
(a),  (d), (k)
8
Using MATLAB in the solution and analysis of differential equations
□
□
✓
□
✓
(a),  (d), (k)
9
Using MATLAB in the solution and analysis of transfer functions
□
✓
□
□
✓
(a),  (f),  (d),  (i)
10
Working and writing a report on a term project
□
□
✓
□
□
(a),  (f),  (d),  (i)
11
Step response analysis
□
□
□
□
✓Table 4
Course outcomes, mapping with ABET and students direct assessment tools for the course MATH2220-linear algebra and engineering applications.
Relevant ABET outcomes
Course outcome number
MATH2220 course outcomes
Assessment   tools
Assignment
Quizzes
Labs
Misc
Exams
(a),   (e),  (j),  (k)
1
Find the reduced form, the row and column spaces, and the rank of a given matrix
□
✓
□
□
□
(a),   (e),  (j),  (k)
2
Solve a system of homogeneous or nonhomogeneous linear equations
□
✓
□
□
✓
(a),   (e),  (j),  (k)
3
Find the inverse of a matrix and the solution of a nonsingular system using determinants
□
✓
□
□
✓
(a),   (e),  (j),  (k)
4
Find the eigenvalues, eigenvectors and and diagonalize a matrix
□
✓
□
□
✓
(a),   (e),  (j),  (k)
5
Analyze orthogonal and symmetric matrices
□
✓
□
□
✓
(a),   (e),  (j),  (k)
6
Analyze power series, exponential, and trigonometric complex functions
□
✓
□
□
✓
(a),   (e),  (j),  (k)
7
Analyze complex logarithms and powers
□
✓
□
□
✓
(a),   (e),  (j),  (k)
8
Find the integral of a complex function over a closed path using Cauchy's theorem
□
□
□
□
✓
(a),   (e),  (j),  (k)
9
Find the singularities and analyze the residue theorem
□
□
□
□
✓
(a), (k), (e), (j)
10
Some applications of the residue theorem
□
✓
□
□
□As can be seen from Tables1–4, the direct assessment tools used in our four math courses are the ones typical for assessing such courses globally, namely, the tools based on problems solving (i.e., exams of different kinds, quizzes, and homework assignments). Some other miscellaneous assessment tools that may come in the form of a course project can also be used in some of the courses for some of the courses outcomes but at a smaller scale than the typical exams-based tools.The ABET criterion “(h),” which is not mapped to any of our four math courses is related to the broad education necessary to understand the impact of engineering solutions in a global and societal context, and this is something traditionally difficult to fulfill in a math course at this level in the curriculum. Fulfilling this ABET criterion, however, is a very important and essential requirement in the other courses that are more of a basic science nature and in the subsequent parts of the engineering curricula of the different programs in the COE, which students will be exposed to after they get specialized after finishing the required doze of ERU courses.Moreover, ABET outcome “g” is related to communication skills and it is addressed here in the math courses only at a small scale where it is only mapped at this moment to the first two of our math courses and is assessed in activities like labs which in this context comes under common tutorial and problems solving sessions and homework assignments where in part students use computers to work on some certain task together as a team.A collective summary of the course outcomes of the four different math courses as mapped to ABET criteria ((a)–(k)) is given in Table5 at one glance. ABET criteria (a) (knowledge), (e) (engineering problems), (k) (techniques, skills, and tools) are the ones most evident to be mapped to our math courses outcomes to a very clear extent and by a good number of course outcomes in each of the four courses. These three ABET criteria ((a), (e), and (k)) are the ones that one would most logically directly think of as being mapped to a typical math course since they are related directly to problems solving, a very explicit feature in a math course. The remaining ABET criteria mapped to the four math courses are addressed in different other courses outcomes but (as mentioned above) to a lesser extent than the criteria (a), (e), and (k).Table 5
ABET outcomes versus math courses outcomes.
ABET outcomes
MATH1110
MATH1120
MATH2210
MATH2220
(a)
O1–O5,  O7,  O8
O1–O10
O1–O10
O1–O7
(b)
O1,  O5
O1,  O2,  O5
(c)
O4,  O5
O1,  O7
O4–O7
(d)
O6
O3
O8–O11
(e)
O1–O4,  O7,  O8
O2,  O4–O6,  O9,  O10
O1–O7
O1–O7
(f)
O6
O10,  O11
(g)
O6
O3
(h)
(i)
O5
O1
O10,  O11
(j)
O10
O1–O3
O1–O7
(k)
O2–O4,   O7,  O8
O2–O4,  O6–O10
O1–O9
O1–O7The course that the least maps to ABET criteria is the fourth course (MATH2220: linear algebra and engineering applications). Nevertheless, it at least maps well to the three ABET criteria (a), (e), and (k) that are highlighted above to be major outcomes relevant for a typical math course. Moreover, this linear algebra course is a higher level and inherently more specific and specialized course than the other three courses and as such is expected to map to a lesser number of ABET criteria. Nonetheless, it should be noted that generally not all of the courses in a curriculum need to address all ABET criteria to the same extent, but the whole curriculum of a program needs of course to show strength and well balanced mapping in the different courses of the curriculum as a whole, a requirement that is well fulfilled by our four math courses and by the whole ERU curriculum as a whole unit; going through the details of the whole curriculum of our ERU is out of the scope of this paper.


## 3. Learning Assessment of the Level of Satisfaction of Mathematics Courses Outcomes

In this section, we provide a learning assessment of the level of satisfaction of the outcomes of the different mathematics courses and based on the mapping between these outcomes and ABET criteria, we can indirectly judge the performance and the level of satisfaction of the ABET criteria in our math courses offered by the ERU at UAEU. The assessment tools, as given in Tables1–4, are homework assignments, quizzes, labs activity and reports, exams, and other miscellaneous assessment tools such as team projects and in-class activities, as judged to be needed by the Instructor teaching the course.


### 3.1. Methodology and Data Collection

A major purpose of this study is to assess the part of the curricula of the engineering programs that is related to the mathematics courses offered for engineering students at the UAEU and to see the extent of the satisfaction of the ABET criteria ((a)–(k)) in this part.


#### 3.1.1. Participants, Materials, and Instruments

This study is designed for the College of Engineering students at the UAEU who are taking the mathematics courses offered by the ERU-UAEU. This study is based on the feedback from two different kinds of assessment tools, direct and indirect tools. A direct tool is the one that gives a direct measure on the student performance through exams, homework assignment, project report, and so forth, while the indirect tool is the one that gives feedback on the student performance based on surveys and opinions in questionnaires and in this study this is taken based on feedback received from students’ surveys. The students’ opinions collected in surveys conducted at the end of each semester aim to see how the students judge their fulfillment of the course outcomes, regardless from their actual scores based on the direct assessment tools used. It should be noted that the score the students give in these surveys is not considered as part of the overall score the students attain in the course and is used solely as feedback to guide the improvement of the course material delivery. One of the purposes of collecting this indirect feedback from the students in each course is to see any possible discrepancy and mismatch between the actual learning direct assessment and the way how students see themselves fulfilling the course. This would help the instructor to better prepare their course material and the way they design the direct assessment tools for their courses in the subsequent semesters; the course is offered so that a better and more representing assessment of the course can be done.


#### 3.1.2. Data Collection

The data collected based on the two assessment tools types described above (direct and indirect) that are reported in this paper cover a period of three consecutive semesters. These data are provided by the ERU/UAEU.


#### 3.1.3. Indirect Tool

A sample of the questionnaire used to collect the indirect assessment results in the form of students’ opinion is given in Table6 for the course MATH2210: differential equations and engineering applications. In this questionnaire, the students are asked to evaluate their abilities, attributes, and skills listed in the table. The survey asks students to evaluate each course outcome based on a 1–5 scale, where 1 is very low (poor) and 5 is very high (excellent). The students should check the appropriate box that corresponds to the extent they feel the course material delivered and the way it has been delivered during the semester has helped them to achieve the course outcomes specified for the course in the syllabus.Table 6
Sample for the indirect assessment students’ survey used in the course MATH2210-differential equations and engineering applications.
Course intended outcome(by completing this course, students are able to)
Very low
↔
Very high
1
2
3
4
5
(
1) Analytically solve first-order differential equations
□
□
□
□
□
(2) Analytically solve second-order differential equations both homogenous and nonhomogenous
□
□
□
□
□
(3) Analytically solve differential equations using the Laplace transform
□
□
□
□
□
(4) Model thermal systems
□
□
□
□
□
(5) Model translational mechanical systems
□
□
□
□
□
(6) Model rotational mechanical systems
□
□
□
□
□
(7) Model electrical systems
□
□
□
□
□
(8) Use MATLAB in the solution and analysis of differential equations
□
□
□
□
□
(9) Use MATLAB in the solution and analysis of transfer functions
□
□
□
□
□
(10) Work and write a report on a term project
□
□
□
□
□
(11) Step response analysis
□
□
□
□
□


### 3.2. Analysis of the Collected Data

#### 3.2.1. Learning Assessment

The level of satisfaction of the different course outcomes of the four math courses under consideration is discussed in this subsection based on the direct assessment measures adopted in these courses. Table7 presents these direct measure data. The scores reported are on a scale from 1 to 5. The target in the math courses is 70% (3.5/5). The score range from 4 to 5 is considered to be at the “excellent” level. For the score range below 4 but above 2, the level of satisfaction is considered to be “ability.” Below 2 the level of satisfaction of the outcome is judged to be “unsatisfactory” and immediate remedial actions must be taken.Table 7
Learning assessment of course outcomes based on the direct measures given in Tables1–4 (NA: not applicable, Values in italic are below the outcomes-achievement passing criteria).
Course
Sem.
#St.
O1Av (SD)
O2Av (SD)
O3Av (SD)
O4Av (SD)
O5Av (SD)
O6Av (SD)
O7Av (SD)
O8Av (SD)
O9Av (SD)
O10Av (SD)
O11Av (SD)
MATH1110
Spring 2011
21
3.9(0.7)
3.5(0.8)
3.8(0.5)
3.6(0.6)
4.9(0.4)
4.9(0.4)
4.1(0.6)
4.0(0.6)
NA
NA
NA
Fall 2011
29
4.0(0.9)
3.8(0.9)
4.0(0.7)
3.9(0.9)
3.4
(1.1)
3.4
(1.1)
4.0(0.9)
3.7(0.8)
NA
NA
NA
Spring 2012
33
4.2(0.5)
4.1(0.7)
3.9(0.6)
3.9(0.8)
3.8(0.8)
3.8(0.8)
3.9(0.8)
3.8(0.7)
NA
NA
NA
3 sem.
average
83
4.03
3.8
3.9
3.8
4.03
4.03
4
3.8
NA
NA
NA
MATH1120
Spring 2011
29
3.3
(0.8)
3.2
(1.2)
3.5(1.2)
3.5(0.9)
2.7
(0.9)
3.3
(1.2)
3.3
(1.2)
3.4
(1.1)
3.3
(1.4)
3.2
(1.4)
NA
Fall 2011
17
3.1
(0.7)
3.3
(0.8)
3.4(0.9)
3.5(1.1)
2.9
(0.9)
3.4
(1.1)
3.2
(0.8)
3.4
(1.2)
3.1
(0.5)
3.4
(0.9)
NA
Spring 2012
32
4.3(0.7)
3.9(0.4)
4.4(0.8)
3.5(0.6)
3.3
(0.7)
3.4
(0.6)
3.3
(0.9)
3.4
(1)
2.90
(1.3)
2.90
(1.3)
NA
3 sem.
average
78
3.6
3.4
3.6
3.5
2.9
3.33
3.3
3.4
3.2
3.1
NA
MATH2210
Spring 2011
29
4(0.5)
3.8(0.6)
4(0.4)
4(0.7)
3.8(1.1)
3.8(0.7)
3.7(0.7)
4.2(0.8)
4(0.7)
4.3(0.9)
3.3
(0.9)
Fall 2011
20
4.00(0.7)
3.90(0.6)
3.40
(0.7)
3.30
(0.9)
4.40(0.8)
3.60(0.4)
3.50(0.3)
4.30(0.6)
3.20
(0.7)
3.80(0.3)
4.00(0.8)
Spring 2012
21
4.3(0.4)
4.10(0.9)
3.30
(0.3)
4.30(0.5)
4.50(0.6)
4.70(0.5)
4.50(0.3)
4.4(0.2)
4.50(0.7)
4.7(0.2)
3.7(0.9)
3 sem.
average
70
4.1
3.93
3.6
3.9
4.2
4.03
3.9
4.3
3.9
4.3
3.67
MATH2220
Spring 2011
17
4.5(0.5)
4.50(0.5)
3.70(0.7)
3.80(0.7)
4.00(1)
4.20(1.1)
3.80(0.8)
3.20
(1.1)
3.30
(1.2)
3.8(0.8)
NA
Fall 2011
18
3.90(0.9)
3.30
(0.8)
4.20(0.6)
3.90(0.6)
4.10(0.7)
4.00(0.7)
3.60(1.2)
3.80(0.9)
3.80(0.6)
3.60(0.7)
NA
Spring 2012
17
4.5(0.3)
4.5(0.3)
4.5(0.4)
3.4
(0.4)
3.4
(0.7)
3.7(0.6)
3.7(0.6)
3.90(0.6)
3.9(0.6)
3.5(0.6)
NA
3 sem.
average
52
4.3
4.1
4.13
3.7
3.8
4
3.7
3.6
3.7
3.63
NAFigure1 shows the average level of satisfaction of the course outcomes over three consecutive semesters for all of the math courses. From Table 7 and Figure 1, most outcomes for the course MATH1110 (Calculus 1) are satisfied to the level of high ability (above 3.5) with some of them touching the border of the “excellent” level (outcomes 1, 5, 6, and 7). For the course MATH1120 (Calculus II), none of the outcomes of that course are fulfilled to the excellent level, contrary to the course MATH1110. Most of the outcomes of this course are satisfied to the level of good ability (3 to 3.5), with only one outcome being satisfied to the level of ability but in the lower ability range below 3 (outcome 5). By referring to Table 2, it can be seen that this outcome is assessed mainly by major exams such as midterm exam and final exam. It should be mentioned that the course MATH1120 (Calculus II) contains several new topics, which are never covered in high school, such as vector-value functions and functions of several variables. This course is designed to prepare the engineering students for advanced courses in different engineering disciplines.Figure 1
The average level of satisfaction of the course outcomes over three semesters for all math courses.Although the level of satisfaction of most outcomes for MATH1120 is judged as “ability” and above, our target (as mentioned earlier) is a level of no less than 3.5/5 (70 percent) and so the result reported for most outcomes should be triggering the actions of the instructors teaching the course to do some action to improve the performance of students on these outcomes and/or to revise the way these outcome are assessed. This is considered as part of our commitment to continuous improvement efforts. The level of satisfaction of outcomes like O5 and O10 in MATH1120 (Calculus II), as judged based on the used direct assessment tools; although this should not be the only trigger for improvement actions by different entities and committees in the ERU, it can also attract the attention of the focus groups and the continuous quality committees in any of the five engineering programs in the COE and may cause these engineering programs to give their feedback to the ERU on the impact such a level of satisfaction of the outcomes might have on the subsequent courses in these programs.According to Table7, the level of satisfaction of the outcomes of the other two courses, namely, MATH2210 (differential equations) and MATH2220 (linear algebra), is similar to that for MATH1110, namely, high ability (3.5/5 and above), with a good number of their outcomes reaching the “excellent” level with clear saturation.Figure2 shows more detailed breakdown of a graphical sample of the performance level of the different outcomes for the course MATH1110 (Calculus I) over the three consecutive semesters investigated. Such data aims at showing the improvement scenarios over the semesters to show a trend and is considered as a means aiding quality monitoring and control. Generally, the performance of all outcomes of the course is consistent and is at the high ability level. The two spikes of O5 and O6 in Spring 11 are to be considered as high compared to their values in the subsequent two semesters. However, considering the performance in the three semesters in isolation from the parameters such as the number of sections offered and the number of students in the classes in the different sections over the semesters with other possible parameters being also important and worth considering is not enough to draw final conclusive conclusions on the trend of improvement for action to be taken. Anyway, a larger number of semesters need to be watched to see more representative trends, from statistical view point. Nevertheless, the overall trend shown in Figure 1 is sufficient for the sake of the present paper to make preliminary judgment on the overall achievement in the different outcomes and enough to explain the continuous quality monitoring and control approach adopted at the ERU and in the different engineering programs at UAEU.Figure 2
The level of satisfaction of the course outcomes over three semesters for the course MATH1110 (Calculus I for engineers).


#### 3.2.2. Opinion Assessment

In this subsection the data obtained based on the indirect assessment tool, namely, the students’ opinion, are presented and discussed. Table8 below shows the details of the average score of each of the course outcomes for the four courses, based on students’ surveys conducted (see the sample survey questionnaire given in Table 6). The same scale used for learning assessment is used here, where the scores reported are on a scale from 1 to 5. The target in the math courses is 70% (3.5/5). The score range from 4 to 5 is considered to be at the “excellent” level. For the score range below 4 but above 2, the level of satisfaction is considered to be “ability.” Below 2 the level of satisfaction of the outcome is judged to be “unsatisfactory” and immediate remedial actions must be taken.Table 8
Opinion assessment of course outcomes based on indirect measures “students survey” (NA: not applicable, Values in italic are below the outcomes-achievement passing criteria).
Course
Sem.
#St.
O1Av (SD)
O2Av (SD)
O3Av (SD)
O4Av (SD)
O5Av (SD)
O6Av (SD)
O7Av (SD)
O8Av (SD)
O9Av (SD)
O10Av (SD)
O11Av (SD)
MATH1110
Spring 2011
21
4.8(0.4)
5(0.0)
4.7(0.4)
4.4(0.8)
4.5(0.9)
4.7(0.4)
4.8(0.4)
4.9(0.3)
NA
NA
NA
Fall 2011
29
4.45(0.6)
4.38(0.7)
4.59(0.7)
4.45(0.6)
4.21(0.9)
4.41(0.7)
4.45(0.6)
4.38(0.7)
NA
NA
NA
Spring 2012
33
4.6(0.5)
4.5(0.7)
4.50(0.7)
3.90(1.1)
4.00(0.9)
4.2(1)
4.5(0.6)
4.5(0.8)
NA
NA
NA
3 sem. average
83
4.62
4.62
4.60
4.25
4.24
4.44
4.58
4.59
NA
NA
NA
MATH1120
Spring 2011
29
4.11(1)
4(0.7)
2.5
(1.4)
4.5(0.8)
4.5(0.6)
4.3(0.7)
4.3(0.7)
4.2(0.8)
4.4(0.7)
4.2(0.7)
NA
Fall 2011
17
4.2(1)
4.1(1.1)
2.2
(1.4)
4.4(0.7)
4.6(0.7)
4.4(1)
4.2(1)
4.1(1)
4.3(1)
4.1(0.9)
NA
Spring 2012
32
4.5(0.7)
4.27(0.8)
2.04
(1.5)
4.38(0.9)
4.50(0.7)
4.12(0.8)
4.19(0.9)
3.85(1)
4.38(0.8)
4.35(0.9)
NA
3 sem. average
78
4.24
4.09
2.15
4.28
4.55
4.32
4.2
4.02
4.34
4.19
NA
MATH2210
Spring 2011
29
4.4(0.5)
4.05(0.6)
3.9(1.2)
4.15(1)
4.1(0.7)
3.95(0.9)
4.42(0.9)
4.10(0.5)
4.42(1.2)
4.36(0.8)
4.57(0.0)
Fall 2011
20
4.00(0.7)
4.25(0.9)
4.25(0.7)
4.25(1)
4.63(0.7)
4.25(1)
3.50(0.8)
4.38(0.8)
4.13(0.9)
4.50(0.9)
4.00(0.7)
Spring 2012
21
4.44(0.5)
4.33(0.5)
4.39(0.7)
4.61(0.5)
4.44(0.5)
4.44(0.5)
4.17(0.5)
4.44(0.6)
4.06(0.8)
4.33(0.6)
4.0(0.5)
3 sem. average
70
4.28
4.21
4.18
4.34
4.39
4.21
4.03
4.31
4.20
4.40
4.19
MATH2220
Spring 2011
17
5.00(0.0)
4.71(0.5)
4.64(0.5)
4.64(0.5)
4.43(0.5)
4.57(0.7)
4.50(0.7)
4.29(0.7)
4.21(0.7)
4.20(0.7)
NA
Fall 2011
18
4.30(0.7)
4.40(0.5)
4.20(0.7)
4.60(0.5)
4.60(0.5)
4.00(0.7)
3.70(0.7)
3.40
(1)
4.60(0.5)
4.60(0.5)
NA
Spring 2012
17
4.64(0.5)
4.45(0.5)
4.54(0.5)
4.54(0.5)
4.45(0.5)
3.91(0.8)
3.73(0.8)
3.45
(0.5)
4.00(0.0)
4.10(0.0)
NA
3 sem. average
52
4.65
4.52
4.46
4.56
4.49
4.16
3.98
3.71
4.27
4.3
NAAs can be seen from Table8, almost all scores are at the excellent achievement level (above 4). The exception is the score of outcome 3 for the course MATH1120 which is at a really low level over the three semesters investigated. This outcome addresses the use of computers to enhance visualization and solve calculus problems in several variables. The students might have faced difficulty of some kind or some barrier that caused them to collectively report such a low opinion of understanding of the part of the course related to this outcome, even though, in the actual score they achieved in the assignment (direct assessment tool) related to this part, they achieved a score indicating an ability level at the higher end of the range (3.6). In the last one of the three investigated semesters the score based on the direct assessment measures was even showing an improving trend on this outcome (from 3.5 growing to 4.4) regardless of the students opinions. Anyway, it looks that the single assessment tool used to assess this outcome, namely, assignments (see Table 2), is not sufficient to come up with final conclusion and remedial actions to bring the mental status of the students to a different opinion from what had been reported by them over the three consecutive semesters investigated. This kind of feedback from the students’ opinion, although the relevant direct measure used indicates a reasonable scenario, should have triggered an improvement action that can be in the form of diversifying the direct assessment tools and/or revising the kind of assignments used in the tool used to generate the data reported in Table 7 for that outcome and possibly can lead to addressing new more attractive methods of material delivery that might cause better perception of this part of the course by the students.


#### 3.2.3. Comparison between Opinion Assessment and Learning Assessment

A comparison between the outcomes assessment results based on the different direct assessment tools used in the courses (see Tables1–4) and the indirect assessment based on the students’ opinion is given in Figure 3 for all four math courses. Figure 3 presents the overall average score for the outcomes of the courses for the three semesters investigated. It can be seen that both the direct and the indirect measures give the same overall final feedback, namely, good ability and an acceptable achievement level based on both measuring tools (direct and indirect). The gap between the results obtained from the students’ assessment and the direct assessment tools is observed to be relatively small for the four courses. Yet, the students, overall, still believe more in their ability to fulfill the course outcomes than what they really achieved based on the direct measuring tools used. This is a positive sign on their healthy attitude, but they still need to be more realistic in their self-evaluation, especially in certain outcomes of the course where Figure 3 shows the average score for all outcomes that may mask some exaggeration in students’ self-evaluation for some outcomes (see Tables 7 and 8 for full details). Alternatively, the instructors, after reflecting on the students’ self-evaluation, need to reconsider the direct assessment tools used in the courses and try to get their students more acquainted and exposed to more frequent and diverse assessment measures in order to continually bridge the gap between the students assessment and the other direct assessment tools.Figure 3
Overall average score for the outcomes of all math courses based on learning and opinion assessments.A correlation analysis was performed to identify how the direct measures relate to indirect measures as follows.For each outcome, two variables were considered.(i)
V1 is the variable of the averages of the scores in the outcome obtained from the direct measure. For instance, for outcome O1, V1 can take the following values: 3.9, 4, and 4.2.
(ii)
V2 is the variable of the averages from the indirect measure; for instance, for outcome O1, V2 can take the following values: 4.8, 4.45, and 4.6.Then the correlation between the two variables V1 and V2 was determined. In Table 9, the reader can find for each course the correlations of the different outcomes.Table 9
Correlations between direct and indirect measures.
Course
O1
O2
O3
O4
O5
O6
O7
O8
O9
O10
O11
MATH1110
−0.4
−0.8
−0.6
−0.4
0.8
0.8
0.8
1.0
NA
NA
NA
MATH1120
1.0
1.0
−1.0
−0.9
−1.0
0.2
−0.4
−0.9
−1.0
NA
MATH2210
0.6
0.9
−1.0
0.6
0.9
0.7
0.4
0.9
0.0
−1.0
−0.9
MATH2220
0.9
0.6
−0.3
0.8
0.5
0.9
0.9
−1.0
0.0
0.0
NAOne can see from Table9 that for the MATH1110 the direct and indirect measures are positively correlated for outcome O8 (perfectly) and also for O5, O6, and O7. They are negatively correlated for the remaining outcomes of this course.For the MATH1120 outcomes O1 and O2 and perfectly positively correlated while O3, O6, and O10 are perfectly inversely correlated. The correlation of outcome O4 is not calculated since the first variable is constant. For outcome O7, the direct and indirect measures are not correlated. For the MATH2210 as well as for the MATH2220, the direct and indirect measures are not correlated at all for outcomes O9. The two measures for the other outcomes are correlated. In fact, O3 and O10 from MATH2210 and O8 from MATH2220 are perfectly inversely correlated. The measures of O11 for MATH2210 are also inversely correlated and O3 from MATH2220 is slightly negatively correlated. The direct and indirect measures of the remaining outcomes are positively correlated.


### 3.3. ABET Outcomes ((a)–(k)) Level of Satisfaction Base Don Courses Outcomes Results

In this subsection, we summarize and define a conclusive criteria based on which we judge the achievement of a certain course outcome. In addition, we relate the ABET outcomes satisfaction level to the satisfaction level of the courses outcomes.


#### 3.3.1. Outcomes Achievement

The following outcomes-achievement passing criteria are defined.Criterion 1: Direct Achievement of the Outcome. An outcome is directly achieved if the average score for this outcome based on the direct assessment tools used in the course is at least 70% (3.5/5).Criterion 2: Indirect Achievement of the Outcome. An outcome is indirectly achieved if the average score for this outcome based on the indirect assessment tool represented by the students opinions is at least 70% (3.5/5).Criteria 3: Overall Achievement of an Outcome. An outcome is judged as overall achieved if it is directly and indirectly achieved according to criteria 1 and 2 above.According to the learning assessment presented in Table7, the course that is most concerning in regards of outcomes achievement is MATH1120. The course outcomes O5 to O10 received scores below 70 percent (3.5/5), which do not qualify these outcomes, according to the above criteria, to be considered achieved, although most of the scores for these outcomes are above 65 percent. In course outcomes 1 and 2 of that course, the scores show improvements where in Spring 11 and Fall 11 the scores of these two outcomes were nonqualifying but improved in Spring 12. This may indicate that the relevant instructors’ focus group in the ERU looked into the deficiency in the level of achievement of the outcomes and came up with some remedial actions, as dictated by the continuous improvement requirements imposed in the COE at UAEU to ever keep a standard educational level and to maintain ABET accreditation. These remedial actions suggested by the course focus group include increasing the tutorial sessions, providing the students with extra worksheets and modifying the course syllabus to meet exactly the needs of the students in the subsequent engineering courses.The only course among the four that according to the above adopted achievement criteria received almost no deficiencies is MATH1110. As for MATH 2210 and 2220, they received much less concerning scores, as compared to MATH1120. By relaxing the above criteria and considering the passing grade to be 65 percent; for example, much of the concerns above would be alleviated. However, justification to do so and feedback on detailed actions to improve the situation of outcomes achievements especially in MATH 1120 are required before attempting to relax the criteria.When considering the scores presented in Table8 based on the students’ feedback, one can see a much better picture than the one obtained from Table 7. The students look more optimistic about their ability to achieve the outcomes and the only serious concerning scores appear in the cells of outcome 3 of the course MATH1120.The summary of the level of achievement of the different outcomes satisfaction according to the criteria adopted above is given in Table10 and basically is decided based on the scores of Table 7 based on the direct assessment measures. At first glance, such matrix would give an easy access and insight to places in the curriculum where action is immediately needed for improvement. MATH1120 clearly is such a place where the math focus group in the ERU must be considering and monitoring over the semester to see the performance as per remedial actions taken.Table 10
Judgment on outcomes achievement based on adopted criteria for directly assessment outcomes (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
Semester
O1
O2
O3
O4
O5
O6
O7
O8
O9
O10
O11
MATH1110
Spring 2011
AC
AC
AC
AC
AC
AC
AC
AC
NA
NA
NA
Fall 2011
AC
AC
AC
AC
NAC
NAC
AC
AC
NA
NA
NA
Spring 2012
AC
AC
AC
AC
AC
AC
AC
AC
NA
NA
NA
MATH1120
Spring 2011
NAC
NAC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
Fall 2011
NAC
NAC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
Spring 2012
AC
AC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
MATH2210
Spring 2011
AC
AC
AC
AC
AC
AC
AC
AC
AC
AC
NAC
Fall 2011
AC
AC
NAC
NAC
AC
AC
AC
AC
NAC
AC
AC
Spring 2012
AC
AC
NAC
AC
AC
AC
AC
AC
AC
AC
AC
MATH2220
Spring 2011
AC
AC
AC
AC
AC
AC
AC
NAC
NAC
AC
NA
Fall 2011
AC
NAC
AC
AC
AC
AC
AC
NAC
AC
AC
NA
Spring 2012
AC
AC
AC
NAC
NAC
AC
AC
NAC
AC
AC
NA


#### 3.3.2. Curriculum Achievement (ABET Achievement)

By considering the mapping between the course outcomes and ABET criteria ((a)–(k)) presented in Tables1–5 for the four courses investigated, we come up hereby with assessment for the ABET ((a)–(k)) criteria level of satisfaction by our four math courses. The adopted criteria are summarized below.Criteria 4: ABET Outcome Directly Achieved. An ABET outcome is directly achieved if 70% of its related mathematics courses outcomes are directly achieved.Criteria 5: ABET Outcome Indirectly Achieved. An ABET outcome is indirectly achieved if 70% of its related mathematics courses outcomes are indirectly achieved.Criteria 6: ABET Outcome Achieved. An ABET outcome is achieved if it is directly and indirectly achieved.Tables11, 12, and 13 present the results on the ABET outcomes achievement. Although the above ABET outcomes achievement criteria seem somewhat stringent at first glance, especially for ABET outcomes that are covered only by a limited number of course outcomes hence giving less chance of passing by being less tolerant in accepting lack of achievements in any of the course outcomes (the case, for example, of ABET outcome (c), being only related to O4 and O5, or ABET outcome d being related to only O6 in the course MATH1110 (see Table 5)), the results reported in Table 11 show acceptable level of ABET outcomes satisfaction for three of our math courses to a good extent. The only course suffering from clear deficiencies regarding ABET satisfaction is apparently the MATH1120, which is highlighted already in the previous discussions calling hence for serious remedial actions.Table 11
Average ABET outcomes achievement level based on learning assessment tools (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(H)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
NAC
NAC
NAC
AC
NAC
NA
AC
NA
NAC
NAC
NAC
MATH2210
AC
NA
AC
AC
AC
AC
NA
NA
AC
NAC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACTable 12
Average ABET outcomes achievement level based on the indirect assessment tool “students opinion” (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
AC
AC
AC
NAC
AC
NA
AC
NA
AC
AC
AC
MATH2210
AC
AC
AC
AC
AC
AC
NA
NA
AC
AC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACTable 13
Overall average ABET outcomes achievement.
Course
(a)
(b)
(c)
(D)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
NAC
NAC
NAC
NAC
NAC
NA
AC
NA
NAC
NAC
NAC
MATH2210
AC
NA
AC
AC
AC
AC
NA
NA
AC
NAC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACOnce more, the students’ judgment on outcomes leads to having ABET outcomes being achieved to a good extent in all four courses, but this is not enough to grant overall achievement of ABET outcomes based on direct and indirect tools where the one more conservative of these tools hence determinant, which is the one based on the direct assessment tools, will dominate the final judgment.


## 3.1. Methodology and Data Collection

A major purpose of this study is to assess the part of the curricula of the engineering programs that is related to the mathematics courses offered for engineering students at the UAEU and to see the extent of the satisfaction of the ABET criteria ((a)–(k)) in this part.


### 3.1.1. Participants, Materials, and Instruments

This study is designed for the College of Engineering students at the UAEU who are taking the mathematics courses offered by the ERU-UAEU. This study is based on the feedback from two different kinds of assessment tools, direct and indirect tools. A direct tool is the one that gives a direct measure on the student performance through exams, homework assignment, project report, and so forth, while the indirect tool is the one that gives feedback on the student performance based on surveys and opinions in questionnaires and in this study this is taken based on feedback received from students’ surveys. The students’ opinions collected in surveys conducted at the end of each semester aim to see how the students judge their fulfillment of the course outcomes, regardless from their actual scores based on the direct assessment tools used. It should be noted that the score the students give in these surveys is not considered as part of the overall score the students attain in the course and is used solely as feedback to guide the improvement of the course material delivery. One of the purposes of collecting this indirect feedback from the students in each course is to see any possible discrepancy and mismatch between the actual learning direct assessment and the way how students see themselves fulfilling the course. This would help the instructor to better prepare their course material and the way they design the direct assessment tools for their courses in the subsequent semesters; the course is offered so that a better and more representing assessment of the course can be done.


### 3.1.2. Data Collection

The data collected based on the two assessment tools types described above (direct and indirect) that are reported in this paper cover a period of three consecutive semesters. These data are provided by the ERU/UAEU.


### 3.1.3. Indirect Tool

A sample of the questionnaire used to collect the indirect assessment results in the form of students’ opinion is given in Table6 for the course MATH2210: differential equations and engineering applications. In this questionnaire, the students are asked to evaluate their abilities, attributes, and skills listed in the table. The survey asks students to evaluate each course outcome based on a 1–5 scale, where 1 is very low (poor) and 5 is very high (excellent). The students should check the appropriate box that corresponds to the extent they feel the course material delivered and the way it has been delivered during the semester has helped them to achieve the course outcomes specified for the course in the syllabus.Table 6
Sample for the indirect assessment students’ survey used in the course MATH2210-differential equations and engineering applications.
Course intended outcome(by completing this course, students are able to)
Very low
↔
Very high
1
2
3
4
5
(
1) Analytically solve first-order differential equations
□
□
□
□
□
(2) Analytically solve second-order differential equations both homogenous and nonhomogenous
□
□
□
□
□
(3) Analytically solve differential equations using the Laplace transform
□
□
□
□
□
(4) Model thermal systems
□
□
□
□
□
(5) Model translational mechanical systems
□
□
□
□
□
(6) Model rotational mechanical systems
□
□
□
□
□
(7) Model electrical systems
□
□
□
□
□
(8) Use MATLAB in the solution and analysis of differential equations
□
□
□
□
□
(9) Use MATLAB in the solution and analysis of transfer functions
□
□
□
□
□
(10) Work and write a report on a term project
□
□
□
□
□
(11) Step response analysis
□
□
□
□
□


## 3.1.1. Participants, Materials, and Instruments

This study is designed for the College of Engineering students at the UAEU who are taking the mathematics courses offered by the ERU-UAEU. This study is based on the feedback from two different kinds of assessment tools, direct and indirect tools. A direct tool is the one that gives a direct measure on the student performance through exams, homework assignment, project report, and so forth, while the indirect tool is the one that gives feedback on the student performance based on surveys and opinions in questionnaires and in this study this is taken based on feedback received from students’ surveys. The students’ opinions collected in surveys conducted at the end of each semester aim to see how the students judge their fulfillment of the course outcomes, regardless from their actual scores based on the direct assessment tools used. It should be noted that the score the students give in these surveys is not considered as part of the overall score the students attain in the course and is used solely as feedback to guide the improvement of the course material delivery. One of the purposes of collecting this indirect feedback from the students in each course is to see any possible discrepancy and mismatch between the actual learning direct assessment and the way how students see themselves fulfilling the course. This would help the instructor to better prepare their course material and the way they design the direct assessment tools for their courses in the subsequent semesters; the course is offered so that a better and more representing assessment of the course can be done.


## 3.1.2. Data Collection

The data collected based on the two assessment tools types described above (direct and indirect) that are reported in this paper cover a period of three consecutive semesters. These data are provided by the ERU/UAEU.


## 3.1.3. Indirect Tool

A sample of the questionnaire used to collect the indirect assessment results in the form of students’ opinion is given in Table6 for the course MATH2210: differential equations and engineering applications. In this questionnaire, the students are asked to evaluate their abilities, attributes, and skills listed in the table. The survey asks students to evaluate each course outcome based on a 1–5 scale, where 1 is very low (poor) and 5 is very high (excellent). The students should check the appropriate box that corresponds to the extent they feel the course material delivered and the way it has been delivered during the semester has helped them to achieve the course outcomes specified for the course in the syllabus.Table 6
Sample for the indirect assessment students’ survey used in the course MATH2210-differential equations and engineering applications.
Course intended outcome(by completing this course, students are able to)
Very low
↔
Very high
1
2
3
4
5
(
1) Analytically solve first-order differential equations
□
□
□
□
□
(2) Analytically solve second-order differential equations both homogenous and nonhomogenous
□
□
□
□
□
(3) Analytically solve differential equations using the Laplace transform
□
□
□
□
□
(4) Model thermal systems
□
□
□
□
□
(5) Model translational mechanical systems
□
□
□
□
□
(6) Model rotational mechanical systems
□
□
□
□
□
(7) Model electrical systems
□
□
□
□
□
(8) Use MATLAB in the solution and analysis of differential equations
□
□
□
□
□
(9) Use MATLAB in the solution and analysis of transfer functions
□
□
□
□
□
(10) Work and write a report on a term project
□
□
□
□
□
(11) Step response analysis
□
□
□
□
□


## 3.2. Analysis of the Collected Data

### 3.2.1. Learning Assessment

The level of satisfaction of the different course outcomes of the four math courses under consideration is discussed in this subsection based on the direct assessment measures adopted in these courses. Table7 presents these direct measure data. The scores reported are on a scale from 1 to 5. The target in the math courses is 70% (3.5/5). The score range from 4 to 5 is considered to be at the “excellent” level. For the score range below 4 but above 2, the level of satisfaction is considered to be “ability.” Below 2 the level of satisfaction of the outcome is judged to be “unsatisfactory” and immediate remedial actions must be taken.Table 7
Learning assessment of course outcomes based on the direct measures given in Tables1–4 (NA: not applicable, Values in italic are below the outcomes-achievement passing criteria).
Course
Sem.
#St.
O1Av (SD)
O2Av (SD)
O3Av (SD)
O4Av (SD)
O5Av (SD)
O6Av (SD)
O7Av (SD)
O8Av (SD)
O9Av (SD)
O10Av (SD)
O11Av (SD)
MATH1110
Spring 2011
21
3.9(0.7)
3.5(0.8)
3.8(0.5)
3.6(0.6)
4.9(0.4)
4.9(0.4)
4.1(0.6)
4.0(0.6)
NA
NA
NA
Fall 2011
29
4.0(0.9)
3.8(0.9)
4.0(0.7)
3.9(0.9)
3.4
(1.1)
3.4
(1.1)
4.0(0.9)
3.7(0.8)
NA
NA
NA
Spring 2012
33
4.2(0.5)
4.1(0.7)
3.9(0.6)
3.9(0.8)
3.8(0.8)
3.8(0.8)
3.9(0.8)
3.8(0.7)
NA
NA
NA
3 sem.
average
83
4.03
3.8
3.9
3.8
4.03
4.03
4
3.8
NA
NA
NA
MATH1120
Spring 2011
29
3.3
(0.8)
3.2
(1.2)
3.5(1.2)
3.5(0.9)
2.7
(0.9)
3.3
(1.2)
3.3
(1.2)
3.4
(1.1)
3.3
(1.4)
3.2
(1.4)
NA
Fall 2011
17
3.1
(0.7)
3.3
(0.8)
3.4(0.9)
3.5(1.1)
2.9
(0.9)
3.4
(1.1)
3.2
(0.8)
3.4
(1.2)
3.1
(0.5)
3.4
(0.9)
NA
Spring 2012
32
4.3(0.7)
3.9(0.4)
4.4(0.8)
3.5(0.6)
3.3
(0.7)
3.4
(0.6)
3.3
(0.9)
3.4
(1)
2.90
(1.3)
2.90
(1.3)
NA
3 sem.
average
78
3.6
3.4
3.6
3.5
2.9
3.33
3.3
3.4
3.2
3.1
NA
MATH2210
Spring 2011
29
4(0.5)
3.8(0.6)
4(0.4)
4(0.7)
3.8(1.1)
3.8(0.7)
3.7(0.7)
4.2(0.8)
4(0.7)
4.3(0.9)
3.3
(0.9)
Fall 2011
20
4.00(0.7)
3.90(0.6)
3.40
(0.7)
3.30
(0.9)
4.40(0.8)
3.60(0.4)
3.50(0.3)
4.30(0.6)
3.20
(0.7)
3.80(0.3)
4.00(0.8)
Spring 2012
21
4.3(0.4)
4.10(0.9)
3.30
(0.3)
4.30(0.5)
4.50(0.6)
4.70(0.5)
4.50(0.3)
4.4(0.2)
4.50(0.7)
4.7(0.2)
3.7(0.9)
3 sem.
average
70
4.1
3.93
3.6
3.9
4.2
4.03
3.9
4.3
3.9
4.3
3.67
MATH2220
Spring 2011
17
4.5(0.5)
4.50(0.5)
3.70(0.7)
3.80(0.7)
4.00(1)
4.20(1.1)
3.80(0.8)
3.20
(1.1)
3.30
(1.2)
3.8(0.8)
NA
Fall 2011
18
3.90(0.9)
3.30
(0.8)
4.20(0.6)
3.90(0.6)
4.10(0.7)
4.00(0.7)
3.60(1.2)
3.80(0.9)
3.80(0.6)
3.60(0.7)
NA
Spring 2012
17
4.5(0.3)
4.5(0.3)
4.5(0.4)
3.4
(0.4)
3.4
(0.7)
3.7(0.6)
3.7(0.6)
3.90(0.6)
3.9(0.6)
3.5(0.6)
NA
3 sem.
average
52
4.3
4.1
4.13
3.7
3.8
4
3.7
3.6
3.7
3.63
NAFigure1 shows the average level of satisfaction of the course outcomes over three consecutive semesters for all of the math courses. From Table 7 and Figure 1, most outcomes for the course MATH1110 (Calculus 1) are satisfied to the level of high ability (above 3.5) with some of them touching the border of the “excellent” level (outcomes 1, 5, 6, and 7). For the course MATH1120 (Calculus II), none of the outcomes of that course are fulfilled to the excellent level, contrary to the course MATH1110. Most of the outcomes of this course are satisfied to the level of good ability (3 to 3.5), with only one outcome being satisfied to the level of ability but in the lower ability range below 3 (outcome 5). By referring to Table 2, it can be seen that this outcome is assessed mainly by major exams such as midterm exam and final exam. It should be mentioned that the course MATH1120 (Calculus II) contains several new topics, which are never covered in high school, such as vector-value functions and functions of several variables. This course is designed to prepare the engineering students for advanced courses in different engineering disciplines.Figure 1
The average level of satisfaction of the course outcomes over three semesters for all math courses.Although the level of satisfaction of most outcomes for MATH1120 is judged as “ability” and above, our target (as mentioned earlier) is a level of no less than 3.5/5 (70 percent) and so the result reported for most outcomes should be triggering the actions of the instructors teaching the course to do some action to improve the performance of students on these outcomes and/or to revise the way these outcome are assessed. This is considered as part of our commitment to continuous improvement efforts. The level of satisfaction of outcomes like O5 and O10 in MATH1120 (Calculus II), as judged based on the used direct assessment tools; although this should not be the only trigger for improvement actions by different entities and committees in the ERU, it can also attract the attention of the focus groups and the continuous quality committees in any of the five engineering programs in the COE and may cause these engineering programs to give their feedback to the ERU on the impact such a level of satisfaction of the outcomes might have on the subsequent courses in these programs.According to Table7, the level of satisfaction of the outcomes of the other two courses, namely, MATH2210 (differential equations) and MATH2220 (linear algebra), is similar to that for MATH1110, namely, high ability (3.5/5 and above), with a good number of their outcomes reaching the “excellent” level with clear saturation.Figure2 shows more detailed breakdown of a graphical sample of the performance level of the different outcomes for the course MATH1110 (Calculus I) over the three consecutive semesters investigated. Such data aims at showing the improvement scenarios over the semesters to show a trend and is considered as a means aiding quality monitoring and control. Generally, the performance of all outcomes of the course is consistent and is at the high ability level. The two spikes of O5 and O6 in Spring 11 are to be considered as high compared to their values in the subsequent two semesters. However, considering the performance in the three semesters in isolation from the parameters such as the number of sections offered and the number of students in the classes in the different sections over the semesters with other possible parameters being also important and worth considering is not enough to draw final conclusive conclusions on the trend of improvement for action to be taken. Anyway, a larger number of semesters need to be watched to see more representative trends, from statistical view point. Nevertheless, the overall trend shown in Figure 1 is sufficient for the sake of the present paper to make preliminary judgment on the overall achievement in the different outcomes and enough to explain the continuous quality monitoring and control approach adopted at the ERU and in the different engineering programs at UAEU.Figure 2
The level of satisfaction of the course outcomes over three semesters for the course MATH1110 (Calculus I for engineers).


### 3.2.2. Opinion Assessment

In this subsection the data obtained based on the indirect assessment tool, namely, the students’ opinion, are presented and discussed. Table8 below shows the details of the average score of each of the course outcomes for the four courses, based on students’ surveys conducted (see the sample survey questionnaire given in Table 6). The same scale used for learning assessment is used here, where the scores reported are on a scale from 1 to 5. The target in the math courses is 70% (3.5/5). The score range from 4 to 5 is considered to be at the “excellent” level. For the score range below 4 but above 2, the level of satisfaction is considered to be “ability.” Below 2 the level of satisfaction of the outcome is judged to be “unsatisfactory” and immediate remedial actions must be taken.Table 8
Opinion assessment of course outcomes based on indirect measures “students survey” (NA: not applicable, Values in italic are below the outcomes-achievement passing criteria).
Course
Sem.
#St.
O1Av (SD)
O2Av (SD)
O3Av (SD)
O4Av (SD)
O5Av (SD)
O6Av (SD)
O7Av (SD)
O8Av (SD)
O9Av (SD)
O10Av (SD)
O11Av (SD)
MATH1110
Spring 2011
21
4.8(0.4)
5(0.0)
4.7(0.4)
4.4(0.8)
4.5(0.9)
4.7(0.4)
4.8(0.4)
4.9(0.3)
NA
NA
NA
Fall 2011
29
4.45(0.6)
4.38(0.7)
4.59(0.7)
4.45(0.6)
4.21(0.9)
4.41(0.7)
4.45(0.6)
4.38(0.7)
NA
NA
NA
Spring 2012
33
4.6(0.5)
4.5(0.7)
4.50(0.7)
3.90(1.1)
4.00(0.9)
4.2(1)
4.5(0.6)
4.5(0.8)
NA
NA
NA
3 sem. average
83
4.62
4.62
4.60
4.25
4.24
4.44
4.58
4.59
NA
NA
NA
MATH1120
Spring 2011
29
4.11(1)
4(0.7)
2.5
(1.4)
4.5(0.8)
4.5(0.6)
4.3(0.7)
4.3(0.7)
4.2(0.8)
4.4(0.7)
4.2(0.7)
NA
Fall 2011
17
4.2(1)
4.1(1.1)
2.2
(1.4)
4.4(0.7)
4.6(0.7)
4.4(1)
4.2(1)
4.1(1)
4.3(1)
4.1(0.9)
NA
Spring 2012
32
4.5(0.7)
4.27(0.8)
2.04
(1.5)
4.38(0.9)
4.50(0.7)
4.12(0.8)
4.19(0.9)
3.85(1)
4.38(0.8)
4.35(0.9)
NA
3 sem. average
78
4.24
4.09
2.15
4.28
4.55
4.32
4.2
4.02
4.34
4.19
NA
MATH2210
Spring 2011
29
4.4(0.5)
4.05(0.6)
3.9(1.2)
4.15(1)
4.1(0.7)
3.95(0.9)
4.42(0.9)
4.10(0.5)
4.42(1.2)
4.36(0.8)
4.57(0.0)
Fall 2011
20
4.00(0.7)
4.25(0.9)
4.25(0.7)
4.25(1)
4.63(0.7)
4.25(1)
3.50(0.8)
4.38(0.8)
4.13(0.9)
4.50(0.9)
4.00(0.7)
Spring 2012
21
4.44(0.5)
4.33(0.5)
4.39(0.7)
4.61(0.5)
4.44(0.5)
4.44(0.5)
4.17(0.5)
4.44(0.6)
4.06(0.8)
4.33(0.6)
4.0(0.5)
3 sem. average
70
4.28
4.21
4.18
4.34
4.39
4.21
4.03
4.31
4.20
4.40
4.19
MATH2220
Spring 2011
17
5.00(0.0)
4.71(0.5)
4.64(0.5)
4.64(0.5)
4.43(0.5)
4.57(0.7)
4.50(0.7)
4.29(0.7)
4.21(0.7)
4.20(0.7)
NA
Fall 2011
18
4.30(0.7)
4.40(0.5)
4.20(0.7)
4.60(0.5)
4.60(0.5)
4.00(0.7)
3.70(0.7)
3.40
(1)
4.60(0.5)
4.60(0.5)
NA
Spring 2012
17
4.64(0.5)
4.45(0.5)
4.54(0.5)
4.54(0.5)
4.45(0.5)
3.91(0.8)
3.73(0.8)
3.45
(0.5)
4.00(0.0)
4.10(0.0)
NA
3 sem. average
52
4.65
4.52
4.46
4.56
4.49
4.16
3.98
3.71
4.27
4.3
NAAs can be seen from Table8, almost all scores are at the excellent achievement level (above 4). The exception is the score of outcome 3 for the course MATH1120 which is at a really low level over the three semesters investigated. This outcome addresses the use of computers to enhance visualization and solve calculus problems in several variables. The students might have faced difficulty of some kind or some barrier that caused them to collectively report such a low opinion of understanding of the part of the course related to this outcome, even though, in the actual score they achieved in the assignment (direct assessment tool) related to this part, they achieved a score indicating an ability level at the higher end of the range (3.6). In the last one of the three investigated semesters the score based on the direct assessment measures was even showing an improving trend on this outcome (from 3.5 growing to 4.4) regardless of the students opinions. Anyway, it looks that the single assessment tool used to assess this outcome, namely, assignments (see Table 2), is not sufficient to come up with final conclusion and remedial actions to bring the mental status of the students to a different opinion from what had been reported by them over the three consecutive semesters investigated. This kind of feedback from the students’ opinion, although the relevant direct measure used indicates a reasonable scenario, should have triggered an improvement action that can be in the form of diversifying the direct assessment tools and/or revising the kind of assignments used in the tool used to generate the data reported in Table 7 for that outcome and possibly can lead to addressing new more attractive methods of material delivery that might cause better perception of this part of the course by the students.


### 3.2.3. Comparison between Opinion Assessment and Learning Assessment

A comparison between the outcomes assessment results based on the different direct assessment tools used in the courses (see Tables1–4) and the indirect assessment based on the students’ opinion is given in Figure 3 for all four math courses. Figure 3 presents the overall average score for the outcomes of the courses for the three semesters investigated. It can be seen that both the direct and the indirect measures give the same overall final feedback, namely, good ability and an acceptable achievement level based on both measuring tools (direct and indirect). The gap between the results obtained from the students’ assessment and the direct assessment tools is observed to be relatively small for the four courses. Yet, the students, overall, still believe more in their ability to fulfill the course outcomes than what they really achieved based on the direct measuring tools used. This is a positive sign on their healthy attitude, but they still need to be more realistic in their self-evaluation, especially in certain outcomes of the course where Figure 3 shows the average score for all outcomes that may mask some exaggeration in students’ self-evaluation for some outcomes (see Tables 7 and 8 for full details). Alternatively, the instructors, after reflecting on the students’ self-evaluation, need to reconsider the direct assessment tools used in the courses and try to get their students more acquainted and exposed to more frequent and diverse assessment measures in order to continually bridge the gap between the students assessment and the other direct assessment tools.Figure 3
Overall average score for the outcomes of all math courses based on learning and opinion assessments.A correlation analysis was performed to identify how the direct measures relate to indirect measures as follows.For each outcome, two variables were considered.(i)
V1 is the variable of the averages of the scores in the outcome obtained from the direct measure. For instance, for outcome O1, V1 can take the following values: 3.9, 4, and 4.2.
(ii)
V2 is the variable of the averages from the indirect measure; for instance, for outcome O1, V2 can take the following values: 4.8, 4.45, and 4.6.Then the correlation between the two variables V1 and V2 was determined. In Table 9, the reader can find for each course the correlations of the different outcomes.Table 9
Correlations between direct and indirect measures.
Course
O1
O2
O3
O4
O5
O6
O7
O8
O9
O10
O11
MATH1110
−0.4
−0.8
−0.6
−0.4
0.8
0.8
0.8
1.0
NA
NA
NA
MATH1120
1.0
1.0
−1.0
−0.9
−1.0
0.2
−0.4
−0.9
−1.0
NA
MATH2210
0.6
0.9
−1.0
0.6
0.9
0.7
0.4
0.9
0.0
−1.0
−0.9
MATH2220
0.9
0.6
−0.3
0.8
0.5
0.9
0.9
−1.0
0.0
0.0
NAOne can see from Table9 that for the MATH1110 the direct and indirect measures are positively correlated for outcome O8 (perfectly) and also for O5, O6, and O7. They are negatively correlated for the remaining outcomes of this course.For the MATH1120 outcomes O1 and O2 and perfectly positively correlated while O3, O6, and O10 are perfectly inversely correlated. The correlation of outcome O4 is not calculated since the first variable is constant. For outcome O7, the direct and indirect measures are not correlated. For the MATH2210 as well as for the MATH2220, the direct and indirect measures are not correlated at all for outcomes O9. The two measures for the other outcomes are correlated. In fact, O3 and O10 from MATH2210 and O8 from MATH2220 are perfectly inversely correlated. The measures of O11 for MATH2210 are also inversely correlated and O3 from MATH2220 is slightly negatively correlated. The direct and indirect measures of the remaining outcomes are positively correlated.


## 3.2.1. Learning Assessment

The level of satisfaction of the different course outcomes of the four math courses under consideration is discussed in this subsection based on the direct assessment measures adopted in these courses. Table7 presents these direct measure data. The scores reported are on a scale from 1 to 5. The target in the math courses is 70% (3.5/5). The score range from 4 to 5 is considered to be at the “excellent” level. For the score range below 4 but above 2, the level of satisfaction is considered to be “ability.” Below 2 the level of satisfaction of the outcome is judged to be “unsatisfactory” and immediate remedial actions must be taken.Table 7
Learning assessment of course outcomes based on the direct measures given in Tables1–4 (NA: not applicable, Values in italic are below the outcomes-achievement passing criteria).
Course
Sem.
#St.
O1Av (SD)
O2Av (SD)
O3Av (SD)
O4Av (SD)
O5Av (SD)
O6Av (SD)
O7Av (SD)
O8Av (SD)
O9Av (SD)
O10Av (SD)
O11Av (SD)
MATH1110
Spring 2011
21
3.9(0.7)
3.5(0.8)
3.8(0.5)
3.6(0.6)
4.9(0.4)
4.9(0.4)
4.1(0.6)
4.0(0.6)
NA
NA
NA
Fall 2011
29
4.0(0.9)
3.8(0.9)
4.0(0.7)
3.9(0.9)
3.4
(1.1)
3.4
(1.1)
4.0(0.9)
3.7(0.8)
NA
NA
NA
Spring 2012
33
4.2(0.5)
4.1(0.7)
3.9(0.6)
3.9(0.8)
3.8(0.8)
3.8(0.8)
3.9(0.8)
3.8(0.7)
NA
NA
NA
3 sem.
average
83
4.03
3.8
3.9
3.8
4.03
4.03
4
3.8
NA
NA
NA
MATH1120
Spring 2011
29
3.3
(0.8)
3.2
(1.2)
3.5(1.2)
3.5(0.9)
2.7
(0.9)
3.3
(1.2)
3.3
(1.2)
3.4
(1.1)
3.3
(1.4)
3.2
(1.4)
NA
Fall 2011
17
3.1
(0.7)
3.3
(0.8)
3.4(0.9)
3.5(1.1)
2.9
(0.9)
3.4
(1.1)
3.2
(0.8)
3.4
(1.2)
3.1
(0.5)
3.4
(0.9)
NA
Spring 2012
32
4.3(0.7)
3.9(0.4)
4.4(0.8)
3.5(0.6)
3.3
(0.7)
3.4
(0.6)
3.3
(0.9)
3.4
(1)
2.90
(1.3)
2.90
(1.3)
NA
3 sem.
average
78
3.6
3.4
3.6
3.5
2.9
3.33
3.3
3.4
3.2
3.1
NA
MATH2210
Spring 2011
29
4(0.5)
3.8(0.6)
4(0.4)
4(0.7)
3.8(1.1)
3.8(0.7)
3.7(0.7)
4.2(0.8)
4(0.7)
4.3(0.9)
3.3
(0.9)
Fall 2011
20
4.00(0.7)
3.90(0.6)
3.40
(0.7)
3.30
(0.9)
4.40(0.8)
3.60(0.4)
3.50(0.3)
4.30(0.6)
3.20
(0.7)
3.80(0.3)
4.00(0.8)
Spring 2012
21
4.3(0.4)
4.10(0.9)
3.30
(0.3)
4.30(0.5)
4.50(0.6)
4.70(0.5)
4.50(0.3)
4.4(0.2)
4.50(0.7)
4.7(0.2)
3.7(0.9)
3 sem.
average
70
4.1
3.93
3.6
3.9
4.2
4.03
3.9
4.3
3.9
4.3
3.67
MATH2220
Spring 2011
17
4.5(0.5)
4.50(0.5)
3.70(0.7)
3.80(0.7)
4.00(1)
4.20(1.1)
3.80(0.8)
3.20
(1.1)
3.30
(1.2)
3.8(0.8)
NA
Fall 2011
18
3.90(0.9)
3.30
(0.8)
4.20(0.6)
3.90(0.6)
4.10(0.7)
4.00(0.7)
3.60(1.2)
3.80(0.9)
3.80(0.6)
3.60(0.7)
NA
Spring 2012
17
4.5(0.3)
4.5(0.3)
4.5(0.4)
3.4
(0.4)
3.4
(0.7)
3.7(0.6)
3.7(0.6)
3.90(0.6)
3.9(0.6)
3.5(0.6)
NA
3 sem.
average
52
4.3
4.1
4.13
3.7
3.8
4
3.7
3.6
3.7
3.63
NAFigure1 shows the average level of satisfaction of the course outcomes over three consecutive semesters for all of the math courses. From Table 7 and Figure 1, most outcomes for the course MATH1110 (Calculus 1) are satisfied to the level of high ability (above 3.5) with some of them touching the border of the “excellent” level (outcomes 1, 5, 6, and 7). For the course MATH1120 (Calculus II), none of the outcomes of that course are fulfilled to the excellent level, contrary to the course MATH1110. Most of the outcomes of this course are satisfied to the level of good ability (3 to 3.5), with only one outcome being satisfied to the level of ability but in the lower ability range below 3 (outcome 5). By referring to Table 2, it can be seen that this outcome is assessed mainly by major exams such as midterm exam and final exam. It should be mentioned that the course MATH1120 (Calculus II) contains several new topics, which are never covered in high school, such as vector-value functions and functions of several variables. This course is designed to prepare the engineering students for advanced courses in different engineering disciplines.Figure 1
The average level of satisfaction of the course outcomes over three semesters for all math courses.Although the level of satisfaction of most outcomes for MATH1120 is judged as “ability” and above, our target (as mentioned earlier) is a level of no less than 3.5/5 (70 percent) and so the result reported for most outcomes should be triggering the actions of the instructors teaching the course to do some action to improve the performance of students on these outcomes and/or to revise the way these outcome are assessed. This is considered as part of our commitment to continuous improvement efforts. The level of satisfaction of outcomes like O5 and O10 in MATH1120 (Calculus II), as judged based on the used direct assessment tools; although this should not be the only trigger for improvement actions by different entities and committees in the ERU, it can also attract the attention of the focus groups and the continuous quality committees in any of the five engineering programs in the COE and may cause these engineering programs to give their feedback to the ERU on the impact such a level of satisfaction of the outcomes might have on the subsequent courses in these programs.According to Table7, the level of satisfaction of the outcomes of the other two courses, namely, MATH2210 (differential equations) and MATH2220 (linear algebra), is similar to that for MATH1110, namely, high ability (3.5/5 and above), with a good number of their outcomes reaching the “excellent” level with clear saturation.Figure2 shows more detailed breakdown of a graphical sample of the performance level of the different outcomes for the course MATH1110 (Calculus I) over the three consecutive semesters investigated. Such data aims at showing the improvement scenarios over the semesters to show a trend and is considered as a means aiding quality monitoring and control. Generally, the performance of all outcomes of the course is consistent and is at the high ability level. The two spikes of O5 and O6 in Spring 11 are to be considered as high compared to their values in the subsequent two semesters. However, considering the performance in the three semesters in isolation from the parameters such as the number of sections offered and the number of students in the classes in the different sections over the semesters with other possible parameters being also important and worth considering is not enough to draw final conclusive conclusions on the trend of improvement for action to be taken. Anyway, a larger number of semesters need to be watched to see more representative trends, from statistical view point. Nevertheless, the overall trend shown in Figure 1 is sufficient for the sake of the present paper to make preliminary judgment on the overall achievement in the different outcomes and enough to explain the continuous quality monitoring and control approach adopted at the ERU and in the different engineering programs at UAEU.Figure 2
The level of satisfaction of the course outcomes over three semesters for the course MATH1110 (Calculus I for engineers).


## 3.2.2. Opinion Assessment

In this subsection the data obtained based on the indirect assessment tool, namely, the students’ opinion, are presented and discussed. Table8 below shows the details of the average score of each of the course outcomes for the four courses, based on students’ surveys conducted (see the sample survey questionnaire given in Table 6). The same scale used for learning assessment is used here, where the scores reported are on a scale from 1 to 5. The target in the math courses is 70% (3.5/5). The score range from 4 to 5 is considered to be at the “excellent” level. For the score range below 4 but above 2, the level of satisfaction is considered to be “ability.” Below 2 the level of satisfaction of the outcome is judged to be “unsatisfactory” and immediate remedial actions must be taken.Table 8
Opinion assessment of course outcomes based on indirect measures “students survey” (NA: not applicable, Values in italic are below the outcomes-achievement passing criteria).
Course
Sem.
#St.
O1Av (SD)
O2Av (SD)
O3Av (SD)
O4Av (SD)
O5Av (SD)
O6Av (SD)
O7Av (SD)
O8Av (SD)
O9Av (SD)
O10Av (SD)
O11Av (SD)
MATH1110
Spring 2011
21
4.8(0.4)
5(0.0)
4.7(0.4)
4.4(0.8)
4.5(0.9)
4.7(0.4)
4.8(0.4)
4.9(0.3)
NA
NA
NA
Fall 2011
29
4.45(0.6)
4.38(0.7)
4.59(0.7)
4.45(0.6)
4.21(0.9)
4.41(0.7)
4.45(0.6)
4.38(0.7)
NA
NA
NA
Spring 2012
33
4.6(0.5)
4.5(0.7)
4.50(0.7)
3.90(1.1)
4.00(0.9)
4.2(1)
4.5(0.6)
4.5(0.8)
NA
NA
NA
3 sem. average
83
4.62
4.62
4.60
4.25
4.24
4.44
4.58
4.59
NA
NA
NA
MATH1120
Spring 2011
29
4.11(1)
4(0.7)
2.5
(1.4)
4.5(0.8)
4.5(0.6)
4.3(0.7)
4.3(0.7)
4.2(0.8)
4.4(0.7)
4.2(0.7)
NA
Fall 2011
17
4.2(1)
4.1(1.1)
2.2
(1.4)
4.4(0.7)
4.6(0.7)
4.4(1)
4.2(1)
4.1(1)
4.3(1)
4.1(0.9)
NA
Spring 2012
32
4.5(0.7)
4.27(0.8)
2.04
(1.5)
4.38(0.9)
4.50(0.7)
4.12(0.8)
4.19(0.9)
3.85(1)
4.38(0.8)
4.35(0.9)
NA
3 sem. average
78
4.24
4.09
2.15
4.28
4.55
4.32
4.2
4.02
4.34
4.19
NA
MATH2210
Spring 2011
29
4.4(0.5)
4.05(0.6)
3.9(1.2)
4.15(1)
4.1(0.7)
3.95(0.9)
4.42(0.9)
4.10(0.5)
4.42(1.2)
4.36(0.8)
4.57(0.0)
Fall 2011
20
4.00(0.7)
4.25(0.9)
4.25(0.7)
4.25(1)
4.63(0.7)
4.25(1)
3.50(0.8)
4.38(0.8)
4.13(0.9)
4.50(0.9)
4.00(0.7)
Spring 2012
21
4.44(0.5)
4.33(0.5)
4.39(0.7)
4.61(0.5)
4.44(0.5)
4.44(0.5)
4.17(0.5)
4.44(0.6)
4.06(0.8)
4.33(0.6)
4.0(0.5)
3 sem. average
70
4.28
4.21
4.18
4.34
4.39
4.21
4.03
4.31
4.20
4.40
4.19
MATH2220
Spring 2011
17
5.00(0.0)
4.71(0.5)
4.64(0.5)
4.64(0.5)
4.43(0.5)
4.57(0.7)
4.50(0.7)
4.29(0.7)
4.21(0.7)
4.20(0.7)
NA
Fall 2011
18
4.30(0.7)
4.40(0.5)
4.20(0.7)
4.60(0.5)
4.60(0.5)
4.00(0.7)
3.70(0.7)
3.40
(1)
4.60(0.5)
4.60(0.5)
NA
Spring 2012
17
4.64(0.5)
4.45(0.5)
4.54(0.5)
4.54(0.5)
4.45(0.5)
3.91(0.8)
3.73(0.8)
3.45
(0.5)
4.00(0.0)
4.10(0.0)
NA
3 sem. average
52
4.65
4.52
4.46
4.56
4.49
4.16
3.98
3.71
4.27
4.3
NAAs can be seen from Table8, almost all scores are at the excellent achievement level (above 4). The exception is the score of outcome 3 for the course MATH1120 which is at a really low level over the three semesters investigated. This outcome addresses the use of computers to enhance visualization and solve calculus problems in several variables. The students might have faced difficulty of some kind or some barrier that caused them to collectively report such a low opinion of understanding of the part of the course related to this outcome, even though, in the actual score they achieved in the assignment (direct assessment tool) related to this part, they achieved a score indicating an ability level at the higher end of the range (3.6). In the last one of the three investigated semesters the score based on the direct assessment measures was even showing an improving trend on this outcome (from 3.5 growing to 4.4) regardless of the students opinions. Anyway, it looks that the single assessment tool used to assess this outcome, namely, assignments (see Table 2), is not sufficient to come up with final conclusion and remedial actions to bring the mental status of the students to a different opinion from what had been reported by them over the three consecutive semesters investigated. This kind of feedback from the students’ opinion, although the relevant direct measure used indicates a reasonable scenario, should have triggered an improvement action that can be in the form of diversifying the direct assessment tools and/or revising the kind of assignments used in the tool used to generate the data reported in Table 7 for that outcome and possibly can lead to addressing new more attractive methods of material delivery that might cause better perception of this part of the course by the students.


## 3.2.3. Comparison between Opinion Assessment and Learning Assessment

A comparison between the outcomes assessment results based on the different direct assessment tools used in the courses (see Tables1–4) and the indirect assessment based on the students’ opinion is given in Figure 3 for all four math courses. Figure 3 presents the overall average score for the outcomes of the courses for the three semesters investigated. It can be seen that both the direct and the indirect measures give the same overall final feedback, namely, good ability and an acceptable achievement level based on both measuring tools (direct and indirect). The gap between the results obtained from the students’ assessment and the direct assessment tools is observed to be relatively small for the four courses. Yet, the students, overall, still believe more in their ability to fulfill the course outcomes than what they really achieved based on the direct measuring tools used. This is a positive sign on their healthy attitude, but they still need to be more realistic in their self-evaluation, especially in certain outcomes of the course where Figure 3 shows the average score for all outcomes that may mask some exaggeration in students’ self-evaluation for some outcomes (see Tables 7 and 8 for full details). Alternatively, the instructors, after reflecting on the students’ self-evaluation, need to reconsider the direct assessment tools used in the courses and try to get their students more acquainted and exposed to more frequent and diverse assessment measures in order to continually bridge the gap between the students assessment and the other direct assessment tools.Figure 3
Overall average score for the outcomes of all math courses based on learning and opinion assessments.A correlation analysis was performed to identify how the direct measures relate to indirect measures as follows.For each outcome, two variables were considered.(i)
V1 is the variable of the averages of the scores in the outcome obtained from the direct measure. For instance, for outcome O1, V1 can take the following values: 3.9, 4, and 4.2.
(ii)
V2 is the variable of the averages from the indirect measure; for instance, for outcome O1, V2 can take the following values: 4.8, 4.45, and 4.6.Then the correlation between the two variables V1 and V2 was determined. In Table 9, the reader can find for each course the correlations of the different outcomes.Table 9
Correlations between direct and indirect measures.
Course
O1
O2
O3
O4
O5
O6
O7
O8
O9
O10
O11
MATH1110
−0.4
−0.8
−0.6
−0.4
0.8
0.8
0.8
1.0
NA
NA
NA
MATH1120
1.0
1.0
−1.0
−0.9
−1.0
0.2
−0.4
−0.9
−1.0
NA
MATH2210
0.6
0.9
−1.0
0.6
0.9
0.7
0.4
0.9
0.0
−1.0
−0.9
MATH2220
0.9
0.6
−0.3
0.8
0.5
0.9
0.9
−1.0
0.0
0.0
NAOne can see from Table9 that for the MATH1110 the direct and indirect measures are positively correlated for outcome O8 (perfectly) and also for O5, O6, and O7. They are negatively correlated for the remaining outcomes of this course.For the MATH1120 outcomes O1 and O2 and perfectly positively correlated while O3, O6, and O10 are perfectly inversely correlated. The correlation of outcome O4 is not calculated since the first variable is constant. For outcome O7, the direct and indirect measures are not correlated. For the MATH2210 as well as for the MATH2220, the direct and indirect measures are not correlated at all for outcomes O9. The two measures for the other outcomes are correlated. In fact, O3 and O10 from MATH2210 and O8 from MATH2220 are perfectly inversely correlated. The measures of O11 for MATH2210 are also inversely correlated and O3 from MATH2220 is slightly negatively correlated. The direct and indirect measures of the remaining outcomes are positively correlated.


## 3.3. ABET Outcomes ((a)–(k)) Level of Satisfaction Base Don Courses Outcomes Results

In this subsection, we summarize and define a conclusive criteria based on which we judge the achievement of a certain course outcome. In addition, we relate the ABET outcomes satisfaction level to the satisfaction level of the courses outcomes.


### 3.3.1. Outcomes Achievement

The following outcomes-achievement passing criteria are defined.Criterion 1: Direct Achievement of the Outcome. An outcome is directly achieved if the average score for this outcome based on the direct assessment tools used in the course is at least 70% (3.5/5).Criterion 2: Indirect Achievement of the Outcome. An outcome is indirectly achieved if the average score for this outcome based on the indirect assessment tool represented by the students opinions is at least 70% (3.5/5).Criteria 3: Overall Achievement of an Outcome. An outcome is judged as overall achieved if it is directly and indirectly achieved according to criteria 1 and 2 above.According to the learning assessment presented in Table7, the course that is most concerning in regards of outcomes achievement is MATH1120. The course outcomes O5 to O10 received scores below 70 percent (3.5/5), which do not qualify these outcomes, according to the above criteria, to be considered achieved, although most of the scores for these outcomes are above 65 percent. In course outcomes 1 and 2 of that course, the scores show improvements where in Spring 11 and Fall 11 the scores of these two outcomes were nonqualifying but improved in Spring 12. This may indicate that the relevant instructors’ focus group in the ERU looked into the deficiency in the level of achievement of the outcomes and came up with some remedial actions, as dictated by the continuous improvement requirements imposed in the COE at UAEU to ever keep a standard educational level and to maintain ABET accreditation. These remedial actions suggested by the course focus group include increasing the tutorial sessions, providing the students with extra worksheets and modifying the course syllabus to meet exactly the needs of the students in the subsequent engineering courses.The only course among the four that according to the above adopted achievement criteria received almost no deficiencies is MATH1110. As for MATH 2210 and 2220, they received much less concerning scores, as compared to MATH1120. By relaxing the above criteria and considering the passing grade to be 65 percent; for example, much of the concerns above would be alleviated. However, justification to do so and feedback on detailed actions to improve the situation of outcomes achievements especially in MATH 1120 are required before attempting to relax the criteria.When considering the scores presented in Table8 based on the students’ feedback, one can see a much better picture than the one obtained from Table 7. The students look more optimistic about their ability to achieve the outcomes and the only serious concerning scores appear in the cells of outcome 3 of the course MATH1120.The summary of the level of achievement of the different outcomes satisfaction according to the criteria adopted above is given in Table10 and basically is decided based on the scores of Table 7 based on the direct assessment measures. At first glance, such matrix would give an easy access and insight to places in the curriculum where action is immediately needed for improvement. MATH1120 clearly is such a place where the math focus group in the ERU must be considering and monitoring over the semester to see the performance as per remedial actions taken.Table 10
Judgment on outcomes achievement based on adopted criteria for directly assessment outcomes (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
Semester
O1
O2
O3
O4
O5
O6
O7
O8
O9
O10
O11
MATH1110
Spring 2011
AC
AC
AC
AC
AC
AC
AC
AC
NA
NA
NA
Fall 2011
AC
AC
AC
AC
NAC
NAC
AC
AC
NA
NA
NA
Spring 2012
AC
AC
AC
AC
AC
AC
AC
AC
NA
NA
NA
MATH1120
Spring 2011
NAC
NAC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
Fall 2011
NAC
NAC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
Spring 2012
AC
AC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
MATH2210
Spring 2011
AC
AC
AC
AC
AC
AC
AC
AC
AC
AC
NAC
Fall 2011
AC
AC
NAC
NAC
AC
AC
AC
AC
NAC
AC
AC
Spring 2012
AC
AC
NAC
AC
AC
AC
AC
AC
AC
AC
AC
MATH2220
Spring 2011
AC
AC
AC
AC
AC
AC
AC
NAC
NAC
AC
NA
Fall 2011
AC
NAC
AC
AC
AC
AC
AC
NAC
AC
AC
NA
Spring 2012
AC
AC
AC
NAC
NAC
AC
AC
NAC
AC
AC
NA


### 3.3.2. Curriculum Achievement (ABET Achievement)

By considering the mapping between the course outcomes and ABET criteria ((a)–(k)) presented in Tables1–5 for the four courses investigated, we come up hereby with assessment for the ABET ((a)–(k)) criteria level of satisfaction by our four math courses. The adopted criteria are summarized below.Criteria 4: ABET Outcome Directly Achieved. An ABET outcome is directly achieved if 70% of its related mathematics courses outcomes are directly achieved.Criteria 5: ABET Outcome Indirectly Achieved. An ABET outcome is indirectly achieved if 70% of its related mathematics courses outcomes are indirectly achieved.Criteria 6: ABET Outcome Achieved. An ABET outcome is achieved if it is directly and indirectly achieved.Tables11, 12, and 13 present the results on the ABET outcomes achievement. Although the above ABET outcomes achievement criteria seem somewhat stringent at first glance, especially for ABET outcomes that are covered only by a limited number of course outcomes hence giving less chance of passing by being less tolerant in accepting lack of achievements in any of the course outcomes (the case, for example, of ABET outcome (c), being only related to O4 and O5, or ABET outcome d being related to only O6 in the course MATH1110 (see Table 5)), the results reported in Table 11 show acceptable level of ABET outcomes satisfaction for three of our math courses to a good extent. The only course suffering from clear deficiencies regarding ABET satisfaction is apparently the MATH1120, which is highlighted already in the previous discussions calling hence for serious remedial actions.Table 11
Average ABET outcomes achievement level based on learning assessment tools (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(H)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
NAC
NAC
NAC
AC
NAC
NA
AC
NA
NAC
NAC
NAC
MATH2210
AC
NA
AC
AC
AC
AC
NA
NA
AC
NAC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACTable 12
Average ABET outcomes achievement level based on the indirect assessment tool “students opinion” (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
AC
AC
AC
NAC
AC
NA
AC
NA
AC
AC
AC
MATH2210
AC
AC
AC
AC
AC
AC
NA
NA
AC
AC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACTable 13
Overall average ABET outcomes achievement.
Course
(a)
(b)
(c)
(D)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
NAC
NAC
NAC
NAC
NAC
NA
AC
NA
NAC
NAC
NAC
MATH2210
AC
NA
AC
AC
AC
AC
NA
NA
AC
NAC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACOnce more, the students’ judgment on outcomes leads to having ABET outcomes being achieved to a good extent in all four courses, but this is not enough to grant overall achievement of ABET outcomes based on direct and indirect tools where the one more conservative of these tools hence determinant, which is the one based on the direct assessment tools, will dominate the final judgment.


## 3.3.1. Outcomes Achievement

The following outcomes-achievement passing criteria are defined.Criterion 1: Direct Achievement of the Outcome. An outcome is directly achieved if the average score for this outcome based on the direct assessment tools used in the course is at least 70% (3.5/5).Criterion 2: Indirect Achievement of the Outcome. An outcome is indirectly achieved if the average score for this outcome based on the indirect assessment tool represented by the students opinions is at least 70% (3.5/5).Criteria 3: Overall Achievement of an Outcome. An outcome is judged as overall achieved if it is directly and indirectly achieved according to criteria 1 and 2 above.According to the learning assessment presented in Table7, the course that is most concerning in regards of outcomes achievement is MATH1120. The course outcomes O5 to O10 received scores below 70 percent (3.5/5), which do not qualify these outcomes, according to the above criteria, to be considered achieved, although most of the scores for these outcomes are above 65 percent. In course outcomes 1 and 2 of that course, the scores show improvements where in Spring 11 and Fall 11 the scores of these two outcomes were nonqualifying but improved in Spring 12. This may indicate that the relevant instructors’ focus group in the ERU looked into the deficiency in the level of achievement of the outcomes and came up with some remedial actions, as dictated by the continuous improvement requirements imposed in the COE at UAEU to ever keep a standard educational level and to maintain ABET accreditation. These remedial actions suggested by the course focus group include increasing the tutorial sessions, providing the students with extra worksheets and modifying the course syllabus to meet exactly the needs of the students in the subsequent engineering courses.The only course among the four that according to the above adopted achievement criteria received almost no deficiencies is MATH1110. As for MATH 2210 and 2220, they received much less concerning scores, as compared to MATH1120. By relaxing the above criteria and considering the passing grade to be 65 percent; for example, much of the concerns above would be alleviated. However, justification to do so and feedback on detailed actions to improve the situation of outcomes achievements especially in MATH 1120 are required before attempting to relax the criteria.When considering the scores presented in Table8 based on the students’ feedback, one can see a much better picture than the one obtained from Table 7. The students look more optimistic about their ability to achieve the outcomes and the only serious concerning scores appear in the cells of outcome 3 of the course MATH1120.The summary of the level of achievement of the different outcomes satisfaction according to the criteria adopted above is given in Table10 and basically is decided based on the scores of Table 7 based on the direct assessment measures. At first glance, such matrix would give an easy access and insight to places in the curriculum where action is immediately needed for improvement. MATH1120 clearly is such a place where the math focus group in the ERU must be considering and monitoring over the semester to see the performance as per remedial actions taken.Table 10
Judgment on outcomes achievement based on adopted criteria for directly assessment outcomes (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
Semester
O1
O2
O3
O4
O5
O6
O7
O8
O9
O10
O11
MATH1110
Spring 2011
AC
AC
AC
AC
AC
AC
AC
AC
NA
NA
NA
Fall 2011
AC
AC
AC
AC
NAC
NAC
AC
AC
NA
NA
NA
Spring 2012
AC
AC
AC
AC
AC
AC
AC
AC
NA
NA
NA
MATH1120
Spring 2011
NAC
NAC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
Fall 2011
NAC
NAC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
Spring 2012
AC
AC
NAC
AC
NAC
NAC
NAC
NAC
NAC
NAC
NA
MATH2210
Spring 2011
AC
AC
AC
AC
AC
AC
AC
AC
AC
AC
NAC
Fall 2011
AC
AC
NAC
NAC
AC
AC
AC
AC
NAC
AC
AC
Spring 2012
AC
AC
NAC
AC
AC
AC
AC
AC
AC
AC
AC
MATH2220
Spring 2011
AC
AC
AC
AC
AC
AC
AC
NAC
NAC
AC
NA
Fall 2011
AC
NAC
AC
AC
AC
AC
AC
NAC
AC
AC
NA
Spring 2012
AC
AC
AC
NAC
NAC
AC
AC
NAC
AC
AC
NA


## 3.3.2. Curriculum Achievement (ABET Achievement)

By considering the mapping between the course outcomes and ABET criteria ((a)–(k)) presented in Tables1–5 for the four courses investigated, we come up hereby with assessment for the ABET ((a)–(k)) criteria level of satisfaction by our four math courses. The adopted criteria are summarized below.Criteria 4: ABET Outcome Directly Achieved. An ABET outcome is directly achieved if 70% of its related mathematics courses outcomes are directly achieved.Criteria 5: ABET Outcome Indirectly Achieved. An ABET outcome is indirectly achieved if 70% of its related mathematics courses outcomes are indirectly achieved.Criteria 6: ABET Outcome Achieved. An ABET outcome is achieved if it is directly and indirectly achieved.Tables11, 12, and 13 present the results on the ABET outcomes achievement. Although the above ABET outcomes achievement criteria seem somewhat stringent at first glance, especially for ABET outcomes that are covered only by a limited number of course outcomes hence giving less chance of passing by being less tolerant in accepting lack of achievements in any of the course outcomes (the case, for example, of ABET outcome (c), being only related to O4 and O5, or ABET outcome d being related to only O6 in the course MATH1110 (see Table 5)), the results reported in Table 11 show acceptable level of ABET outcomes satisfaction for three of our math courses to a good extent. The only course suffering from clear deficiencies regarding ABET satisfaction is apparently the MATH1120, which is highlighted already in the previous discussions calling hence for serious remedial actions.Table 11
Average ABET outcomes achievement level based on learning assessment tools (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(H)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
NAC
NAC
NAC
AC
NAC
NA
AC
NA
NAC
NAC
NAC
MATH2210
AC
NA
AC
AC
AC
AC
NA
NA
AC
NAC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACTable 12
Average ABET outcomes achievement level based on the indirect assessment tool “students opinion” (AC: achieved, NAC: not achieved, and NA: not applicable).
Course
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
AC
AC
AC
NAC
AC
NA
AC
NA
AC
AC
AC
MATH2210
AC
AC
AC
AC
AC
AC
NA
NA
AC
AC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACTable 13
Overall average ABET outcomes achievement.
Course
(a)
(b)
(c)
(D)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
MATH1110
AC
AC
AC
AC
AC
AC
AC
NA
AC
NA
AC
MATH1120
NAC
NAC
NAC
NAC
NAC
NA
AC
NA
NAC
NAC
NAC
MATH2210
AC
NA
AC
AC
AC
AC
NA
NA
AC
NAC
AC
MATH2220
AC
NA
NA
NA
AC
NA
NA
NA
NA
AC
ACOnce more, the students’ judgment on outcomes leads to having ABET outcomes being achieved to a good extent in all four courses, but this is not enough to grant overall achievement of ABET outcomes based on direct and indirect tools where the one more conservative of these tools hence determinant, which is the one based on the direct assessment tools, will dominate the final judgment.


## 4. Recommendations

Overall, the results of this study are very good and promising. They show that the majority of the ABET-outcomes are “mathematically” achieved according to our criteria of performance. Nonetheless, it is clear from the previous section that we need to give more attention to the MATH1120 Calculus II for engineers, namely, outcome “O5” which is“identify the equations of common surfaces such as sphere, cone, paraboloid.” In fact, despite using the 60% performance indicator, the “O5” was not directly achieved in the spring of 2011 and fall of 2011. There was an improvement in the spring of 2012, which is a good sign, but it was not enough to the achievement according to the 70% performance indicator. In addition, we need to stress more on the outcomes “O9:calculate line integrals” and “O10:understand the notion of the curl and divergence of a vector field” of the MATH1120 since they were not 60%-directly achieved. Furthermore, the outcome “O3:use computers to enhance visualization and solve calculus problems in several variables” was not 60%-indirectly achieved. On the other hand, all the other outcomes were 60%-achieved (directly and indirectly). The following are some recommendations for improving the learning teaching processes of mathematics courses offered to the engineering students based on this study:(i)
offering more exercises and problems in class and as homework for the outcomes that help students to gain competency in autonomy, responsibility, self-development, and role in context such as
(a)
the ability to think critically in analyzing engineering problems (e.g., “O5” in MATH1110-Calculus I for engineers and “O1” in MATH1120-Calculus II for engineers),
(b)
the ability to work effectively with others (e.g., “O6” in MATH1110-Calculus I for engineers),
(c)
using computers and new technologies to enhance visualization and solve calculus problems (e.g., “O3” in MATH1120-Calculus II for engineers and “O8-O9” in MATH2210-differential equations and engineering applications),
(ii)
increasing the assessment methods for the outcomes that help student to acquire knowledge in mathematics, especially for the MATH1120-Calculus II for engineers where almost all the outcomes were 70%-unachieved, while for the others courses the results were good.Largely speaking, one can say that the findings show a very good students-instructors performance.


## 5. Conclusions

This paper gives a representative sample of data and based on that explains the methodology adopted in the COE-UAEU for monitoring and controlling the educational and learning quality and performance in the MATH courses offered at UAEU for engineering students and the extent of satisfaction of ABET outcomes in these courses. Four math courses are offered at UAEU for general engineering students before they get specialized, namely, MATH1110: Calculus I for engineers, MATH1120: Calculus II for engineers, MATH2210: differential equations and engineering applications, and MATH2220: linear algebra and engineering applications. Mapping between the outcomes of these math courses and the engineering ABET criteria ((a)–(k)) showed that most of the ABET-outcomes ((a)–(k)) are to a great extent in-line with the outcomes of our mathematics courses. This encourages us to use the achievement of the courses outcomes as an assessment tool for the engineering-ABET outcomes. A course outcome achievement passing criteria is defined based on direct and indirect assessment tools that require the achievement of a score of 3.5/5 (70 percent) for each outcome for that outcome to be considered well realized. Hereby the indirect tools addressed are in the form of surveys conducted to take the opinion of the students on the level of outcomes achievement as seen by them. The direct tools used are the ones based on students’ scores in exams and any other learning activities such as homework assignments, quizzes, and course projects. Based on the data collected and analyzed in this study, it was found that most courses meet the passing criteria except MATH1120 (Calculus II), which was attributed to the high intensity of the course content that contains completely new topics to the students. As a part of the continuous improvement process, different improvement actions were suggested by the course focus group such as more tutorial sessions, extra worksheets, and modifying the course syllabus to meet exactly the needs of the students in the subsequent engineering courses.On the other hand, this study shows that monitoring the performance of courses outcomes over a number of semesters is a means of showing a trend but some abnormalities in the results in some semesters and possible deviations from the general trend cannot be considered alone as a source of feedback to trigger quality improvement actions in isolation from other important factors such as the number of sections offered for the course and the number of students in the classes in the different sections over the semesters. The feedback obtained from the direct assessment tools used although is comparable to the feedback obtained from the indirect students-opinion-based tool, the result of the students survey shows that the students evaluate higher their level of appreciation and satisfaction of the courses outcomes than what they achieve based on the direct assessment tools in reality. To bridge the gap between the results of the direct and indirect assessment tools, revisiting the direct assessment tools and perhaps the way the course material is delivered is required by the course instructor.More stringent criteria on the ABET outcomes satisfaction are adopted and this shows that the course MATH1120 is most susceptible to lack of realization of ABET criteria. The other three courses meet ABET in a reasonable way according to the adopted criteria.



---

*Source: 735325-2015-03-02.xml*