# Development of an Integrated Connectedness Model to Evaluate the Effectiveness of Teaching and Learning

Salah Al-Hamad, Laszlo E. Kollar, Taimoor Asim, Rakesh Mishra

► To cite this version:

Salah Al-Hamad, Laszlo E. Kollar, Taimoor Asim, Rakesh Mishra. Development of an Integrated Connectedness Model to Evaluate the Effectiveness of Teaching and Learning. 15th Conference on e-Business, e-Services and e-Society (I3E), Sep 2016, Swansea, United Kingdom. pp.707-716, 10.1007/978-3-319-45234-0_64. hal-01702200

HAL Id: hal-01702200

https://inria.hal.science/hal-01702200

Submitted on 6 Feb 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Development of an Integrated Connectedness Model to evaluate the effectiveness of Teaching and Learning

Salah Al-Hamad<fnref n="1" />, Laszlo E. Kollar<fnref n="2" />, Taimoor Asim<fnref n="1" />,*, and Rakesh Mishra<fnref n="1" />

**Abstract.** Use of blended learning system in a structured manner results in achieving higher order skills in cognitive domain. However, there is a need to quantify skills improvements as the students go through lower level skills to higher level skills. To ascertain this progression, it is necessary to develop a mathematical model of learning which can indicate the effectiveness of teaching and learning methods on skills improvement. In this study, mathematical learning model has been developed, which predicts the students' knowledge, depending on the amount of instruction they receive. It is expected that this model will enable development of direct correlation between teaching and learning methods, and the skills level attained by the students. The model is applied for different categories of learning. Parameters in the model are determined after least-square fitting has been applied on observed student learning data.

## 1 Introduction

Teaching and learning processes that are being followed globally by education providers consist of conventional face-to-face approach. The globalisation, along with the interdependence of various economies, has resulted in creating an extra dimension to the higher order of skills requirements. Hence, there is a need to develop new teaching and learning methodologies that can comply with the ever increasing demands of the industry, regarding the skills of engineering students.

The present paper summarizes existing learning models, and proposes an integrated connectedness model (ICM) that is applicable to indicate and compare the effectiveness of different teaching and learning methods on skills improvement. The model is validated by using data available in the literature to quantify the usefulness of the teaching and learning process in cognitive learning [1]. The old version of levels of

<footnote n="1">¹School of Computing & Engineering, University of Huddersfield, Queensgate, Huddersfield, UK (HD1 3DH)
{u0417721, t.asim, r.mishra}@hud.ac.uk</footnote>

<footnote n="2">²Savaria Institute of Technology, University of West Hungary, Karolyi G. ter 4, Szombathely, Hungary (H-9700)
kollar.laszlo@nyme.hu</footnote>
taxonomy was assumed in the study cited, and in the validation of the proposed model as well. Then, cognitive learning evaluation, assuming the new version of levels of taxonomy, through the ICM have been carried out and presented in this paper.

## 2 Connectedness Model

Real learning usually involves some learning of both (i) students learn by memorization, and that learning is independent of prior knowledge, and (ii) students learn new knowledge by constructing an association between new and some prior knowledge. The first is known as Pure Memory Model while the latter is called Simple Connectedness Model. Pritchard et al [2] developed a model which interpolates between, and even beyond, these two models. This model is called Connectedness Model, and the parameter that establishes the relationship between the pure models is called the connectedness parameter, denoted by $\gamma$. Connectedness Model can be effectively used for all skill levels of cognitive domain. This model determines students' knowledge $K_T$ as a function of the amount of teaching or instruction i. Thus, $K_T$ represents the fraction of the material that is known by the student, and another parameter, $A_T$ represents what is unknown. $K_T$ and $A_T$ vary between 0 and 1. Initially $K_T = 0$, because students do not know anything about the subject to be learnt, or in other words, the subject knowledge is unknown, i.e. $A_T = 1$. Students are supposed to learn the subject completely by the end of the course, hence $K_T = 1$ and $A_T = 0$ at the end of the teaching period in the case of an ideal student. The governing differential equation takes the following form:

$$ \frac{dA_T(i)}{di} = -A_T(i)(\delta_{con}\gamma(1 - A_T(i)) + \delta_{mem}(1 - \gamma)) \quad (1) $$

The model involves a differential equation for $dA_T/di$, i.e. for the rate of change of unknown knowledge. The equations are based on $A_T$, because given instructions are generally related to what students do not know. However, once the solution for $A_T$ is found, $K_T$ can easily be obtained. The parameter that expresses the probability that something taught sticks in the student's mind is the sticking coefficient $\delta$, hence $\delta_{con}$ and $\delta_{mem}$ are the sticking coefficients from the pure memory and
simple connectedness models. The solution for the known knowledge can be written as follows:

$$K_T(i) = \frac{(1-K_{T0})(\delta_{mem}(1-\gamma)+\delta_{con}\gamma)}{(1-K_{T0})\delta_{con}\gamma+(\delta_{mem}(1-\gamma)+K_{T0}\delta_{con}\gamma)\exp(\delta_{mem}(1-\gamma)i+\delta_{con}\gamma i)} \quad (2)$$

where students' knowledge, during the teaching period, depends on their initial knowledge $K_{T0}$, which can be obtained by pre-instruction test scores, and can be used as input in the model. The model is equivalent to the pure memory model for $\gamma=0$, and it is equivalent to the simple connect model for $\gamma=1$. Pure memory model is particularly applicable to lowest skills in cognitive domain. The equation for the rate of change of unknown knowledge can be written as follows:

$$\frac{dA_T(i)}{di} = -\delta_{mem}A_T(i) \quad (3)$$

The solution for the known knowledge is obtained as follows:

$$K_T(i) = \frac{1}{1+\frac{(1-K_{T0})\exp(-\delta_{con}i)}{K_{T0}}} \quad (4)$$

## 3 Development of an Integrated Connectedness Model (ICM) for different learning domains

The learning domains are distinguished in cognitive learning as knowledge, comprehension, application, analysis, synthesis, and evaluation. The proposed model relates the test results of any domain to the test results of the preceding domain. The model assumes identical importance to each of these domains, i.e. the total knowledge that may be gained in each domain takes 1/6 of the total knowledge in the subject that students learn. The initial knowledge for the knowledge test is assumed to be $K_{T0,Kn}=0$, i.e. students do not know anything about the subject that they are about to learn. The knowledge gained in the knowledge domain is 1/6 of the total knowledge in the subject, and it is essential for gaining knowledge in the comprehension domain. Therefore, the initial knowledge for the comprehension test is $K_{T0,co}=1/6 K_{T,Kn}(i_{ins})$, where $K_{T,Kn}(i_{ins})$ is the known knowledge in the knowledge domain at the end of the learning period. The initial knowledge can be determined similarly for all the other domains, ending with the initial
knowledge for evaluation, which is K<sub>T0,ev</sub>=5/6 K<sub>T,sy</sub>(i<sub>ins</sub>) with K<sub>T,sy</sub>(i<sub>ins</sub>) being the known knowledge in the synthesis domain at the end of learning period.

The model also assumes that learning in knowledge domain is inde-
pendent of prior knowledge, learning in the evaluation domain is based
purely on association between prior and new knowledge, whereas
learning in the other domains is a combination of both types. First, an
appropriate model has to be chosen, and then the sticking coefficients
δ<sub>mem</sub> and δ<sub>con</sub> as well as the connectedness parameter γ have to be de-
termined. In practice, since the sticking coefficients always appear in
the products δ<sub>i</sub> in the models, the products δ<sub>mem</sub> and δ<sub>con</sub> are deter-
mined and used in further calculations. These parameters are deter-
mined by fitting the solution in the chosen model on test data. Conse-
quently, the combined model is constructed as follows (see figure 1).
Learning in the knowledge domain is modelled by the pure memory
model, and the product δ<sub>mem</sub> is determined. Learning in the evaluation
domain is modelled by the simple connect model, and the product δ<sub>con</sub>
is determined. Then, the same sticking coefficients δ<sub>mem</sub> and δ<sub>con</sub> in the
connectedness model are used to simulate learning in the remaining
four domains, and the connectedness parameter β is determined for
each of the four domains.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Flowchart of combined model for cognitive learning</figcaption></figure>

4 Teaching/Learning Groups

45 students were selected from the three mechanical engineering spe-
cialisations i.e. Automotive, Industrial Maintenance and Manufacturing
Engineering. The students have then been divided into three separate
groups. Each group consists of 15 students (5 students from each spe-
cialization), where the students have almost similar abilities. The de-
tails of these groups are as follows:

**Group 1:** Demonstration under the watchful eyes of the instructor (Teacher Centred, Interactive Dependent)

Teachers in this group serve as the centre of knowledge in both theory and practical sessions, and are primarily responsible for directing the learning process. During assessment phase (student's participation and demonstrations), teachers were focused almost exclusively on what had the students learned. Students were viewed as empty vessels, whereas the teacher imparts learning into these vessels within a given time period. Furthermore, learning was viewed as additive process [3].

**Group 2:** Students learn using computers and the teacher facilitates the process (Student Centred, Interactive Independent).

In this method, students were considered as knowledgeable and they
can bring about engagement and personal responsibility in learning [4].
This supports the idea of knowledge construction by learners through
their use of prior knowledge and experience, which assists them to
shape meaning and acquire new knowledge. The mechanical engineer-
ing teachers, during their preparation, observed that in constructive
learning students participated in class and they may have a wide range
of previous learning experiences, which enable teachers to select teach-
ing/learning methods at higher level of skills for optimal learning [5].

**Group 3: Project Base Group, work with and without supervision (In-teractive)**

In this collaborative-interactive approach the lecturer provides computer
tutorials including videos and animations, which show the students how
to use tutorial instructions in order to warm-up to the lecture with the
use of tutorial tasks and questions [6-7]. The lecturer intends to use two
ways of communication between the teacher and the students, combined
with active learning to increases understanding. The method was estab-
lished in cooperative environment where students work together. It al-
lows learning to continue after the class session. Students teach each
other. The most effective way to learn is to actually teach, because this requires the highest degree of mental processing (high level of thinking skills) and greatly increases the likelihood that long-term memories will be produced. The tutorial was provided with motivational animations to stimulate team work, and it has a greater likelihood of being incorporated into long-term memory.

## 5 Cognitive Skills Evaluation Techniques

The learner should achieve proficiency in lower levels of cognition, and then progress through higher levels. This analysis is similar to the one carried out by Zywno [8] for electrical engineering students. In mechanical engineering modules, various levels have been identified as per the developed model of cognitive level skills related to the knowledge (recall data), comprehension (understand information), application (applying knowledge to the new situation), analysis (separating information into part of better understanding), evaluating (justify a stand or decision by appraising, arguing, defending, judging, selecting, supporting, valuing and evaluating) and creating (create new product or point of view by assembling, constructing, creating, designing, developing and formulating idea).

Lecturer marked the students during maintenance and production of six tasks in Automotive, Industrial Maintenance and Manufacturing. The quality of students' results for each activity is determined by comparing their products with the checklist and awarding learning ability indicators for each student and task. The learning ability indicator shows how well the student has performed a certain task by comparing his/her application results with the checklist. Figure 2a shows the correlation between learning ability indicator (average marks obtained in the examination before entering this course) and the marks obtained for the three groups in the examination of the knowledge cognition level. Most groups show considerable improvement in knowledge but final marks for group 2 students are uniformly distributed between 80% and 95%. This indicates that student-centred approach has increased the level of achievement of learning outcomes for this heterogeneous group of students. The final marks for group 3 are spread between 65% and 85%. Hence, the interactive teaching and learning methods have produced a slight increase in the final marks but not too much like group 2.
Figure 2b presents the variation of students' marks in the examination of analysis cognition level. The final marks for group 2 are concentrated in the interval 75% to 95% so their level of achievement is the same as in previous cases (knowledge, comprehension, application). Furthermore, the students' final marks from group 1 are in the interval 55% to 80%. Hence, the teacher-centred approach does not increase the marks significantly at analysis cognition level. Figure 2c presents the variation of students' marks in the examination of evaluation cognition level. The students were evaluated for their abilities in analyzing and evaluating the machining operation and procedure and selecting, preparing tools and equipment and using measuring instruments facilities to calculate missing dimensions of engineering application. This also requires students to be capable of analyzing and verifying the manufacturing operations, assembling different parts to create prototype in final shape, designing a new shape and modifying one shape to another shape, arranging machine tools, materials and instruments for final manufacture and engineering maintenance preparation of parts and tools. The students' final marks from group 1 are clustered around the interval 55% to 65%. The teacher-centered approach does not enable the development of appropriate students' skills for evaluation cognitive level. Furthermore, students from group 1 obtained the lowest marks in comparison to those from group 2. Group 3 show less marks than the previous cases (knowledge, comprehension, application and analyses). This shows that the combination students-centred approached is far more useful in delivering learning outcomes at higher level of developed cognition skills.

<figure><img src="image_3.png" /><figcaption>**Fig. 2a.** Comparison between teaching methods in Knowledge cognition level</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 2b.** Comparison between teaching methods in Analysis cognition level</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Fig. 2c.** Comparison between teaching methods in Evaluation cognition level</figcaption></figure>

**6 Cognitive Learning Evaluation through ICM**

The combined model is applied here for the case of imparting cognitive learning skills through blended learning system. The procedure described in Section 3 is followed for different learning domains. The learning domains for this case are the following: knowledge, comprehension, application, analysis, evaluation and creating. The product $\delta_{\text{mem}}^i$ for the knowledge domain as well as the product $\delta_{\text{con}}^i$ for the creating domain were determined to be 1.33 and 0.57 for Group 1, 2.01 and 0.71 for Group 2 and 1.45 and 0.49 for Group 3 respectively. First, the product $\delta_{\text{mem}}^i$ was determined from the pure memory model only, using the test results obtained for the knowledge domain and assuming no initial knowledge. Then, the product $\delta_{\text{con}}^i$ was calculated from the simple connect model using the test results obtained for the creating domain and using test results obtained for the evaluation domain as input. The dependence of post-instruction knowledge on initial knowledge in the creating domain is shown in figure 3 together with the test data used for fitting.

Once the sticking coefficients are known, the connectedness parameter was also determined for each learning domain, and results are summarised in table 1. The connectedness parameter increases for the different domains from knowledge to creating, because the more advanced the students' learning in the subject, the more association they can construct between new and prior knowledge. The values that did not follow this trend are the connectedness parameters for the comprehension and analysis domains for Group 3. The value for the comprehension domain is negative, which means that the normalized gain slightly decreases
with increasing pre-instruction test scores. This can happen when students with higher pre-instruction scores exert less effort, whereas students with lower pre-instruction scores make more effort to improve their results [2]. Furthermore, this group was not exposed to a very structured learning environment as instructor's input was least with this group. This might have caused skills development that cannot be explained from the model that has been used. The trends observed justify the choice of the model. If students learnt by memorization only or by constructing only association between new and some prior knowledge, then the values of the connectedness parameters would be close to 0 or 1, respectively.

<figure><img src="image_6.png" /><figcaption>Fig. 3. Curve fitting on data for creating (simple connect model)</figcaption></figure>

<table><caption>**Table 1.** Connectedness parameters for all the learning domains in cognitive learning</caption><thead><tr><th></th><th>Group 1</th><th>Group 2</th><th>Group 3</th></tr></thead><tbody><tr><td>Knowledge</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Comprehension</td><td>-0.02</td><td>0.12</td><td>-0.37</td></tr><tr><td>Application</td><td>0.12</td><td>0.33</td><td>0.25</td></tr><tr><td>Analysis</td><td>0.64</td><td>0.48</td><td>0.18</td></tr><tr><td>Evaluation</td><td>0.94</td><td>0.66</td><td>1.03</td></tr><tr><td>Creating</td><td>1</td><td>1</td><td>1</td></tr></tbody></table>

The post-instruction knowledge as a function of initial knowledge is shown in figure 4. The normalized gains for the knowledge domain and
for the creating domain are obtained from the pure memory model and the simple connected model, respectively, and they are shown in figure 5. The normalized gains are calculated for the remaining four learning domains by using the connectedness model, and they are presented in figure 6. It can be seen that the knowledge as well as the normalized gain is always highest for Group 2. The knowledge and the normalised gain are lowest for Group 1 in the knowledge, comprehension and analysis domains, whereas these are lowest for Group 3 for the application, evaluation and creating domains. Thus, the teaching method applied for Group 3 is more effective at lower level skills, but the method applied for Group 1 is more effective at higher level skills. However, the most effective teaching method in all the cases is the one that has been applied with Group 2.

<figure><img src="image_7.png" /><figcaption>Fig. 4. Curve fitting on data (connectedness model) (a) comprehension (b) application (c) analysis (d) evaluation</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 5. Normalized gain for (a) knowledge domain (pure memory model) (b) creating domain (simple connect model)</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Fig. 6.** Normalized gain for (a) comprehension (b) evaluation domains (connectedness model)</figcaption></figure>

## Conclusions

Mathematical models have been developed in the present study that quantifies the learning process at microscopic level within cognitive skills domains. In contrast to the macroscopic (conventional) evaluation methods used throughout the world, these models provide a much clearer picture of the teaching/learning taking place at different skills' levels enabling a better control over the quality of teaching and learning process. These models can be further modified in order to apply them to other fields of education.

The above study has clearly indicated that integrated connectedness model (ICM) represents skills development in cognitive skills domains fairly well. ICM can be used to monitor effectiveness of the teaching and learning strategies through well-developed assessment strategies. It can also dictate development of teaching and learning materials by providing important feedback on the effectiveness.

## References

1. Abdulrasool. S., Mishra. R.: Using Computer Technology Tools to Improve the Teaching-Learning Process in Technical and Vocational Education: Mechanical Engineering Subject Area. Int. J. Comp. Tech. 15 (9), 155-168 (2008)
2. Pritchard. D.E., Lee. Y.J.: Mathematical learning models that depend on prior knowledge and instructional strategies. Physical Review Special Topics–Physics Education Research. 4, 010109 (2008)

3. Mei-Y. L., Xlnhqng. L., Dongyu. C., Meilu.: Impacts of Teaching Approaches on Learning Approaches of Construction Engineering Students: A Comparative Study between Hong Kong and Mainland China. J. Eng. Ed. (2008)

4. Watters. D.J., Watters. J.J.: Approaches to learning by students in the biological sciences: Implications for teaching. Int. J. Sc. Ed. 29 (1), 19-43 (2007)

5. Mehmet. S.: Blended Learning model in mechanical manufacturing Training. Afr. J. Bus. Man. 4 (12), 2520-2526 (2010)

6. Mumcu. F.K. Usluel. Y.K.: ICT in Vocational and Technical Schools: Teachers' Instructional, Managerial and Personal Use Matters. Turk. Onl. J. Ed. Tech. 9 (2010)

7. Richard. E., Mayer.: Short video about the Role of video and its design for effective teaching and learning in higher education. University of California, Santa Barbara, USA (2011)

8. Zywno. S.M.: Hypermedia Instruction and Learning Outcomes at Different Levels of Bloom's Taxonomy of Cognitive Global. J. Eng. Ed. 7 (2003)