# Toward a Unified Framework for Evaluating Online Education Quality

Ibrahim Tanko Gampine, Bassirou Niang, Kossi Kawedia Yakoubou

► To cite this version:

Ibrahim Tanko Gampine, Bassirou Niang, Kossi Kawedia Yakoubou. Toward a Unified Framework for Evaluating Online Education Quality. Electronic Journal of eLearning, 2025, Volume 23 (Issue 4), pp.37-52. 10.34190/ejel.23.4.4280 . hal-05326895

HAL Id: hal-05326895

https://hal.science/hal-05326895v1

Submitted on 22 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Toward a Unified Framework for Evaluating Online Education Quality

Ibrahim Tanko Gampine<fnref n="1" />, Bassirou Niang<fnref n="1" />,<fnref n="2" /> and Kossi Kawedia Yakoubou<fnref n="3" />

ibrahim.gampine@univ-montp3.fr

bassirou.niang@univ-thies.sn

kawekos@yahoo.fr

https://doi.org/10.34190/ejel.23.4.4280

An open access article under **CC Attribution 4.0**

**Abstract:** E-Learning has become a global phenomenon. It makes learning more accessible and acquisition of new skills and knowledge easier. In sub-Saharan Africa, however, online qualifications are often the subject of controversy regarding their recognition. This is clear evidence of unsuitable e-Learning systems, as well as the limited relevance of the programs they offered in addressing the Africans' context-specific needs. Despite the multitude of studies on the quality of online education, inconsistencies in findings make not only comparisons between studies difficult but also complicate the assessment of quality online education. To address this issue, this study integrated the Kirkpatrick with DeLone & McLean models to identify core quality dimensions. Furthermore, this study clarified the context-specific requirements of the identified dimensions. Ten hypotheses were tested using online survey questionnaires administered to four higher education institutions via Qualtrics. The findings supported eight hypotheses and rejected two. This model highlights the critical role played by system quality, the quality of course content, faculty and institutional support in enhancing learning. Furthermore, the model establishes a clear cause-and-effect pathway useful in addressing poor learning outcomes. We discussed the implications of the findings in the context of sub-Saharan Africa. The model is simple, theoretically sound, and comprehensive for real-life applications. Specifically, this study highlighted the importance of both formative and summative evaluations. Further qualitative studies on the context-specific requirements of the dimensions would be desirable.

**Keywords:** Quality education, e-Learning, Online learning, Kirkpatrick model, DeLone & McLean model, Quality assessment, Developing country

## 1. Introduction

Following the outbreak of the COVID-19 pandemic, e-Learning has become more important than ever. Online education plays an important role in developing human resources. In Africa, the e-Learning market is growing faster due to the rapid adoption of mobile devices, digital technologies, and internet services. In 2024, the market was valued at USD 3,411.38 million and it is projected to grow at a compound annual growth rate of 19.20% between 2025 and 2034. It is expected to reach an impressive USD 19,755.71 million by 2034, (e-Learning Africa, 2025). Revenue is expected to grow at an annual rate, resulting in a projected market volume of US$1.49 billion by 2029 (Statista, 2025). This growth highlights the immense potential of e-Learning to meet the diverse educational needs of Africa's population. However, ensuring quality remains a critical obstacle to the continued growth of e-Learning. Kotler (2019) defines quality as the provision of a distinctive product or service that satisfies the user's needs. According to the United Nations International Children's Emergency Fund (UNICEF, 2024), quality education means providing learners with access to basic literacy and numeracy without fear of exclusion.

By contrast, Green (1994) defines quality in education as producing graduates who meet the demands of private and public sector organisations. Lee-Post (2009) defines online learning as technology-mediated education dependent on quality of design and delivery. The increasing shift from traditional to online higher education (Hafeez, Naureen, & Sultan, 2022) makes discourse on quality assurance crucial. Barbour and Clark (2016) reported low performance among students in online education. One needs to determine the antecedents of quality online education and their contextual relevance in addressing the problems of quality online learning. While some existing studies have used subjective measures, others have adopted existing models (Barteit et al., 2020; Esfijani, 2018) to identify the antecedents of quality e-Learning. However, each study has a distinct framework with different quality dimensions (Khan et al., 2023; Tan, Chan, & Mohd Said, 2021). This makes assessment and comparison between studies challenging. Therefore, this calls for the need of a standardized

<footnote n="1">¹Université Paul Valéry (UFR4), laboratoire CORHIS, Route de Mendes, Montpellier France</footnote>

<footnote n="2">²Université de Thiès, Sénégal</footnote>

<footnote n="3">³Université de Kara, laboratoire LaRSEG, Kara, Togo</footnote>
assessment framework for online education. To address this problem, we asked the following research questions:

RQ1: What are the critical dimensions of quality e-Learning education in Sub-Saharan Africa?

RQ2: How do these dimensions interact to influence perceived quality education?

RQ3: Can integrating Kirkpatrick and DeLone & McLean into an input-process-output framework provides a more consistent and comprehensive assessment model?

The effect of Western education remains apparent, despite the efforts made to contextualise the African educational curricula. This creates the need for an educational curriculum that addresses Africa context-specific needs (Masenya, 2021). Based on this assumption, we conceptualize quality education as an education that is accessible, relevant, and satisfactory to the needs of stakeholders. We approach this problem by anchoring on a theoretical framework that integrates the Kirkpatrick with the DeLone and McLean model. The Kirkpatrick model has gained popularity as a framework for measuring the effectiveness of teaching and learning in higher education (Cohen, Nørgård, & Mor, 2020). Likewise, DeLone and McLean (2003) model is useful in assessing the quality of information systems. Further, we draw insight into the input-process-output framework (Galais et al., 2021) to classify the dimensions of the two models; this is because of the interdependencies between the identified constructs (Tamkin et al., 2002). This study equally discussed the relevance of the identified dimensions with respect to the African context.

This study is significant, as it is the first to integrate the Kirkpatrick model with the DeLone and McLean model. Specifically, this integrated model links the technical system to other learning interventions such as course content, faculty, and institutional support. This model highlights the critical role of both formative and summative evaluations. Learners' satisfaction with system quality, quality of course content, faculty, and institutional support enhance the acquisition of knowledge. Besides, viewing quality education as a system emphasizes the need to classify quality dimensions based on the input, process, and output levels when addressing quality-related problems. This study lays the foundation for future studies to explore the potential of this integrated model in the assessment of quality online education. Unlike previous studies, this integrated framework supports both formative and summative evaluations. This implies that poor-quality education can be attributed to inadequate system quality, information quality, faculty or institutional support.

According to Statista, over 130 million children and youth in Sub-Saharan Africa were out of school as of 2022. This study highlights the importance of system quality, information quality, faculty or institutional support and their contextual relevance in the design and delivery of e-Learning in sub-Saharan Africa. Therefore, this study contributes to the achievement of the Sustainable Development Goal n°4, which is the commitment to promoting inclusive quality education and learning opportunities. This model is simple, theoretically based, and comprehensive for real-life applications. It provides clarity and consistency in how the various dimensions interact to produce quality education. The framework is a useful tool for instructional designers and quality assessment professionals. They can draw insights from the core quality dimensions with the three different levels of assessment when addressing quality-related challenges. We organized this study into three sections. The first section presents the theoretical background of the study. The second section outlines the methodological approach, data collection procedure, and analysis techniques. The final section presents the discussion, implications, and perspectives for future research.

## 2. Theoretical Background

### 2.1 Inconsistencies in Online Quality Education Assessment

We found inconsistencies in the studies of online quality education assessment. While some studies anchored on the Kirkpatrick model to assess quality online education (Quinton et al., 2022; Cahapay, 2021), others relied on the D&M model (Çelik & Ayaz, 2021; Lee-Post, 2009). Besides, some authors used other theoretical models (Almaiah et al., 2022; Tan, Chan, & Mohd Said, 2021) or subjectively selected dimensions (Altun & Johnson (2023; Safdar et al., 2020). In a meta-analysis of the literature, Isfijani (2018) revealed the lack of consensus on online education assessment. Besides, Tan, Chan, & Mohd Said (2021) found quality instruction, online interaction, and instructional and technical support as the factors that enhance the quality of online higher education. In a quantitative study of 273 university students in Pakistan, Safdar et al. (2020) found comfortability in computer usage, self-motivation, confidence and time spent on learning as important to quality online education. Though their study contributes to research by outlining employee-level antecedents, it failed to anchor on a theoretical framework. This implies that the choice of their antecedents was subjective.
Moreover, Marciniak (2018) draws insights from existing models to administer a questionnaire to 23 Spanish online education experts. She found training objectives, student profiles, training contents, learning activities, online teacher profiles, didactic materials, learning strategies, learning assessments, and tutorials as critical online dimensions. This study outlined the importance of institutional-level antecedents, but it provides no evidence of causal relationships. Conversely, in a qualitative study of 8 online education programme directors in the USA, Altun and Johnson (2023) found that students' needs, changes in education, the educational system, leadership, continuous improvement, the integration of learning and teaching theories, research on online education, and the quality of instructors contribute to achieving quality education. Similarly, in a thematic content analysis Wright et al. (2023) found course design, instructor facilitation, quality online infrastructure, and student engagement as key to achieving quality education. Despite the inclined bias and subjectivity, these studies contribute to literature by suggesting dimensions that are not evident in the existing models.

On the other hand, a systematic review of literature by Hafeez, Naureen & Sultan (2022) highlighted the importance of learner-teacher interaction, learner's prompt feedback, support service, and appropriate technology. In a study inspired by the D&M model, Çelik and Ayaz (2021) collected data from 882 university students in Turkey to assess the effectiveness of online learning using system quality, information quality, and service quality through the mediating role of system use and user satisfaction. These studies contribute to existing literature by emphasising the importance of technical and non-technical dimensions in enhancing quality education. The Online Learning Consortium (OLC, 2024) developed a balanced scorecard that outlines institutional support and technology support, course development or instructional design, course structure, teaching & learning, social and student engagement, faculty support, student support, and evaluations and assessment as dimensions to quality online education. Though the above review contributes to our understanding of dimensions that contribute to quality education, the inconsistencies in quality dimensions make quality assessment difficult for professionals and complicate comparison between studies. Furthermore, there is limited evidence of their applicability in the context of sub-Saharan Africa.

## 2.2 Quality Education Assessment Using the Kirkpatrick Evaluation Model

According to Smidt et al. (2009), a training evaluation model assesses whether a training intervention meets the needs of organising institutions and the training participants. There exist many learning evaluation models but the best known and the most widely used is the Kirkpatrick model (Alsalamah & Callinan, 2022; Cahapay, 2021). The overview of publications of the Kirkpatrick model shows that research in the area is still active and growing (Alsalamah & Callinan, 2022). It provides a holistic understanding of the key dimensions required to ensure high-quality training outcomes (Alsalamah & Callinan, 2021). Recent studies have explored training evaluation using the Kirkpatrick model (Mosquera et al., 2023; Peters & Gohlich, 2023; Quinton et al., 2022; Alsalamah & Callinan, 2021). Inspired by this model, Quinton et al. (2022) evaluated young people's reactions to learning in the context of community-based development learning in the United Kingdom. The findings of data collected from 301 young people living in housing services showed that higher-level engagement leads to positive reactions, whilst enjoyment positively predicted learning outcomes through the mediating effect of transfer intention. Whilst this study contributes to highlighting the role of programme enjoyment and the students' level of engagement in enhancing learning, the choice of its dimensions remains rather subjective and outside the control of the learning institution.

Furthermore, Alsalamah and Callinan (2021) evaluated reaction to learning, learning, behaviour and training outcomes of training programmes for 250 female head teachers and 12 supervisors in Saudi Arabia. Their approach helps determine the strengths and weaknesses of a training evaluation process, but it failed in examining the relationships between the learning antecedents (training content, training method, trainer skills, and training environment) and the organisational bottom line. Moreover, Peters and Gohlich (2023) adapted the Kirkpatrick's evaluation model to examine reaction, learning, transfer and transformative learning outcomes using 29 postgraduate students at a German university. The findings suggest that the group coaching was effective on all levels and fostered transformative learning. On the other hand, Rahmawati and Lukito (2020) evaluated the effectiveness of employee training in a hospital. The results of data collected from 55 employees found that the four-level hypothesis has a significant impact on the training programme effectiveness. Mosquera, Suarez, & Guerrero (2023) assessed learning efficiency using an experimental design of 82 programming students divided into an experimental and control groups. The findings showed no differences in the submission time between the experimental and control groups. However, the results revealed differences in the values of grades. This study contributes to assessing learning by using pre-tests and post-tests, but it failed to outline learning antecedents and any causal links between them and grades. In addition, the sample size of these studies is inadequate to warrant any generalisability.
Studies based on the Kirkpatrick model have helped us to understand how to measure learning by emphasising the importance of evaluating learning at upper and lower levels. However, their main limitation is that they failed to consider learning antecedents and their causal relationships. This limits their ability to probe the critical factors that impede on learning effectiveness (Aluko & Shonubi, 2014). The studies inspired by the Kirkpatrick model focus more on the levels of measurement. They ignore the role of the learning antecedents (training content, trainer skills, training environment and training methods, etc.) and their effect at each level. The trainee's or learner's feelings toward the course content, instructor ability, training environment, material, and learning activities are assessed at the reaction level (Abdulghani et al., 2014). Cahapay (2021) criticizes the Kirkpatrick model for its rigidity and focus on the lower-level criteria but also the paucity of evidence on the causal chains among the levels. The consideration of context in the design and delivery of learning effectiveness in higher education is critical (Cahapay, 2021). Few studies tested the Kirkpatrick model in an African context using university students as their target population. The Kirkpatrick model is simple and pragmatic (Tamkin, Yarnall, & Kerrin, 2002); it offers the opportunity for adaptation (Capahay, 2021). We present the Kirkpatrick model and their corresponding objectives in Table 1 below.

<table><caption>Table 1: The four levels of the Kirkpatrick training model</caption><thead><tr><td>Levels</td><td>Objectives</td></tr></thead><tbody><tr><td>Level 1 (reaction)</td><td>To assess the participants thoughts of the programme content, teaching material, learning environment, instructor, etc. It is normally measured using questionnaires.</td></tr><tr><td>Level 2 (learning)</td><td>To assess changes in knowledge, skills, or attitude concerning the training objectives; it is objectively measured using performance tests.</td></tr><tr><td>Level 3 (behaviour)</td><td>To assess changes in job behaviour resulting from the program, to identify whether the learning is being applied. Assessment methods include observation and productivity data.</td></tr><tr><td>Level 4 (Outcomes)</td><td>To assess the bottom-line contribution of the training program. Methods include measuring costs, quality, and return on investment (ROI).</td></tr></tbody></table>

Source: Adapted from Tamkin et al. (2002).

## 2.3 Quality Education Assessment Using the DeLone & McLean Model

Information systems have become pervasive and critical for the survival of today's organizations. Laudon and Laudon (2009) define an information system as a combination of hardware and software technologies that collect, store, and process data into useful information for decision-making purposes. DeLone and McLean (2003) models have six different dimensions. They were system quality, information quality, use, user satisfaction, individual impact, and organizational outcome. A later study refined the model to integrate service quality and replaced organizational impact with net benefits. The key information systems success factors are organized at three levels (Lee-Post, 2009). The first level addresses design quality, the second level measures delivery quality, and the final level assesses the outcome. We present the dimensions of the D&M model in Table 2 below.

<table><caption>Table 2: The dimensions of D&M model</caption><thead><tr><td>S#</td><td>Construct</td><td>Measurement</td></tr></thead><tbody><tr><td>1.</td><td>System quality</td><td>To evaluate ease of use, system flexibility, system reliability, and ease of learning, as well as system features of intuitiveness, sophistication, flexibility, and response times.</td></tr><tr><td>2.</td><td>Information quality</td><td>To assess relevance, understandability, accuracy, conciseness, completeness, understandability, timeliness, and usability.</td></tr><tr><td>3.</td><td>Service quality</td><td>To analyse the responsiveness, accuracy, reliability, technical competence, and empathy of the IT personnel staff.</td></tr><tr><td>4.</td><td>Use</td><td>To evaluate the amount of use, frequency of use, nature of use, appropriateness of use, extent of use, and purpose of use.</td></tr><tr><td>5.</td><td>User satisfaction</td><td>To evaluate user feelings or satisfaction</td></tr><tr><td>6.</td><td>Net benefit</td><td>To analyse improved decision-making, improved productivity, increased sales, cost reductions, improved profits, market efficiency, consumer welfare, creation of jobs, and economic development.</td></tr></tbody></table>

Source: Adapted from D&M model (2016).
## 2.4 Integrating Kirkpatrick to DeLone & McLean Models

DeLone and McLean's model was used in measuring system effectiveness (Çelik & Ayaz, 2022), while the Kirkpatrick model assesses learning effectiveness (Smidt et al., 2009). The Kirkpatrick and DeLone & McLean models were exclusively used to measure learning effectiveness in higher education (Mosquera et al., 2023; Çelik and Ayaz, 2022; Mkinga & Mandari, 2020). As D&M model, the key dimensions of the Kirkpatrick model are also interdependent (Tamkin, Yarnall, & Kerrin, 2002). This implies that each model is a system in its own with a structured organism composed of interdependent components. Luhmann, Baecker, & Gilgen, (2013) defines a system as a set of interconnected things forming a complete whole with input, process, and output feedback. The objective of this study was to identify the critical dimensions of online quality education assessment using the D&M and Kirkpatrick models. As a result, we classified the components of each model based on the input-process-output model (Galais et al., 2021).

Inputs are the primary means used to create value (Lannelongue, Gonzalez-Benito, & Gonzalez-Benito, 2015). At the input level, Kirkpatrick's reaction level represents the system quality, information, and service quality. A reaction is the learner's perception of the effectiveness of interventions in delivering a training program (Rahmawati & Lukito, 2020). Based on the context of this study, we operationalized the service quality to "faculty and institutional support" and the information quality to "quality of course content." Recent studies have highlighted the impact of system quality on information system effectiveness (Lufti, 2023; Almaiah et al., 2022). Other studies have reported a positive and significant relationship between technical and institutional support (Hafeez, Naureen, & Sultan, 2022); Tan, et al., (2021). Besides, some components of the integrated model are classified at the process level. For instance, Dennis and Meredith (2000) described the process as a value-addition mechanism. The dimensions of the two models that illustrate value addition are use, user satisfaction, learning, and behavioural change.

Based on the context of this study, we adopted learner satisfaction and acquisition of knowledge as the process dimensions. We adapted the user's satisfaction to learner satisfaction and learning and behavioural change to acquisition of knowledge. Studies have examined learning as a mediating variable (Quinton, et al., 2022), while others use user satisfaction (Almaiah, et al., 2022; Çelik and Ayaz, 2022; Ali & Jaafrey, 2017). Finally, an outcome is expressed either qualitatively or quantitatively (Lannelongue, Gonzalez-Benito, & Gonzalez-Benito, 2015). The most widely used e-Learning outcomes are information system success, organizational performance, quality education, and net benefits (Alsalamah & Callinan, 2022; Delone & Mclean, 2003). For the context of this study, we adopted quality education as the dependent variable. We present the dimensions of the integrated model in Table 3 below.

<table><caption>Table 3: The proposed integrated model</caption><thead><tr><td>Characteristics</td><td>D&M Model</td><td>Kirkpatrick Model</td><td>Integrated Model</td></tr></thead><tbody><tr><td>inputs</td><td>system quality, information quality, and service quality</td><td>Reaction</td><td>Reaction; satisfaction with system quality, course content, teacher and institutional support.</td></tr><tr><td>Process</td><td>Use, user satisfaction</td><td>Learning and Behavioural Change</td><td>Acquisition of knowledge, and learner's satisfaction,</td></tr><tr><td>outputs</td><td>net benefits</td><td>Organizational Outcome</td><td>Quality education</td></tr></tbody></table>

Source: Literature review

Based on the above taxonomy and the review of literature, this study proposes six ideal online learning dimensions. They are system quality, quality of course content, faculty and institutional support, acquisition of knowledge, learner satisfaction, and quality education. The proposed theoretical model is shown in Figure 1:
<figure><img src="image_2.png" /><figcaption>Figure 1: Theoretical model</figcaption></figure>

Based on the above model, we formulate the following hypotheses:

H1: System quality positively relates to quality education.

H2: Quality of course content positively relates to quality education.

H3: Faculty and institutional support positively relate to quality education.

H4: Acquisition of knowledge mediates the relationship between system quality and quality education.

H5: Acquisition of knowledge mediates the relationship between quality of course content and quality education.

H6: Acquisition of knowledge mediates the relationship between faculty and institutional support and quality education.

H7: Learner satisfaction mediates the relationship between system quality and quality education.

H8: Learner satisfaction mediates the relationship between quality of course content and quality education.

H9: Learner satisfaction mediates the relationship between faculty and institutional support and quality education.

H10: Learner satisfaction positively affects acquisition of knowledge.

# 3. Research Methodology

## 3.1 Population and Sampling

This study adopted a quantitative approach. We conveniently administered questionnaires to four (4) higher educational institutions in Africa. These institutions are from Ghana, Burkina Faso, and Senegal. We opted for this sampling technique, as we were unable to randomly select participants. Because the selected institutions only agree to administer the questionnaire on their online platforms, this limits randomization. Easterby-Smith, Thorpe & Jackson (2018) describe convenience sampling as the use of an accessible sample in a population. They note that its sample characteristics limit its generalisability to a study population but can still be useful depending on the purpose of the study. This aligns with the objective of this study, which is to examine the relations between the identified constructs in the study context but not to generalize the findings. The choice of institutions is based on their strong online educational experience and blended teaching and learning delivery at the graduate and undergraduate levels.

The ethical guidelines for data collection and processing were strictly adhered to. Students were asked to voluntarily take part in the survey. They were also assured of the confidentiality of their responses. An online
survey instrument was pilot tested before being administered to students via Qualtrics. The results of the pilot study showed Cronbach Alpha values for all studied constructs higher than 0.70. The link to the questionnaire was then published on the universities' online platforms upon approval from the various administrations. Regular reminders were sent, and the data collection lasted two months, from February to April 2024. 369 responses were received, and 36 were not useful to be part of the analysis, so they were discarded. Only 333 valid responses were included for analysis. The greater number of respondents are undergraduates (n = 175), representing 52.55% of the population. Most participants are from public institutions (n = 268), representing 55.85%. The respondents' demographic data are presented in Table 4 below.

<table><caption>Table 4: The respondents' demographic data</caption><thead><tr><th>Variables</th><th>Classification</th><th>Frequency</th><th>Percentage %</th></tr></thead><tbody><tr><td rowspan="3">Education level</td><td>Bachelor</td><td>175</td><td>52,55</td></tr><tr><td>Master</td><td>158</td><td>47,45</td></tr><tr><td>Total</td><td>333</td><td>100,00</td></tr><tr><td rowspan="3">Sex</td><td>Male</td><td>195</td><td>58,56</td></tr><tr><td>Female</td><td>137</td><td>41,44</td></tr><tr><td>Total</td><td>333</td><td>100,00</td></tr><tr><td rowspan="3">Age</td><td>20 to 25 years</td><td>245</td><td>73,57</td></tr><tr><td>25 and above</td><td>88</td><td>26,43</td></tr><tr><td>Total</td><td>333</td><td>100,00</td></tr><tr><td rowspan="4">Country</td><td>Ghana (2)</td><td>183</td><td>54,95</td></tr><tr><td>Senegal (1)</td><td>85</td><td>25,53</td></tr><tr><td>Burkina Faso (1)</td><td>65</td><td>19,52</td></tr><tr><td>Total</td><td>333</td><td>100,00</td></tr><tr><td rowspan="3">Education sub-sector</td><td>Public</td><td>268</td><td>80,48</td></tr><tr><td>Private</td><td>65</td><td>53,15</td></tr><tr><td>Total</td><td>333</td><td>100,00</td></tr></tbody></table>

Source: Field data

## 3.2 Study Instrument

The six constructs were measured using previously validated scales. The first variable, system quality (SQ), was developed by Lee-Post (2009) and measured using five items. The items assessed the user's friendliness, reliability, and security of the information system. The second variable, the quality of course content (QOCC), was developed by Lee-Post (2009) and measured using four items, which assess the relevance and usefulness of course content. The third variable, faculty and institutional support (FIS) is developed by Hafeez, Naureen, & Sultan, (2022) and measured using four items that assess the availability and consistency of support from faculty and administration staff. The fourth variable, acquisition of knowledge (AK), is measured using four items identified from the literature; they assess completion of learning activities and ability to transfer learning. The fifth variable, learner satisfaction (LS), was developed by Lee-Post (2009) and measured using four items, which assess time spent learning, joy, and satisfaction in learning. The sixth variable, quality education (QE), is developed from Aledo-Ruiz, Martínez-Caro, & Santos-Jaén (2022) is measured using five items, which assess the quality of faculty members and infrastructure, the image, and pride for being associated with these institutions. Respondents were asked to rate the extent to which they agree with the questionnaire items on a five-point Likert scale, ranging from 1 = 'totally disagree' to 5 = 'totally agree.' Microsoft Excel was used to enter the coded data.

## 3.3 Data Analysis Techniques

Before performing the descriptive statistics, we used the Statistical Package for Social Sciences (SPSS) version 22 to look for outliers and missing data. This process ensures that the data to be analysed is clean and of high quality. Finally, SMARTPLS version 4 was used to evaluate the measurement and structural models. SMARTPLS does not require data to be normally distributed. Besides, Smart PLS has a user-friendly interface and supports
small sample sizes (Adam, 2015). Compared to the covariance-based SEM, PLS-SEM provides a more consistent item loading that boost the reliability and validity of factors. However, in a path analysis, the two approaches produce similar results (Dash & Paul, 2021). Therefore, we proceeded to assess the measurement model for validity and reliability of the studied constructs. Finally, the structural models were used to estimate the formulated hypotheses. We controlled for the other demographic variables, such as country, sub-sector, and type of institution, because they are not of interest to the study's objectives (Frost, 2019).

# 4. Results of this Study

## 4.1 Measurement Model Analysis

We conducted confirmatory factor analyses (CFA) to test the psychometric properties of all the variables. The aim was to examine whether there was sufficient convergent and discriminant validity among all constructs. According to Hair et al. (2021), factor loading, average variance extracted (AVE), and composite reliability (CR) are critical tests to use to assess convergent validity. Apart from the faculty and institutional support, the composite reliability values for all the other constructs exceeded the minimum threshold of 0.70. The average variance extracted for all constructs was also higher than .50. Based on the values of the AVE, which were all greater than 0, it is concluded that the reliability of the measurement model is established. Results of the goodness of measures are presented in Table 5 below.

<table><caption>Table 5: The results of goodness of measures</caption><thead><tr><th>Construct</th><th>Items</th><th>Loading</th><th>CR</th><th>AVE</th><th>VIF</th></tr></thead><tbody><tr><td rowspan="4">Acquisition of knowledge</td><td>AK1</td><td>0,808</td><td>0,764</td><td>0,574</td><td>1.856</td></tr><tr><td>AK2</td><td>0,719</td><td></td><td></td><td>1.849</td></tr><tr><td>AK4</td><td>0,72</td><td></td><td></td><td>1.288</td></tr><tr><td>AK5</td><td>0,656</td><td></td><td></td><td>1.266</td></tr><tr><td rowspan="4">Quality education</td><td>AR1</td><td>0,846</td><td>0,723</td><td></td><td>1.491</td></tr><tr><td>AR2</td><td>0,821</td><td></td><td>0,53</td><td>1.402</td></tr><tr><td>AR4</td><td>0,667</td><td></td><td></td><td>1.256</td></tr><tr><td>AR5</td><td>0,678</td><td></td><td></td><td>1.260</td></tr><tr><td rowspan="3">Faculty and institutional support</td><td>FIS1</td><td>0,663</td><td>0,59</td><td>0,546</td><td>1.107</td></tr><tr><td>FIS3</td><td>0,736</td><td></td><td></td><td>1.245</td></tr><tr><td>FIS4</td><td>0,81</td><td></td><td></td><td>1.300</td></tr><tr><td rowspan="4">Learner satisfaction</td><td>LS1</td><td>0,666</td><td>0,743</td><td>0,565</td><td>1.546</td></tr><tr><td>LS2</td><td>0,722</td><td></td><td></td><td>1.420</td></tr><tr><td>LS3</td><td>0,74</td><td></td><td></td><td>1.407</td></tr><tr><td>LS4</td><td>765</td><td></td><td></td><td>1.519</td></tr><tr><td rowspan="4">Quality of course content</td><td>QOCC1</td><td>0,759</td><td>0,741</td><td>0,552</td><td>1.434</td></tr><tr><td>QOCC2</td><td>0,724</td><td></td><td></td><td>1.445</td></tr><tr><td>QOCC3</td><td>0,793</td><td></td><td></td><td>1.463</td></tr><tr><td>QOCC4</td><td>0,686</td><td></td><td></td><td>1.311</td></tr><tr><td rowspan="4">System quality</td><td>SQ2</td><td>0,739</td><td>0,743</td><td>0,565</td><td>1.524</td></tr><tr><td>SQ3</td><td>0,706</td><td></td><td></td><td>1.413</td></tr><tr><td>SQ4</td><td>0,815</td><td></td><td></td><td>1.704</td></tr><tr><td>SQ5</td><td>0,742</td><td></td><td></td><td>1.480</td></tr></tbody></table>

Source: Statistical output

## 4.2 Discriminant Validity

Discriminant validity assesses how distinct a construct is from other constructs. It assesses how indicators represent only a single construct. The Heterotrait Monotrait (HTMT) ratios and the Fornell-Larcker criterion are
the measurement criteria for discriminant validity. The decision rule for HTMT is that its ratio must be lower than a cut-off value of 0.85 (Henseler, et al., 2015). The Fornell-Larcker criterion decision rule is that the square root of the AVEs should be greater than the correlations of the constructs (Henseler, et al., 2015). For the HTMT, constructs that are conceptually more distinct have lower and more conservative threshold values of less than 0.85 (Henseler et al., 2015). The observed values in Table 6 below indicate that this study has acceptable convergent and discriminant validity.

<table><caption>Table 6: The results of discriminant validity</caption><thead><tr><th rowspan="2"></th><th colspan="6">Heterotrait monotrait</th><th colspan="6">Fornell-Larcker criterion</th></tr><tr><th>AK</th><th>QE</th><th>FIS</th><th>LS</th><th>QOCC</th><th>SQ</th><th>AK</th><th>QE</th><th>FIS</th><th>LS</th><th>QOCC</th><th>SQ</th></tr></thead><tbody><tr><th>AK</th><td></td><td></td><td></td><td></td><td></td><td></td><td>0.757</td><td></td><td></td><td></td><td></td><td></td></tr><tr><th>QE</th><td>0.738</td><td></td><td></td><td></td><td></td><td></td><td>0.395</td><td>0.728</td><td></td><td></td><td></td><td></td></tr><tr><th>FIS</th><td>1.035</td><td>0.721</td><td></td><td></td><td></td><td></td><td>0.689</td><td>0.406</td><td>0.739</td><td></td><td></td><td></td></tr><tr><th>LS</th><td>0.520</td><td>0.871</td><td>0.660</td><td></td><td></td><td></td><td>0.384</td><td>0.720</td><td>0.367</td><td>0.752</td><td></td><td></td></tr><tr><th>QOCC</th><td>0.826</td><td>0.616</td><td>0.816</td><td>0.644</td><td></td><td></td><td>0.621</td><td>0.456</td><td>0.601</td><td>0.483</td><td>0.743</td><td></td></tr><tr><th>SQ</th><td>0.767</td><td>0.669</td><td>0.774</td><td>0.695</td><td>0.745</td><td></td><td>0.578</td><td>0.488</td><td>0.516</td><td>0.517</td><td>0.662</td><td>0.752</td></tr></tbody></table>

Source: Statistical output

## 4.3 The Goodness of Fit Model Test

The goodness-of-fit model was tested using R², F², and Q² values. The R² represents the variance explained in each of the endogenous variables, and it is a measure of the explanatory power (Shmueli & Koppius, 2011); it is also referred to as in-sample predictive power (Rigdon, 2012). The threshold values for endogenous constructs are 0.26 = substantial, 0.13 = moderate, and 0.02 = weak (Cohen, 1988). F² complements the R² test by assessing a change in R² when an exogenous variable is removed from the model. It is also referred to as effect size. According to Cohen (1988), the effect size is small at 0.02, medium at 0.15, and high at 0.35. Q² is a predictive relevance test that uses in-samples and holdout samples to measure whether a model has predictive relevance. A Q² value above 0 shows the model has predictive relevance. Table 7 below shows that all the endogenous latent variables had an R² value ranging from 0.304–0.563; therefore, the developed model is at a higher level for predicting endogenous variables. Table 7 below also shows that all endogenous latent variables had an F² value in the range of 0.003–0.089; therefore, the developed model has a high predictive relevance. Finally, Table 7 below shows that all endogenous latent variables had a Q² value ranging from 0.185 to 0.0563. Therefore, we can conclude that this model has predictive relevance. The results of the fit measures are presented in Table 7 below:

<table><caption>Table 7: The results of fit measures</caption><thead><tr><th>Exogenous construct</th><th>Endogenous Construct</th><th>R-square</th><th>F-square</th><th>Q-square</th></tr></thead><tbody><tr><td>SQ</td><td>AK</td><td>0,563</td><td>0,051</td><td>0.563</td></tr><tr><td>QOCC</td><td>LS</td><td>0,304</td><td>0,089</td><td>0.185</td></tr><tr><td>FIS</td><td></td><td></td><td>0,048</td><td></td></tr><tr><td></td><td></td><td></td><td>0.293</td><td></td></tr><tr><td></td><td></td><td></td><td>0.003</td><td></td></tr><tr><td></td><td></td><td></td><td>0,034</td><td></td></tr><tr><td>AK</td><td>QE</td><td>0,535</td><td>0,036</td><td>0.180</td></tr><tr><td>LS</td><td></td><td></td><td>0,083</td><td></td></tr></tbody></table>

Source: Statistical output

## 4.4 Testing of Hypotheses

Our study used bootstrapping of 5000 samples to test the relationship among latent variables. Significance is assessed through the p-value, T statistics, and coefficient of relationship. After confirming the goodness of the measurement model, we proceeded to examine the studied structural model. A relationship is significant if its
p-value is < 0.05 and T statistics are > 1.96 (t-value two-tail, α 5%), whilst a p-value > 0.05 and T statistics < 1.96 (t-value two-tail, α 5%) mean that, the correlation is not significant (Hair, et al., 2021). Out of the ten formulated hypotheses, eight were validated against two rejected. The direct association between system quality and quality education is supported (β=0.254; p ≤ 0.000). Similarly, the direct association between quality of course content and quality education was accepted (β= 0.179; p ≤ 0.000). Likewise, the relationship between learner's satisfaction and acquisition of knowledge was confirmed (β=0.384, p ≤ .000). In contrast, the direct relationship between faculty and institutional support and quality education was rejected (β= 0.102; p = 0.051). Besides, this study has examined the mediating roles of acquisition of knowledge and learner satisfaction in the relationships between the independent variables and the dependent variable.

The first mediating hypothesis that acquisition of knowledge mediates the relationship between system quality and quality education is supported (β = 0.028, t = t=2.103 p ≤ 0.036). The second mediating hypothesis that learner satisfaction mediates the relationship between system quality and quality education is also supported (β = 0.226, t = t=4.702, p ≤ 0.000). The third mediating hypothesis is that acquisition of knowledge mediates the relationship between quality of course content and quality education is supported (β = 0.030, t = t=2.098, p ≤ 0.036). The fourth mediating hypothesis that learner satisfaction mediates the relationship between quality of course content and quality education is supported (β = 0.149, t = t=2.953, p ≤ 0.003). The fifth mediating hypothesis, that acquisition of knowledge mediates the relationship between faculty and institutional support and quality education, is supported (β = 0.064, t = t=2.215, p ≤ 0.027). However, the final mediating hypothesis that learner satisfaction mediates the relationship between faculty and institutional support and quality education is found not to be significant (β = 0.039, t = t=2.215, p ≤ 0.337); hence, this hypothesis is rejected. We present the results of the formulated hypothesized in Table 8 below.

<table><caption>Table 8: The results of formulated hypotheses</caption><thead><tr><td>Hypotheses</td><td>Original sample (O)</td><td>Sample mean (M)</td><td>Standard deviation (STDEV)</td><td>T statistics (IO/STDEV)</td><td>P values</td><td>Decision</td></tr></thead><tbody><tr><td>SQ-&gt;QE</td><td>0.254</td><td>0.200</td><td>0.060</td><td>3.381</td><td>0.000</td><td>Accepted</td></tr><tr><td>QOCC-&gt;QE</td><td>0.179</td><td>0.179</td><td>0.051</td><td>3.511</td><td>0.000</td><td>Accepted</td></tr><tr><td>FIS-&gt;QE</td><td>0.102</td><td>0.102</td><td>0.052</td><td>1.954</td><td>0.051</td><td>Rejected</td></tr><tr><td>LS-&gt;AK</td><td>0.384</td><td>0.030</td><td>0.068</td><td>4.999</td><td>0.000</td><td>Accepted</td></tr><tr><td>SQ-&gt;AK-&gt;QE</td><td>0.028</td><td>0.027</td><td>0.013</td><td>2.103</td><td>0.036</td><td>Accepted</td></tr><tr><td>QOCC-&gt;AK-&gt;QE</td><td>0.030</td><td>0.029</td><td>0.014</td><td>2.098</td><td>0.036</td><td>Accepted</td></tr><tr><td>FIS-&gt;AK-&gt;QE</td><td>0.064</td><td>0.064</td><td>0.029</td><td>2.215</td><td>0.027</td><td>Accepted</td></tr><tr><td>SQ-&gt;LS-&gt;QE</td><td>0.226</td><td>0.229</td><td>0.048</td><td>4.702</td><td>0.000</td><td>Accepted</td></tr><tr><td>QOCC-&gt;LS-&gt;QE</td><td>0.149</td><td>0.151</td><td>0.050</td><td>2.953</td><td>0.003</td><td>Accepted</td></tr><tr><td>FIS-&gt;LS-&gt;QE</td><td>0.039</td><td>0.039</td><td>0.044</td><td>0.883</td><td>0.377</td><td>Rejected</td></tr></tbody></table>

Source: Statistical output
The structural model of this study is presented in figure 2:

<figure><img src="image_3.png" /></figure>

Source: Statistical output

<caption>**Figure 2: Statistical model**</caption>

**5. Discussion**

First, the objective of this study was to identify critical dimensions of quality in online education and then discuss their contextual relevance. To this end, the study integrated the Kirkpatrick with DeLone and McLean models to identify core quality dimensions. The integrated model shows the importance of system quality, quality of course content, faculty and institutional support, acquisition of knowledge, learner satisfaction, and quality education. Previous studies highlighted the role of system quality, information quality, and instructional and technical support (Wright, et al, 2023; Almaiah, et al, 2022; DeLone & Maclean, 2016). The design and implementation requirements of quality education are not universal but need to take account of the context-specific realities. In line with the sub-Saharan Africa context, we argue that learning systems should be optimized for mobile phones, since these are more accessible in Africa than computers. Furthermore, the technical system design must integrate lightweight materials such as compressed videos and downloadable PDF files or offer offline access to content due to the high cost of internet bandwidth in this region. Africa is a multicultural context with different ethnic groups and dialects, so instructional design must account for the local languages and dialects or offer translation subtitles. Africa needs a homegrown educational system that addresses the local challenges (Masenya, 2021; Cahapay, 2021).

In addition, the high level of illiteracy in this region requires an instructional design that accounts for varying literacy levels, using visuals and videos for low-level literacy users. Besides, the content design must also reflect the local realities, practices, and values, with more focus on collaborative learning than individualized learning. For instance, community-learning opportunities could be provided through group discussion and local mentorship. The course content must be free from bias and be inclusive of norms and cultural practices. It can be interactive and engaging, present real-life local scenarios. This indigenization of learning content will enhance learning outcomes by making it accessible and relevant (Cahapay, 2021; Masenya, 2021). The second objective of this study was to examine how the identified dimensions interact to influence perceived quality education. The dimensions of this study were classified into input, process, and output levels and operationalized as independent, mediating, and dependent variables (Galais et al., 2021). The relationships at each level are tested. Our findings show the direct effects of information systems on quality education (β=0.254; p ≤ 0.000) and that of course content on quality education (β=0.179; p ≤ 0.000). This implied that students perceived quality digital infrastructure and relevant course content as catalysts to quality education. Previous studies corroborated these findings (Khan et al., 2023; Lufti, 2017; DeLone & McLean, 2016) but without emphasis on its critical role.
However, the direct effect of faculty and institutional support on quality education was not supported (β=0.254; p ≤ 0.051). This finding contradicts the assumption of the existing literature that quality service as well as administrative and faculty support enhance organizational bottom lines (Altun & Johnson, 2023; Tan, Chan, & Mohd Said, 2021). The influence of both system quality and quality of course content might have overshadowed the support factors. Sometimes, the administrative staff are overburden with the day-to-day duties. This affects the quality of service they provide to students. In sub-Saharan Africa, often constrained by financial resources, educational service providers employ personnel who lack the requisite skills and competencies to perform faculty and administrative roles. These low-paid staff do not only lack the knowledge but also the motivation and commitment to support learners. There is a need for the top management of the concerned institutions to consistently examine their faculty and administrative support policies and strategies. This will ensure that they provide the support that meets the needs of their students. They need to motivate and provide training and development opportunities for their personnel. Studies have rather tested use, intention to use, and user satisfaction as the mediating variables (Lufti, 2023; Ali & Jaafrey, 2017). We contribute to the existing literature by testing the mediating roles of acquisition of knowledge and learner's satisfaction. Except for the mediating role of learner satisfaction in the relationship between faculty and institutional support, all the other mediating hypotheses were confirmed. The insignificant mediating effect of learner satisfaction in the relationship between faculty and institutional support and quality education implies the negative perceived role of administrative and faculty support by the students. This contradicts findings from existing literature (Lufti, 2023; Ali & Jaafrey, 2017). The context specific needs might explain these differences. Considering the contextual realities, students might have no clue of the support in question or merely such services are not up to the student's expectation. Therefore, it is important to motivate, train and develop the academic support staff.

Moreover, this study aims to investigate whether integrating the dimensions of Kirkpatrick and DeLone & McLean into an input-process-output framework would provide a more consistent and comprehensive assessment model for sub-Saharan Africa. This assumption is confirmed as most of the relationships were confirmed and supported by the study model (R², F², and Q values in table 7). Lufti (2023) argues that the D&M model has consistently proved to be unsuccessful in predicting behaviour. Besides, Aluko and Shonubi (2014) have criticized the Kirkpatrick model for failing to propose factors that impede learning transfer. This implies that none of the models can sufficiently explain the quality of e-Learning education. Kirkpatrick focuses on learning outcomes whilst D&M focuses on information system success. Thus, integrating these models reinforces their capacity in assessing quality online education. The combination of these models offers a holistic view to assessing both technical and non-technical interventions. Besides, the integration of the models highlights the importance of both formative and summative evaluations when assessing quality education. This establishes a clear cause-and-effect pathway. Moreover, it provides insights on how these critical dimensions interact together to produce a desirable outcome. This integrated model outlines the dimensions at the organizational level that are subject to institutional reconfiguration to enhance quality learning.

## 5.1 Implications and Limitations

Existing studies have used subjective measures to identify quality dimensions (Safdar et al., 2020; Marciniak, 2018). Others have borrowed dimensions from existing models (Lufti, 2023; Alsalamah & Callinan, 2022;). To the best of our knowledge, however, no study has examined the integration of the Kirkpatrick model to the D&M model. This study therefore extends existing literature by integrating complementary models: one that assesses learning evaluation and one that evaluates information system quality. This integrated model remedies inconsistencies in the existing literature by providing a unified and structured model that addresses gaps, overlaps, and misalignment in previous studies (Cahapay, 2021; Esfijani, 2018). In addition, using the input-process-output framework to classify quality dimensions enhances comparability among studies. Theoretically, this study highlighted the importance of contextual relevance in the formulation of e-Learning strategy. In practice, this model supports both formative and summative evaluations, as poor learning outcomes can be traced back to inadequate system quality. Similarly, quality technical infrastructure can lead to better learning outcomes.

The practical implication of this study is that policy and decision-makers at e-Learning institutions in Africa should advocate for online learning solutions that address a mix of technological, cultural, linguistic, pedagogical, and socio-economic realities of the region. As fully online qualifications are often subject to controversies for recognition in Africa, policy and decision makers must ensure that their course content conforms to local and regional standards. Adopting a technical system that is optimized for mobile phones and is energy and bandwidth efficient will enhance access to education. These outlined elements, coupled with context-specific instructional design that integrates local languages, can enhance access and the relevance of the online
programmes. Quality assessment professionals could approach quality improvement from a system point of view with input-process-output analysis. Malfunctions at the process level, such as student dissatisfaction or poor performance, can result to bigger reputational problems. Finally, by recommending a context-specific design and delivery of e-Learning in a developing context, this study contributes to the achievement of the Sustainable Development Goal n°4, which is the commitment to promoting the delivery of inclusive but quality education and development opportunities. This model is simple, theoretically based, and comprehensive for real-life applications.

Despite the valuable insights gained from this study, several limitations warrant consideration. First, the use of self-reported questionnaires may introduce potential bias and inaccuracy. However, we have used cross-country data and controlled for this problem using Harman's single factor test to assess the total variance of the items. Common method variance remains a potential concern for studies that collect data using a self-reported questionnaire (Podsakoff, MacKenzie, & Podsakoff, 2012). The result reveals a value of 32.45% of total variance, which is less than the 50% threshold. This indicated that this study was free from common method bias. Future studies could use this integrated model with objective measures or collect data from multiple sources. They could also collect time series data. Additionally, the cross- sectional design may limit our ability to establish causality between the identified variables. Future studies could employ longitudinal methods with objective measures to provide a more nuanced understanding of these relationships in different contexts. Besides, the use of a convenient sampling technique limits the generalizability of the findings (Easterby-Smith, Thorpe, & Jackson, 2018). Future studies could adopt a simple random sampling technique using a larger sample size to test this model. Furthermore, our classification of dimensions using the input-process-output framework (Galais et al., 2021) may be somewhat subjective. Therefore, other studies could use different methods to categorize the dimensions of the Kirkpatrick and Delone & McLean models. Finally, as most of the respondents were undergraduates, our findings may not be generalizable to the graduate level.

## 5.2 Conclusion

Stakeholders are greatly concerned about the quality of online higher education assessment. Considering the growth of online education and its strategic importance for human resource development, it is critical to understand the critical dimensions that contribute to quality education. However, this has not been the case, given that we have a multitude of quality dimensions and assessment models. This makes quality assessment challenging and comparability between studies difficult. As a result, this study's objective was to identify the critical quality dimensions of online learning through the integration of the Kirkpatrick to DeLone & McLean models. Six dimensions, namely information quality, quality of course content, faculty and institutional support, acquisition of knowledge, learner satisfaction, and quality education were identified. Furthermore, the objective of this study was to propose a theoretically based quality assessment model based on the input-process-output framework. The findings of this study implied the appropriateness and suitability of the model in supporting both formative and summative evaluations. The integrated model offers a comprehensive lens to assess both technical and non-technical interventions of learning outcomes. This study emphasizes the critical role of the institutional-level dimensions, as learner-level dimensions such as time taken to study, learner's motivation are beyond the control of the training institutions. The thesis of this study is that certain dimensions are essential and should be consistent across studies. The integration of Kirkpatrick to D&M models revealed that these dimensions are course content, quality system, faculty, and institutional support. Adapting these dimensions to the region's specific needs, will enhance learning outcomes, bring economic and societal benefits to the region, and improve the recognition of online learning. The identified dimensions do not constitute an exhaustive list of institutional-level factors. Other factors, such as the learning environment, vision and mission statements, and ethical considerations, can be considered as moderating variables.

## Acknowledgements

We would like to acknowledge the support of the concerned institutions for making this research project possible.

**Ethics statement:** Ethical review and approval were not required for this study in accordance with the local legislation and institutional requirements. Participation in the questionnaire was entirely voluntary, and all respondents provided informed consent before participating. No personal identifying information was collected, and data were analyzed anonymously to ensure participants' confidentiality.

**AI statement:** No AI was used at any point in the research, writing, or creating of this paper.
References

Abdulghani, H. M., Shaik, S. A., Khamis, N., Al-Drees, A. A., Irshad, M., Khalil, M. S., & Isnani, A. (2014). Research methodology workshops evaluation using Kirkpatrick's model: translating theory into practice. *Medical Teacher*, 36(sup1), S24-S29.

Adam, A., M. (2015). Statistics for business research: A Guide for non-statisticians. First Edition, ISBN-13: 978-1505358391.
Global Research Publishers.

Aledo-Ruiz, M. D., Martínez-Caro, E., & Santos-Jaén, J. M. (2022). The influence of corporate social responsibility on students' emotional appeal in the HEIs: The mediating effect of reputation and corporate image. *Corporate Social Responsibility and Environmental Management*, 29(3), 578-592.

Ali, B., & Jaafreh, A. (2017). Evaluation of information system success: applied delone and MClean information system success model in context banking system in KSA. *Assurance in Education*, 32(1), 1-16. Available at [https://doi.org/10.1108/QAE-05-2023-00836](https://doi.org/10.1108/QAE-05-2023-00836). [Accessed: 07 February 2024].

Almaiah, M. A., Al-Rahmi, A. M., Alturise, F., Alrawad, M., Alkhalaf, S., Lutfi, A., Al-Rahmi, W. M., & Awad, A. B. (2022).
Factors influencing the adoption of Internet banking: An integration of ISSM and UTAUT with price value and
perceived risk. *Frontiers in Psychology*, 13. Available at [https://doi.org/10.3389/fpsyg.2022.919198](https://doi.org/10.3389/fpsyg.2022.919198). [Accessed: 01
September 2022]

Alsalamah, A., & Callinan, C. (2022). The Kirkpatrick model for training evaluation: Bibliometric analysis after 60 years (1959–2020). *Industrial and Commercial Training*, 54(1), 36–63. Available at [https://doi.org/10.1108/ICT-12-2020-0115](https://doi.org/10.1108/ICT-12-2020-0115). [Accessed: 03 May 2024]

Altun, S. A., & Johnson, T. (2023). Online education directors' perceptions of leadership, management and administrative roles. In *Global Perspectives on the Internationalization of Higher Education* (pp. 300-314). IGI Global.

Aluko, F. R., & Shonubi, O. K. (2014). Going beyond Kirkpatrick's training evaluation model: The role of workplace factors in distance learning transfer. *Africa Education Review*, 11(4), 638–657. Available at [https://doi.org/10.1080/18146627.2014.935007](https://doi.org/10.1080/18146627.2014.935007). [Accessed: 05 May 2022]

Barbour, M., & Clark, T. (2016). Cases of quality: Case studies of the approval and evaluation of K-12 online and blended providers. In *In Society for Information Technology & Teacher Education International Conference* (pp. 809-815).
Association for the Advancement of Computing in Education (AACE).

Barteit, S., Guzek, D., Jahn, A., Bärnighausen, T., Jorge, M. M., & Neuhann, F. (2020). Evaluation of e-Learning for medical education in low and middle-income countries: A systematic review. *Computers & Education*, 145, 103726.

Cahapay, M. (2021). Kirkpatrick Model: Its limitations as used in higher education evaluation. *International Journal of Assessment Tools in Education*, 8(1), 135-144. Available at [https://doi.org/10.21449/ijate.856143](https://doi.org/10.21449/ijate.856143). [Accessed: 20 February 2024]

Çelik, K., & Ayaz, A. (2022). Validation of the Delone and McLean information systems success model: A study on student information system. *Education and Information Technologies*, 27(4), 4709–4727. Available at [https://doi.org/10.1007/s10639-021-10798-4](https://doi.org/10.1007/s10639-021-10798-4). [Accessed: 07 May 2024]

Cohen, A., Nørgård, R. T., & Mor, Y. (2020). Hybrid learning spaces—Design, data, didactics. *British Journal of Educational Technology*, 51(4).

Cohen, J. (1988). Statistical Power analysis for the behavioural sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

Dash, G., & Paul, J. (2021). CB-SEM vs PLS-SEM methods for research in social sciences and technology forecasting. *Technological Forecasting and Social Change*, 173, 121092

Delone, W. H., & Mclean, E. R. (2016). Information systems success measurement. *Foundations and Trends in Information Systems*, 2(1), 1–116. Available at [https://doi.org/10.1561/2900000005](https://doi.org/10.1561/2900000005). [Accessed: 09 May 2024]

Delone, W., & Mclean, E. (2003). The DeLone and McLean model of information systems success: a ten-year update. *Journal of Management Information Systems*, 19, 9–30. Available at [https://doi.org/10.1080/07421222.2003.11045748](https://doi.org/10.1080/07421222.2003.11045748). [Accessed: 11 May 2024]

Dennis, D. R., & Meredith, J. R. (2000). An analysis of process industry production and inventory management systems. *Journal of operations management*, 18(6), 683-699.

Easterby-Smith, M., Thorpe, R., & Jackson, P. R. (2018). Management research. Sage. CA, Sage, 2018, 440 pp., ISBN 978-1-5264-2479-2, ISBN 978-1-5264-2479-8

Learning Africa (2025). Growth of Africa's e-Learning market. Published on January 29, 2025, 12:00 pm. Available at [https://ela-newsportal.com/africa-e-Learning-market-transforming-education-across-the-continent/#:~:text=Africa%20is%20experiencing](https://ela-newsportal.com/africa-e-Learning-market-transforming-education-across-the-continent/#:~:text=Africa%20is%20experiencing). [Accessed: 03 May 2024]

Efijani, A. (2018). Measuring quality in online education: A Meta-synthesis. *American Journal of Distance Education*, 32(1), 57-73. Available at [https://doi.org/10.1080/08923647.2018.1417658](https://doi.org/10.1080/08923647.2018.1417658). [Accessed: 02 May 2024]

Frost, J. (2019). Introduction to statistics. *Statistics Pennsylvania, State College, PA 16801. Jim Publishing ISBN 978-1-7354311-0-9.*

Galais, C., Fernández-Martínez, J. L., Font, J., & Smith, G. (2021). Testing the input-process-output model of public participation. *European Journal of Political Research*, 60(4), 807-828.

Green, D. (1994). What is quality in higher education? Taylor & Francis, 1900 Frost Road, Bristol, PA 19007-1598. Available at [https://eric.ed.gov/?id=ED415723](https://eric.ed.gov/?id=ED415723). [Accessed: 11 May 2024]
Hafeez, M., Naureen, S., & Sultan, S. (2022). Quality indicators and models for online learning quality assurance in higher education. *Electronic Journal of E-Learning, 20*(4), Article 4. Available at https://doi.org/10.34190/ejel.20.4.2553.
[Accessed: 21 May 2024]

Hair Jr, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S. (2021). *Partial least squares structural equation modelling (PLS-SEM) using R: A workbook* (p. 197). Springer Nature.

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modelling. *Journal of the academy of marketing science*, 43, 115-135.

Khan, E. A., Cram, A., Wang, X., Tran, K., Cavaleri, M., & Rahman, M. J. (2023). Modelling the impact of online learning quality on students' satisfaction, trust and loyalty. *International Journal of Educational Management*, 37(2), 281-299.

Kotler, P. (2019). The market for transformation. *Journal of Marketing Management*, 35(5-6), 407-409.

Lannelongue, G., Gonzalez-Benito, J., & Gonzalez-Benito, O. (2015). Input, output, and environmental management productivity: effects on firm performance. *Business Strategy and the Environment*, 24(3), 145-158.

Laudon, K., & Laudon, J. (2009). Management information systems: International Edition, 11/E. London, Uk: Pearson Higher Education.

Lee-Post, A. (2009). E-Learning success model: An information systems Perspective. *Electronic Journal of E-Learning, 7*(1), 61-70.

Luhmann, N., Baecker, D., & Gilgen, P. (2013). Introduction to systems theory (p. 63). Cambridge: Polity.

Lutfi, A. (2023). Factors affecting the success of accounting information systems from the lens of DeLone and McLean IS model. *International Journal of Information Management Data Insights*, 3(2), 100202. Available at
https://doi.org/10.1016/j.ijimei. [Accessed: 07 May 2024]

Marciniak, R. (2018). Quality assurance for online higher education programs: Design and validation of an integrative assessment model applicable to Spanish universities. *International Review of Research in Open and Distributed Learning*, 19(2).

Martin, F., & Bolliger, D. U. (2022). Developing an online learner satisfaction framework in higher education through a systematic review of research. *International Journal of Educational Technology in Higher Education*, 19(1), 50. Available at https://doi.org/10.1186/s41239-022-00355-5. [Accessed: 03 May 2024]

Masenya, M. J. (2021). Toward a relevant de-colonized curriculum in South Africa: suggestions for a way forward. *SAGE Open*, 11(4). Available at https://doi.org/10.1177/21582440211052559 (Original work published 2021) [Accessed: 07 May 2024]

Mkinga, M., & Mandari, H. (2020). Evaluating student's information system success using DeLone and McLean's Model: Student's Perspective. *Journal of International Technology and Information Management*, 29(2), 24-42. Available at
https://doi.org/10.58729/1941-6679.1447. [Accessed: 07 May 2024]

Mosquera, J. M. L., Suarez, C. G. H., & Guerrero, V. A. B. (2023). Effect of flipped classroom and automatic source code evaluation in a CS1 programming course according to the Kirkpatrick evaluation model. *Education and Information Technologies*, 28(10), 13235-13252. Available at https://doi.org/10.1007/s10639-023-11678-9. [Accessed: 05 May 2024]

Online Learning Consortium (2024). Administration of online programs. Available at
https://onlinelearningconsortium.org/consulting/quality-scorecards/administration-of-online-programs/. [Accessed:
07 May 2024]

Peters, B., & Göhlich, M. (2024). Transformative learning through group coaching. *Journal of Transformative Education*, 22(2), 157-179. Available at https://doi.org/10.1177/15413446231178897. [Accessed: 05 May 2024]

Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. *Annual review of psychology*, 63(1), 539-569.

Praslova, L. (2010). Adaptation of Kirkpatrick's four-level training criteria to assessment of learning outcomes and program evaluation in Higher Education. *Educational Assessment, Evaluation and Accountability*, 22(3), 215-225. Available at
https://doi.org/10.1007/s11092-010-9098-7. [Accessed: 05 May 2024]

Quinton, M. L., Tidmarsh, G., Parry, B. J., & Cumming, J. (2022). A Kirkpatrick model process evaluation of reactions and learning from my strengths training for life™. *International Journal of Environmental Research and Public Health*, 19(18). Available at https://doi.org/10.3390/ijerph191811320. [Accessed: 22 May 2024]

Rahmawati, W. (2020). Evaluating the effectiveness of training with the Kirkpatrick method (study of Semen Padang Hospital employees). *Economica*, 8(2), 117-129. Available at https://doi.org/10.22202/economica.2020.v8.i2.3649.
v8.i2.3649. [Accessed: 20 May 2024]

Rigdon, E. E. (2012). Rethinking partial least squares path modelling: In praise of simple methods. *Long range planning*, 45(5-6), 341-358.

Safdar, G., Rauf, A., Ullah, R., & Rehman, A. (2020). Exploring Factors Leading to Quality Online Learning in the Era of Covid-19: A Correlation Model Study. *Universal Journal of Educational Research*, 8, 7324-7329. Available at
https://doi.org/10.13189/. [Accessed: 03 May 2024]

Shmueli, G., & Koppius, O. R. (2011). Predictive analytics in information systems research. * MIS quarterly*, 553-572.

Smidt, A., Balandin, S., Sigafoos, J., & Reed, V. A. (2009). The Kirkpatrick model: A useful tool for evaluating training outcomes. *Journal of Intellectual & Developmental Disability*, 34(3), 266-274. Available at
https://doi.org/10.1080/13668250903093125. [Accessed: 06 May 2024]
Statista (2025). Online education Africa. Available at https://www.statista.com/outlook/emo/online-education/africa?srsltid=AfmBOorOWN-W0EMxtXOvF3iCR-5xJgMaDx3D8CiXOV2zojaq1alF689. [Accessed: 10 September 2025]

Tamkin, P., Yarnall, J., & Kerrin, M. (2002). Kirkpatrick and Beyond: A Review of Models of Training Evaluation. Brighton, England: Institute for Employment Studies. Available at https://www.employment-studies.co.uk/system/files/resources/files/392.pdf. [Accessed: 20 February 2024]

Tan, K. H., Chan, P. P., & Mohd Said, N.-E. (2021). Higher education students’ online instruction perceptions: A quality virtual learning environment. *Sustainability*, 13(19), Article 19. https://doi.org/10.3390/su131910840. [Accessed: 20 February 2024]

United Nation International Children Education Fund Unicef South Asia (2024). Quality of education. Available at https://www.unicef.org/rosa/what-we-do/quality-education. [Accessed: 02 May 2024]

Wright, A. C., Carley, C., Alarakyia-Jivani, R., & Nizamuddin, S. (2023). Features of high-quality online courses in higher education: A Scoping Review. *Online Learning*, 27(1), 46-70.