# Students' Perceptions of e-Assessment

Bastian Küppers, Ulrik Schroeder

► To cite this version:

Bastian Küppers, Ulrik Schroeder. Students' Perceptions of e-Assessment. Open Conference on Computers in Education (OCCE), Jun 2018, Linz, Austria. pp.275-284, 10.1007/978-3-030-23513-0_27. hal-02370930

HAL Id: hal-02370930

https://inria.hal.science/hal-02370930

Submitted on 19 Nov 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Students' Perceptions of e-Assessment

## A Case Study from Germany

Bastian Küppers

IT Center, RWTH Aachen University, Germany, kueppers@itc.rwth-aachen.de

Ulrik Schroeder

Learning Technologies Research Group, RWTH Aachen University, Germany,
schroeder@cs.rwth-aachen.de

**Abstract.** In order to verify common findings in the literature regarding the conception of e-assessment among students, we carried out a survey based on common findings. Our survey, which has been carried out over several higher education institutes, enhances the already existing findings by adding new facets. The achieved results are promising in that students seem to be open-minded regarding e-assessment, which is in line with the findings in the already existing literature. However, there are some open points that have to be resolved in a reliable way in order to completely convince the students of the opportunities offered by e-assessment.

**Keywords.** Computer-based Assessment, e-Assessment, BYOD, Cheating

## 1 Introduction

If e-assessment is to be introduced into the examination system of an institute of higher education (IHE), it is not only the staff of the latter that have to accept this type of assessment, but also the students [1], especially when it comes to e-assessment on students' devices (BYOD) [2]. Therefore, it is of interest for IHEs that are willing to introduce e-assessment, to be aware of the possible limiting factors from the students' points of view, to tailor the e-assessment system and the process of integration to the students' needs.

To verify that the findings regarding the students' points of view in the literature are valid for our institution, we carried out our own survey about e-assessment, BYOD scenarios and cheating in examinations.

The paper is structured as follows: in the second section, we give a brief overview of the findings already presented in the literature. In the third section, we discuss the setup of our survey, followed by a discussion of the achieved results in the fourth section. The paper closes with a summary and an outlook.
## 2 Related research

There is a lot of literature about students' perceptions of e-assessment, which has been written over the last years. Most of these papers focus on a particular IHE, e.g. Saudi Electronic University, Saudi Arabia [3] and Dow University of Health, Pakistan [4]. Some papers focus even on a single study course, e.g. Polytechnic Institute of Porto, Portugal, Marketing Degree [5], University College London, UK, Chemical Engineering [6], Hong Kong Polytechnic University, Hong Kong, Rehabilitation Sciences) [7] and Kocaeli University, Turkey, Desktop Publishing [8]. The findings reported in these papers testified generally positive students' attitudes regarding e-assessment.

For the course of this paper, the most important publication is “e-Exams with student owned devices: Student voices” by Matthew Hillier [2], since his paper focuses on a BYOD scenario. There are many interesting findings about students’ perceptions not only regarding e-assessment, but especially about their perceptions regarding e-assessment on their own devices. However, even Hillier’s research was conducted only in one IHE.

## 3 Design of the survey

We constructed our survey based on the findings in a previous paper [2], to answer our research question: **Which factors influence students' perceptions of e-assessment?**

We anticipated that the perception of e-assessment is influenced by:

* gender

* age

* the study programme (science, technology, engineering and mathematics (STEM) versus humanities, for example)

* technology affinity

* the stage of study (Bachelor versus Master level)

Since we expected the results to be additionally influenced by the general technology affinity of the students, we incorporated another questionnaire as part of our survey to be able to distinguish technology accepting students and technology reluctant students. This questionnaire is the TA-EG questionnaire by Karrer et al. [9], which is designed to measure technology affinity. The items of the TA-EG questionnaire have been reordered to eliminate effects that could originate from the clustered answers of the original questionnaire. Additionally, unlike existing surveys, we wanted to carry out the survey at multiple IHEs and for different study courses.

Altogether, this resulted in the, originally German, survey as shown in Table 1. The survey was carried out mainly with students of RWTH Aachen University and FH Aachen University of Applied Sciences, but also students at Maastricht University, Alpen-Adria-University Klagenfurt, TU Berlin, FOM Hochschule für Oekonomie und Management (Study Centre Aachen) and Albstadt-Sigmaringen University were invited to participate. The study programmes mentioned explicitly in the survey are the main study programmes, which are related computer science courses at those universities.
<table><caption>Table 1. The Survey (translated to English).</caption><thead><tr><th>Part</th><th>Item</th><th>Scale</th></tr></thead><tbody><tr><td rowspan="3">General</td><td>Age</td><td>3 Options<sup>1</sup></td></tr><tr><td>Study Programme</td><td>9 Options<sup>2</sup></td></tr><tr><td>Gender</td><td>2 Options</td></tr><tr><td rowspan="18">TA-EG</td><td>I like to have new electronic devices.</td><td rowspan="18">Five-level Likert Scale (5LLS)</td></tr><tr><td>Electronic devices cause illness.</td></tr><tr><td>I like to go to stores for electronic devices.</td></tr><tr><td>I (would) have problems understanding electronic and computer magazines.</td></tr><tr><td>Electronic devices provide a high standard of living.</td></tr><tr><td>Electronic devices lead to intellectual impoverishment.</td></tr><tr><td>Electronic devices make many things more complicated.</td></tr><tr><td>I inform myself about electronic devices, even if I have no intention to buy them.</td></tr><tr><td>Electronic devices make you independent.</td></tr><tr><td>I enjoy trying out electronic devices.</td></tr><tr><td>Electronic devices make everyday life easier for me.</td></tr><tr><td>Electronic devices increase security.</td></tr><tr><td>Electronic devices reduce personal contact between people.</td></tr><tr><td>I know most of the functions of the electronic devices I own.</td></tr><tr><td>I am thrilled when a new electronic device comes onto the market.</td></tr><tr><td>Electronic devices cause stress.</td></tr><tr><td>I know about electronic devices.</td></tr><tr><td>It is easy for me to learn how to operate an electronic device.</td></tr><tr><td>Electronic devices help to obtain information.</td></tr><tr><td rowspan="5">e-Assessment</td><td>E1: I think it is very good to have electronic examinations in my studies.</td><td rowspan="5">5LLS</td></tr><tr><td>E2: I think that electronic examinations are a good complement to paper-based examinations.</td></tr><tr><td>E3: I think that electronic examinations are a good substitute to paper-based examinations.</td></tr><tr><td>E4: I see advantages of electronic examinations, namely...</td></tr><tr><td>E5: I see disadvantages of electronic examinations, namely...</td></tr><tr><td rowspan="3">BYOD</td><td>B1: I find it very advantageous if electronic examinations are carried out on my own electronic device (laptop).</td><td>5LLS</td></tr><tr><td>B2: I see the following advantages in using my own electronic device (laptop) for an examination, namely...</td><td>3 Options<sup>5</sup></td></tr><tr><td>B3: I see the following disadvantages in using my own electronic device (laptop) for an examination, namely....</td><td>3 Options<sup>6</sup></td></tr><tr><td rowspan="2">Fraud</td><td>C1: I think that cheating in paper-based examinations can be done very easily.</td><td rowspan="2">5LLS</td></tr><tr><td>C2: I think that cheating in electronic examinations can be done very easily.</td></tr></tbody></table>
The options for the items noted in Table 1 are:

1. < 18; 18 – 25; > 25

2. Bachelor of Computer Science, Master of Computer Science, Scientific Programming, Technomathematics, Bachelor of Technical Communication, Master of Technical Communication, Bachelor of Computer Science (Teacher), Master of Computer Science (Teacher), Other (free text)

3. Faster Correction, More Realistic Examinations, More Diverse Examination Tasks, Other (free text)

4. Security, Usability, Fairness, Other (free text)

5. Familiar Device, Location-independent Examinations, Other (free text)

6. Security, Differences Between Devices, Other (free text)

## 4 Analysis of the results

In total, 408 students responded to the survey with demographics as shown in Table 2.

<table><caption>**Table 2.** Demographics of the participating students.</caption><thead><tr><th></th><th>Male</th><th>Female</th><th>Not Answered</th><th>Σ</th></tr></thead><tbody><tr><th>&lt; 18</th><td>1.2 %</td><td>0.25 %</td><td>0 %</td><td><strong>1.5 %</strong></td></tr><tr><th>18 – 25</th><td>60.3 %</td><td>16.67 %</td><td>0.5 %</td><td><strong>77.5 %</strong></td></tr><tr><th>&gt; 25</th><td>14 %</td><td>6.4 %</td><td>0 %</td><td><strong>20.3 %</strong></td></tr><tr><th>Not Answered</th><td>0.46 %</td><td>0 %</td><td>0.25 %</td><td><strong>0.7 %</strong></td></tr><tr><th>Σ</th><td><strong>75.96 %</strong></td><td><strong>23.3 %</strong></td><td><strong>0.74 %</strong></td><td><strong>100 %</strong></td></tr></tbody></table>

About three quarters of the participating students were male and one quarter were female. A similar distribution can be seen for the age, where about three quarters were aged between 18 and 25 years and nearly a fifth of the students were aged above 25 years.

The students came from a variety of study programmes, as can be seen from Figure 1. Other programmes of study included artificial intelligence, engineering and physics. So, despite individual students studying in programmes like economics and literature, the vast majority of the study programmes were related to a STEM topic. Therefore, it is not surprising that the results of the TA-EG questionnaire did not allow for identifying subgroups with different affinities regarding technology.

The plots in Figure 2 refer to the original grouping of the TA-EG questionnaire, which has four groups: Enthusiasm, Competency, Positive Attitude, and Negative Attitude. The five subplots in each of these plots refers to a question in the corresponding group of questions of the TA-EG questionnaire. Please note that every item in the TA-EG questionnaire in our survey used a five-level Likert scale ranging from 1 (“Strongly agree”) to 5 (“Strongly disagree”). The overall variance of these items was 0.76, which accounts for the indistinguishability of different subgroups.
<figure><img src="image_2.png" /><figcaption>**Figure 1.** Distribution of study programmes.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Violin plots [10] of the TA-EG sub-questionnaire.</figcaption></figure>

Since there were too few students enrolled in a study course that were not from the STEM field, the collected data were not suitable to answer whether the study course influenced the students’ perceptions of e-assessment. The absence of those students may be caused by the decision to carry out the survey via an online portal, which may have biased the results so that only students participated who had an affinity for technology. However, that cannot be concluded from the data.
## 4.1. Influences of gender and age

To examine the influence of gender, age, and study level (bachelor or master) the data set was split into subsets accordingly. These subsets were then tested for significant differences with a Fisher test [11]. The results for the Likert-scaled questions can be found in Table 3.

<table><caption>**Table 3.** p-values for the Fisher test.</caption><thead><tr><th rowspan="2"></th><th colspan="3">p-Value</th></tr><tr><th>Gender</th><th>Age</th><th>Study Level</th></tr></thead><tbody><tr><td>E1</td><td>0.04536 &lt; 0.05</td><td>0.0951 &lt; 0.1</td><td>0.1661</td></tr><tr><td>E2</td><td>0.4115</td><td>0.002503 &lt; 0.01</td><td>0.0534 &lt; 0.1</td></tr><tr><td>E3</td><td>0.6161</td><td>0.14</td><td>0.7155</td></tr><tr><td>B1</td><td>0.2079</td><td>0.211</td><td>0.1844</td></tr><tr><td>C1</td><td>0.5356</td><td>0.6287</td><td>0.3458</td></tr><tr><td>C2</td><td>0.5694</td><td>0.5445</td><td>0.185</td></tr></tbody></table>

Given these p-values, conclusions about the influence of gender, age and study level are possible to a certain extent. Regarding question E1, it seems that women are more hesitant to accept e-assessment as part of the examination system. In addition, students between 18 and 25 years seem to be more positive about e-assessment than students of other ages. For question E2, the age again makes a difference, as students older than 25 years seem to be less convinced that e-assessment is a good complement to paper-based examinations compared to younger students. The same tendency is revealed when considering the question about whether the study level influences the perception of e-assessment. Students that are enrolled in a master's programme seem to be more reluctant regarding e-assessment than students in a bachelor's programme. If this tendency is caused by the progress in the studies or by age, again, is a crucial factor that cannot be concluded from the data, which is shown in **Figure 3**.
<figure><img src="image_4.png" /></figure>

E1, Gender

$$ \mu_f = 2.31, \sigma_f^2 = 1.39 $$

$$ \mu_m = 2.05, \sigma_m^2 = 1.08 $$

$$ \mu_o = 2.12, \sigma_o^2 = 1.2 $$

<figure><img src="image_5.png" /></figure>

E1, Age

$$ \mu_{<18} = 2.5, \sigma_{<18}^2 = 1.9 $$

$$ \mu_{18-25} = 2.06, \sigma_{18-25}^2 = 1.17 $$

$$ \mu_{>25} = 2.30, \sigma_{>25}^2 = 1.16 $$

$$ \mu_o = 2.12, \sigma_o^2 = 1.2 $$

<figure><img src="image_6.png" /></figure>

E2, Age

$$ \mu_{<18} = 1.5, \sigma_{<18}^2 = 0.3 $$

$$ \mu_{18-25} = 1.91, \sigma_{18-25}^2 = 1.1 $$

$$ \mu_{>25} = 2.16, \sigma_{>25}^2 = 0.99 $$

$$ \mu_o = 1.97, \sigma_o^2 = 1.1 $$

<figure><img src="image_7.png" /></figure>

E2, Study Level

$$ \mu_b = 1.81, \sigma_b^2 = 0.68 $$

$$ \mu_m = 1.99, \sigma_m^2 = 1.15 $$

$$ \mu_o = 1.97, \sigma_o^2 = 1.1 $$

<figure><img src="image_8.png" /><figcaption>**Figure 4.** Violin plots of questions E3, B1, C1, and C2.</figcaption></figure>
# 5 Discussion

The achieved results from the survey show a rather clear picture. The students would like to have electronic examinations in their studies, but not necessarily as a replacement for paper-based examination, but rather as a complementary approach, as the answers of questions E2 and E3 (see Figure 4) suggest. This perception of e-assessment is understood through advantages, which cover topics like faster correction (75.98%, E4), more realistic assignments (62.74%, E4), more diverse examination tasks (45.34%, E4), and readability (which was stated in free text comments). However, students are also concerned about disadvantages, like security (41.67%, E5), usability (42.64%, E5), and fairness (34.56%, E5). Additionally, technical difficulties and the subsequent loss of already solved assignments are mentioned very often in the comments. Overall, less than half of the students sees disadvantages in e-assessment; however, especially when it comes to a BYOD approach, the students are afraid that technical difficulties may lead to a handicap for them or that they have to have a capable device on their own. Still, the tendency seems to be positive regarding a BYOD approach (see B1 in Figure 4), as students see the advantage of a familiar device (89.7%, B2). However, due to the reported concerns, it is very important to have a reasonable backup strategy for these situations. As we have discussed elsewhere [12], it is important to regularly have backups during an e-assessment, so that a student can simply switch to an emergency device provided by the IHE in case her own device breaks down. These emergency devices could also be used for students that cannot afford a device on their own in order to enable these students to participate in electronic examinations. Additionally, the topic of fairness is important to the students, as they state differences between the students' devices as the main concern when utilising BYOD (82.84%, B3). Furthermore, topics like security or cheating are of importance for the students. The students are rather split about the risk of cheating in paper-based examinations; however, there is a tendency that students think that it is easier to cheat in electronic examinations (see C1 and C2 in Figure 4). Therefore, new ways of reducing the risk of cheating in electronic examinations have to be found, as we have discussed elsewhere [13] and presented an approach to security [14].

Age is seemingly a factor; it does influence the perception of e-assessment in line with the concept of *Digital Natives* introduced by Marc Prensky [15]. He claims that “[t]oday’s students have not just changed incrementally from those of the past”, but underwent a drastic change of attitude, because “the arrival and rapid dissemination of digital technology in the last decades of the 20th century [was] an event which changes things so fundamentally that there is absolutely no going back”. The evidence gained from the survey suggests a similar conclusion, because there is a statistically significant difference between students over the age of 25 years in comparison to younger students. Shelley White states in her article “The Generation Z effect” [16] that “Gen Z is loosely accepted as people born in the mid- to late-1990s and later. (According to the Pew Research Center in the United States, the last Gen Y was born in 1997, while Statistics Canada says Gen Z starts with people born in 1993)”. The timespan mentioned in her article is exactly in line with our findings of the age that has an influence on the perception of e-assessment.

Gender having an influence on the perception of e-assessment is actually not surprising, as many studies show that women seem to have a lower confidence in using
technology in general than men, for example Kadijevich [17], Kahveci [18], and Yau and Cheng [19], whether this is justified or not. Therefore, it is reasonable to assume that the same tendency can be observed when examining the perception of e-assessment.

# 6 Summary and outlook

In order to identify factors that influence students' perceptions of e-assessment, we carried out our own survey based on the findings in a previous paper [2]. However, we extended our survey over multiple IHEs to gain a broader view. The results are promising, in that students seem to be open-minded regarding e-assessment, which is in line with the findings in the already existing literature. However, there are open points that have to be reliably resolved in order to convince the students completely of e-assessment. Therefore, more research is needed to uncover all the open questions that exist among the students as well as to find solutions to these open questions. Further research could also tackle the question as to whether affinity to technology and the field of study have a direct influence on the perception of e-assessment. In addition, it could be further investigated if the influence of the study level is indeed significant, due either to further progress in studying or if there is a hidden correlation between age and level of study.

# References

1. Terzis, V., Economides, A.A.: The acceptance and use of computer based assessment. Computers & Education. 4(56), 1032-1044 (2011)

2. Hillier, M.: e-Exams with student owned devices: Student voices. Proceedings of the International Mobile Learning Festival 2015: Mobile Learning, MOOCs and 21stCentury learning, Hong Kong SAR, China. 582-608 (2015)

3. Alsadoon, H.: Students' Perceptions of E-Assessment at Saudi Electronic University. The Turkish Online Journal of Educational Technology. 16(1), 147-153 (2017)

4. Jawaid, M., Moosa, F.A., Jaleel, F., Ashraf, J.: Computer Based Assessment (CBA): Perception of residents at Dow University of Health Sciences. Pakistan Journal of Medical Sciences. 30(4), 688-691 (2014)

5. Babo, R., Azevedo, A., Suhonen, J.: Students' Perceptions about Assessment Using an e-Learning Platform. In D.G. Sampson, R. Huang, G.-J. Hwang, T.-C. Liu, N.-S. Chen, Kinshuk, C.-C. Tsai (eds.) Proceedings of the IEEE 15th International Conference on Advanced Learning Technologies, Hualien, Taiwan, 244-246 (2015)

6. Sorensen, E.: Implementation and student perceptions of e-assessment in a Chemical Engineering module. European Journal of Engineering Education. 38(2), 172-185 (2013)

7. Hodgson, P., Pang, M.Y.C.: Effective Formative E-Assessment of Student Learning: A Study on a Statistics Course. Assessment & Evaluation in Higher Education. 37(2), 215-225 (2012)

8. Özden, M.Y., Ertürk, I., Sanli, R.: Students' Perceptions of Online Assessment: A Case Study. Journal of Distance Education. 19(2), 77-92 (2004)
9. Karrer, K., Glaser, C., Clemens, C., Bruder, C.: Technikaffinität erfassen – der Fragebogen TA-EG. Der Mensch im Mittelpunkt technischer Systeme. 8. Berliner Werkstatt Mensch-Maschine-Systeme. 196-201 (2009)

10. Hintze, J.L., Nelson, R.D.: Violin plots: A box plot-density trace synergism. The American Statistician. 52(2), 181-184 (1998)

11. Upton, G.J.G.: Fisher's Exact Test. Journal of the Royal Statistical Society. 155(3), 395-402 (1992)

12. Küppers, B., Politze, M., Schroeder, U.: Reliable e-Assessment with git - Practical Considerations and Implementation. EUNIS 2017 Book of Proceedings, Münster, Germany. 253-262 (2017)

13. Küppers, B., Kerber, F., Meyer, U., Schroeder, U.: Beyond Lockdown: Towards Reliable e-Assessment. GI-Edition - Lecture Notes in Informatics. P273, 191-196 (2017)

14. Küppers, B., Politze, M., Zameitat, R., Kerber, F., Schroeder, U.: Practical Security for Electronic Examinations on Students' Devices. In Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Computing. SAI 2018. Advances in Intelligent Systems and Computing, 857. Springer, Cham, 290-306 (2019)

15. Prensky, M.: Digital Natives, Digital Immigrants. On the Horizon. 9(5), 1-6 (2001)

16. White, S.: The Generation Z effect. https://www.theglobeandmail.com/news/national/education/canadian-university-report/the-genz-effect/article26898388/ (2018) [accessed 2019-01-15]

17. Kadijevich, D.: Gender Differences in Computer Attitude among Ninth-Grade Students. Journal of Educational Computing Research. 22(2), 145-154 (2000)

18. Kahveci, M.: Students' perceptions to use technology for learning: Measurement integrity of the modified Fennema-Sherman attitudes scales. Turkish Online Journal of Educational Technology. 9(1), 185-201 (2010)

19. Yau, H.K., Cheng, A.L.F.: Gender Difference of Confidence in Using Technology for Learning. Journal of Technology Studies. 38(2), 74-79 (2012)