Source document: https://docs-lawep.s3.us-east-2.amazonaws.com/1697740422888.pdf (Georgia)
Problems of Distance Assessment of Students during the COVID-19 Pandemic at BSMA Abdul Kakhidze, Avtandil Gegenava, Eldar Mskhaladze, Joni Babilodze, Teimuraz Chokharadze Batumi State Maritime Academy, 53 Rustaveli Avenue, Batumi, 6004, Georgia E-mail: a.kakhidze@bsma.edu.ge Abstract.

Due to the restrictions caused by the covid-19 pandemic situation, higher education institutions of Georgia, including Batumi State Maritime Academy, had to move to distance learning within a limited time frame.
Remotely was carried out not only teaching (lectures and practical works) but assessment processes as well.

The paper discusses the disadvantages of distance assessment based on a one-year survey (done by the authors of the given article), analyses the students’ mid-term and final assessment results before and during the pandemic period and suggests possible ways to eliminate them.

Keywords: distance education; distance assessment; preventing cheating.
More than one year has passed since the World Health Organization declared the spreading of Covid 19 as a world pandemic situation (March 11, 2020).

Almost every country of the world had to develop and activate restrictions against the fast spreading of the disease.

Limitations were carried out in the congregate area as it represents the highest risk of the rapid virus spreading after direct contact with the already diseased person.

Free movement was also restricted.


Protection people from the pandemic, most countries have tightened and limited the access of foreign citizens to their territories.

These restrictions have been carried out even among the EU countries where the citizens of EU member states are allowed to move freely within the EU.
The restrictions caused by the Covid-19 pandemic have created a new, previously unfamiliar reality in the world, which has harmed the economies of citizens and countries and, of course, the education system.

Due to the epidemic situation, the business had to resort and use such tools that minimize the congestion of people in one physical area and provide distance communication while performing professional activities.
Existed reality has given a huge opportunity to develop electronic, internet platforms for business management and communication.

17 Restrictions imposed due to the alarming spread of the Covid-19 epidemic have also affected educational processes.

The education system has faced unusual challenges in every country.

In particular, the implementation of the educational program, based on the student's industrial training, was at risk.
Maritime education is among them where complete and appropriate training of students includes on-board cadet and various simulation training.

They are conducted (carried out) in specialized simulation laboratories installed in the maritime academy.
Even the universities having well-developed e-learning resources still consider the traditional, so-called face-to-face learning a more effective form of teaching when a teacher and a student communicate in a classroom, while e-learning is considered as a secondary, auxiliary form.

[1].
[1].

Professors of Illinois University Bill Cope and Mary Kalantzis critically discuss the weaknesses and negative features of traditional forms of teaching in their scientific paper “COVID- 19: Why higher ed may never be the same”.

They believe that even without the restrictions caused by the COVID-19, forms of traditional teaching would face structural changes, and one of the reasons for this is the increase of the cost [1].
On March 21, 2020, an emergency condition was declared in Georgia to stop the spreading of the COVID-19 pandemic.

Based on this fact, the implementation of the traditional face-to-face educational process in classrooms, laboratories, simulation classes, etc., was restricted at schools and high educational institutions.

Remarkably, BSMA managed to transfer the educational process (lectures and practical training) from face-to-face to distance learning platforms in very little time.
The methodological side of effective distance learning has been left out of proper attention due to the existence of other immediately solved issues.

However, it should also be noted that the Maritime Academy has prepared a distance learning platform and trained its academic staff in a very short period, and the academic staff has also been able to digitize the necessary training resources for lectures and practical works very quickly.
The distance learning platform was used to deliver learning materials smoothly to the students.

In the process of online teaching, the lecture format has not been changed in general (in fact), while conducting the practice works with students were significantly limited, as there was no opportunity for students to be actively involved in training courses, where practical tasks are solved in writing forms (at the whiteboard) by demonstrating to the group.
The problem was mainly caused by 2 reasons: the student was using a smartphone internet to attend the course (most students do not have a proper computer device) or the student was unable to write quickly on the virtual board of the ZOOM platform (BSMA uses the ZOOM platform for distance learning).

18 The most visible indicator of the effectiveness of any teaching, including the distance learning process, is the assessment of students’ knowledge and skills.
All courses are designed only with appropriate forms of evaluation, given in the course syllabus.

The choice of assessment forms depends on the course objectives and the content of the learning outcomes set in the course.

When the assessment forms of student knowledge and skills are chosen correctly, then it is easy for the professor to identify student learning strengths and weaknesses and improve the quality of teaching based on them.
The accelerated transfer of the learning process from face-to-face to the distance was carried out without any methodological training at BSMA, i.e., lectures and practical works/training conducted by the teachers took place almost the same way as in the classroom, face-to-face teaching process.
Many educational researchers believe that such kind of copping of face-to-face learning modalities is less effective in distance learning: „Often reluctantly and in a piecemeal fashion, many colleges have tried to migrate their traditional practices online.

They have made awkward attempts to replicate the traditional classroom with video lectures, e-textbooks, online tests, and learning management systems that look like old-fashioned syllabuses.
The result is often a step back into all that was wrong with didactic modes of teaching“[1].

A similar point of view can be expressed about the classroom and distance assessment methods used for evaluating a student's knowledge and skills.
Assessment in the classroom is carried out under the overall control of the teacher, while in the case of distance assessment the supervision of the professor to a similar degree is associated with fulfillment of several prerequisites, the first of all, all students should have appropriate computer equipment and good quality Internet, which is practically impossible to a significant number of students due to their economic conditions.
Surveys showed that most of the students attending the classes online used smartphones and mobile internet, which means that their possibilities for full-fledged participation in the learning process were very limited.

To avoid unwanted disorientation of students in the extreme conditions caused by pandemic restrictions and to minimize their discomfort, the structure of pre-pandemic final assessment has remained for most of the learning courses.
In particular, the student’s final assessment is formed by summing up the Midterm Assessment and Finale Examination.

A midterm assessment is formed by summing up the rating assessment, performed by the teacher, and computerized adaptive testing - CAT conducted by the assessment and examination 19 center.

A midterm assessment is formed by the student based on the course assimilation during the semester, which is carried out by the course teacher.
At the end of the semester, after the completion of the educational process, the final assessment is obtained by the student only after getting the written examination (Final Examination) score.

Exam questions are selected based on the learning outcomes of the study course.

A limited amount of time is allocated for the completion of the examination task and the examination is organized by the assessment and examination center of Maritime Academy.
Because of the increasing interest and importance of distance learning, the pros and cons of distance assessment of a student's knowledge are often discussed in the scientific literature, and the most relevant issue is the degree of reliability of distance assessment.

Neil C. Rowe from Naval postgraduate School remarks that: "When a student scores well for an online assessment, does that mean that they know the material?
This question is becoming increasingly important as online distance-learning programs become popular.

While traditional paper-and-pencil assessments of students can be done in distance-learning programs, it is appealing to think that technology can both teach material and assess learning.
The traditional assessment also requires costs: the time of human proctors, care in the control of the assessment materials before and after administration, and grading effort, all of which are simplified in online assessment.

But can we trust the results?” And there he answers that “Unfortunately, often we cannot” [2].

A similar opinion is expressed by other researchers, for example: Ford believes that everyone lies at one time or another [3].
Some researchers point out that academic cheating is a common occurrence in education [4; 5; 6].

Bushweller presents very alarming statistics in his paper, according to which 70% of American high school seniors admit to cheating on at least one test, and 95% of the students who said they cheated were never caught [7].

Many other studies can be used to prove that academic cheating is a common phenomenon in higher education.
Some researchers believe that the cases of cheating are even higher when the communication between the examiner and the student is weakened due to poor internet or poor computer equipment.

Unfortunately, such cases are often accessed by unscrupulous students „Cheating also has been observed to increase as the bandwidth (information per second) of the communications channel between assessor and assessee decreases; that is, people who feel more "distant" cheat more“ [8, 9].
Cases of cheating in the examination process existed even before the advent of distance learning but were not so many as in distance assessment, and the main reason for this is the weakness 20 of the examiner's control over the assessment process.

The distance assessment in the examination process is much weaker because of the examiner's improper control than face-to-face assessment in the classroom.
Due to the inability of the examiner to establish total control, the student may use various prohibited acts for passing the exam:   in advance prepared, readymade notes for correct answers; readymade answers received via internet resources;  help of a groupmate student either online or through direct communication;  help of the third person in the same way as given in the previous paragraph;  help of another person to take an exam instead of him/her (in case if there is not any type of face identification and/or IP control), etc.
Beside the above-mentioned fact, a student also tries to present so-called "unexpected factors" to avoid the already scheduled exam in order not to get an undesirable mark (grade) and asks to retake (repass) it later.

These factors are:  Bad internet;  Emergency power cutoff;  Bad or poor-quality internet connection device (smartphone, poor quality computer, no copier, no video camera, or audio device, etc.

);  Poor social or living conditions;  Illness or incapacity; etc.
Diagrams of 7 different learning courses in the Maritime Education Programme are given below, showing and comparing the assessment process before the pandemic (blue diagrams) and during the pandemic (orange diagrams) period of studies.

The data of a large group of students (on average 200 students per course) are used for comparison.

The number of students is presented in percentage according to the received points (marks) and categories.

21 Mathematics-1 2019-2020 I 2020-2021 I .

2 9 3 .

9 6 3 .
2 9 3 .

9 6 3 .

2 2 5 .

4 0 5 5 6 .

3 5 5 .

4 4 2 5 9 .

.

5 5 2 .

4 0 2 .

2 1 3 1 2 5 2 .

3 7 1 .

3 3 2 1 2 .

9 7 1 .

3 0 1 6 7 .

3 8 .

.

3 0 1 7 6 .

.

7 2 1 6 8 .

0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - Diagram 1.
The number of students not admitted to the final exam increased by 4%; The number of students who failed the final exam decreased by 15%; The number of students who failed the course decreased by 10.2%; Midterm assessment: 45 points - writing exam; 15 points - CAT.

Final exam: 40 points - Math.

tasks.

Mathematics-2 2018-2019 II 2019-2020 II .

9 4 6 .

5 6 5 .

4 5 3 .

3 7 2 1 8 .

8 7 .

.

5 9 6 .

4 9 5 .

8 7 2 .

6 1 2 .

3 9 1 6 1 .

9 2 2 .

1 1 2 .

7 7 1 .

9 3 1 .

4 1 2 .

6 4 1 .

9 2 2 .


4 3 1 .

6 2 1 .

5 0 1 6 9 .

1 6 .

0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - Diagram 2.
The number of students not admitted to the final exam has practically not changed; The number of students who failed the final exam decreased by 6.8%; The number of students who failed the course decreased by 6.2%; Midterm assessment: 45 points - writing exam; 15 points - CAT.

Final exam: 40 points - Math.

tasks.

22 Physics-1 2019-2020 I 2020-2021 I .

4 0 7 .

5 2 5 .

9 8 4 .

7 2 3 .

6 3 4 .

6 3 4 .

4 1 4 .

6 7 3 .

3 6 3 .

8 8 1 5 1 .

6 8 1 .

2 1 1 .

1 1 1 .

2 2 2 .

9 7 1 .

2 2 2 .


9 6 1 .

4 5 1 3 6 .

6 6 .

6 4 3 .

5 1 .

0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - Diagram 3.
The number of students not admitted to the final exam decreased by 3.8%; The number of students who failed the final exam decreased by 17.7%; The number of students who failed the course decreased by 16.2%; Midterm assessment: 30 points - writing exam; 15-point - laboratory work; 15 points - CAT; Final exam: 40 points - Physics tasks.

.

6 1 6 .

4 1 4 .

6 7 4 .

4 9 2 .

9 5 2 3 2 1 1 9 Physics-2 2018-2019 II 2019-2020 II .

1 4 6 .

1 2 6 .

6 4 3 .

8 8 2 .

9 4 1 0 1 .

2 6 2 .

8 5 1 .

7 3 2 .


8 8 1 .

8 5 1 2 1 9 7 3 .

8 6 .

7 4 .

0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - Diagram 4.
The number of students not admitted to the final exam increased by 2%; The number of students who failed the final exam increased by 2.9%; The number of students who failed the course increased by 5.8%; Midterm assessment: 30 points - writing and oral exam; 15-point - laboratory work; 15 points - CAT; Final exam: 40 points - Physics tasks.

23 Astronavigation-1 2019-2020 I 2020-2021 I .

5 2 6 .

5 2 5 .

8 7 5 .

6 6 5 .

7 3 3 .

8 5 2 .

8 3 1 .

7 1 1 .

4 6 1 3 2 .

.

8 9 3 7 2 .

2 6 2 .

2 7 1 .


2 3 2 .

3 3 1 .

9 9 1 .

2 0 2 .

8 5 1 .

9 2 1 .

9 0 1 1 8 .

.

8 6 1 .

5 5 1 0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - Diagram 5.
The number of students not admitted to the final exam increased by 2.1%; The number of students who failed the final exam decreased by 14.1%; The number of students who failed the course decreased by 9%; Midterm assessment: 45 points - writing exam; 15 points - CAT; Final exam: 17 points - theoretical question; 23 points - Astronavigation tasks.

Astronavigation-2 2018-2019 II 2019-2020 II .

4 3 6 .

2 3 5 .

7 8 5 .

2 3 5 .

4 8 3 1 3 4 8 .

6 5 .

.

7 6 1 5 6 .

.

8 4 3 .

1 0 3 .

1 4 2 .

7 2 1 .


2 0 2 .

2 0 2 .

2 0 2 5 1 .

3 8 1 .

8 2 1 .

6 1 2 4 7 .

.

3 5 1 .

2 2 1 0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - Diagram 6.
The number of students not admitted to the final exam decreased by 2.8%; The number of students who failed the final exam decreased by 10.2%; The number of students who failed the course decreased by 11.4%; Midterm assessment: 45 points - writing exam; 15 points - CAT; Final exam: 40 points - Astronavigation tasks.

24 Maritime Radiocomunication and Radionavigation Systems 2018-2019 II 2019-2020 II .

5 5 9 .

3 1 7 .

1 8 6 .

4 8 6 .

5 5 2 .

1 6 2 9 5 .

2 3 .

.

9 2 1 2 2 .

2 2 .

.

7 8 1 .


8 4 1 8 9 1 3 1 .

1 3 .

9 0 1 .

1 8 1 .

5 8 1 0 - 2 0 2 1 - 4 0 4 1 - 6 0 0 - 1 5 1 6 - 3 0 3 1 - .

4 4 2 5 6 .

B 7 3 .

6 0 1 A Diagram 7.


The number of students not admitted to the final exam increased by 2.7%; The number of students who failed the final exam decreased by 10.7%; The number of students who failed the course decreased by 6.8%; The number of students with 31-40 points has increased.

Midterm assessment: 45 points - writing exam; 15 points - CAT; Final exam: 40 points - theoretical questions.
Conclusions: The presented statistics brightly show the changes of assessment distribution in the different students’ groups before the pandemic, before starting a distance learning, and during the pandemic period, when the teaching and assessment process was conducted (carried out) remotely:  Decreased the number of students who were not able to take a minimum competency for admission to the final exam (except Math-1, Physics-2, and Astronavigation-1);  Decreased the number of students who failed the final exam (except Physics-2);  Decreased the number of students who failed the learning course (except Physics-2) The improved changes in the distribution of students assessment can be explained:  During the distance learning, the students assimilated the learning material better and reached proper learning outcomes, which does not seem real;  In the process of distance assessment, the students used unpermitted resources and achieved the desired points (grades) by using various illegal actions;  The difference in assessment (marks) is high in the exams, where students’ only theoretical knowledge was evaluated, and relatively low in exams where the student was required to solve different types of quantitative tasks; 25 As mentioned above, the improved changes in the assessment results of students are likely to be caused by uncontrolled usage of prohibited resources during distance learning by students, so to reduce and/or avoid inconsistencies during the assessment process, we consider to:  Pay more attention to students explaining to them the unacceptability and harm of cheating in examination papers;  Develop and implement effective mechanisms for cheating detection;  Use such kind of distance assessment forms, where the effect of cheating decreases to the minimum;  Use such kind of distance assessment forms that minimize the effect of cheating;  Develop the combined, interrelated methods of distance learning and assessment;  If possible, it is recommended to conduct exams not remotely but face to face, under the supervision of an examiner at university.
References [1] Cope B., Kalantzis M. “COVID-19: Why higher ed may never be the same”, April 2, 2020. https://universitybusiness.com/covid-19-why-higher-ed-may-never-be-the-same/; [2] Rowe, Neil C. “Cheating in Online Student Assessment: Beyond Plagiarism”.

On-Line Journal of Distance Learning Administration, Summer 2004. http://hdl.handle.net/10945/36015; [3] Ford, C. (1996).

Lies!

Lies!!

Lies!!!

The psychology of deceit.

Washington, DC: The American Psychiatric Press; [4] Cizek, G. J.

(1999).
(1999).

Cheating on tests: how to do it, detect it, and prevent it.

Mahwah, NJ: Lawrence Erlbaum; [5] Dick, M., Sheard, J., Bareiss, C., Carter, J., Joyce, D., Harding, T., & Laxer, C. (2003, June).

Addressing student cheating: definitions and solutions.

ACM SIGCSE Bulletin, 35(2), 172- 184; [6] Lathrop, A., & Foss, K. (2000).

Student cheating and plagiarism in the Internet era: a wake- up call.

Englewood, CO: Libraries Unlimited; [7] Bushweller, K. (1999, April).

Generation of cheaters.


The American School Board Journal, 186(4), pp.

24-32; 26 [8] Moore, M., Lockee, B.

& Burton, J.

(2002).

Measuring Success: Evaluation Strategies for Distance Education.

Educause Quarterly, 25-1, 20-26.

Retrieved November 17, 2013 from http://www.editlib.org/p/92835; [9] Burgoon, J., Stoner, M., Bonito, J., & Dunbar, N. (2003, January).

Trust and deception in mediated communication.

36th Hawaii Intl.

Conf.

on Systems Sciences, 44a.

27