## 1 Introduction  
As a result of the COVID-19 pandemic that emerged at the end of 2019, E-learning or web-based distance learning platforms have become a viable alternative to facilitate the learning process [1]. Within this learning process, knowledge assessment plays a pivotal role in ensuring effective teaching [2]. Open-ended questions have been identified as a valuable method for determining students' level of knowledge and encouraging them to express their thoughts, perspectives, and experiences in their own words [1]. By inviting open-ended responses, teachers gain a more accurate and comprehensive insight into how students grasp domain-specific knowledge [3]. However, manually scoring these responses can introduce inconsistencies, as scoring may vary among markers or from one student to another [2]. Additionally, expecting a single definitive response to an open-ended question proves challenging for teachers, due to variations in students' vocabulary and writing structures [4]. This can lead to subjective judgements about the answers and compromise the objectivity of the assessment process [5]. According to [PERSON] et al. in Ref. [6], short answers have the following characteristics: the answer should not inferred just from the question's words (requiring external knowledge); the answer should be given in natural language; the length of the answer typically spans from one phrase to one paragraph; the content of the answer is relevant to the subject domain; and the answer is structured as closed-ended yet is not rigidly defined. However, some short-answer questions require students to express their subjective viewpoints within a defined context. Hence, short-answer questions also refer to semi-open-ended questions [7]. The grading system for short answers poses inherent challenges compared to automated multiple-choice grading systems. It is essential to thoroughly examine the nuances and variations in these answers to ensure accurate assessment [8]. The advancements in natural language processing (NLP) and machine learning applications have spurred interest among educators in creating exams comprising open-ended questions that can be automatically evaluated for a large number of students [5]. Automatic short answer grading (ASAG) is an emerging field of research, reflecting the educational sector's increasing adoption of technology to aid students and professionals. ASAG systems hold potential as valuable resources for educators, facilitating the enhanced integration of open-ended questions and providing more objective assessments for both formative and summative evaluations [9]. ASAG functions by analyzing students' answers in relation to a given question and the desired answer, as illustrated in Figure 1.  
The recent advancements in natural language processing (NLP) and deep learning have introduced promising methodologies and frameworks capable of addressing various tasks. Across numerous NLP tasks, including ASAG, language models (LMs) have demonstrated considerable success. In modern approaches, LMs are trained using neural networks. Initial neural models were based on recurrent neural networks (RNNs), like long short-term memory networks (LSTMs and BiLSTM) [3]. The development of large language models like BERT (Bidirectional Encoder Representations from Transformers), based on the transformer architecture, and the increasing adoption of transfer learning have been instrumental in constructing custom ASAG systems [11]. Figure 1: ASAG pipeline [10]. Transformer employs self-attention for natural language processing, enabling the parallel computation of input and output vectors and addressing the sequential processing limitations of recurrent neural network (RNN), convolutional neural network (CNN), and long short-term memory (LSTM) approaches [12]. However, this self-attention mechanism can be computationally expensive. In the literature, ASAG studies often measure success by high correlations and a minimal loss value on standard benchmark tests using widely accessible datasets [5].