# Design and Implementation of a Machine Learning-Based English Intelligent Test System

Authors: Jing Guo
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Examination is a particularly important part of the students’ whole learning process. Therefore, this paper designs and develops an online examination system. This paper studies an examination recommendation algorithm based on a machine learning neural network model. The algorithm can recommend problems for students according to students’ error history, error prone, and knowledge poor problems. Experiments also show that the recommendation algorithm studied in this paper improves the recommendation effect compared with several other similar algorithms.


---

## Body

## 1. Introduction

The development of the Internet and the development of Internet plus strategy have led to the reform in the educational field, breaking through the traditional education mode in time and space and reducing the consumption of [1] of manpower and material resources. For example, examination is a particularly important part of students’ learning process, but the traditional paper-based examination involves the process of manually writing questions, printing test papers, students’ examination, manual scoring, and result statistics. The whole process may be very labor-intensive and cost-effective [2]. At present, network education is booming, and there are a large number of online examination systems. The emergence of a large number of online examination systems has well solved the disadvantages of traditional paper examination [3]. However, in the era of big data, the online education industry is also facing information overload, and a large number of resources are not fully utilized [4]. The problem is that information is overloaded and a large number of resources are not fully utilized. Some online platforms on the market and many universities provide practice and examination functions but provide the same access rights for all students. However, they provide the same problems to all students, ignoring their personal learning needs [5]. The existing examination system in the market is mainly profit oriented, paying more attention to the aesthetic design of the interface and the expansion of functions. In contrast, many university examination systems are designed to evaluate students’ recent learning, so they pay more attention to the core online examination function [6]. On the other hand, many university examination systems are designed to evaluate students’ recent learning, so they focus more on the relatively simple core online examination function. For these systems, with the accumulation of question bank, students can practice at random. The system is not designed to measure students’ recent learning, so it focuses more on the core online examination function, which is relatively simple [7]. Students are unable to find their own problems in the huge problem bank, and even a large number of problems have not been selected, which leads to the students’ half-hearted efforts [8].The development of information technology and the popularity of the Internet have brought about a massive amount of information, and we have entered the era of big data [9]. While the popularity and growth of information has brought convenience to users, it has also brought the problem of information overload [10]. Faced with such a huge amount of data, both users and information service providers will encounter great challenges. It is difficult for users to retrieve information that is useful or interesting to them [11]; it is also not easy for information service providers to make their information stand out [12].Recommendation algorithms are the core component of recommendation systems. Traditional recommendation algorithms include content-based recommendation algorithms, collaborative filtering algorithms [13], and hybrid recommendations, among which collaborative filtering is the most widely used. With the development of deep learning and the increase of dataset size, traditional recommendation algorithms can no longer well solve the problems of data sparsity, cold start, and other problems, in addition to the accuracy of the learned model cannot meet the requirements. Therefore, the current academic and industrial research recommendation system algorithms are mostly based on deep learning [14].Nowadays, personalized education is an important research topic. Based on consideration of the above issues, this paper’s recommendation algorithm is applied to the developed online examination system. The main focus is on the test practice module of the online examination system. The paper also focuses on the test practice module of the online examination system and researches a personalized recommendation algorithm that can recommend different questions for different students according to their history of wrong questions. The system can recommend different quality questions for different students based on their history of wrong questions, making the practice less blind and more focused, thus improving learning efficiency [15]. This helps both students and teachers to a certain extent. This will help both students and teachers to a certain extent, and will also contribute to the development of personalized education.


## 2. Related Work

Since 2006, when deep learning was proposed by Hinton et al., it has gradually been widely noticed and applied by the scientific community, and it has made outstanding achievements in image processing, natural language processing, etc. Especially in recent years, deep learning has been researched and applied quite extensively [12]. An earlier application of deep learning to recommendation algorithms used restricted Boltzmann bodies to build models, but the scale of connections between neural network layers was too large. Most of the current research in recommendation algorithms is built on deep learning. A recommendation algorithm based on multilayer neural networks was proposed by [13] and applied to YouTube video recommendation. [14] proposed a memory-based collaborative filtering algorithm for neural networks that exploits the interaction relationship to user items and combines neighboring users in a nonlinear way through the memory layer to make the representation of users more accurate with some improvement in effectiveness. However, only first-order connectivity is considered in the combination of near-neighboring users, without tapping into deeper user feature information. In order to solve the sparsity problem faced by traditional collaborative filtering, a hierarchical Bayesian model based on deep learning is proposed, adding certain auxiliary information to represent the content information in a neural network, taking into account complex nonlinear relationships and enhancing the vector representation performance. However, the form of inner product still used in the prediction layer is not sufficient to reveal the complex nonlinear relationships between user and item interactions [16]. In response to Wang’s research, [15] proposed an NCF model using a nonlinear neural network structure instead of inner product operations to enhance the interaction to capture the nonlinear feature interactions between users and items. [17] is an industrial application that employs multiple convolutional layers on the item graph for Pinterest image recommendation and thus captures relationships at the level of items rather than collective user behavior. [18] proposes a spectral convolution operation to discover all possible connectivities between users and items in the spectral domain, which allows the discovery of ri connections between user-item pairs through the feature decomposition of the graph adjacency matrix, but is complex, very time-consuming, and does not support large-scale recommendation schemes [19, 20].


## 3. Neural Graph Collaborative Filtering Algorithm

The neural graph collaborative filtering (NGCF) algorithm is an algorithm proposed in a paper published in 2019. Based on implicit feedback data, the core idea of the algorithm is to apply deep learning techniques to recommender system algorithms and exploit the higher-order connectivity of the user-item interaction graph to extract deeper features of users and items, which are used to enhance the embedding feature representation of users and items. This improves the performance of the algorithm by providing a more accurate representation of users and items than traditional collaborative filtering algorithms.


### 3.1. The Concept of Higher-Order Connectivity

In order to extract synergistic signals, this can be achieved using a higher-order connectivity graph structure for user-item interaction. The higher-order connectivity diagram is shown in Figure1.Figure 1
Higher-order connectivity ofu1.Figure1 shows the u1 expanded tree structure. Higher-order connectivity refers to paths that reach u1 from any node and are greater than or equal to 1. The connectivity of this structure contains rich synergistic signals. For example, path u1⟵i2⟵u2 contains behavioral similarities between users u1 and u2, i2 as both interact with each other, while path u1⟵i2⟵u2⟵i4 suggests that u1 has the potential to interact with i4.It can be seen that the higher-order connectivity structure contains a lot of valuable information. Therefore, it is of great importance to utilize it to integrate into the model representation.


### 3.2. Neural Graph Collaborative Filtering Model

The neurograph collaborative filtering model designs a neural network approach to propagation for recursion, thus enabling the integration of higher-order connectivity information into the embedding. Specifically, this is achieved by designing an embedding propagation layer that is primarily responsible for fusing interaction information into the user and item embedding, and then by adding multiple such propagation layers, the higher-order connected collaborative signals can be captured. In Figure1, the behavioral similarities between layers can be captured by adding two additional layers. By adding three layers, it is possible to capture the potential interest in of interest.The neural graph collaborative filtering model has three main components: an embedding layer, multiple embedding propagation layers, and a prediction layer. The overall structure of the model is shown in Figure2.
(1)
Embedding layer: the main purpose is to map the sparse feature vectors of users and items into a dense type and generate the initial user and item embedding. a matrix can be constructed as an embedding lookup table, as in(1)E=eu1,⋯,euNei1,⋯,eiM,Figure 2
Structure of the neural graph collaborative filtering model.whereeu∈Rdei∈Rd denotes the initial embedding of users and items and d denotes the dimensionality. In the above set, the first N items represent users and the last M items represent items
(2)
Embedding propagation layer: each propagation layer consists of two phases: message construction and message synthesis(1)
Message construction: for a useru, and an item i with which an interaction has occurred, the composition is in terms of a user-item pair u,i, and the message from i to u is defined as(2)mu⟵i=1NuNiW1ei+W2ei.eu,where1/NuNi is the Laplace parametrization, which represents the decay factor of the propagation process; Nu and Ni represent the number of items that user u has interacted with and the number of users that have interacted with item i, respectively; W1 and W2 represent the two weight matrices, which are the parameters that the model needs to learn; ei and eu represent the initial imbedding; and and • represents the element-by-element product of vector operator
(2)
Message synthesis: in order to obtain the embedding of useru optimized after propagation, it is necessary to synthesize the messages from all items to u that have interacted with user u. The synthesis function is defined in(3)eu1=LeakyRelumu⟵u+∑i∈Numu⟵i,whereeu1 represents the embedding of user u after one layer of propagation, LeakyRelu is the activation function, and mu⟵u indicates that the self-connected signal retains the features mu⟵u=W1e before propagation.Similar to obtaining the embedding of useru above, the embedding representation of item i can also be obtained after a layer of propagation ei1, also in the above two stages.


## 3.1. The Concept of Higher-Order Connectivity

In order to extract synergistic signals, this can be achieved using a higher-order connectivity graph structure for user-item interaction. The higher-order connectivity diagram is shown in Figure1.Figure 1
Higher-order connectivity ofu1.Figure1 shows the u1 expanded tree structure. Higher-order connectivity refers to paths that reach u1 from any node and are greater than or equal to 1. The connectivity of this structure contains rich synergistic signals. For example, path u1⟵i2⟵u2 contains behavioral similarities between users u1 and u2, i2 as both interact with each other, while path u1⟵i2⟵u2⟵i4 suggests that u1 has the potential to interact with i4.It can be seen that the higher-order connectivity structure contains a lot of valuable information. Therefore, it is of great importance to utilize it to integrate into the model representation.


## 3.2. Neural Graph Collaborative Filtering Model

The neurograph collaborative filtering model designs a neural network approach to propagation for recursion, thus enabling the integration of higher-order connectivity information into the embedding. Specifically, this is achieved by designing an embedding propagation layer that is primarily responsible for fusing interaction information into the user and item embedding, and then by adding multiple such propagation layers, the higher-order connected collaborative signals can be captured. In Figure1, the behavioral similarities between layers can be captured by adding two additional layers. By adding three layers, it is possible to capture the potential interest in of interest.The neural graph collaborative filtering model has three main components: an embedding layer, multiple embedding propagation layers, and a prediction layer. The overall structure of the model is shown in Figure2.
(1)
Embedding layer: the main purpose is to map the sparse feature vectors of users and items into a dense type and generate the initial user and item embedding. a matrix can be constructed as an embedding lookup table, as in(1)E=eu1,⋯,euNei1,⋯,eiM,Figure 2
Structure of the neural graph collaborative filtering model.whereeu∈Rdei∈Rd denotes the initial embedding of users and items and d denotes the dimensionality. In the above set, the first N items represent users and the last M items represent items
(2)
Embedding propagation layer: each propagation layer consists of two phases: message construction and message synthesis(1)
Message construction: for a useru, and an item i with which an interaction has occurred, the composition is in terms of a user-item pair u,i, and the message from i to u is defined as(2)mu⟵i=1NuNiW1ei+W2ei.eu,where1/NuNi is the Laplace parametrization, which represents the decay factor of the propagation process; Nu and Ni represent the number of items that user u has interacted with and the number of users that have interacted with item i, respectively; W1 and W2 represent the two weight matrices, which are the parameters that the model needs to learn; ei and eu represent the initial imbedding; and and • represents the element-by-element product of vector operator
(2)
Message synthesis: in order to obtain the embedding of useru optimized after propagation, it is necessary to synthesize the messages from all items to u that have interacted with user u. The synthesis function is defined in(3)eu1=LeakyRelumu⟵u+∑i∈Numu⟵i,whereeu1 represents the embedding of user u after one layer of propagation, LeakyRelu is the activation function, and mu⟵u indicates that the self-connected signal retains the features mu⟵u=W1e before propagation.Similar to obtaining the embedding of useru above, the embedding representation of item i can also be obtained after a layer of propagation ei1, also in the above two stages.


## 4. Intelligent Paper Test System

This system largely fulfills the design requirements described above (with the exception of support for automated marking machines, which is not included at this time to reduce costs). The subsystems in Figure3 are briefly described below:
(1)
Parameter input moduleFigure 3
Main system.It can accept various control conditions input by the user and put various control conditions in the automatic grouping of papers for reasonable and flexible matching, which is an important guarantee of the system’s scientificity and flexibility. It is worth mentioning that this module can dynamically reflect the important information of the question bank in the system, such as various assessment points, the amount of questions of various assessment points, and generate question papers accordingly.(2)
Automated paper assembling moduleThe core part of the system, its efficiency is closely related to the structure of the question bank. A well-designed automatic paper-forming module should be able to interface well with the parameter input module and complete the function of randomly generating examination papers under strict control.The output module of the system can output test papers directly to a printer or to Word for typesetting and printing. It is particularly worth mentioning that the organic combination of the system and MS Word, borrowing Word’s powerful typesetting functions and spell checking and grammar checking tools, makes it possible for the user to have a complete and beautiful examination paper in front of him.(3)
System output moduleThe system output module can output test papers directly to the printer or to Word for typesetting and printing. It is particularly worth mentioning that the organic combination of the system and MS Word, using Word’s powerful typesetting functions and spell checking and grammar checking tools, makes it possible to present the user with a complete and beautiful examination paper [21, 22].
(4)
System maintenance moduleThe maintenance module of the system can be divided into two submodules, one for the maintenance of test papers and the other for the maintenance of question banks.The main purpose of the maintenance module is to modify test errors and save them back to the original question bank, which is in line with user habits, as users are generally reluctant to maintain the question bank directly, and there is some assurance of the security of the question bank.The question bank maintenance module can add, delete, and change questions, and it has the ability to add new questions and new question types, so it can adapt to the changing standards of the National Level 4 and 6 examinations. Practice has proven that a dynamic and scalable system is a viable system.(5)
Automatic test moduleThis is where the user can conduct an on-machine test of the objective part of the set of test papers (including listening tests). The system here fully simulates the real examination room, e.g., listening can only be listened to once, other test questions can be referred to repeatedly; there is a clock to show the time, etc.(6)
Storage and loading moduleThe main purpose of this module is to allow users to use the system for on-board testing and paper maintenance. The web-based version can remotely load teacher-generated test papers for examinations [23].
(7)
Information statistics moduleHere, the user can access a variety of dynamic information about the test papers: the types of questions they contain, the number of questions in each type, the distribution of assessment priorities, the difficulty factor, etc. The user can even perform statistics such as pass rates and merit rates, in the simple result management system provided as an add-on to this module.The flowchart for randomly selected questions is essentially a random selection of questions under a variety of conditions, see Figure4.Figure 4
Flow chart of randomly selected questions.The focus is on random selection, and the key is on condition control. Two types of question banks have been designed: special and general. The dedicated question bank contains a few typical question types already available in the system, such as reading comprehension. One of the difficulties of the system is that the parameter input module should allow the user to control the criteria for new question types.About the interface between the system and Word in the output module, the system queries the registration information on the user’s machine to detect whether Word is installed and where it is located and then uses OLE Automation to communicate with it [24, 25].In order to take advantage of the system’s on-board testing and paper maintenance capabilities, the system provides a dedicated storage and loading module in addition to text files to hold the papers. This storage can of course be done by storing a single paper in a single file, but this would result in a flood of files and inconvenience to manage, so the system uses a single file to store multiple papers.


## 5. Experiments and Data

### 5.1. Dataset Selection

In order to complete the test recommendation function, the first step is to obtain the data. The data can be obtained from the exam system database wrong question record table; you need to collect the student id, question id, whether the question is wrong (0 or 1), and all students’ wrong question record data. In this paper, the recommendation data interface is reserved for the recommendation exercise module of the examination system, which can obtain data from the database and call the recommendation algorithm. However, as the system prototype implemented in this paper is still in the testing stage, the data of real students’ question records are being collected and a large amount of data cannot be obtained. In order to ensure the accuracy and effectiveness of the recommendation algorithm, two publicly available datasets, Amazon-book and Gowalla, which can be mapped to the database structure of the prototype, were selected for experimentation to verify the effectiveness of the recommendation algorithm.The descriptions of the dataset selection are as follows:(1)
These two datasets are publicly available and have been widely used in the study of recommendation algorithms(2)
In fact, the recommendation algorithm in this paper focuses on the implementation of the recommendation function and the accuracy of the algorithm’s recommendations and is not specific to a particular scenario, as shown in Table1Table 1
Format of data collected.
Dataset
Data collection format
Student error data
Student ID, title JD, whether it is wrong
Gowalla
User ID, location ID, check in
Amazon-book
User ID, book ID, purchase or notThe table above shows the format of the data recorded for real student errors in the system and the format of the records for the two datasets. For each dataset, in order to perform the algorithm experiments, data processing needs to be performed first.The two datasets differ in size, sparsity, etc. and the statistics for both are shown in Table2.Table 2
Statistical information on the dataset.
Dataset
Users
Items
Interactions
Density
Gowalla
27458
40685
1027424
0.000074
Amazon-book
54231
95478
2948712
0.00059
(1) Gowalla dataset: this dataset is a check-in dataset from the Gowalla website, which records the check-in time and location information of users on the social networking site, including user ID, check-in time, latitude, longitude, and location ID. Users can share their location through check-in.


### 5.2. Comparative Analysis of Results

The model is improved based on NGCF as shown in Figure5 for the examination evaluation of different students. The user and item embedding is first initialized and an attention mechanism is introduced in the propagation layer. Then, a combination of variable weighted averaging is applied to the interaction information, and after multiple layers of propagation, the embedding is obtained after optimization of each layer. Finally, these embeddings, together with the initial embedding, are connected through a cascade layer.Figure 5
Assessment of different student examinations.In addition, to validate the effectiveness of the algorithmic models, the model NGCF-Att from this paper was compared with the three algorithms BPRMF, NCF, and NGCF. For a fair comparison, all algorithms were optimized using the BPR loss function.The recall@20 and ndcg@20 evaluation metrics for the four algorithms on the Gowalla and Amazon-book datasets are shown in Table3.Table 3
Comparison of the results of the four algorithms.
Dataset
Algorithm name
Recall@20
ndcg@20
Gowalla
BPRMF
0.1234
0.1475
NCF
0.1324
0.1992
NGCF
0.1427
0.2147
NGCF-Att
0.1424
0.2213
Amazon-book
BPRMF
0.0234
0.0257
NCF
0.0541
0.5331
NGCF
0.0321
0.0547
NGCF-Att
0.0257
0.0654The comparison of the results in the above table shows that the other three models based on deep learning are slightly better than the traditional matrix decomposition MF model, which reflects the optimization of the traditional MF to varying degrees.Secondly, the index of NGCF-Att will be slightly higher than that of NCF and NGCF, because it combines the advantages of the two models and introduces an attention mechanism to improve network efficiency.Therefore, we can draw the following conclusions:(1)
The research of recommendation algorithm based on deep learning is an important way to improve the performance of recommendation system(2)
Inner product has some limitations. Using neural network instead of inner product can capture the complex nonlinear interaction between users and items(3)
The attention mechanism assigns different weights to different items of each user, which is more in line with the realistic level of users’ preference for different items. Variable weight learning improves the performance of the model and makes it easier to explain


## 5.1. Dataset Selection

In order to complete the test recommendation function, the first step is to obtain the data. The data can be obtained from the exam system database wrong question record table; you need to collect the student id, question id, whether the question is wrong (0 or 1), and all students’ wrong question record data. In this paper, the recommendation data interface is reserved for the recommendation exercise module of the examination system, which can obtain data from the database and call the recommendation algorithm. However, as the system prototype implemented in this paper is still in the testing stage, the data of real students’ question records are being collected and a large amount of data cannot be obtained. In order to ensure the accuracy and effectiveness of the recommendation algorithm, two publicly available datasets, Amazon-book and Gowalla, which can be mapped to the database structure of the prototype, were selected for experimentation to verify the effectiveness of the recommendation algorithm.The descriptions of the dataset selection are as follows:(1)
These two datasets are publicly available and have been widely used in the study of recommendation algorithms(2)
In fact, the recommendation algorithm in this paper focuses on the implementation of the recommendation function and the accuracy of the algorithm’s recommendations and is not specific to a particular scenario, as shown in Table1Table 1
Format of data collected.
Dataset
Data collection format
Student error data
Student ID, title JD, whether it is wrong
Gowalla
User ID, location ID, check in
Amazon-book
User ID, book ID, purchase or notThe table above shows the format of the data recorded for real student errors in the system and the format of the records for the two datasets. For each dataset, in order to perform the algorithm experiments, data processing needs to be performed first.The two datasets differ in size, sparsity, etc. and the statistics for both are shown in Table2.Table 2
Statistical information on the dataset.
Dataset
Users
Items
Interactions
Density
Gowalla
27458
40685
1027424
0.000074
Amazon-book
54231
95478
2948712
0.00059
(1) Gowalla dataset: this dataset is a check-in dataset from the Gowalla website, which records the check-in time and location information of users on the social networking site, including user ID, check-in time, latitude, longitude, and location ID. Users can share their location through check-in.


## 5.2. Comparative Analysis of Results

The model is improved based on NGCF as shown in Figure5 for the examination evaluation of different students. The user and item embedding is first initialized and an attention mechanism is introduced in the propagation layer. Then, a combination of variable weighted averaging is applied to the interaction information, and after multiple layers of propagation, the embedding is obtained after optimization of each layer. Finally, these embeddings, together with the initial embedding, are connected through a cascade layer.Figure 5
Assessment of different student examinations.In addition, to validate the effectiveness of the algorithmic models, the model NGCF-Att from this paper was compared with the three algorithms BPRMF, NCF, and NGCF. For a fair comparison, all algorithms were optimized using the BPR loss function.The recall@20 and ndcg@20 evaluation metrics for the four algorithms on the Gowalla and Amazon-book datasets are shown in Table3.Table 3
Comparison of the results of the four algorithms.
Dataset
Algorithm name
Recall@20
ndcg@20
Gowalla
BPRMF
0.1234
0.1475
NCF
0.1324
0.1992
NGCF
0.1427
0.2147
NGCF-Att
0.1424
0.2213
Amazon-book
BPRMF
0.0234
0.0257
NCF
0.0541
0.5331
NGCF
0.0321
0.0547
NGCF-Att
0.0257
0.0654The comparison of the results in the above table shows that the other three models based on deep learning are slightly better than the traditional matrix decomposition MF model, which reflects the optimization of the traditional MF to varying degrees.Secondly, the index of NGCF-Att will be slightly higher than that of NCF and NGCF, because it combines the advantages of the two models and introduces an attention mechanism to improve network efficiency.Therefore, we can draw the following conclusions:(1)
The research of recommendation algorithm based on deep learning is an important way to improve the performance of recommendation system(2)
Inner product has some limitations. Using neural network instead of inner product can capture the complex nonlinear interaction between users and items(3)
The attention mechanism assigns different weights to different items of each user, which is more in line with the realistic level of users’ preference for different items. Variable weight learning improves the performance of the model and makes it easier to explain


## 6. Conclusions

At present, the system has been used in different colleges and universities and is widely popular because of its powerful function suitable for English teaching and has strong promotion value. It is believed that the networking of the system will facilitate the use of Intranet for multimedia teaching and paperless examination process of institutions.



---

*Source: 5875380-2022-05-24.xml*