# Intelligent Scoring of English Composition by Machine Learning from the Perspective of Natural Language Processing

Authors: Dan Zhang; Xiaorong Yuan
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

To eliminate the shortcomings of the current English composition scoring, this study intends to propose an automatic scoring model by artificial intelligence (AI) technology—machine learning (ML)—from the perspective of natural language processing (NLP). According to the sentence judgment criteria and scoring criteria of English composition, and with a combination of the n-gram model and decision tree (DT) algorithm as technical support, a composition scoring system is established with the content of the composition as the extraction feature. The random forest algorithm is taken as the training algorithm of the model. By using different maximum feature extraction numbers to test the model using the random forest training method, it is found that when the number of features is 11, the model can play the best prediction performance. Similarly, different learning rate values are used to verify the composition scoring rate of the model. It is found that when the learning rate is 6E-6, the model can play the best scoring performance. Therefore, it is believed that this automatic composition scoring model can be applied to English composition scoring in schools, and some theoretical concepts are put forward for intelligent education. The proposed composition scoring system by N-gram model and DT algorithm and trained by the random forest algorithm has high performance, which can be applied in the automatic scoring of students’ English composition in schools, and provides a basis for the educational applications of ML under AI.


---

## Body

## 1. Introduction

Natural language processing techniques have been used in the literature by numerous researchers to extract features from essays in the datasets. In recent years, the number of relevant studies on AI technology has gradually increased [1], and people began to explore how to apply computer technologies such as ML and AI to students’ teaching [2], including a machine scoring system using AI equipment to correct students’ homework or for composition [3]. However, it is evident that there has been no breakthrough in research on the use of technology during this period. At present, the ML scoring system using AI almost uniformly extracts and evaluates the topic expression information of the composition according to the semantics of the composition [4], and the information of some words or sentences may be ignored. Moreover, for the same composition, due to the different subjective concepts of reviewers, their scoring results will also be different [5]. The same composition may need more than one person to evaluate, so it will take a lot of time, and students need a lot of time to understand their composition’s score, which is not real-time. Therefore, the present research work mainly focuses on an English composition scoring system by ML under AI technology.After determining the experimental objectives, how to achieve them needs to be considered from many aspects. Vögelin et al. studied the impact of lexical features on teachers’ judgment of English argumentative papers and found that texts with greater lexical diversity and complexity had been more positively evaluated in terms of their overall quality and analysis criteria “grammar” and “thesis framework” [6]. When studying the automatic scoring system of composition, the primary is to study the factors affecting the content of the composition. When designing a narrative review composition scoring scale, Baethge et al. summarized and found that the explanation of composition importance, the purpose of review, reference and presentation, evidence level, and results were very important for the evaluation of composition structure [7]. In the research field of automatic scoring of English composition, there have been research results and theories using various intelligent technologies. For example, Ramesh and Sanampudi found that it was a great challenge to evaluate a composition by considering the correlation between content and prompt, the development of ideas, cohesion, and coherence. However, few researchers focused on content-based assessment, while many researchers focused on style-based assessment [8]. Ghanta developed a machine learning model for performing automated essay scoring and evaluating their performance. In this research, the available essay dataset were used for testing and training the efficiency of the considered techniques. Feature extraction form essays in the datasets were performed using natural language processing techniques [9].Shin and Gierl compared the protocols between human evaluators when studying the establishment of a composition scoring system using vector machine (VM) and convolutional neural network (CNN). The results show that the model performs better, which means that the model produces more comparable results with human evaluators [10]. Beseiso and others proposed a neural network model by a transformer. By deep learning (DL), the composition scoring results are compared with human raters. It is concluded that the proposed model is superior to the existing composition scoring methods in terms of scores. The analysis proves the applicability of the proposed model in automatic composition scoring at the level of higher education [11]. From the above research, it can be seen that the current automatic composition scoring system is mostly by the combination of neural network and DL technology. Although there are many studies, there is no breakthrough.Nevertheless, this study intends to design and implement an automatic scoring system for English composition using innovative technology. This proposed algorithm capitalizes on the weaknesses of the existing algorithms and presents a system that improves the accuracy and efficiency in the context of natural language processing. From the perspective of NLP, this study will use a language processing model of Chinese language model n-gram, introduce methods in the scoring of Chinese composition into the scoring of English composition, and propose a scoring model for text content feature extraction combined with the DT in ML. The random forest algorithm is used to train the model, the quadratic weighted value is used to judge the feasibility of the algorithm, and then different learning rates are used to judge the learning ability most suitable for different situations of the model, and the learning situation most suitable for the model is selected to provide some opinions for solving the subjective impact of students’ composition scoring.


## 2. Materials and Methods

### 2.1. The Scoring Characteristics of English Composition

As the composition is a subject with a large proportion of subjectivity, it can reflect students’ comprehensive ability in the examination, including logical expression ability, correct writing ability of words and sentences, and their mastery of composition structure [12], which can express and convey their views according to open topics. Generally speaking, the topics of English composition have several characteristics as shown in Table 1 [13].Table 1
Characteristics of English composition test questions.
Summary of characteristicsSpecific meaningWord featuresEnglish has very strict requirements for word spelling, such as tense, singular, plural, and personalityPhrase featuresPhrases are composed of many forms, such as noun phrases, verb phrases, and prepositional phrases. Each phrase generally has a fixed collocation, which ensures the characteristics of English form in the structureSyntactic featuresSimilar to Chinese, English sentences need to form a fixed and complete structure, and there are also very complex and rich sentencesStructure characteristicsSentences and paragraphs composed of words need to be logically connected, and the coherent connection is the basis of smooth passageBy summarizing the contents of Table1, it can also be precisely concluded that when scoring English compositions, it is necessary to evaluate the content and structure of the compositions from four aspects as shown in Table 2 [14].Table 2
Four aspects of English composition scoring.
Scoring criterionSpecific scoring contentsComposition vocabularyIt is an index to judge whether there are too many repetitive nouns in the text, and count the vocabulary of the composition, to judge the student’s mastery of word spelling and memory abilityComposition statementStudents’ mastery of relevant knowledge can be judged from the structure of sentences or the number and situation of complex sentence patterns. Generally speaking, compositions with too simple sentence patterns will be judged as low score compositionsComposition structureFrom the structure of the composition, this index is used to judge whether the students’ description meets the standard and whether the cohesion of paragraphs and sentence patterns is reasonableComposition logicThis index is used to judge whether the students’ English logical expression meets the standard from the content expressed in the composition, and analyze whether the content is related to the topic without departing from the topicTherefore, the scoring of English composition is very subjective, excluding the consideration of whether the spelling of sentences and words is correct. At present, most of the scoring standards in the examination are from the perspective of linguistics and do not give a set of specific scoring rules from the aspect of content expression. Therefore, the scoring of English composition generally needs to be carried out from the scoring process of experts [15]. As shown in Figure 1, the expert scoring structure is widely used at present.Figure 1
Scoring structure of the expert composition.


### 2.2. Related Concepts of NLP

In the field of NLP [16], lexical analysis is an important work, which is generally completed by using part of speech tagging technology. Analyzing the syntax of sentence patterns can analyze the number of words or phrases, and using filter words can remove redundant or inconsistent words [17]. Figure 2 illustrates the parsing process of the following sentence “my dog also loves playing ball.”Figure 2
Example of the parsing process.In Figure2, ROOT represents the next sentence to be analyzed, NP and NN mean nouns and common nouns, VR indicates verbs or phrases, ADVP stands for adverb phrases, PRP denotes person, RB refers to adverbs, and VBZ and VBG point to verbs, to judge the mastery of the pattern of this sentence.


### 2.3.N-Gram Language Model

N-Gram is a model that statistics language through probability [18]. The analysis method is to predict and analyze the last word and sentence through the previous word and sentence. Given a sequence of “I love,” according to the prediction results of the model and the prediction results according to the maximum likelihood value, such as “you = 40%” and “her = 1%,” these probabilities add up to 100%. Given a sentence pattern as s=w1w2,…,wn, the following equation solves the probability of this sentence:(1)ps=pw1pw2|w1pw3|w1w2…pwn|w1…wn−1.The parametersw1w2,…,wn represent each word in the sentence. To solve the probability of the occurrence of a specific sentence pattern, n parameters must be estimated. Due to the increasing number of subsequent words, the parameters involved in the calculation will also increase, and the calculation process will be very complex. Therefore, the Markov hypothesis [19] is introduced to make the solved words related to the previous words, and (1) can be changed as follows:(2)ps=∏i=1npwn|w1w2…wn−1≈∏i=1npwi|wi−2wi−1.The following equation demonstrates how to solve the parameterpwi|wi−2wi−1 in (2):(3)pwi|wi−2wi−1≈cwi−2wi−1wicwi−2wi−1,where cwi−2wi−1wi represents the number of occurrences of three words wi−2wi−1wi in the corresponding corpus and cwi−2wi−1 indicates the number of occurrences of two words wi−2wi−1 in the corpus. With a large enough corpus, such as the web, we can compute these counts and estimate the probability from equation (3). For the case where the probability is 0 due to the nonoccurrence of a single word, the smoothing method is generally used, that is, the method of reducing the frequency of the existing words, and then assigning the reduced probability to other nonoccurrence words. Theoretically, the more words in the calculation of the model, the better the effect. But the amount of calculation also increases gradually. The basic intuition of the n-gram model is that instead of computing the probability of a word given its entire history, we can approximate the history by just the last few words of a text.


### 2.4. Machine Learning

Before we go deep into the details of how to apply these machine learning and AI tools in NLP and text analysis, let’s clarify certain basic ideas. Machine learning for NLP and text analytics involves a set of statistical techniques for identifying parts of speech, entities, sentiment, and other aspects of the text. The techniques can be expressed as a model that is applied to other text, also known as supervised machine learning. The most common ML methods for NLP are as follows: support vector machines, decision tree, and neural networks. ML [20] generally involves knowledge disciplines in many fields, which is work that needs experience. The “experience” here is generally expressed in the form of data. Therefore, ML is an algorithm that reflects the model through data, that is, a learning algorithm, which includes a learning algorithm called DT [21], which is summarized by name. DT is a decision algorithm similar to a tree structure. Case expression is whether an English composition is excellent or not, which is judged by a variety of factors (content, logical expression, sentence smoothness). Figure 3 signifies the structure of DT, which uses leaves and branches to represent the connection between nodes.Figure 3
Schematic diagram of DT structure.The basic work of the DT is to select the basic root node, which can accommodate all training samples, and then select the optimal characteristics [22]. According to the sample characteristics, the training set samples are divided into subsets, so that each subset is the optimal node under specific conditions. If it can be grouped, it can correspond to leaf nodes; if not, it needs to be further divided and divided again to form nodes. The selection of optimal features is usually referred to by information gain.Information gain is generally by entropy. Assuming thatX should be a random variable with limited values, if P(X − xi) = pi, i = 1, 2, …, n, the entropy can be defined by(4)HX=−∑i=1npilogpi.Since the value of entropy is only related to theX distribution [23] and has nothing to do with the specific value, X can be changed to h (P), as signified by(5)Hp=−∑i=1npilogpi.Because the information gain is the reduced value of information complexity under certain conditions, if the information gaing(C, B) of feature B to training sample set C is the information entropy H (c) of D minus the difference of entropy H(C|B), then the following equation describes the relations between them:(6)gC,B=HC−HC|B.Another method for selecting the optimal features is the Iterative Dichotomiser 3 (ID3) algorithm [24], which is a process of selecting features through information gain. Table 3 lists its calculation steps.Table 3
Steps of ID3 algorithm.
Step sequenceStep content1Feature set B and training set C should be input, and the threshold should be set toε.2If all instances in C belong to the same class Ck, thent is a single node tree. The class Ck needs to be marked as a node class and t should be returned.3IfB = φ, T is a tree with a single node. This single node should be marked directly. A mark should be made on the Ck that appears most frequently in C and return t.4The calculation should be made on the value with reduced complexity of feature information in feature set B, and the feature Bg is selected, which has the most reduced complexity.5If the feature Bg is selected, the information complexity reduction value is less than the thresholdε. Then, T is taken as a tree with a single node, and Ck with the most strength in C is taken as the marker point, and T is returned.6C Should be divided into subsets Ci with multiple nonempty sets, referring to bg =ai, if bg has each possible value ai. After child nodes are formed, the one with the largest number of instances in Ci is taken as a marker to build the node and child nodes into a tree T, and return T.7For the ith subnode, all the features except Bg are used as the feature set, and Ci is used as the training example set. Steps 1 to 5 are repeated recursively to get the subtreeT and return T.8OutputT.To avoid the phenomenon that the ID3 algorithm tends to favor feature nodes with more values, the information gain is taken as the solution. The ratio of the information gaing(C, B) to the entropy HB(C) of the training example set c about the value of feature B is the corresponding information gain ratio gR(C, B). The following equation illustrates the process:(7)gRC,B=gC,BHBC.The following equation expresses the calculation ofHB(C):(8)HBC=−∑i=1nCiClog2CiC.Parametern in (8) is the number of characteristic B values.


### 2.5. Feature Extraction from Text Content

Text content feature extraction [25] generally uses vector space as the basic method. Vector space assumes that the composition content is only related to the occurrence frequency of specific sentence units in the text, but independent of position and order; that is, the composition content representation of vector space is calculated through the sentence unit frequency. In a paragraph of text d, there are n different feature items f1, f2, …, fn. The text can represent the feature item set d(f1, f2, …, fn), fi refers to the feature item, 1≤i ≤ n, and the feature items are irrelevant and independent of order. The calculation is implemented on the feature item weight wi of fi in the text, and f1, f2, …, fn is taken as an n-dimensional coordinate system. Then, w1w2,…,wn are the corresponding dimension values, so text d is taken as an n-dimensional vector with w1w2,…,wn in the n-dimensional space. The following equation defines the calculation of d:(9)V⇀d=w1,w2,…wn.where V⇀d indicates the vector of text d in n-dimensional space.In vector space, the most widely used way to get the weight is through the term frequency-inverse document frequency method. Term frequency (TF) and inverse document frequency (IDF) [26] are used to judge the importance of words. If the same word appears more frequently in the same composition, it appears less frequently in other compositions. Then, the larger the IDF value, and vice versa, the smaller the IDF. In the composition di, if the kth characteristic term ti,k appears ni,k times in the composition, its TF function can be expressed by(10)TFi,k=ni,k.However, because different compositions have different lengths and numbers of words, to unify the standard, it is necessary to standardize the frequency of words. The following equation is used to realize the standardization:(11)TFi,k=ni,k∑mnm,k.The number of IDF values indicates the frequency of occurrence of characteristic termsti,k in text d. The following equation demonstrates its calculation:(12)IDFi,k=logDd:ti,k∈d.Assuming there areM compositions, where the characteristic items ti,k appear in the mi,k compositions, then there is(13)IDFi,k=logM|mi,k+α.Parameterα is the constant of experience, generally taken as 0.01. For the characteristic terms ti,k, the following is used to calculate its characteristic weight:(14)wi,k=TFi,k∗IDFi,k.TF-IDF method can comprehensively consider the discrimination ability and frequency of feature items, and professional vocabulary can more clearly express the theme of the composition. Figure4 manifests the flow diagram of this feature extraction algorithm using vector space.Figure 4
Flowchart of feature extraction by vector space.


### 2.6. The Training Method for Random Forest

The random forest training method is an extended form of bootstrap aggregation (Bagging) [27]. It uses the funded sampling technology to select n samples from the training sample set in the way of taking out and putting back, to form a new sample training set to train the model, and then generates n DTs to form the random forest model and test the results of the data. In essence, it is an optimized version of the DT algorithm. Each DT in the random forest is formed by using independent samples, and then, these trees are combined concurrently. The computing power of a single DT is weak, but after a large number of DTs forming groups, each sample will choose the result with the highest probability after passing through any DT. Table 4 signifies the establishment steps of random forest.Table 4
Establishment steps of random forest.
Step no.Step content1There aren samples selected from the training sample set in the form of taking back2In the feature set,k features are randomly selected, and the DT is established combined with the samples selected in step 13Steps 1-2 are repeated totallym times to establish m DTs to form a random forest4For the new data group, after each DT is classified, it is voted to determine the classification directionFigure5 illustrates the scoring structure for English composition using random forest training, in which the training sets are Train_xk and Train_yk, and the number of DTs in test set_Test_xN2,M is all K.Figure 5
English composition scoring structure by random forest.The following equation aims at calculating the mean value of the results from the DTs in Figure5.(15)Testyi=1k∑k=1KTkTesti.Here, K is the number of DTs.


## 2.1. The Scoring Characteristics of English Composition

As the composition is a subject with a large proportion of subjectivity, it can reflect students’ comprehensive ability in the examination, including logical expression ability, correct writing ability of words and sentences, and their mastery of composition structure [12], which can express and convey their views according to open topics. Generally speaking, the topics of English composition have several characteristics as shown in Table 1 [13].Table 1
Characteristics of English composition test questions.
Summary of characteristicsSpecific meaningWord featuresEnglish has very strict requirements for word spelling, such as tense, singular, plural, and personalityPhrase featuresPhrases are composed of many forms, such as noun phrases, verb phrases, and prepositional phrases. Each phrase generally has a fixed collocation, which ensures the characteristics of English form in the structureSyntactic featuresSimilar to Chinese, English sentences need to form a fixed and complete structure, and there are also very complex and rich sentencesStructure characteristicsSentences and paragraphs composed of words need to be logically connected, and the coherent connection is the basis of smooth passageBy summarizing the contents of Table1, it can also be precisely concluded that when scoring English compositions, it is necessary to evaluate the content and structure of the compositions from four aspects as shown in Table 2 [14].Table 2
Four aspects of English composition scoring.
Scoring criterionSpecific scoring contentsComposition vocabularyIt is an index to judge whether there are too many repetitive nouns in the text, and count the vocabulary of the composition, to judge the student’s mastery of word spelling and memory abilityComposition statementStudents’ mastery of relevant knowledge can be judged from the structure of sentences or the number and situation of complex sentence patterns. Generally speaking, compositions with too simple sentence patterns will be judged as low score compositionsComposition structureFrom the structure of the composition, this index is used to judge whether the students’ description meets the standard and whether the cohesion of paragraphs and sentence patterns is reasonableComposition logicThis index is used to judge whether the students’ English logical expression meets the standard from the content expressed in the composition, and analyze whether the content is related to the topic without departing from the topicTherefore, the scoring of English composition is very subjective, excluding the consideration of whether the spelling of sentences and words is correct. At present, most of the scoring standards in the examination are from the perspective of linguistics and do not give a set of specific scoring rules from the aspect of content expression. Therefore, the scoring of English composition generally needs to be carried out from the scoring process of experts [15]. As shown in Figure 1, the expert scoring structure is widely used at present.Figure 1
Scoring structure of the expert composition.


## 2.2. Related Concepts of NLP

In the field of NLP [16], lexical analysis is an important work, which is generally completed by using part of speech tagging technology. Analyzing the syntax of sentence patterns can analyze the number of words or phrases, and using filter words can remove redundant or inconsistent words [17]. Figure 2 illustrates the parsing process of the following sentence “my dog also loves playing ball.”Figure 2
Example of the parsing process.In Figure2, ROOT represents the next sentence to be analyzed, NP and NN mean nouns and common nouns, VR indicates verbs or phrases, ADVP stands for adverb phrases, PRP denotes person, RB refers to adverbs, and VBZ and VBG point to verbs, to judge the mastery of the pattern of this sentence.


## 2.3.N-Gram Language Model

N-Gram is a model that statistics language through probability [18]. The analysis method is to predict and analyze the last word and sentence through the previous word and sentence. Given a sequence of “I love,” according to the prediction results of the model and the prediction results according to the maximum likelihood value, such as “you = 40%” and “her = 1%,” these probabilities add up to 100%. Given a sentence pattern as s=w1w2,…,wn, the following equation solves the probability of this sentence:(1)ps=pw1pw2|w1pw3|w1w2…pwn|w1…wn−1.The parametersw1w2,…,wn represent each word in the sentence. To solve the probability of the occurrence of a specific sentence pattern, n parameters must be estimated. Due to the increasing number of subsequent words, the parameters involved in the calculation will also increase, and the calculation process will be very complex. Therefore, the Markov hypothesis [19] is introduced to make the solved words related to the previous words, and (1) can be changed as follows:(2)ps=∏i=1npwn|w1w2…wn−1≈∏i=1npwi|wi−2wi−1.The following equation demonstrates how to solve the parameterpwi|wi−2wi−1 in (2):(3)pwi|wi−2wi−1≈cwi−2wi−1wicwi−2wi−1,where cwi−2wi−1wi represents the number of occurrences of three words wi−2wi−1wi in the corresponding corpus and cwi−2wi−1 indicates the number of occurrences of two words wi−2wi−1 in the corpus. With a large enough corpus, such as the web, we can compute these counts and estimate the probability from equation (3). For the case where the probability is 0 due to the nonoccurrence of a single word, the smoothing method is generally used, that is, the method of reducing the frequency of the existing words, and then assigning the reduced probability to other nonoccurrence words. Theoretically, the more words in the calculation of the model, the better the effect. But the amount of calculation also increases gradually. The basic intuition of the n-gram model is that instead of computing the probability of a word given its entire history, we can approximate the history by just the last few words of a text.


## 2.4. Machine Learning

Before we go deep into the details of how to apply these machine learning and AI tools in NLP and text analysis, let’s clarify certain basic ideas. Machine learning for NLP and text analytics involves a set of statistical techniques for identifying parts of speech, entities, sentiment, and other aspects of the text. The techniques can be expressed as a model that is applied to other text, also known as supervised machine learning. The most common ML methods for NLP are as follows: support vector machines, decision tree, and neural networks. ML [20] generally involves knowledge disciplines in many fields, which is work that needs experience. The “experience” here is generally expressed in the form of data. Therefore, ML is an algorithm that reflects the model through data, that is, a learning algorithm, which includes a learning algorithm called DT [21], which is summarized by name. DT is a decision algorithm similar to a tree structure. Case expression is whether an English composition is excellent or not, which is judged by a variety of factors (content, logical expression, sentence smoothness). Figure 3 signifies the structure of DT, which uses leaves and branches to represent the connection between nodes.Figure 3
Schematic diagram of DT structure.The basic work of the DT is to select the basic root node, which can accommodate all training samples, and then select the optimal characteristics [22]. According to the sample characteristics, the training set samples are divided into subsets, so that each subset is the optimal node under specific conditions. If it can be grouped, it can correspond to leaf nodes; if not, it needs to be further divided and divided again to form nodes. The selection of optimal features is usually referred to by information gain.Information gain is generally by entropy. Assuming thatX should be a random variable with limited values, if P(X − xi) = pi, i = 1, 2, …, n, the entropy can be defined by(4)HX=−∑i=1npilogpi.Since the value of entropy is only related to theX distribution [23] and has nothing to do with the specific value, X can be changed to h (P), as signified by(5)Hp=−∑i=1npilogpi.Because the information gain is the reduced value of information complexity under certain conditions, if the information gaing(C, B) of feature B to training sample set C is the information entropy H (c) of D minus the difference of entropy H(C|B), then the following equation describes the relations between them:(6)gC,B=HC−HC|B.Another method for selecting the optimal features is the Iterative Dichotomiser 3 (ID3) algorithm [24], which is a process of selecting features through information gain. Table 3 lists its calculation steps.Table 3
Steps of ID3 algorithm.
Step sequenceStep content1Feature set B and training set C should be input, and the threshold should be set toε.2If all instances in C belong to the same class Ck, thent is a single node tree. The class Ck needs to be marked as a node class and t should be returned.3IfB = φ, T is a tree with a single node. This single node should be marked directly. A mark should be made on the Ck that appears most frequently in C and return t.4The calculation should be made on the value with reduced complexity of feature information in feature set B, and the feature Bg is selected, which has the most reduced complexity.5If the feature Bg is selected, the information complexity reduction value is less than the thresholdε. Then, T is taken as a tree with a single node, and Ck with the most strength in C is taken as the marker point, and T is returned.6C Should be divided into subsets Ci with multiple nonempty sets, referring to bg =ai, if bg has each possible value ai. After child nodes are formed, the one with the largest number of instances in Ci is taken as a marker to build the node and child nodes into a tree T, and return T.7For the ith subnode, all the features except Bg are used as the feature set, and Ci is used as the training example set. Steps 1 to 5 are repeated recursively to get the subtreeT and return T.8OutputT.To avoid the phenomenon that the ID3 algorithm tends to favor feature nodes with more values, the information gain is taken as the solution. The ratio of the information gaing(C, B) to the entropy HB(C) of the training example set c about the value of feature B is the corresponding information gain ratio gR(C, B). The following equation illustrates the process:(7)gRC,B=gC,BHBC.The following equation expresses the calculation ofHB(C):(8)HBC=−∑i=1nCiClog2CiC.Parametern in (8) is the number of characteristic B values.


## 2.5. Feature Extraction from Text Content

Text content feature extraction [25] generally uses vector space as the basic method. Vector space assumes that the composition content is only related to the occurrence frequency of specific sentence units in the text, but independent of position and order; that is, the composition content representation of vector space is calculated through the sentence unit frequency. In a paragraph of text d, there are n different feature items f1, f2, …, fn. The text can represent the feature item set d(f1, f2, …, fn), fi refers to the feature item, 1≤i ≤ n, and the feature items are irrelevant and independent of order. The calculation is implemented on the feature item weight wi of fi in the text, and f1, f2, …, fn is taken as an n-dimensional coordinate system. Then, w1w2,…,wn are the corresponding dimension values, so text d is taken as an n-dimensional vector with w1w2,…,wn in the n-dimensional space. The following equation defines the calculation of d:(9)V⇀d=w1,w2,…wn.where V⇀d indicates the vector of text d in n-dimensional space.In vector space, the most widely used way to get the weight is through the term frequency-inverse document frequency method. Term frequency (TF) and inverse document frequency (IDF) [26] are used to judge the importance of words. If the same word appears more frequently in the same composition, it appears less frequently in other compositions. Then, the larger the IDF value, and vice versa, the smaller the IDF. In the composition di, if the kth characteristic term ti,k appears ni,k times in the composition, its TF function can be expressed by(10)TFi,k=ni,k.However, because different compositions have different lengths and numbers of words, to unify the standard, it is necessary to standardize the frequency of words. The following equation is used to realize the standardization:(11)TFi,k=ni,k∑mnm,k.The number of IDF values indicates the frequency of occurrence of characteristic termsti,k in text d. The following equation demonstrates its calculation:(12)IDFi,k=logDd:ti,k∈d.Assuming there areM compositions, where the characteristic items ti,k appear in the mi,k compositions, then there is(13)IDFi,k=logM|mi,k+α.Parameterα is the constant of experience, generally taken as 0.01. For the characteristic terms ti,k, the following is used to calculate its characteristic weight:(14)wi,k=TFi,k∗IDFi,k.TF-IDF method can comprehensively consider the discrimination ability and frequency of feature items, and professional vocabulary can more clearly express the theme of the composition. Figure4 manifests the flow diagram of this feature extraction algorithm using vector space.Figure 4
Flowchart of feature extraction by vector space.


## 2.6. The Training Method for Random Forest

The random forest training method is an extended form of bootstrap aggregation (Bagging) [27]. It uses the funded sampling technology to select n samples from the training sample set in the way of taking out and putting back, to form a new sample training set to train the model, and then generates n DTs to form the random forest model and test the results of the data. In essence, it is an optimized version of the DT algorithm. Each DT in the random forest is formed by using independent samples, and then, these trees are combined concurrently. The computing power of a single DT is weak, but after a large number of DTs forming groups, each sample will choose the result with the highest probability after passing through any DT. Table 4 signifies the establishment steps of random forest.Table 4
Establishment steps of random forest.
Step no.Step content1There aren samples selected from the training sample set in the form of taking back2In the feature set,k features are randomly selected, and the DT is established combined with the samples selected in step 13Steps 1-2 are repeated totallym times to establish m DTs to form a random forest4For the new data group, after each DT is classified, it is voted to determine the classification directionFigure5 illustrates the scoring structure for English composition using random forest training, in which the training sets are Train_xk and Train_yk, and the number of DTs in test set_Test_xN2,M is all K.Figure 5
English composition scoring structure by random forest.The following equation aims at calculating the mean value of the results from the DTs in Figure5.(15)Testyi=1k∑k=1KTkTesti.Here, K is the number of DTs.


## 3. Experimental Results and Discussion

### 3.1. Verification of the Effect of Feature Number on the Training Performance of the Model

In this link, the prediction accuracy effect of the model under the random forest training algorithm is tested by changing the maximum number of features in the model. The number of feature numbers is 5,7,9,10,11,12,14,16 parameter feature numbers, which allows a single DT to use the maximum number of features. The accuracy value is judged by the quadratic weighted value. Figure6 presents the test results.Figure 6
Influence of maximum feature number on prediction accuracy of the random forest model.Figure6 reveals that with the increase in the maximum number of features, the accuracy of the sentence prediction model under the random forest training algorithm initially increases and then decreases. When the maximum number of features reaches 11, the prediction accuracy of the whole model is the highest, which is 0.8983. When the maximum number of features continues to increase, the prediction accuracy of the model declines. The reason is that when the maximum number of features is at a small value, increasing the number of features can make each DT in the model have more optional features, and the prediction ability of the DT will be enhanced. However, when the number of features of the DT reaches the maximum range, the difference of each DT will be reduced. Therefore, the prediction ability of each DT begins to decline. Therefore, to ensure the maximum prediction ability and accuracy of the model, the maximum number of features should be kept at about 11. To further enhance the prediction ability of the model, there are three strategies proposed here. They are to further enrich the sentences in the corpus, and the more training samples, the better the prediction effect of the model. Adding other language or grammar detection modules, the current prediction model cannot detect grammar deeply. Other AI technologies (neural network (neural network, DL) are introduced for further training and prediction. The use of a neural network can avoid manual feature extraction and improve the accuracy of the model from the feature extraction part.


### 3.2. The Scoring Rate Test of the Model for English Composition under Different Learning Rates

In this link, different learning rates are used to test the scoring effect of the model on a total of seven compositions in English composition scoring. The learning rates are 4E-5, 6E-6, and 6E-7, respectively. The secondary weighted value is still used to judge the scoring rate of the model. Figure7 displays the test results.Figure 7
English composition scoring rate of the model under different learning rates.Figure7 implies that when using the scoring model to score seven English compositions under three different learning rates, through the test, when the learning rate is 6E-6, the scoring rate of the model reaches the maximum and shows the best performance, and its average scoring rate reaches 0.81. If the learning rate is smaller or greater than 6e-6 is selected for scoring, the scoring performance of the model has decreased. If the learning rate is too large, the model may not be able to converge and score English compositions. Therefore, with the learning rate of 6E-6, the scoring performance of the model can reach the maximum. The present work proposes two measures to improve the performance of this link. The first is to establish a sentence corpus specially applied to the model to enrich the sample size; the second is to study feature extraction in different aspects from the perspective of linguistics.


## 3.1. Verification of the Effect of Feature Number on the Training Performance of the Model

In this link, the prediction accuracy effect of the model under the random forest training algorithm is tested by changing the maximum number of features in the model. The number of feature numbers is 5,7,9,10,11,12,14,16 parameter feature numbers, which allows a single DT to use the maximum number of features. The accuracy value is judged by the quadratic weighted value. Figure6 presents the test results.Figure 6
Influence of maximum feature number on prediction accuracy of the random forest model.Figure6 reveals that with the increase in the maximum number of features, the accuracy of the sentence prediction model under the random forest training algorithm initially increases and then decreases. When the maximum number of features reaches 11, the prediction accuracy of the whole model is the highest, which is 0.8983. When the maximum number of features continues to increase, the prediction accuracy of the model declines. The reason is that when the maximum number of features is at a small value, increasing the number of features can make each DT in the model have more optional features, and the prediction ability of the DT will be enhanced. However, when the number of features of the DT reaches the maximum range, the difference of each DT will be reduced. Therefore, the prediction ability of each DT begins to decline. Therefore, to ensure the maximum prediction ability and accuracy of the model, the maximum number of features should be kept at about 11. To further enhance the prediction ability of the model, there are three strategies proposed here. They are to further enrich the sentences in the corpus, and the more training samples, the better the prediction effect of the model. Adding other language or grammar detection modules, the current prediction model cannot detect grammar deeply. Other AI technologies (neural network (neural network, DL) are introduced for further training and prediction. The use of a neural network can avoid manual feature extraction and improve the accuracy of the model from the feature extraction part.


## 3.2. The Scoring Rate Test of the Model for English Composition under Different Learning Rates

In this link, different learning rates are used to test the scoring effect of the model on a total of seven compositions in English composition scoring. The learning rates are 4E-5, 6E-6, and 6E-7, respectively. The secondary weighted value is still used to judge the scoring rate of the model. Figure7 displays the test results.Figure 7
English composition scoring rate of the model under different learning rates.Figure7 implies that when using the scoring model to score seven English compositions under three different learning rates, through the test, when the learning rate is 6E-6, the scoring rate of the model reaches the maximum and shows the best performance, and its average scoring rate reaches 0.81. If the learning rate is smaller or greater than 6e-6 is selected for scoring, the scoring performance of the model has decreased. If the learning rate is too large, the model may not be able to converge and score English compositions. Therefore, with the learning rate of 6E-6, the scoring performance of the model can reach the maximum. The present work proposes two measures to improve the performance of this link. The first is to establish a sentence corpus specially applied to the model to enrich the sample size; the second is to study feature extraction in different aspects from the perspective of linguistics.


## 4. Conclusion

There are problems that the scoring criteria for students’ English composition are inaccurate, the scoring rules are too subjective, and most of the scoring criteria take the composition content as the starting point. To put forward a scoring system for English composition, initially, the present work discusses the characteristics of grammar, sentence, and structure of English composition and expounds on the scoring criteria for English composition. By the n-gram model, combined with the DT algorithm under ML, a composition scoring system for text content feature extraction is established, and then the random forest algorithm is introduced to train the model. Then, the present work extracts a large number of nontext features that can reflect the quality of students’ compositions. Simultaneously, according to Bai Ji’s parameter adjustment experience, a set of ML model parameter adjustment methods is summarized, so that the model parameters can be adjusted to the optimal direction, efficiently and quickly, and further improve the prediction results of the model. In the following test link, different maximum feature numbers are used to test the prediction performance of the model under the random forest training algorithm. It is found that the feature number with the value of 11 can make the model play the best prediction ability. Then, different learning rate values are used to test the performance of the English composition scoring rate of the model. The results show that when the learning rate is 6E-6, the model can play the best scoring performance. Therefore, the conclusion is that this composition scoring system by N-gram model and DT algorithm and trained by random forest algorithm has high performance, which can be applied in the automatic scoring of students’ English composition in schools, and provides a certain basis for the educational applications of ML under AI. Besides, the shortcomings are in the first part of the results. At present, the model has insufficient judging ability for grammar, which needs to be further optimized. In the follow-up work, attention will be made to introducing neural networks and other technologies to improve the performance of grammar scoring.



---

*Source: 9070272-2022-05-06.xml*