# Research on Network Oral English Teaching System Based on Machine Learning

Authors: Xiaowei Liu
Journal: Security and Communication Networks
 (2022)

---

## Abstract

With the development of computer science and technology and the advancement of language teaching and learning methods, computer-assisted language learning technology has made it possible to solve this problem. In the area of oral learning, some computer-assisted language learning systems at home and abroad mainly focus on the learning of words and grammar and only have one or two evaluation indicators as the basis of evaluation, which have certain functional defects and can only give an overall rating to learners’ pronunciation. To address these problems, this paper takes the English speech of Chinese college students as the research object and improves the traditional computerized English pronunciation quality evaluation method by considering multiplicative evaluation indexes, such as pitch, speed, rhythm, and intonation; i.e., pitch evaluation is based on frequency central feature parameters, speech speed evaluation based on speech duration, rhythm evaluation. The method of pitch, speed, rhythm, and intonation evaluation has been experimentally verified to be reliable. Furthermore, considering the weight of the above multiplicative evaluation indexes.


---

## Body

## 1. Introduction

With global integration and the increasing internationalization of China, the demand for English learning in China is growing rapidly [1]. And the lack of good English teachers and traditional classroom teaching cannot meet the needs of English learning due to the constraints of time and location. The combination of all these reasons has made English teaching and learning a major problem for the people of China. English learning has also become one of the hot spots in the field of education. With the development of computer science and technology and advances in language teaching and learning methods, Computer-Assisted Language Learning (CALL) technology has made it possible to solve this problem [2].Facing a series of challenges brought by the rapid change of knowledge content and the increasingly fierce international competition for highly qualified talents, it is an important and urgent issue for students to change their learning styles and improve the quality of learning. In order to become qualified citizens in the 21st century, students must not only have a deep grasp of the subject matter but also learn to learn and have better problem-solving, higher-order thinking, independent thinking, and knowledge application skills. Therefore, in order to meet this requirement of social development, teachers of ideology and politics classes must change their teaching methods in the teaching process, give full-play to the main position of students in the learning process, provoke students to think and research deeply about the knowledge and content they have learned and promote students' deep learning [3]. The problem-solving based on real situations is towards deep learning practice exploration, which is inseparable from the high-quality classroom teaching problem design of ideology and politics class teachers. It can be said that good problems are favorable pushers to promote students’ deep learning and play an important role [4, 5].On the one hand, the majority of English learners in China still use language repeaters, MP3 players, cell phones, and other portable devices to assist in learning spoken English, but these methods are not able to perform However, none of these devices can recognize the pronunciation, and they are limited to the function of following the pronunciation, and they cannot directly give the learners reasonable and objective pronunciation evaluation and feedback guidance. Feedback and guidance cannot be given directly to the learners [6]. Moreover, due to the limitation of technology, some CALL systems at home and abroad mainly focus on the learning of words and grammar. The system has only one or two evaluation indexes as the basis for evaluation, which has certain functional defects and can only give learners an overall rating. This has certain functional shortcomings and can only give learners an overall rating [7]. Since self-learners are limited by their own English level, it is difficult for them to find their pronunciation errors by only one overall score, so that they can correct them. It is difficult for self-learners to find their own pronunciation errors and correct them. On the other hand, in the evaluation of spoke English, the current English speaking test is still based on the main score. The current English speaking test is still dominated by manual scoring which is highly subjective, with different standards and slow speed. Due to the different knowledge and experience of the scoring experts, and even the different status of the same expert, the evaluation of the same pronunciation may be biased [8].The contributions of this paper are as follows:This paper eliminates the influence of the human vocal organ itself and high-order harmonic distortion of speech signal acquisition equipment on speech signal quality. A speech preprocessing scheme is designed to obtain a smoother and uniform speech signal, which can provide better parameters for speech feature extraction, so as to improve the quality of speech processing.In this paper, a deep belief network is designed. The weights of the generated model are obtained through pretraining, and then the network is fine-tuned by back-propagation algorithm to obtain a network model with better performance. The initialization scheme is designed to initialize the weights of multi-layer perceptron for DBN.The experimental results show that the evaluation methods of intonation, speed, rhythm, and intonation used in this paper are reliable. Furthermore, considering the weight of the above multi-parameter evaluation indexes, a reasonable and objective evaluation model of English pronunciation quality is constructed by regression analysis.


## 2. Related Work

Domestically, in terms of voice pronunciation quality evaluation, the Institute of Automation of the Chinese Academy of Sciences, the Chinese Academy of Social Sciences Institute of Linguistics, University of Science and Technology of China, Tsinghua University, Microsoft Asia Research Institute, and Anhui Sci-Tech Co. Ltd. have been conducting related researches and have achieved corresponding results. [9] added rhyme factors to the original monophonic and trisyllabic models to construct a rhyme modeling approach, thus improving the performance of pronunciation quality evaluation. [10–12] solved the confusion degree problem between the probability space and the target pronunciation acoustic model by studying the phoneme-related frame-regularized logarithmic posterior probability and its transformation, which led to a significant improvement of pronunciation quality evaluation performance. [13] proposed a new algorithm introducing the GMM-UBM model in phoneme articulation quality evaluation, constructing phoneme-independent feature distribution model, and scoring better than other algorithms, which are close to the scoring correlation of experts. [14] proposed a new computational strategy to apply linguistic rules in the log posterior probability algorithm, and its correlation between machine scoring and human scoring reached 0.795, which is 9% better than the original algorithm. [15] proposed a comprehensive pronunciation quality evaluation algorithm based on MFCC and LSP parameters and an objective scoring algorithm. [16] proposed a new pronunciation quality evaluation algorithm-PASS (phone-based automatic score for l2 speech quality), and successfully applied it to the call system for English learning people- -the interactive language learning system of Tsinghua University. [17] conducted a comprehensive evaluation study of pronunciation quality by comparing the pitch, rate, stress, rhythm, and intonation of the utterances to be evaluated with the standard utterances of the corpus, and achieved better results. In contrast, because the theoretical analysis of ANN is more difficult and cannot explain the temporal dynamic characteristics of speech signal well; it is easier to overfit when training the learning network model, more difficult to adjust the model parameters, and requires a lot of experience and skills, and the speed is slower, and the effect is not better than other methods when there are fewer levels (less than or equal to 3) [18–20], so the shallow artificial neural network did not make a big breakthrough in this period and development [21].At the end of December 2012, Baidu released its voice assistance and claimed to have successfully applied DNN to its voice recognition service. Baidu released Baidu Voice Assistant and claimed to have successfully applied DNN to the voice recognition service of Baidu Voice Assistant [22]. Subsequently, Cloud Voice claimed that they used DNN-based models to replace the original Gaussian Mixture Models (GMM) based models to upgrade the backend of the public speech cloud service, and after the upgrade, the speech recognition error rate was reduced by more than 30% and the recognition speed was also greatly improved [23]. Since the background speech recognition service of Sogou Speech Assistant is provided by Cloud Voice, the technical staff of Sogou confirmed that the new version has greatly improved the user experience, which further proves that DNN technology can significantly improve the performance of speech recognition.In a word, the concept of deep learning is currently hot, providing high accuracy and high speed for speech recognition.


## 3. Speech Signal Preprocessing

The purpose of these operations is to eliminate the effects on speech signal quality due to high harmonic distortion, high frequencies, and aliasing of the human vocal cords themselves and the speech signal acquisition equipment [24].


### 3.1. Preaggravation

The filter response function is shown in equation:(1)Hz=1−αz−1,0.9≤α≤1.0.The input speech signalxn as:(2)yn=xn−αxn−1.


### 3.2. Split Frame

It can be regarded as a quasi-steady-state process because the speech signal is relatively stable within a short-time range (generally 10∼30 ms); i.e., the speech signal has short-time smoothness [25,26].


### 3.3. Add Window

To enhance the speech waveform around samplen and attenuate the rest of the waveform calculated by the following formula:(3)Qn=∑m=−∞∞Tsnωn−m,here T [ ] denotes some transformation, either linear or nonlinear, Qn is a time series obtained after all segments are processed [27], rectangular and Hanning windows, which are defined as:(1)
Hanning Window(4)ωn=0.54−0.46cos2πnN−1,0≤n≤N−1,0.(2)
Rectangular window(5)ωn=1,0≤n≤N−1,0.(3)
Hanning Window(6)ωn=0.51−cos2πnN−1,0≤n≤N−1,0.The rectangular window has a narrow main flap with high-frequency resolution, but also because it has a high side flap easily leads to more serious interference between adjacent harmonics, and sometimes superimposed and sometimes canceled within the adjacent harmonic interval to produce serious leakage [28].The endpoint detection results for the sentence will be in the place where we always put it. Are shown in Figure1 the results, it can be seen that the endpoint detection with the double threshold comparison method has better results.Figure 1
sentence will be placed where we often put it.


## 3.1. Preaggravation

The filter response function is shown in equation:(1)Hz=1−αz−1,0.9≤α≤1.0.The input speech signalxn as:(2)yn=xn−αxn−1.


## 3.2. Split Frame

It can be regarded as a quasi-steady-state process because the speech signal is relatively stable within a short-time range (generally 10∼30 ms); i.e., the speech signal has short-time smoothness [25,26].


## 3.3. Add Window

To enhance the speech waveform around samplen and attenuate the rest of the waveform calculated by the following formula:(3)Qn=∑m=−∞∞Tsnωn−m,here T [ ] denotes some transformation, either linear or nonlinear, Qn is a time series obtained after all segments are processed [27], rectangular and Hanning windows, which are defined as:(1)
Hanning Window(4)ωn=0.54−0.46cos2πnN−1,0≤n≤N−1,0.(2)
Rectangular window(5)ωn=1,0≤n≤N−1,0.(3)
Hanning Window(6)ωn=0.51−cos2πnN−1,0≤n≤N−1,0.The rectangular window has a narrow main flap with high-frequency resolution, but also because it has a high side flap easily leads to more serious interference between adjacent harmonics, and sometimes superimposed and sometimes canceled within the adjacent harmonic interval to produce serious leakage [28].The endpoint detection results for the sentence will be in the place where we always put it. Are shown in Figure1 the results, it can be seen that the endpoint detection with the double threshold comparison method has better results.Figure 1
sentence will be placed where we often put it.


## 4. The Proposed Deep Learning Model

DBN is used to pretrain the weights of the generated model and then fine-tune the network with a back-propagation algorithm to obtain a better-performing network model. Numerous experiments have shown that internalizing the weight of a multilayer perceptron with a DBN of the appropriate configuration often gives much better results than random initialization [29]. DBN as shown in Figure 2. Essentially, the DBN is trained layer by layer on the RBM to obtain a global better initial parameter, thus improving the network performance. A large number of experiments also demonstrate that DBN can solve the problems of traditional BP networks: the need for a large set of training samples with markers, slow convergence, inappropriate parameter selection leading to the network falling into a local optimum.Figure 2
DBN model.Then the wake-sleep algorithm is used to adjust all the weight. The cognitive and the generative are made to agree as much as possible; i.e., the topmost representation of the generative is able to recover the bottom node as correctly as possible [30].The training process of Deep Learning is as follows:(1)
Unsupervised hierarchical training of each layer parameters using unsolicited data (calibrated data are also possible). The main difference from traditional neural networks is that this step is equivalent to the feature learning process [31].(2)
Top-down supervised learning (that is, by going through the data with labels for training. The errors are transmitted top-down layer by layer to fine-tune the model parameters).Supervised learning is used to further tune the parameters of the whole multilayer model. Unlike the random initialization process of traditional neural networks, the initialized parameters of DL are obtained by learning the structure of the input data in the first step and are not randomly initialized, thus the initial values are closer to the global optimum, and thus better results can be achieved.


## 5. Experimental Simulation and Result Analysis

### 5.1. Data Source

This paper uses the Spoken Arabic Digit dataset from the UCI machine learning repository, constructed by the Automatic Signaling Laboratory at Badji–Mokhtar University. The dataset is the pronunciation of Arabic digits after extraction of 13-order MFCC feature parameters, and consists of a total of 8800 speech data (88 individuals pronouncing 10 Arabic digits, each repeated 10 times), pronounced by 44 men and 44 women between 16 and 40 years old.Before MFCC feature parameter extraction, the parameters to be set are sampling rate 16 KHz, 16-bit encodings, hamming window plus window function [32].English sentence data source, the subjects of this paper were university students of our university, 24 in total, 15 male and 9 females. The subjects were recorded using the recording software CoolEdit, with a sampling rate of 16 KHz and 16-bit codings. A total of 10 sentences were recorded, all of which were commonly spoken English sentences.


### 5.2. Speech Recognition Experiment

In order to verify the effectiveness of the model in this paper, we experimentally compare the recognition rate of this model with other models for nonperson-specific solitary word recognition. In order to verify the effectiveness of this model, we compare the recognition rate of this model with other models for nonperson-specific isolated word recognition. The Spoken ArabicDigit dataset from the UCI machine learning repository is used, which consists of 8800 Arabic digital speech data (88 individuals pronounce 10 Arabic digits, each digit is repeated 10 times). 6600 pronunciation from the first 66 individuals are used as the training set, and 2200 pronunciation from the last 22 individuals is used as the test set. The experimental software is Matlab R2013a [33].At present, there is no unified standard for the selection of parameters, such as the number of hidden layers, the number of nodes in each hidden layer, the number of samples for batch processing, and the number of iterations of BP networks in the deep belief network model. The parameters of the DBN model constructed in this paper are obtained by experimental comparison and tuning.In the process of adjusting the weights by error back propagation of the DBN model using BP network, the number of samples per batch is appropriately reduced, the number of iterations is increased, and the recognition rate of the model can be improved by training a small number of iterations, which, of course, is accompanied by the increase of the model building time. It is worth noting that the smaller the number of samples per batch, the higher the number of iterations, and the higher the recognition rate, but the two do not monolithically line. Too few samples and too many iterations may also cause overfitting of the model and reduce the performance of the model.For the same Spoken Arabic Digit dataset from the UCI machine learning library, [21] proposed a new Tree Distributions Approximation based on Graphical Tree Structure (TDA-GTS) and is similar to the Tree Distributions Approximation based on Maximum Weight Spanning Tree (TDA-MWST), the traditional Discrete Hidden Markov Model (DHMM), and the Continuous Continuous Markov Model (CMM). Model (DHMM), and a Continuous Density Hidden Markov Model (CDHMM) are compared and the recognition effect is improved. [22] proposed a K-means algorithm based on selective weights and Thresholds (KASWT), and compared it with the BP_Adaboost algorithm, and the recognition effect was improved.Here, the model in this paper is compared with the above models, and the comparison results of their recognition rates are shown in Table1.Table 1
Comparison of recognition rates under different models.
Model/indexRecognition rate (%)DHMM90.79CDHMM94.09TCA-MWST93.16BP_Adaboost93.09KASWT92.68Paper model96.64As shown in Table1, the recognition rate of the DBN model constructed in this paper is 96.64%, which is better than the above models. Therefore, the DBN-based speech recognition model established in this paper is reasonable and effective and can be further used in speech pronunciation quality evaluation.


### 5.3. Speech Evaluation Experiment

The purpose of the speech evaluation experiment is to verify the performance of the English speech pronunciation quality evaluation model and method proposed in this paper, the method is as follows:(7)AConsistencyrate=Number of samples consistent between machine evaluation and manual evaluationTotal number of samples.The adjacent agreement rate is the ratio of the sum of the number of samples in which the machine evaluation and the manual evaluation agree and are adjacent to the total number of samples. Here, “adjacent” is defined as the difference of one level between the ratings of a machine and manual evaluations. The specific calculation method is as follows:(8)AAdjacent consistencyrate=N+ATotal number of samples.


### 5.4. Manual Evaluation

According to the pronunciation quality characteristics, we set four different levels for different evaluation indexes (pitch, speed, rhythm, and intonation) and the overall evaluation situation. The detailed evaluation grades and corresponding evaluation standards are shown in Table2.Table 2
Artificial evaluation level and evaluation criteria.
GradeIntonationSpeed of speechRhythmIntonationPopulationAThe content is complete and accurateModerate speedStress, accurate pronunciation, and a strong sense of rhythmAccurate and natural intonationExcellent pronunciationBThe pronunciation is relatively clear and fluentSpeak a little faster (slower)Stress pronunciation is more accurate and has a good sense of rhythmThe intonation is more accurate and naturalThe pronunciation is generally goodCThere are pronunciation errors that affect understandingSpeak fast (slow)The accent pronunciation is general and has a certain sense of rhythmThe intonation is basically accurate, but not natural enoughGeneral grasp of pronunciationDThere are serious pronunciation errors affecting understandingSpeak too fast (slow)Accent pronunciation is wrong, the number of accents is too much (less), and the sense of rhythm is poorThe intonation is inaccurate and unnaturalPoor overall grasp of pronunciationThe manual evaluation was carried out by two experienced university English teachers. They evaluated each of the 24 They evaluated each of the 10 recorded common English utterances of our university students, including 4 evaluation indicators of intonation, speed, rhythm, and the four evaluation indexes and the overall evaluation were.Considering that the subjectivity of the teacher in the manual evaluation process may have an impact on the evaluation results, this paper uses the Pearson correlation coefficient to test the reliability of the manual evaluation results.Furthermore, the evaluation results of the two teachers were averaged (rounded) to obtain different students’ different sentences for each evaluation index and the overall score as the final manual evaluation result.


### 5.5. Testing of the Evaluation Model

On the basis of testing the credibility of the above evaluation indexes, this paper furthermore considers the multi-criteria indexes of pitch, speed, rhythm, and intonation and their weights, and uses regression analysis to build a reasonable and objective English pronunciation quality evaluation model for college students. The paper furthermore considers the multi-covariate indicators of pronunciation quality and their weight and uses regression analysis to build a reasonable and objective evaluation model of English pronunciation quality for college students.The regression analysis method uses mathematical and statistical methods to establish statistical models to study the statistical relationships between the variables of objective things, through a large number of tests and Through a large number of tests and observations of objective things, the statistical regularities hidden in those seemingly uncertain phenomena are searched for model predictions, as in equation:(9)Score=AccuracyScore×0.44+SpeedScore×0.106+Rhythmscore×0.341+IntonationScore×0.312−0.397.TheF-test of the significance of multiple accuracy score, speed score, rhythm score, and intonation score are clear about the random variable score as a whole. Significant impact. According to the analysis of variance table calculated, indicating that the regression equation is significant, and the score has a significant linear relationship with accuracy score, speed score, rhythm score, and intonation score, that is, the probability of error in the judgment that the four evaluation indexes have a significant linear impact on the total score is only 0.003 as shown in Table 3.Table 3
Overall evaluation experimental results.
Index/difference levelConsistency rate (%)Adjacent consistency rate (%)PearsonIntonation87.51000.722In the regression analysis method, thet-test was used to test the significance of the regression coefficients. The table of regression coefficients calculated by SPSS software shows that at the significance level α = 0.05, AccuracyScore, SpeedScore, RhythmScore, and IntonationScore passed the significance test at the significance level of α = 0.05, indicating that all four evaluation indicators had a significant effect on the total score have a significant effect on the total score.Furthermore, equation (9) was used to evaluate a total of 240 sentences for 10 sentences for 24 students. The experimental results are shown in Figure 3, the distribution of different sample values is different. For example, the distribution of sample value 5 is 6, which is also what we expect. 210 samples have the same rating between machines, and no samples have the difference of two or three levels; as shown in Table 4, the overall agreement rate between machine and manual evaluation is 87.5%, and the adjacent agreement rate is as high as 100%, which indicates that machine evaluation and manual evaluation have a strong correlation, indicating that machine evaluation has a strong correlation with manual evaluation.Figure 3
Graph of the overall evaluation difference between machine and human.Table 4
Selected experimental data.
Sentence/indexIntonationSpeed of speechRhythmIntonationPopulation01Machine ratingAABAManual ratingAABA02Machine ratingAABAManual ratingAABA03Machine ratingAABAManual ratingACBA11Machine ratingABBAManual ratingAABA12Machine ratingAABAManual ratingABAASome of the experimental data are shown in Table4.The experimental results are very good, but after thinking again, I think it may be related to the following factors: (1) the amount of data in this paper is small, and linear regression analysis is used to fit the model; (2) the average score of the two English teachers is used as the final score of the manual evaluation, and the rounding method makes the score high; (3) this paper uses a four-level scoring method for evaluation, and the “adjacent” in the adjacent agreement rate is defined as the difference between the levels of machine evaluation and manual evaluation by one level, which to a certain extent increases the adjacent agreement rate.In equation (9), the weight of pitch is the largest, the weight of rhythm and intonation is the second, and the weight of the speech rate is the smallest. After arriving at the above experimental results, the author communicated with the English speech experts in our school to furthermore examine the reliability of the multi-covariate pronunciation quality evaluation model proposed in this paper.As shown in Figure4, the pronunciation time of different words shows that the complexity of the scheme in this paper is very low and takes little time. This is because we preprocess the speech, making the data easy to use.Figure 4
Pronunciation time of different words.


## 5.1. Data Source

This paper uses the Spoken Arabic Digit dataset from the UCI machine learning repository, constructed by the Automatic Signaling Laboratory at Badji–Mokhtar University. The dataset is the pronunciation of Arabic digits after extraction of 13-order MFCC feature parameters, and consists of a total of 8800 speech data (88 individuals pronouncing 10 Arabic digits, each repeated 10 times), pronounced by 44 men and 44 women between 16 and 40 years old.Before MFCC feature parameter extraction, the parameters to be set are sampling rate 16 KHz, 16-bit encodings, hamming window plus window function [32].English sentence data source, the subjects of this paper were university students of our university, 24 in total, 15 male and 9 females. The subjects were recorded using the recording software CoolEdit, with a sampling rate of 16 KHz and 16-bit codings. A total of 10 sentences were recorded, all of which were commonly spoken English sentences.


## 5.2. Speech Recognition Experiment

In order to verify the effectiveness of the model in this paper, we experimentally compare the recognition rate of this model with other models for nonperson-specific solitary word recognition. In order to verify the effectiveness of this model, we compare the recognition rate of this model with other models for nonperson-specific isolated word recognition. The Spoken ArabicDigit dataset from the UCI machine learning repository is used, which consists of 8800 Arabic digital speech data (88 individuals pronounce 10 Arabic digits, each digit is repeated 10 times). 6600 pronunciation from the first 66 individuals are used as the training set, and 2200 pronunciation from the last 22 individuals is used as the test set. The experimental software is Matlab R2013a [33].At present, there is no unified standard for the selection of parameters, such as the number of hidden layers, the number of nodes in each hidden layer, the number of samples for batch processing, and the number of iterations of BP networks in the deep belief network model. The parameters of the DBN model constructed in this paper are obtained by experimental comparison and tuning.In the process of adjusting the weights by error back propagation of the DBN model using BP network, the number of samples per batch is appropriately reduced, the number of iterations is increased, and the recognition rate of the model can be improved by training a small number of iterations, which, of course, is accompanied by the increase of the model building time. It is worth noting that the smaller the number of samples per batch, the higher the number of iterations, and the higher the recognition rate, but the two do not monolithically line. Too few samples and too many iterations may also cause overfitting of the model and reduce the performance of the model.For the same Spoken Arabic Digit dataset from the UCI machine learning library, [21] proposed a new Tree Distributions Approximation based on Graphical Tree Structure (TDA-GTS) and is similar to the Tree Distributions Approximation based on Maximum Weight Spanning Tree (TDA-MWST), the traditional Discrete Hidden Markov Model (DHMM), and the Continuous Continuous Markov Model (CMM). Model (DHMM), and a Continuous Density Hidden Markov Model (CDHMM) are compared and the recognition effect is improved. [22] proposed a K-means algorithm based on selective weights and Thresholds (KASWT), and compared it with the BP_Adaboost algorithm, and the recognition effect was improved.Here, the model in this paper is compared with the above models, and the comparison results of their recognition rates are shown in Table1.Table 1
Comparison of recognition rates under different models.
Model/indexRecognition rate (%)DHMM90.79CDHMM94.09TCA-MWST93.16BP_Adaboost93.09KASWT92.68Paper model96.64As shown in Table1, the recognition rate of the DBN model constructed in this paper is 96.64%, which is better than the above models. Therefore, the DBN-based speech recognition model established in this paper is reasonable and effective and can be further used in speech pronunciation quality evaluation.


## 5.3. Speech Evaluation Experiment

The purpose of the speech evaluation experiment is to verify the performance of the English speech pronunciation quality evaluation model and method proposed in this paper, the method is as follows:(7)AConsistencyrate=Number of samples consistent between machine evaluation and manual evaluationTotal number of samples.The adjacent agreement rate is the ratio of the sum of the number of samples in which the machine evaluation and the manual evaluation agree and are adjacent to the total number of samples. Here, “adjacent” is defined as the difference of one level between the ratings of a machine and manual evaluations. The specific calculation method is as follows:(8)AAdjacent consistencyrate=N+ATotal number of samples.


## 5.4. Manual Evaluation

According to the pronunciation quality characteristics, we set four different levels for different evaluation indexes (pitch, speed, rhythm, and intonation) and the overall evaluation situation. The detailed evaluation grades and corresponding evaluation standards are shown in Table2.Table 2
Artificial evaluation level and evaluation criteria.
GradeIntonationSpeed of speechRhythmIntonationPopulationAThe content is complete and accurateModerate speedStress, accurate pronunciation, and a strong sense of rhythmAccurate and natural intonationExcellent pronunciationBThe pronunciation is relatively clear and fluentSpeak a little faster (slower)Stress pronunciation is more accurate and has a good sense of rhythmThe intonation is more accurate and naturalThe pronunciation is generally goodCThere are pronunciation errors that affect understandingSpeak fast (slow)The accent pronunciation is general and has a certain sense of rhythmThe intonation is basically accurate, but not natural enoughGeneral grasp of pronunciationDThere are serious pronunciation errors affecting understandingSpeak too fast (slow)Accent pronunciation is wrong, the number of accents is too much (less), and the sense of rhythm is poorThe intonation is inaccurate and unnaturalPoor overall grasp of pronunciationThe manual evaluation was carried out by two experienced university English teachers. They evaluated each of the 24 They evaluated each of the 10 recorded common English utterances of our university students, including 4 evaluation indicators of intonation, speed, rhythm, and the four evaluation indexes and the overall evaluation were.Considering that the subjectivity of the teacher in the manual evaluation process may have an impact on the evaluation results, this paper uses the Pearson correlation coefficient to test the reliability of the manual evaluation results.Furthermore, the evaluation results of the two teachers were averaged (rounded) to obtain different students’ different sentences for each evaluation index and the overall score as the final manual evaluation result.


## 5.5. Testing of the Evaluation Model

On the basis of testing the credibility of the above evaluation indexes, this paper furthermore considers the multi-criteria indexes of pitch, speed, rhythm, and intonation and their weights, and uses regression analysis to build a reasonable and objective English pronunciation quality evaluation model for college students. The paper furthermore considers the multi-covariate indicators of pronunciation quality and their weight and uses regression analysis to build a reasonable and objective evaluation model of English pronunciation quality for college students.The regression analysis method uses mathematical and statistical methods to establish statistical models to study the statistical relationships between the variables of objective things, through a large number of tests and Through a large number of tests and observations of objective things, the statistical regularities hidden in those seemingly uncertain phenomena are searched for model predictions, as in equation:(9)Score=AccuracyScore×0.44+SpeedScore×0.106+Rhythmscore×0.341+IntonationScore×0.312−0.397.TheF-test of the significance of multiple accuracy score, speed score, rhythm score, and intonation score are clear about the random variable score as a whole. Significant impact. According to the analysis of variance table calculated, indicating that the regression equation is significant, and the score has a significant linear relationship with accuracy score, speed score, rhythm score, and intonation score, that is, the probability of error in the judgment that the four evaluation indexes have a significant linear impact on the total score is only 0.003 as shown in Table 3.Table 3
Overall evaluation experimental results.
Index/difference levelConsistency rate (%)Adjacent consistency rate (%)PearsonIntonation87.51000.722In the regression analysis method, thet-test was used to test the significance of the regression coefficients. The table of regression coefficients calculated by SPSS software shows that at the significance level α = 0.05, AccuracyScore, SpeedScore, RhythmScore, and IntonationScore passed the significance test at the significance level of α = 0.05, indicating that all four evaluation indicators had a significant effect on the total score have a significant effect on the total score.Furthermore, equation (9) was used to evaluate a total of 240 sentences for 10 sentences for 24 students. The experimental results are shown in Figure 3, the distribution of different sample values is different. For example, the distribution of sample value 5 is 6, which is also what we expect. 210 samples have the same rating between machines, and no samples have the difference of two or three levels; as shown in Table 4, the overall agreement rate between machine and manual evaluation is 87.5%, and the adjacent agreement rate is as high as 100%, which indicates that machine evaluation and manual evaluation have a strong correlation, indicating that machine evaluation has a strong correlation with manual evaluation.Figure 3
Graph of the overall evaluation difference between machine and human.Table 4
Selected experimental data.
Sentence/indexIntonationSpeed of speechRhythmIntonationPopulation01Machine ratingAABAManual ratingAABA02Machine ratingAABAManual ratingAABA03Machine ratingAABAManual ratingACBA11Machine ratingABBAManual ratingAABA12Machine ratingAABAManual ratingABAASome of the experimental data are shown in Table4.The experimental results are very good, but after thinking again, I think it may be related to the following factors: (1) the amount of data in this paper is small, and linear regression analysis is used to fit the model; (2) the average score of the two English teachers is used as the final score of the manual evaluation, and the rounding method makes the score high; (3) this paper uses a four-level scoring method for evaluation, and the “adjacent” in the adjacent agreement rate is defined as the difference between the levels of machine evaluation and manual evaluation by one level, which to a certain extent increases the adjacent agreement rate.In equation (9), the weight of pitch is the largest, the weight of rhythm and intonation is the second, and the weight of the speech rate is the smallest. After arriving at the above experimental results, the author communicated with the English speech experts in our school to furthermore examine the reliability of the multi-covariate pronunciation quality evaluation model proposed in this paper.As shown in Figure4, the pronunciation time of different words shows that the complexity of the scheme in this paper is very low and takes little time. This is because we preprocess the speech, making the data easy to use.Figure 4
Pronunciation time of different words.


## 6. Conclusions

In terms of speaking evaluation, English speaking exams are still dominated by manual scoring with a strong subjective will, varying standards, and slow speed, with poor repeatability and stability. To address these problems, we take the english speech as the research object, improve the traditional computerized English pronunciation quality evaluation method, consider multi-covariate evaluation indexes, such as pitch, speed, rhythm, and intonation, and uses regression analysis to build a reasonable and objective English pronunciation quality evaluation model.



---

*Source: 3198565-2022-03-10.xml*