# Oral Business English Recognition Method Based on RankNet Model and Endpoint Detection Algorithm

Authors: Xia Xu; Kunxue Xiao
Journal: Journal of Sensors
 (2022)

---

## Abstract

The prosodic feature of “stressed syllables” plays an important role in business English learning. For the majority of nonnative language learners learning English, the pronunciation of native language can easily affect the expression of spoken English. For those who cooperate and communicate in business, if they want to have more idiomatic and accurate spoken English, they need to control the necessary condition of stress first. This puts forward new requirements for the existing speech recognition technology: more accurate recognition of stressed syllables, reading complex and different emotional colors, and helping to correct oral expression. The experimental results show that (1) the higher the Fisher Ratio of the feature, the easier it is to distinguish the stressed syllables; adjusting the weight of features can effectively improve the recognition accuracy. (2) The recognition rate will decrease with the increase of noise. (3) InwMS method can distinguish features better than min-max method, but the recognition rate of stressed syllables is very low. Linear recognition based on single feature is not recommended. (4) The error rates of the two methods are 20.6% and 19.32%. If any feature of the fusion feature is removed, the error recognition rate of the model will increase by at least 3%. (5) For sentence stress recognition based on fusion features, the recognition error rate on RankNet model is as high as 42.51%. The final result of system operation is good, simple, and convenient.


---

## Body

## 1. Introduction

With the rapid development of economic globalization, neither the country nor the individual can stay out of it, and the whole world is closely linked. If people want to seek better cooperation, mutual communication and help are essential and important factors. Combined with the rapid development of computer technology in recent decades, using intelligent systems, software, and other methods to assist in learning business English has been widely popular in people’s study, work, and life. Due to the wide circulation of English, it occupies an important position in economic and trade exchanges. How to learn oral English well and use it in business cooperation is a hot topic that occupies the study list for a long time. In this paper, researchers through the study of a lot of relevant literature summed up the previous experiments to get some empirical conclusions and make use of the discovery to create new value.Compare the syllables and morphemes of simple words and compound words [1]. It is found that lexical parameters and audibility have influence on word speech recognition in monosyllabic speech test [2]. The quality of final vowels of words plays an important role in English major stress [3]. Acoustic cues of word stress promote offline and online language processing to a certain extent [4], because speakers are sensitive to auditory feedback in stressed and unstressed syllables [5]. The effects of stress are analyzed for syllable position, pitch, duration, and amplitude [6]. Using four deep neural network architectures, the problems of phonetic symbols, lexical stress assignment, and syllability can be solved [7]. In order to improve the recognition ability of accented syllables in spoken English, a word stress recognition model is established based on natural language processing and endpoint detection algorithm [8]. According to syllable stress, transfer learning and CNN neural network are used to recognize English phonetic emotion [9]. When the speech cluster is in the middle of the word, the stress has a great influence, and the recognition is more accurate [10]. If you realize stress placement and emphasize syllables when learning English, it is helpful to learn English words [11]. The hybrid algorithm based on HMM and clustering is used to recognize English speech, which improves the recognition speed by 20.1% [12]. After experiments under different parameters, the CNN algorithm optimized by LPC has higher accuracy of English speech recognition [13]. CTC word model for acoustic detection of speech recognition needs to train more data and has better performance [14]. Applying speech recognition to college English listening teaching can effectively improve students’ listening [15].The above documents are explained from different methods and angles. It not only shows the importance and related characteristics of learning oral stress, but also provides us with excellent ideas and good experimental methods for our research. Although there are many speech research results that are good, but so far the speech system targeted is not strong, and speech recognition technology is not perfect and mature. In addition to the inability to accurately identify the position of learners’ stressed pronunciation, learners cannot get a reasonable assessment report of stressed pronunciation, which cannot effectively achieve the purpose of oral business English learning. There is not yet a successful business English learning system for recognizing stressed syllables in spoken English. Most systems are faced with the problems of incomplete recognition function and imperfect performance. Therefore, this paper uses natural language processing and endpoint detection algorithm to integrate the technology of the system, so as to have a better application in speech recognition of stressed syllables.


## 2. Theoretical Basis

### 2.1. Oral Business English

Business English [16], as the name implies, mainly involves business interaction. Under the trend of internationalization, the purpose of this type of language is to help learners adapt to work and life in the workplace. In addition to simple and basic English language learning, learners not only need to improve their English expression level, but also need to understand business terminology, western business management concepts, and conventions; Foreigners’ thinking mode and psychological state are different from our own country, which involves the conceptual requirements at the cultural level. If you want to do business with foreigners well, you need to know the ways and means to cooperate with them and respect their business requirements and standards.Therefore, the functionality of “spoken language” is very important, which has three main components: words, grammar, and sentences. Oral fluency and pronunciation accuracy are very important in real interpersonal communication. As it happens, English is a language of stress and rhythm. It is self-evident that stress plays an important role in learning words and sentences, which helps to lay a solid foundation for rhythm and intonation, and it includes important grammatical and semantic characteristics. Some literature also well illustrates that the core of mastering business English and oral English learning is the stressed syllables of words and sentences. It is shown Figure1:Figure 1
Factor characteristics of stress.


### 2.2. Natural Language Processing

Natural language processing [17] (NLP) is an interdisciplinary subject of intelligence science and English. For machines, digital information is its expression. Through the bridge of NLP, we can create a way for human beings to communicate with machines, so that human beings can understand what machines are communicating and expressing. However, NLP also has five major difficulties: (1) language has no simple and intuitive rules. Its laws are complicated, and it is difficult for ordinary people to understand and observe. (2) Openness of English can be interconnected with other languages. (3) Language is an open collection. Users can discover and create new expressions by themselves. (4) Language is often associated with concrete practice, and it depends on the knowledge of various disciplines. (5) If you want to use a language well and smoothly, you need to base yourself on the environment and context of the language, as shown in Table 1:Table 1
Four applications of NLP.
Typical applications of NLPEmotional analysis [18]The network is full of complex and redundant text information. These messages have different effects for different people or machines. However, if you make use of these texts. It can be found that the emotions they express are nothing more than positive or negative feedbackMachine translationLanguage and language need to be translated before undifferentiated communication can be realized. Nowadays, the application of machine translation is mature, such as Google translation and Baidu translationChat robot [19]It represents the future intelligent application, which promotes the application of chat robots to produce more new values. Using this kind of robot, smart home, intelligent entertainment, and other aspects glow with new vitalitySpeech recognition [20]The application of speech recognition has entered thousands of households and is widely used in medical treatment, communication, home appliances, games, and other industries. Whether it is the voice-to-text function in QQ or WeChat, voice navigation for cars etc., its use is not a complex laboratory high-tech, but a national-level application


### 2.3. Endpoint Detection Algorithm

Speech signal processing is very important. Endpoint detection algorithm technology is used to distinguish speech and nonspeech signals, so as to accurately determine the starting point and ending point of speech. In reality, the real environment of speech recognition is easily affected by noise, so the endpoint detection algorithm should reduce the interference of mute and noise. If the detection of speech signal endpoint is not accurate, it is easy to cause recognition errors, which directly determines the success or failure of the whole speech recognition work in some aspects. The performance of speech recognition can be improved effectively if the collection amount of speech data is effectively processed and the processing time is saved.(1)
Short-term average energy [21](1)En=∑m⟶−∞∞xm⋅ωn−m2.Forhn=ω2n, there are
(2)En=∑m⟶−∞∞x2m⋅hn−m.Short-term average amplitude:(3)Mn=∑m⟶−∞∞xm⋅ωn−m,where ωn is Hamming window and hn can realize framing processing. When the value obtained by En is large, it can be applied to the voiced segment, and when the value obtained by En is small, it can be applied to the unvoiced segment, distinguishing the starting point and the ending point of the speech signal. En and Mn reflect the strength of the speech signal.
(2)
Short-term average zero-crossing rate(4)Zn=∑m⟶−∞∞sgnxn−sgnxn−1⋅ωn−m,sgnxn=1,xn≥0,−1,xn<0.General take:(5)ωn=12N,0≤n≤N−1,0,else.Threshold zero crossing rate [22]:
(6)Zn=12∑m⟶−∞∞sgnxn−T−sgnxn−1−T+sgnxn+T−sgnxn−1+T⋅ωn−m.


### 2.4. Speech Feature Extraction

(1)
Energy characteristics(7)Qn=Txmwn−m.(2)
Duration characteristics. Longer syllables can be judged as stressed syllables(3)
Fundamental frequency characteristics [23]Discrete signal sequence:(8)Rk=∑m⟶−∞∞xm⋅xm+k.Random signal sequence:(9)Rk=limN⟶−∞12N+1∑m⟶−N∞xm⋅xm+k.Autocorrelation function:(10)xk=xn+NP,Rk=Rk+NP.(4)
LPC [24](11)Ri=∑n=−∞+∞Swn+iSwn,Hz=Sz/Uz=G/1−∑k=1Pαkz−k,Sz=∑k=1pαkSn−k+Gun.(5)
Mel cepstrum coefficientsOutput of triangular filter:(12)Yi=∑k=Fi−1Fik−Fi−1Fi−Fi−1Xk+∑k=Fi+1Fi+1Fi−1−kFi+1−FiXk,i+1,2,⋯,24.DCT Transform [25]:
(13)Ck=∑j=124ilogYjcoskj−12π24,k=1,2,⋯,p.First-order difference cepstrum calculation:(14)Δc1m=∑k=−22kcl−km,1≤m≤p.Mel frequency conversion relation:(15)Mel=ln1+f700⋅1000ln1+1000/700.(6)
Fractal dimension feature extractionBox counting dimension:(16)DBs=limε⟶0lnNεln1/ε.Minkowski dimension:(17)DM=limε⟶02−lnAGεlnε,AGε=area∪t∈FGεt,ft.


### 2.5. Algorithm Evaluation Index

(1)
Recognition of stressed syllables of words(18)Rright=Nright/Nright+Nerror,Rerror=Nerror/Nright+Nerror,Rmiss=Nmiss/Nright+Nerror+Nmiss.(2)
Recognition of sentence stress(19)Ri=NiN.Among them,Rright,Rerror,Rmiss,Ri represents the correct recognition rate, wrong recognition rate, omission rate, and rereading recognition accuracy of evaluation indicators, respectively.


## 2.1. Oral Business English

Business English [16], as the name implies, mainly involves business interaction. Under the trend of internationalization, the purpose of this type of language is to help learners adapt to work and life in the workplace. In addition to simple and basic English language learning, learners not only need to improve their English expression level, but also need to understand business terminology, western business management concepts, and conventions; Foreigners’ thinking mode and psychological state are different from our own country, which involves the conceptual requirements at the cultural level. If you want to do business with foreigners well, you need to know the ways and means to cooperate with them and respect their business requirements and standards.Therefore, the functionality of “spoken language” is very important, which has three main components: words, grammar, and sentences. Oral fluency and pronunciation accuracy are very important in real interpersonal communication. As it happens, English is a language of stress and rhythm. It is self-evident that stress plays an important role in learning words and sentences, which helps to lay a solid foundation for rhythm and intonation, and it includes important grammatical and semantic characteristics. Some literature also well illustrates that the core of mastering business English and oral English learning is the stressed syllables of words and sentences. It is shown Figure1:Figure 1
Factor characteristics of stress.


## 2.2. Natural Language Processing

Natural language processing [17] (NLP) is an interdisciplinary subject of intelligence science and English. For machines, digital information is its expression. Through the bridge of NLP, we can create a way for human beings to communicate with machines, so that human beings can understand what machines are communicating and expressing. However, NLP also has five major difficulties: (1) language has no simple and intuitive rules. Its laws are complicated, and it is difficult for ordinary people to understand and observe. (2) Openness of English can be interconnected with other languages. (3) Language is an open collection. Users can discover and create new expressions by themselves. (4) Language is often associated with concrete practice, and it depends on the knowledge of various disciplines. (5) If you want to use a language well and smoothly, you need to base yourself on the environment and context of the language, as shown in Table 1:Table 1
Four applications of NLP.
Typical applications of NLPEmotional analysis [18]The network is full of complex and redundant text information. These messages have different effects for different people or machines. However, if you make use of these texts. It can be found that the emotions they express are nothing more than positive or negative feedbackMachine translationLanguage and language need to be translated before undifferentiated communication can be realized. Nowadays, the application of machine translation is mature, such as Google translation and Baidu translationChat robot [19]It represents the future intelligent application, which promotes the application of chat robots to produce more new values. Using this kind of robot, smart home, intelligent entertainment, and other aspects glow with new vitalitySpeech recognition [20]The application of speech recognition has entered thousands of households and is widely used in medical treatment, communication, home appliances, games, and other industries. Whether it is the voice-to-text function in QQ or WeChat, voice navigation for cars etc., its use is not a complex laboratory high-tech, but a national-level application


## 2.3. Endpoint Detection Algorithm

Speech signal processing is very important. Endpoint detection algorithm technology is used to distinguish speech and nonspeech signals, so as to accurately determine the starting point and ending point of speech. In reality, the real environment of speech recognition is easily affected by noise, so the endpoint detection algorithm should reduce the interference of mute and noise. If the detection of speech signal endpoint is not accurate, it is easy to cause recognition errors, which directly determines the success or failure of the whole speech recognition work in some aspects. The performance of speech recognition can be improved effectively if the collection amount of speech data is effectively processed and the processing time is saved.(1)
Short-term average energy [21](1)En=∑m⟶−∞∞xm⋅ωn−m2.Forhn=ω2n, there are
(2)En=∑m⟶−∞∞x2m⋅hn−m.Short-term average amplitude:(3)Mn=∑m⟶−∞∞xm⋅ωn−m,where ωn is Hamming window and hn can realize framing processing. When the value obtained by En is large, it can be applied to the voiced segment, and when the value obtained by En is small, it can be applied to the unvoiced segment, distinguishing the starting point and the ending point of the speech signal. En and Mn reflect the strength of the speech signal.
(2)
Short-term average zero-crossing rate(4)Zn=∑m⟶−∞∞sgnxn−sgnxn−1⋅ωn−m,sgnxn=1,xn≥0,−1,xn<0.General take:(5)ωn=12N,0≤n≤N−1,0,else.Threshold zero crossing rate [22]:
(6)Zn=12∑m⟶−∞∞sgnxn−T−sgnxn−1−T+sgnxn+T−sgnxn−1+T⋅ωn−m.


## 2.4. Speech Feature Extraction

(1)
Energy characteristics(7)Qn=Txmwn−m.(2)
Duration characteristics. Longer syllables can be judged as stressed syllables(3)
Fundamental frequency characteristics [23]Discrete signal sequence:(8)Rk=∑m⟶−∞∞xm⋅xm+k.Random signal sequence:(9)Rk=limN⟶−∞12N+1∑m⟶−N∞xm⋅xm+k.Autocorrelation function:(10)xk=xn+NP,Rk=Rk+NP.(4)
LPC [24](11)Ri=∑n=−∞+∞Swn+iSwn,Hz=Sz/Uz=G/1−∑k=1Pαkz−k,Sz=∑k=1pαkSn−k+Gun.(5)
Mel cepstrum coefficientsOutput of triangular filter:(12)Yi=∑k=Fi−1Fik−Fi−1Fi−Fi−1Xk+∑k=Fi+1Fi+1Fi−1−kFi+1−FiXk,i+1,2,⋯,24.DCT Transform [25]:
(13)Ck=∑j=124ilogYjcoskj−12π24,k=1,2,⋯,p.First-order difference cepstrum calculation:(14)Δc1m=∑k=−22kcl−km,1≤m≤p.Mel frequency conversion relation:(15)Mel=ln1+f700⋅1000ln1+1000/700.(6)
Fractal dimension feature extractionBox counting dimension:(16)DBs=limε⟶0lnNεln1/ε.Minkowski dimension:(17)DM=limε⟶02−lnAGεlnε,AGε=area∪t∈FGεt,ft.


## 2.5. Algorithm Evaluation Index

(1)
Recognition of stressed syllables of words(18)Rright=Nright/Nright+Nerror,Rerror=Nerror/Nright+Nerror,Rmiss=Nmiss/Nright+Nerror+Nmiss.(2)
Recognition of sentence stress(19)Ri=NiN.Among them,Rright,Rerror,Rmiss,Ri represents the correct recognition rate, wrong recognition rate, omission rate, and rereading recognition accuracy of evaluation indicators, respectively.


## 3. Recognition of Stressed Syllables Based on RankNet

### 3.1. Speech Recognition Process

#### 3.1.1. NLP Corpus Preprocessing

The original speech data of NLP needs to be preprocessed. Mark the grammatical categories of words in sentences. Finally, the proper nouns in the text are identified and divided into blocks. First of all, the English is preprocessed to achieve the desired effect, and then after data cleaning. The text is segmented by using the word segmentation tool. Then the main part of the word is extracted, which is a process of getting the root after removing the prefix of the word. Word form is to use dictionaries to restore complex words to the most basic forms. Then, the grammatical categories of words in sentences are marked. Finally, the proper nouns in the text are identified and divided into blocks, as shown in Figure2:Figure 2
Steps of English corpus preprocessing.


#### 3.1.2. Stress Recognition Process

Compared with the traditional stressed syllable recognition process, it is relatively simple, and its functions are not perfect. In this paper, English words and sentences should be stressed and recognized. Therefore, we specially set up a two-layer stress recognition process, as shown in Figures3 and 4:Figure 3
Layer 1 process: word stress recognition.Figure 4
Layer 2 process: statement stress recognition.The first layer mainly tests words. After feature extraction, phonemes are aligned according to time, and phoneme corpus is labeled to form training sample data. The second layer is sentence stress recognition. When the first layer recognizes the word with each stressed syllable, the stressed syllable kernel of the word is taken out to participate in the recognition of sentence stress.


### 3.2. RankNet Model

RankNet is based on artificial neural network. It mainly solves the ranking from the perspective of probability. This self-learning technique forms a pair of data functions. Its basic processing unit is neuron. It consists of an outputyi and a large number of input objects xi:
(20)Yjt=f∑i=1nwjixi−θj.RankNet’s network model is shown in Figure5:Figure 5
RankNet network architecture.RankNet uses gradient descent algorithm and uses iterative operation to add hidden nodes to increase the adjustable parameters, thus obtaining the approximate exact solution, as shown in Table2:Table 2
RankNet.
RankNetBP neural networkThe input layer of RankNet is mainly composed of pairs of feature vectorsThe input layer of typical BP neural network is very different, that is, feature vectors one by oneThe loss function is mainly the output cross entropy of paired inputs. In back propagation, the output cross entropy loss function of a pair of input eigenvectors is used. According to the gradient of the connection weights of each layer, the connection weights between the input layer and the hidden layer and between the hidden layer and the output layer are adjustedFor BP neural network, its typical loss function is minimum mean square error. The interlayer connection weight is updated according to the error value between the actual output and the expected output


### 3.3. Speech Endpoint Detection

Speech endpoint detection is mainly used in silence, transition, sound and end stages. At different stages, their state transitions are different, whereEH,EL, represent the high and low thresholds value of the short-term energy; RH,RL represent the high and low thresholds of the autocorrelation function value. As shown in Figure 6:Figure 6
Endpoint detection state transition.One threshold is relatively low and sensitive to signal changes; one threshold is higher. The dual-threshold endpoint detection algorithm is mainly based on short-term energy and zero-crossing rate fusion algorithm. Two different thresholds can be determined for speech endpoint detection by these two algorithms. One threshold is relatively low and sensitive to signal changes; one threshold is relatively high; when the double threshold is exceeded for a period of time, the two are properly matched, which means that the starting endpoint of the speech signal can be judged.(21)ZT=minIF,zc+2fzc,ETL=minI1,I2,ETU=5ETL,I1=0.03EMAX−EMIN+EMIN,I2=4EMIN.Among them,IF is generally 25, which represents the empirical value; zc and fzc represent the mean and standard deviation of the zero-crossing rate, respectively. The short-term average energy or average amplitude of each frame is recorded as E; the maximum value and the minimum value are recorded as EMAX and EMIN, respectively.DTW algorithm can save computing time, as shown in Figure7:Figure 7
Efficient DTW algorithm.


### 3.4. Stress Recognition System

In this chapter, we mainly explain the overall framework of stressed syllables in the system. The final system structure diagram includes three sections. The first section is the feature extraction module, which mainly provides preliminary judgment and evaluation results. This module is the data basis of stressed syllables. The second section is a single feature analysis module, in which stressed syllables are judged by linear discriminant function; fractal dimension curve marks position. Finally, the location information is sent to the final section of the speech evaluation. This section can specifically train and analyze the evaluation results of statistics, as shown in Figure8:Figure 8
Improved system architecture framework based on Sphinx-4.


## 3.1. Speech Recognition Process

### 3.1.1. NLP Corpus Preprocessing

The original speech data of NLP needs to be preprocessed. Mark the grammatical categories of words in sentences. Finally, the proper nouns in the text are identified and divided into blocks. First of all, the English is preprocessed to achieve the desired effect, and then after data cleaning. The text is segmented by using the word segmentation tool. Then the main part of the word is extracted, which is a process of getting the root after removing the prefix of the word. Word form is to use dictionaries to restore complex words to the most basic forms. Then, the grammatical categories of words in sentences are marked. Finally, the proper nouns in the text are identified and divided into blocks, as shown in Figure2:Figure 2
Steps of English corpus preprocessing.


### 3.1.2. Stress Recognition Process

Compared with the traditional stressed syllable recognition process, it is relatively simple, and its functions are not perfect. In this paper, English words and sentences should be stressed and recognized. Therefore, we specially set up a two-layer stress recognition process, as shown in Figures3 and 4:Figure 3
Layer 1 process: word stress recognition.Figure 4
Layer 2 process: statement stress recognition.The first layer mainly tests words. After feature extraction, phonemes are aligned according to time, and phoneme corpus is labeled to form training sample data. The second layer is sentence stress recognition. When the first layer recognizes the word with each stressed syllable, the stressed syllable kernel of the word is taken out to participate in the recognition of sentence stress.


## 3.1.1. NLP Corpus Preprocessing

The original speech data of NLP needs to be preprocessed. Mark the grammatical categories of words in sentences. Finally, the proper nouns in the text are identified and divided into blocks. First of all, the English is preprocessed to achieve the desired effect, and then after data cleaning. The text is segmented by using the word segmentation tool. Then the main part of the word is extracted, which is a process of getting the root after removing the prefix of the word. Word form is to use dictionaries to restore complex words to the most basic forms. Then, the grammatical categories of words in sentences are marked. Finally, the proper nouns in the text are identified and divided into blocks, as shown in Figure2:Figure 2
Steps of English corpus preprocessing.


## 3.1.2. Stress Recognition Process

Compared with the traditional stressed syllable recognition process, it is relatively simple, and its functions are not perfect. In this paper, English words and sentences should be stressed and recognized. Therefore, we specially set up a two-layer stress recognition process, as shown in Figures3 and 4:Figure 3
Layer 1 process: word stress recognition.Figure 4
Layer 2 process: statement stress recognition.The first layer mainly tests words. After feature extraction, phonemes are aligned according to time, and phoneme corpus is labeled to form training sample data. The second layer is sentence stress recognition. When the first layer recognizes the word with each stressed syllable, the stressed syllable kernel of the word is taken out to participate in the recognition of sentence stress.


## 3.2. RankNet Model

RankNet is based on artificial neural network. It mainly solves the ranking from the perspective of probability. This self-learning technique forms a pair of data functions. Its basic processing unit is neuron. It consists of an outputyi and a large number of input objects xi:
(20)Yjt=f∑i=1nwjixi−θj.RankNet’s network model is shown in Figure5:Figure 5
RankNet network architecture.RankNet uses gradient descent algorithm and uses iterative operation to add hidden nodes to increase the adjustable parameters, thus obtaining the approximate exact solution, as shown in Table2:Table 2
RankNet.
RankNetBP neural networkThe input layer of RankNet is mainly composed of pairs of feature vectorsThe input layer of typical BP neural network is very different, that is, feature vectors one by oneThe loss function is mainly the output cross entropy of paired inputs. In back propagation, the output cross entropy loss function of a pair of input eigenvectors is used. According to the gradient of the connection weights of each layer, the connection weights between the input layer and the hidden layer and between the hidden layer and the output layer are adjustedFor BP neural network, its typical loss function is minimum mean square error. The interlayer connection weight is updated according to the error value between the actual output and the expected output


## 3.3. Speech Endpoint Detection

Speech endpoint detection is mainly used in silence, transition, sound and end stages. At different stages, their state transitions are different, whereEH,EL, represent the high and low thresholds value of the short-term energy; RH,RL represent the high and low thresholds of the autocorrelation function value. As shown in Figure 6:Figure 6
Endpoint detection state transition.One threshold is relatively low and sensitive to signal changes; one threshold is higher. The dual-threshold endpoint detection algorithm is mainly based on short-term energy and zero-crossing rate fusion algorithm. Two different thresholds can be determined for speech endpoint detection by these two algorithms. One threshold is relatively low and sensitive to signal changes; one threshold is relatively high; when the double threshold is exceeded for a period of time, the two are properly matched, which means that the starting endpoint of the speech signal can be judged.(21)ZT=minIF,zc+2fzc,ETL=minI1,I2,ETU=5ETL,I1=0.03EMAX−EMIN+EMIN,I2=4EMIN.Among them,IF is generally 25, which represents the empirical value; zc and fzc represent the mean and standard deviation of the zero-crossing rate, respectively. The short-term average energy or average amplitude of each frame is recorded as E; the maximum value and the minimum value are recorded as EMAX and EMIN, respectively.DTW algorithm can save computing time, as shown in Figure7:Figure 7
Efficient DTW algorithm.


## 3.4. Stress Recognition System

In this chapter, we mainly explain the overall framework of stressed syllables in the system. The final system structure diagram includes three sections. The first section is the feature extraction module, which mainly provides preliminary judgment and evaluation results. This module is the data basis of stressed syllables. The second section is a single feature analysis module, in which stressed syllables are judged by linear discriminant function; fractal dimension curve marks position. Finally, the location information is sent to the final section of the speech evaluation. This section can specifically train and analyze the evaluation results of statistics, as shown in Figure8:Figure 8
Improved system architecture framework based on Sphinx-4.


## 4. Simulation Experiment Analysis

After a series of preliminary design, the specific process and implementation of each module has been described. Our accented syllable recognition system for oral business English is also basically completed.


### 4.1. Feature Selection and Evaluation

The selection of speech features needs a certain judgment basis. We can optimize the distinction between the two categories and judge the influence of features on the recognition of stressed syllables. The higher the Fisher Ratio numerical results, the stronger the ability to distinguish stressed syllables:(22)γFisher=σbetween+σwithin.Divergent summation of such feature data:(23)σbetween=∑i=1cmki−mk2.Divergent sum of such characteristic data for each class of samples:(24)σwithin=∑i=1c1ni∑c∈ωicki−mki2.We tested eight-dimensional feature vectors and measured the recognition ability of stressed syllables of each feature, judge by Fisher Ratio values in training set and test set.(1)
After testing, we can find that Eavg in the third dimension and FDchr in the eighth dimension have higher Fisher Ratio, both of which are above 0.8. Secondly, the second dimension and the fourth dimension have higher values, and the Fisher Ratio values of Emax and Echr characteristics are both above 0.7. The remaining characteristic values are very low. For individual words, the ability to distinguish stressed syllables from unstressed syllables is poor, as shown in Figure9:(2)
After the stress recognition process of the first layer, the sentence stress recognition of the second layer is carried out, and these eight feature types are still tested. We can see that this time, the highest Fisher Ratio value in the test set is the first dimension, with a value as high as 0.4 duration. The highest value in the training set is the eighth dimension, and Fisher Ratio value is as high as 0.16. Different test sets and training sets have different influences on stressed syllables. This means that the recognition ability of stressed syllables in these two dimensions is very strong. However, the recognition ability and effect of other dimensions are weak, as shown in Figure10:Figure 9
Word stress recognition per dimensional feature.Figure 10
Sentence stress recognition per dimensional feature.


### 4.2. Endpoint Detection Comparison

On MATLAB platform, based on efficient DTW speech recognition system, the experimental environment is tested under three conditions: no noise, 5 dB, and 10 dB. In the experiment, 20 volunteers were invited to record the detected voice data. Among them, men and women are 50%, and each person has 10 phonetic materials. The recognition performance of dual-threshold speech endpoint detection algorithm is tested. In order to better illustrate the advantages of the detection algorithm selected in this paper, we will also use the traditional endpoint detection algorithm for data comparison. The recognition rate calculation formula is as follows:(25)Recognitionrate=Totalnumberofsounds‐numberofmisjudgedsoundsTotalspeechnumber.After simulation experiment, the recognition rate of the two algorithms decreases with the increase of noise. In the same environment, the recognition rate of polysyllabic syllables is generally slightly lower than that of monosyllabic syllables. In the case of no noise, the recognition rates of the algorithms are not much different. However, with the increase of noise environment, the syllable recognition rate of this algorithm is obviously improved, and the effect is obviously superior to the traditional algorithm, as shown in Figure11:Figure 11
Recognition rate of two endpoint detection algorithms.


### 4.3. Stress Syllable Recognition

#### 4.3.1. Recognition of Stressed Syllables Based on Linear Discrimination

We tested eight different phonetic features to study which features are the most effective for recognizing stressed syllables. Among them, the evaluation standard of experimental results is the recognition rate of word stress, as shown in Figure12:Figure 12
Recognition result of linear discrimination.The average horizontal line is set at 70%. It can be found from the figure that the minimum and maximum normalization methods are all higher than the horizontal values, and the results are relatively balanced, so it is impossible to compare the vowel phonemes of a word. While MS normalization method is obviously superior, each feature presents a good degree of discrimination, but their recognition rate of stressed syllables is very low, which cannot reflect the situation of stressed syllables well.


#### 4.3.2. Recognition of Stressed Syllables Based on RankNet

In view of the poor results of the former method, we choose the RankNet model established in this paper, which integrates various features instead of adopting a single feature recognition method, so as to compare vowel phonemes of words and improve the recognition rate of stressed syllables, as shown in Figure13:Figure 13
Recognition of stressed syllables by fusing features.The recognition rate of stressed syllables with fused features has been well improved. The error rates of the two methods are 20.6% and 19.32%, respectively. The robustness of the model is good.Using InwMS normalization method, if any feature is removed, the error recognition rate of the model will increase by at least 3%. When dealing with duration, Eavg and FDchr, the error rate is the highest, which exceeds 24%. It can be seen that the recognition distinction of a single feature recognition is not necessarily the same in feature fusion, and their position is not necessarily important. Features are complementary, as shown in Figure14:Figure 14
Fusion feature-recognition result of removing a feature.


#### 4.3.3. Sentence Stress Recognition Based on RankNet

Under INwMS normalization method, the error recognition rates of accented syllables of A, B, and C are compared. A stands forRerror; B stands for RFerror; C stands for RFerror2. On the test set, the reread recognition error rate of RankNet is 42.51%, as shown in Figure 15:Figure 15
Sentence stress recognition result of fused features.


## 4.1. Feature Selection and Evaluation

The selection of speech features needs a certain judgment basis. We can optimize the distinction between the two categories and judge the influence of features on the recognition of stressed syllables. The higher the Fisher Ratio numerical results, the stronger the ability to distinguish stressed syllables:(22)γFisher=σbetween+σwithin.Divergent summation of such feature data:(23)σbetween=∑i=1cmki−mk2.Divergent sum of such characteristic data for each class of samples:(24)σwithin=∑i=1c1ni∑c∈ωicki−mki2.We tested eight-dimensional feature vectors and measured the recognition ability of stressed syllables of each feature, judge by Fisher Ratio values in training set and test set.(1)
After testing, we can find that Eavg in the third dimension and FDchr in the eighth dimension have higher Fisher Ratio, both of which are above 0.8. Secondly, the second dimension and the fourth dimension have higher values, and the Fisher Ratio values of Emax and Echr characteristics are both above 0.7. The remaining characteristic values are very low. For individual words, the ability to distinguish stressed syllables from unstressed syllables is poor, as shown in Figure9:(2)
After the stress recognition process of the first layer, the sentence stress recognition of the second layer is carried out, and these eight feature types are still tested. We can see that this time, the highest Fisher Ratio value in the test set is the first dimension, with a value as high as 0.4 duration. The highest value in the training set is the eighth dimension, and Fisher Ratio value is as high as 0.16. Different test sets and training sets have different influences on stressed syllables. This means that the recognition ability of stressed syllables in these two dimensions is very strong. However, the recognition ability and effect of other dimensions are weak, as shown in Figure10:Figure 9
Word stress recognition per dimensional feature.Figure 10
Sentence stress recognition per dimensional feature.


## 4.2. Endpoint Detection Comparison

On MATLAB platform, based on efficient DTW speech recognition system, the experimental environment is tested under three conditions: no noise, 5 dB, and 10 dB. In the experiment, 20 volunteers were invited to record the detected voice data. Among them, men and women are 50%, and each person has 10 phonetic materials. The recognition performance of dual-threshold speech endpoint detection algorithm is tested. In order to better illustrate the advantages of the detection algorithm selected in this paper, we will also use the traditional endpoint detection algorithm for data comparison. The recognition rate calculation formula is as follows:(25)Recognitionrate=Totalnumberofsounds‐numberofmisjudgedsoundsTotalspeechnumber.After simulation experiment, the recognition rate of the two algorithms decreases with the increase of noise. In the same environment, the recognition rate of polysyllabic syllables is generally slightly lower than that of monosyllabic syllables. In the case of no noise, the recognition rates of the algorithms are not much different. However, with the increase of noise environment, the syllable recognition rate of this algorithm is obviously improved, and the effect is obviously superior to the traditional algorithm, as shown in Figure11:Figure 11
Recognition rate of two endpoint detection algorithms.


## 4.3. Stress Syllable Recognition

### 4.3.1. Recognition of Stressed Syllables Based on Linear Discrimination

We tested eight different phonetic features to study which features are the most effective for recognizing stressed syllables. Among them, the evaluation standard of experimental results is the recognition rate of word stress, as shown in Figure12:Figure 12
Recognition result of linear discrimination.The average horizontal line is set at 70%. It can be found from the figure that the minimum and maximum normalization methods are all higher than the horizontal values, and the results are relatively balanced, so it is impossible to compare the vowel phonemes of a word. While MS normalization method is obviously superior, each feature presents a good degree of discrimination, but their recognition rate of stressed syllables is very low, which cannot reflect the situation of stressed syllables well.


### 4.3.2. Recognition of Stressed Syllables Based on RankNet

In view of the poor results of the former method, we choose the RankNet model established in this paper, which integrates various features instead of adopting a single feature recognition method, so as to compare vowel phonemes of words and improve the recognition rate of stressed syllables, as shown in Figure13:Figure 13
Recognition of stressed syllables by fusing features.The recognition rate of stressed syllables with fused features has been well improved. The error rates of the two methods are 20.6% and 19.32%, respectively. The robustness of the model is good.Using InwMS normalization method, if any feature is removed, the error recognition rate of the model will increase by at least 3%. When dealing with duration, Eavg and FDchr, the error rate is the highest, which exceeds 24%. It can be seen that the recognition distinction of a single feature recognition is not necessarily the same in feature fusion, and their position is not necessarily important. Features are complementary, as shown in Figure14:Figure 14
Fusion feature-recognition result of removing a feature.


### 4.3.3. Sentence Stress Recognition Based on RankNet

Under INwMS normalization method, the error recognition rates of accented syllables of A, B, and C are compared. A stands forRerror; B stands for RFerror; C stands for RFerror2. On the test set, the reread recognition error rate of RankNet is 42.51%, as shown in Figure 15:Figure 15
Sentence stress recognition result of fused features.


## 4.3.1. Recognition of Stressed Syllables Based on Linear Discrimination

We tested eight different phonetic features to study which features are the most effective for recognizing stressed syllables. Among them, the evaluation standard of experimental results is the recognition rate of word stress, as shown in Figure12:Figure 12
Recognition result of linear discrimination.The average horizontal line is set at 70%. It can be found from the figure that the minimum and maximum normalization methods are all higher than the horizontal values, and the results are relatively balanced, so it is impossible to compare the vowel phonemes of a word. While MS normalization method is obviously superior, each feature presents a good degree of discrimination, but their recognition rate of stressed syllables is very low, which cannot reflect the situation of stressed syllables well.


## 4.3.2. Recognition of Stressed Syllables Based on RankNet

In view of the poor results of the former method, we choose the RankNet model established in this paper, which integrates various features instead of adopting a single feature recognition method, so as to compare vowel phonemes of words and improve the recognition rate of stressed syllables, as shown in Figure13:Figure 13
Recognition of stressed syllables by fusing features.The recognition rate of stressed syllables with fused features has been well improved. The error rates of the two methods are 20.6% and 19.32%, respectively. The robustness of the model is good.Using InwMS normalization method, if any feature is removed, the error recognition rate of the model will increase by at least 3%. When dealing with duration, Eavg and FDchr, the error rate is the highest, which exceeds 24%. It can be seen that the recognition distinction of a single feature recognition is not necessarily the same in feature fusion, and their position is not necessarily important. Features are complementary, as shown in Figure14:Figure 14
Fusion feature-recognition result of removing a feature.


## 4.3.3. Sentence Stress Recognition Based on RankNet

Under INwMS normalization method, the error recognition rates of accented syllables of A, B, and C are compared. A stands forRerror; B stands for RFerror; C stands for RFerror2. On the test set, the reread recognition error rate of RankNet is 42.51%, as shown in Figure 15:Figure 15
Sentence stress recognition result of fused features.


## 5. Conclusion

With the rapid increase of business cooperation, there are more and more people who need spoken English. However, the uneven language teaching is extremely lacking, and there is a big blank market for the system software used for oral English assistance. The research results of this paper show that(1)
in feature selection, the higher the Fisher Ratio is, the better it can distinguish stressed syllables(2)
the double-threshold speech endpoint detection algorithm is obviously superior to the traditional algorithm. Among them, the recognition rate decreases with the increase of environmental noise; the recognition rate of polysyllables is slightly lower than that of monosyllables(3)
min-max method cannot compare the vowel phonemes of words; although InwMS method can distinguish features better, the recognition rate of stressed syllables is low. Therefore, the recognition result of linear judgment of a single feature is of little significance(4)
testing the fusion features improves the recognition rate of stressed syllables. The error rates of the two normalization methods are 20.6% and 19.32%, respectively(5)
if any feature is removed, the error recognition rate of the model tested for fusion features will increase by at least 3%. After individual features are removed, the error rate of feature recognition rises linearly(6)
under INwMS normalization method, for sentence stress recognition based on feature fusion, the rereading recognition error rate of RankNet on the test set is as high as 42.51%The established RankNet algorithm model has stable recognition effect and good reliability. The final experimental training is a good practice of combining experiment with theory, which has more obvious advantages than other traditional methods. The method in this paper still has some shortcomings. For example, the phonetic corpus is small in scale; there will still be missed judgments and misjudgments in noise environment. In the future work, we try some ways to further improve the recognition performance, for example, develop distributed systems to reduce maintenance costs; increase the size of the corpus to make the phonetic representation more extensive; further expand and refine the grammar, rhythm, and rhythm of stressed syllables; improve the speech endpoint detection algorithm to overcome the influence of noise. With the continuous research of speech-related topics, its development will be constantly innovated with the development of the times, and more new ideas will be put into practice to strive for breakthrough progress.



---

*Source: 7426303-2022-05-11.xml*