# Design of English Translation Model Based on Recurrent Neural Network

Authors: Xiaohui Wang
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

In order to improve the accuracy and stability of English translation, this paper proposes an English translation model based on recurrent neural network. Based on the end-to-end encoder-decoder architecture, a recursive neural network (RNN) English machine translation model is designed to promote machine autonomous learning features, transform the distributed corpus data into word vectors, and directly map the source language and target language through the recurrent neural network. Selecting semantic errors to construct the objective function during training can well balance the influence of each part of the semantics and fully consider the alignment information, providing a strong guidance for the training of deep recurrent neural networks. The experimental results show that the English translation model based on recurrent neural network has high effectiveness and stability. Compared with the baseline system, it has improved about 1.51–1.86 BLEU scores.Conclusion. The model improves the performance and quality of English machine translation model, and the translation effect is better.


---

## Body

## 1. Introduction

Since the machine translation task was put forward, a large number of resources have been put into related research, and a wealth of research results have been produced. Machine translation systems have also developed from rule systems based on dictionaries and manual rules, case-based translation systems to statistical machine translation systems. Statistical machine translation can quickly learn translation knowledge from a large amount of data [1]. It has obvious advantages in robustness and translation quality compared with rule-based and case-based methods. It has become the mainstream in machine translation research (as shown in Figure 1). Over the last 10 years, statistical machine translation systems have been intensively studied and a variety of statistical translation models have been proposed, the most important of which is the suggestion of phrase-based translation models. A word-based statistical translation system begins with a lexical correspondence between languages and extracts parts of the translated language from the data as the basic unit of machine translation because a translated phrase pair can directly record a rich translation phenomenon compared to the lexical level. The machine translation system and translation performance have improved significantly [2]. However, phrase-based translation systems also face some problems. Machine translation researchers have done a series of in-depth exploration work. An important attempt is to introduce a large number of lexical and syntactic features to better describe translation phenomena. Vocabulary-based feature is a sparse high-dimensional feature, which is difficult to obtain accurate estimation and prone to overfitting in the case of limited bilingual data, and its ability to describe long-distance, phrase-level translation is limited. Although syntactic features can describe the recursive structure in language to a certain extent, syntactic features depend on the results of syntactic analysis. However, the current syntactic analysis technology is not robust enough to achieve the desired results. In recent years, deep neural network technology has received extensive attention in natural language processing. Deep neural network can automatically learn new representations of various objects in natural language, which puts forward a new idea to solve the problem that sparse features are easy to overfit and lack the ability to describe language structures in natural language processing.Figure 1
Design of English translation model.


## 2. Literature Review

Li and others created the IBM 701 computer to complete the research on English-Russian machine translation. The IBM 701 computer accepted the test of the world’s first machine translation machine and translated several simple Russian sentences into English. This simple test showed the feasibility of people using computers to translate at that time. Since then, people began to study machine translation [3]. Abebe and others deepened the reform of Halliday system grammar and proposed the SCHRDLU model. These new theories and methods of machine translation in the 1970s have opened up a new way of thinking for machine translation [4]. Li and others put forward machine translation models based on statistics and examples, which are based on large-scale real text processing and use corpus processing means to transform corpus into language knowledge base. This is an important revolution in the history of machine translation and also pushes natural language processing to a new stage of rapid development [5]. Aziz and others believe that the basis of language is rules, and the sentences of a language can be derived from its grammar, sentence pattern, and other rules. Therefore, Chomsky gave the theoretical principle of generating sentences from rules. From then on to the 1990s, rule-based machine translation has been the mainstream of machine translation research. Rule-based methods include literal translation, transformational translation, and interlanguage translation [6]. Marquez and others put forward the case-based machine translation method [7]. Its basic idea is to build a case sentence base corresponding to the source language and the target language. For the sentence to be translated, the sentence with the closest distance is matched from the source language corpus, then the target sentence corresponding to the sentence is found, and the target sentence is appropriately modified according to the sentence to be translated, and thus the target sentence of the sentence to be translated is obtained. Kumar and others proposed to integrate the case-based machine translation method with the rule-based method, which greatly improved the translation performance [8].


## 3. Research Methods

### 3.1. Recurrent Neural Network Analysis

In the context of in-depth training, neural networks have unique advantages and play an important role in solving many problems, such as distribution, but neural network transmission has its limitations. The human brain is involved in energy, and distribution is a small part of it [9]. In addition to identifying events, people are able to determine the physical relationship between information resources and information resources, and the length of the data varies. Only a normal neural network can solve these problems.The recursive neural network model is shown in Figure2.Figure 2
Recurrent neural network model.If the intermediate closed-loop link is deleted, it becomes the input layer, potential layer, and release layer in the neural network.vk represents the input at time k; hk represents the hidden form at k time; Wk represents k-time output; U represents the weight matrix from input layer to hidden layer; M represents the weight matrix from hidden layer to hidden layer; and N represents the weight matrix from hidden layer to output layer [10, 11]. When k is 1, set hk to 0, and the specific calculation process is as follows:(1)r1=Uv1+Mh0,h1=fr1,w1=gNh1,where f and g represent the activation function.With the time gradually advancing, the memory stateh1 of time 1 participates in the next prediction activity, as shown in the following formula:(2)r2=Uv2+Mh1,h2=fr2,w2=gNh2.Thus, the specific state at timet is obtained, as shown in the following formula:(3)rk=Uvk+Mhk,hk=frk,wk=gNhk.During training, recurrent neural network is prone to gradient explosion or disappearance, which leads to the fact that the gradient cannot be transmitted continuously in a long sequence, finally making it difficult for the model to capture a long distance. Under normal conditions for gradient bursts, the gradient threshold can be scientifically determined based on model parameter training and can be stopped immediately if the gradient exceeds a specified threshold [12]. For the missing gradient, several methods can be used to solve this problem. For example, initiate the weight value scientifically so that the whole brain does not select the maximum or minimum to avoid gradient disappearance area; sigmoid and tanh can be replaced by relu function according to function; design with integrated protocols such as long short-term memory (LSTM) or GRU.


### 3.2. Framework of English Translation Model Based on Recurrent Neural Network

Figure3 shows the range of standard English translators based on neural network reconstruction.Figure 3
Model framework.The process uses an organic mix of Nginx and web servers to improve developer’s scalability and reliability. When multiple users request, Nginx not only transmits the request results to the server but also completes multiple requests equal to the maximum number of visits against failure. Intermediate processes include intermediate programs and memory database modules [13]. The data sent by the user are processed on average, and the random data are transferred efficiently and quickly through the memory database. Improving the management and coordination of strategic plans can improve the security and high efficiency of data transmission. The decryption process has two identifiers, the GPU and the CPU. In order to reduce the delay of the model response according to many methods of parallel modeling and hybrid decoding, the parallel response of the model is optimized, and each model has a high degree of parallelism and low lag.


### 3.3. English Machine Translation Model Based on LSTM Attention Embedding

According to the analysis of LSTM network model (as shown in Figure4), the output vector of LSTM network model encoding phase is constant. Therefore, a parallel vector is used to encode the source sentence. In actual English machine translation, the input English sequence is a sequence of indefinite length, which leads to the problem that the model cannot fully match the English input sequence when the standard LSTM model is used for machine translation of English, which makes the translation effect unsatisfactory. In addition, because of the different meanings in the translation, showing the sequence of designs in constant numbers with the same amount of attention to the sequence does not help improve the quality of translation [14]. Therefore, in order to solve the above problems, we are studying the LSTM network and carrying out the design of English translation technology based on the LSTM network. First, a set of vectors is used instead of the length to indicate the order of the terms. Then, by dynamically selecting the background vector while creating the temporary object, the translation model pays more attention to the objects that are related to the language during the translation, thus improving the interpretation of the model. The English LSTM translator, built into the monitoring process, consists of three parts: the encoder, the decoder, and the monitoring process. The procedure for calculating the state of the next hidden process of the objective of the model is the same as for the LSTM decoder section, as shown in the following equation:(4)Zi+1=σci,ui,zi,where ui represents the i-th word in the target language sequence and ci represents the background vector of the word i. The back vector of the LSTM model embedded in the face mechanism is not a continuous one but a set of multiple vectors. Therefore, each word in the target language sequence can see its own origin vector.Figure 4
Schematic diagram of LSTM network structure.If the hidden layer state of the encoder at timej is hj, the corresponding background vector can be calculated by(5)Ci=∑j=1Taijhj,where aij represents the weight, i.e., the value of attention to the i-th word of the first language sequence of the i-th word of the sequence of words, which can be calculated by equations (6) and (7).(6)aij=expeij∑k=1Texpeik,(7)eij=azi,hj,where a is a function used to measure the current lag process of the target language and the comparison of word order lag states, and it can be calculated by(8)eij=vTtanhWzZi+Whhj,where v, Wz, and Wh represent the model parameters to be learned. By improving the monitoring procedures in the LSTM network, the model can measure the weight difference at the end of the language, which solves the problem of distance from the LSTM model and improves the standard performance [15, 16].


## 3.1. Recurrent Neural Network Analysis

In the context of in-depth training, neural networks have unique advantages and play an important role in solving many problems, such as distribution, but neural network transmission has its limitations. The human brain is involved in energy, and distribution is a small part of it [9]. In addition to identifying events, people are able to determine the physical relationship between information resources and information resources, and the length of the data varies. Only a normal neural network can solve these problems.The recursive neural network model is shown in Figure2.Figure 2
Recurrent neural network model.If the intermediate closed-loop link is deleted, it becomes the input layer, potential layer, and release layer in the neural network.vk represents the input at time k; hk represents the hidden form at k time; Wk represents k-time output; U represents the weight matrix from input layer to hidden layer; M represents the weight matrix from hidden layer to hidden layer; and N represents the weight matrix from hidden layer to output layer [10, 11]. When k is 1, set hk to 0, and the specific calculation process is as follows:(1)r1=Uv1+Mh0,h1=fr1,w1=gNh1,where f and g represent the activation function.With the time gradually advancing, the memory stateh1 of time 1 participates in the next prediction activity, as shown in the following formula:(2)r2=Uv2+Mh1,h2=fr2,w2=gNh2.Thus, the specific state at timet is obtained, as shown in the following formula:(3)rk=Uvk+Mhk,hk=frk,wk=gNhk.During training, recurrent neural network is prone to gradient explosion or disappearance, which leads to the fact that the gradient cannot be transmitted continuously in a long sequence, finally making it difficult for the model to capture a long distance. Under normal conditions for gradient bursts, the gradient threshold can be scientifically determined based on model parameter training and can be stopped immediately if the gradient exceeds a specified threshold [12]. For the missing gradient, several methods can be used to solve this problem. For example, initiate the weight value scientifically so that the whole brain does not select the maximum or minimum to avoid gradient disappearance area; sigmoid and tanh can be replaced by relu function according to function; design with integrated protocols such as long short-term memory (LSTM) or GRU.


## 3.2. Framework of English Translation Model Based on Recurrent Neural Network

Figure3 shows the range of standard English translators based on neural network reconstruction.Figure 3
Model framework.The process uses an organic mix of Nginx and web servers to improve developer’s scalability and reliability. When multiple users request, Nginx not only transmits the request results to the server but also completes multiple requests equal to the maximum number of visits against failure. Intermediate processes include intermediate programs and memory database modules [13]. The data sent by the user are processed on average, and the random data are transferred efficiently and quickly through the memory database. Improving the management and coordination of strategic plans can improve the security and high efficiency of data transmission. The decryption process has two identifiers, the GPU and the CPU. In order to reduce the delay of the model response according to many methods of parallel modeling and hybrid decoding, the parallel response of the model is optimized, and each model has a high degree of parallelism and low lag.


## 3.3. English Machine Translation Model Based on LSTM Attention Embedding

According to the analysis of LSTM network model (as shown in Figure4), the output vector of LSTM network model encoding phase is constant. Therefore, a parallel vector is used to encode the source sentence. In actual English machine translation, the input English sequence is a sequence of indefinite length, which leads to the problem that the model cannot fully match the English input sequence when the standard LSTM model is used for machine translation of English, which makes the translation effect unsatisfactory. In addition, because of the different meanings in the translation, showing the sequence of designs in constant numbers with the same amount of attention to the sequence does not help improve the quality of translation [14]. Therefore, in order to solve the above problems, we are studying the LSTM network and carrying out the design of English translation technology based on the LSTM network. First, a set of vectors is used instead of the length to indicate the order of the terms. Then, by dynamically selecting the background vector while creating the temporary object, the translation model pays more attention to the objects that are related to the language during the translation, thus improving the interpretation of the model. The English LSTM translator, built into the monitoring process, consists of three parts: the encoder, the decoder, and the monitoring process. The procedure for calculating the state of the next hidden process of the objective of the model is the same as for the LSTM decoder section, as shown in the following equation:(4)Zi+1=σci,ui,zi,where ui represents the i-th word in the target language sequence and ci represents the background vector of the word i. The back vector of the LSTM model embedded in the face mechanism is not a continuous one but a set of multiple vectors. Therefore, each word in the target language sequence can see its own origin vector.Figure 4
Schematic diagram of LSTM network structure.If the hidden layer state of the encoder at timej is hj, the corresponding background vector can be calculated by(5)Ci=∑j=1Taijhj,where aij represents the weight, i.e., the value of attention to the i-th word of the first language sequence of the i-th word of the sequence of words, which can be calculated by equations (6) and (7).(6)aij=expeij∑k=1Texpeik,(7)eij=azi,hj,where a is a function used to measure the current lag process of the target language and the comparison of word order lag states, and it can be calculated by(8)eij=vTtanhWzZi+Whhj,where v, Wz, and Wh represent the model parameters to be learned. By improving the monitoring procedures in the LSTM network, the model can measure the weight difference at the end of the language, which solves the problem of distance from the LSTM model and improves the standard performance [15, 16].


## 4. Result Analysis

### 4.1. Experimental Environment and Parameter Setting

To measure the performance of standardized English translation translators based on the advanced level of LSTM. The evaluation system based on tensorflow framework is developed. Among them, LSTM neural network includes 30,000 large words, 512 large vector word and hidden number, 2 LSTM network set number, 3 general search line, and 128 large package. To improve the quality, the measurements were optimized using the stochastic gradient drop method, the decomposition phase algorithm, and the column search algorithm [17].


### 4.2. Dataset Source and Preprocessing

The study selected 2019 data from the International Speech Language and Translation Assessment Competition (IWSLT) as test data with 220,000 Chinese-English pair sentences, 3 pairs of tests, and 1 pair of small file data [18]. It should be noted that the LSTM English translator cannot be trained directly from the IWSLT 2019 documentation, so the documentation must be converted to word vectors [19]. The study used the term segmentation of data and then applied CBOW to vector segmented data.


#### 4.2.1. Word Segmentation

IWSLT 2019 data contain two parallel sentences in Chinese and English, and the study segmented the experimental data in Chinese and English because of the different ways of segmenting words in Chinese and English. Chinese lexicon uses word segmentation based on statistics [20]. Firstly, according to the composition form of Chinese words, a word is regarded as a combination of multiple fixed words; then, according to the size of the co-occurrence frequency between words in the context of the sentence, the probability of word generation is judged, that is, the credibility of the word. Finally, set the threshold according to the credibility of the word, form the condition of word formation, and determine the word segmentation. For English segmentation, the basic unit of English is the word, so it only needs to be divided directly by space. However, because English has a semicolon, it is necessary to remove semicolons when segmenting words. There are three simple steps you can take to stay away from English. First, rewrite the uppercase and lowercase letters, separating the words and characters at the end of the sentences with spaces, and finally, following the English language completely, write the general sentences using a special template.


#### 4.2.2. Word Vector Representation

Word vectorization refers to the digitization of language symbols, so that language symbols can be input into the model for training and learning. CBOW is used to vectorize words [21]. Assuming that the dictionary size is v, the index set v=0,1,…,v−1 with one-to-one correspondence between the word and the integer in the dictionary is established, and the integer corresponding to the word in the dictionary is the index of the word [22]. If there is a text sequence with a length of T, a time window size of m, and a word at time t of J, the probability of CBOW maximizing the generation of the center word from the background word is as follows:(9)J=∏t=1TPwt−m,…,wt−1,wt+1,…,wt+m.By taking the negative logarithm of the above equation, the loss function can be obtained, i.e., the maximum value of (9) can be calculated by(10)J=∑t=1TlogPwt|wt−m,…,wt+1,…,wt+m.Assuming that the background word vector is expressed asv and the center word vector is expressed as u, then through CBOW training, for the words indexed as i in the dictionary, the vector vi of the word as the background word can be obtained, and the vector as the center word can represent ui.


### 4.3. Evaluation Indicators

This study selected the BLEU value as a measure for the mean quality of the translation design. The greater the cost, the better the translation. The way to calculate BLEU is as follows:(11)BLEU=BP×exp∑n=1Nwnlog10pn,where BP represents the penalty factor; N represents the longest tuple length, usually 4; n represents the number of tuples; wn represents the weight of tuple n; and pn represents the ratio of tuple n.


### 4.4. Model Validation

To measure the effectiveness of the translation standards according to the LSTM color scale, the LSTM model and the LSTM color scale were used to train the experimental data, and the results are shown in Figure5. In Figure 5, the value of BLEU centralized LSTM model is higher than that of the standard LSTM model, which shows that the middle LSTM model has better translation values in longer sentences and demonstration performance. Performance is improving [23].Figure 5
Performance of attention mechanism (RNN changed to LSTM).


### 4.5. Model Comparison

To learn about the effectiveness of LSTM standard English translators, learn how to use LSTM English translator models, RNN English translator models, and GRU-attention English translator models and apply translation. An English translation compares the test data. The training packages and results are shown in Figure6. As shown in Figure 6, the BLEU value of the standard English translator with LSTM is analyzed on machine development. During the test, it is found that the Bleu value of the English translation model based on LSTM is higher than that of the English translation model based on the traditional RNN. The GRU-attention and recursion-based neural network translation model is efficient and stable and improves the score from 1.51 to 1.86 BLEU compared to the core standard.Figure 6
BLEU values of different translation models on the experimental dataset.


## 4.1. Experimental Environment and Parameter Setting

To measure the performance of standardized English translation translators based on the advanced level of LSTM. The evaluation system based on tensorflow framework is developed. Among them, LSTM neural network includes 30,000 large words, 512 large vector word and hidden number, 2 LSTM network set number, 3 general search line, and 128 large package. To improve the quality, the measurements were optimized using the stochastic gradient drop method, the decomposition phase algorithm, and the column search algorithm [17].


## 4.2. Dataset Source and Preprocessing

The study selected 2019 data from the International Speech Language and Translation Assessment Competition (IWSLT) as test data with 220,000 Chinese-English pair sentences, 3 pairs of tests, and 1 pair of small file data [18]. It should be noted that the LSTM English translator cannot be trained directly from the IWSLT 2019 documentation, so the documentation must be converted to word vectors [19]. The study used the term segmentation of data and then applied CBOW to vector segmented data.


### 4.2.1. Word Segmentation

IWSLT 2019 data contain two parallel sentences in Chinese and English, and the study segmented the experimental data in Chinese and English because of the different ways of segmenting words in Chinese and English. Chinese lexicon uses word segmentation based on statistics [20]. Firstly, according to the composition form of Chinese words, a word is regarded as a combination of multiple fixed words; then, according to the size of the co-occurrence frequency between words in the context of the sentence, the probability of word generation is judged, that is, the credibility of the word. Finally, set the threshold according to the credibility of the word, form the condition of word formation, and determine the word segmentation. For English segmentation, the basic unit of English is the word, so it only needs to be divided directly by space. However, because English has a semicolon, it is necessary to remove semicolons when segmenting words. There are three simple steps you can take to stay away from English. First, rewrite the uppercase and lowercase letters, separating the words and characters at the end of the sentences with spaces, and finally, following the English language completely, write the general sentences using a special template.


### 4.2.2. Word Vector Representation

Word vectorization refers to the digitization of language symbols, so that language symbols can be input into the model for training and learning. CBOW is used to vectorize words [21]. Assuming that the dictionary size is v, the index set v=0,1,…,v−1 with one-to-one correspondence between the word and the integer in the dictionary is established, and the integer corresponding to the word in the dictionary is the index of the word [22]. If there is a text sequence with a length of T, a time window size of m, and a word at time t of J, the probability of CBOW maximizing the generation of the center word from the background word is as follows:(9)J=∏t=1TPwt−m,…,wt−1,wt+1,…,wt+m.By taking the negative logarithm of the above equation, the loss function can be obtained, i.e., the maximum value of (9) can be calculated by(10)J=∑t=1TlogPwt|wt−m,…,wt+1,…,wt+m.Assuming that the background word vector is expressed asv and the center word vector is expressed as u, then through CBOW training, for the words indexed as i in the dictionary, the vector vi of the word as the background word can be obtained, and the vector as the center word can represent ui.


## 4.2.1. Word Segmentation

IWSLT 2019 data contain two parallel sentences in Chinese and English, and the study segmented the experimental data in Chinese and English because of the different ways of segmenting words in Chinese and English. Chinese lexicon uses word segmentation based on statistics [20]. Firstly, according to the composition form of Chinese words, a word is regarded as a combination of multiple fixed words; then, according to the size of the co-occurrence frequency between words in the context of the sentence, the probability of word generation is judged, that is, the credibility of the word. Finally, set the threshold according to the credibility of the word, form the condition of word formation, and determine the word segmentation. For English segmentation, the basic unit of English is the word, so it only needs to be divided directly by space. However, because English has a semicolon, it is necessary to remove semicolons when segmenting words. There are three simple steps you can take to stay away from English. First, rewrite the uppercase and lowercase letters, separating the words and characters at the end of the sentences with spaces, and finally, following the English language completely, write the general sentences using a special template.


## 4.2.2. Word Vector Representation

Word vectorization refers to the digitization of language symbols, so that language symbols can be input into the model for training and learning. CBOW is used to vectorize words [21]. Assuming that the dictionary size is v, the index set v=0,1,…,v−1 with one-to-one correspondence between the word and the integer in the dictionary is established, and the integer corresponding to the word in the dictionary is the index of the word [22]. If there is a text sequence with a length of T, a time window size of m, and a word at time t of J, the probability of CBOW maximizing the generation of the center word from the background word is as follows:(9)J=∏t=1TPwt−m,…,wt−1,wt+1,…,wt+m.By taking the negative logarithm of the above equation, the loss function can be obtained, i.e., the maximum value of (9) can be calculated by(10)J=∑t=1TlogPwt|wt−m,…,wt+1,…,wt+m.Assuming that the background word vector is expressed asv and the center word vector is expressed as u, then through CBOW training, for the words indexed as i in the dictionary, the vector vi of the word as the background word can be obtained, and the vector as the center word can represent ui.


## 4.3. Evaluation Indicators

This study selected the BLEU value as a measure for the mean quality of the translation design. The greater the cost, the better the translation. The way to calculate BLEU is as follows:(11)BLEU=BP×exp∑n=1Nwnlog10pn,where BP represents the penalty factor; N represents the longest tuple length, usually 4; n represents the number of tuples; wn represents the weight of tuple n; and pn represents the ratio of tuple n.


## 4.4. Model Validation

To measure the effectiveness of the translation standards according to the LSTM color scale, the LSTM model and the LSTM color scale were used to train the experimental data, and the results are shown in Figure5. In Figure 5, the value of BLEU centralized LSTM model is higher than that of the standard LSTM model, which shows that the middle LSTM model has better translation values in longer sentences and demonstration performance. Performance is improving [23].Figure 5
Performance of attention mechanism (RNN changed to LSTM).


## 4.5. Model Comparison

To learn about the effectiveness of LSTM standard English translators, learn how to use LSTM English translator models, RNN English translator models, and GRU-attention English translator models and apply translation. An English translation compares the test data. The training packages and results are shown in Figure6. As shown in Figure 6, the BLEU value of the standard English translator with LSTM is analyzed on machine development. During the test, it is found that the Bleu value of the English translation model based on LSTM is higher than that of the English translation model based on the traditional RNN. The GRU-attention and recursion-based neural network translation model is efficient and stable and improves the score from 1.51 to 1.86 BLEU compared to the core standard.Figure 6
BLEU values of different translation models on the experimental dataset.


## 5. Conclusion

In conclusion, the new concept of English machine translation template recursion and LSTM color-level innovation is the introduction of a focused mechanism of the standard design of LSTM English translation to improve the presentation content language of native language information, thereby improving the translation and interpretation efficiency of the English translation system. Compared with other English translators, LSTM standard English translation system has better performance and can perform more efficient translation.



---

*Source: 5177069-2022-08-25.xml*