# Research on Cultural Translation Based on Neural Network

Authors: Junya Wang
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

The translation of pragmatic meaning requires the processing of pragmatic implicit meaning, the translator’s responsibility for the readers of the target language to understand the pragmatic meaning, and the domestication and foreignization of cultural images. The way of pragmatic language equivalence is domestication translation to achieve the fluency of language expression. The creation of foreignized translation in order to foster the equitable interchange and dissemination of culture, which is precisely the aim of cultural translation, is the equivalence of the pragmatic meaning of cultural imagery. Neural network (NN) machine translation technology (MTT) simulates the human brain nervous system. Based on deep learning technology, the whole sentence is used as the basic unit of translation, which greatly improves the accuracy of machine translation. This paper selects the relevant texts of Shaanxi’s red tourism culture and uses the NN translation system to translate. It helps to improve the accuracy of the text translation of Shaanxi red tourism culture by analyzing the translation and providing a reference direction for further revision by neural translation users and promote the overseas dissemination of red culture. The part of the source language side that generates a vector representation of a dense and continuous sentence from the input corresponds to the encoder.


---

## Body

## 1. Introduction

In the theory of speech act, pragmatic intention refers to the intention of doing words and deeds, that is, illocutionary force, which is also the pragmatic implicature of speech. Illocutionary force is not only the most important pragmatic purpose of utterance but also the meaning of utterance. In my opinion, in translation activities, context consists of three aspects: cultural context, that is, the cultural background outside the text; situational context, that is, the background knowledge shared by the original reader and the target reader; text context, that is, the information content in a text. Textual context and situational context are shared by both source and target readers, while cultural context varies from language to language. Therefore, pragmatic intention has distinct cultural characteristics. This also concretely explains the view of cultural translation. Translation itself is a cultural issue, and translation studies must transcend the limitations of language and gradually expand from the conversion of pure language literal forms to the translation of cultural connotation, namely, the dynamic interpretation of connotation. It is undoubtedly appropriate to bring translation into the context of cultural studies [1]. Translation is a generalized cultural translation in which the cultural connotation of one language is transformed into another cultural form.Machine translation has always had a large number of users, especially some non-English professionals who prefer to use machine translation. In the early stage, machine translation was mainly based on phrases and statistical translation method [2]. It carried out probability operation through large-scale bilingual or multicorpus and constructed language model according to the similarity degree of various language elements so as to carry out machine translation. Therefore, there were many problems in terms of the accuracy of words and the accuracy of grammar [3]. The quality of its translation is far from that of manual translation in terms of word choice and grammar. With the emergence of NN machine translation technology, the translation quality of machine translation has a qualitative leap [4].In recent years, with the benefit of the “Belt and Road” initiative, Shaanxi’s tourism industry has been booming, and the red tourism culture has developed its own school, so the publicity of red tourism culture has become more and more important. More and more tourism practitioners are using machine translation to translate Chinese texts and make propaganda with the help of this translation [5]. Therefore, NN translation will be more conducive to the external publicity of red tourism culture.NN machine translation (NMT) is a NN model designed by a computer based on the characteristics of the human brain nervous system. The model, like a human brain, has a large number of “neurons” [6]. Each “neuron” undertakes and completes a corresponding simple task, and then these “neurons” cooperate and coordinate to finally complete the complex translation work. At the same time, when each “neuron” completes the task, it first completes the decoding step. During the coding process, some software will simulate the human to obtain the target region that needs attention after passing the global scanning visually, which refers to the attention mechanism. The core objective in the target area is to select the information that is more critical to the current mission objective [7]. This mechanism improves the efficiency and accuracy of machine translation. NN MTT is a kind of deep learning end-to-end large circulation God through network system; to encode the sentence semantic phrases rather than memory, which greatly improve the accuracy of the translation and fluency, it will be a whole sentence as the basic unit of translation, break down barriers of previous translation based on the phrase, and make the translation more accurate and more in line with the language habits of various countries [8].The new machine translation technology has given new vitality to the development of machine translation. In order to understand the application effect of machine translation in practical activities and explore the development prospects of machine translation, the author chooses the translation software Google, Baidu, and Tencent NN translation software which has made achievements in the field of machine translation. Part of the tourism text of Zhaojin, a red tourist attraction in Shaanxi, is used as the test content, three pieces of NN translation software are used to translate it, and their similarities and differences are compared. On the one hand, the translation ability and characteristics of machine translation software can be understood in detail [9]. On the other hand, it provides a correction direction for users who are committed to the translation of red tourism culture and users who often use machine translation [10].Machine translation has brought great convenience to our work and study, but at the same time, because machine translation involves the cross and integration of natural language understanding, artificial intelligence and NN, and other disciplines, it is bound to exist in many technical problems to be further improved and solved. From the above analysis, it is not difficult to see that NN translation has gone beyond the phrase and vocabulary as the translation unit mode. There are still some unsatisfactory aspects of machine translation, but we are very much looking forward to the future of machine translation, and remind ourselves that accurate translation nevertheless requires human participation and that we should not blindly rely on machines when choosing translation software. It is also necessary to choose different translation software for different types of text characteristics. To sum up, machine translation still does not match human translation and still makes major grammatical or word choice mistakes. From the perspective of linguistics, the root cause of these errors lies in the lack of proficiency in the knowledge and understanding of the language system, and the exchange between two languages stays on the surface without going deep into the core of linguistic and cultural differences. From the writing level of machine language, the main reason lies in the deficiency of machine training data, resulting in insufficient information materials stored inside the machine, resulting in mistranslation [11]. In order to solve these problems, the writer believes that the following three aspects can be improved: First, from the technical level, the algorithm of machine in-depth learning should be improved to improve the end-to-end corresponding process and gradually optimize the translation system. Second, a system error correction database is established. Third is interdisciplinary communication so that engineers engaged in natural language processing can cooperate with expert teachers in the field of linguistics to improve and refine the software. At the same time, with the increasingly frequent people-to-people exchanges between countries, the accuracy of machine translation will benefit the external communication of various industries, especially the tourism culture with national characteristics such as red tourism culture.Although neuromachine translation has indeed made great progress in terms of effectiveness, there are still some problems.


### 1.1. The Dictionary Is Incomplete

Neural network machine translation is based on NN, and its dimensions are directly influenced by the size of the dictionary. In order to save space and shorten time, the size of the dictionary is generally under 50,000, and around 30,000 are common. Unregistered words are replaced with UNK identifiers during translation. However, this makes it difficult to capture the source language information accurately and completely. On the target language side, the existence of the UNK identifier can also affect the overall effect of the sentence, making it harder to accurately understand the output statement. The more diverse the parts of speech, the more prominent the problem.


### 1.2. Overtranslation

Attention mechanism is an important point in neural network machine translation (NMT). In the decoding process, each step outputs the target word flexibly by calculating the attention of the target language to each word in the source language. However, there is no auxiliary information in each attention calculation, and there may be a problem that a certain source language word is paid high attention for many times or some words are paid too little attention.


### 1.3. Semantic Inaccuracy of Translation

Word representation in NMT adopts the method of continuous representation. This not only improves the generalization effect of the translation model and the fluency of translation sentences but also tends to cause the problem of semantic inaccuracy of translation.Nowadays, with the continuous improvement of the computing capacity of computers and the explosion of language information, machine translation technology has an increasing impact on our daily life, work, and study. However, how the brain works is still unknown, and neuromachine translation is still far from the goal of “faithfulness, expressiveness, and elegance.” Machine translation remains an important and challenging research area.This paper is an indepth analysis of existing sequence to sequence NN on the basis of machine translation. The subtitles are put forward according to the dataset for improving the model. The objective is to increase the amount of training data and the depth of the optimization of machine translation existing NN algorithm, to further improve for the depth of the short sequence of NMT effect.


## 1.1. The Dictionary Is Incomplete

Neural network machine translation is based on NN, and its dimensions are directly influenced by the size of the dictionary. In order to save space and shorten time, the size of the dictionary is generally under 50,000, and around 30,000 are common. Unregistered words are replaced with UNK identifiers during translation. However, this makes it difficult to capture the source language information accurately and completely. On the target language side, the existence of the UNK identifier can also affect the overall effect of the sentence, making it harder to accurately understand the output statement. The more diverse the parts of speech, the more prominent the problem.


## 1.2. Overtranslation

Attention mechanism is an important point in neural network machine translation (NMT). In the decoding process, each step outputs the target word flexibly by calculating the attention of the target language to each word in the source language. However, there is no auxiliary information in each attention calculation, and there may be a problem that a certain source language word is paid high attention for many times or some words are paid too little attention.


## 1.3. Semantic Inaccuracy of Translation

Word representation in NMT adopts the method of continuous representation. This not only improves the generalization effect of the translation model and the fluency of translation sentences but also tends to cause the problem of semantic inaccuracy of translation.Nowadays, with the continuous improvement of the computing capacity of computers and the explosion of language information, machine translation technology has an increasing impact on our daily life, work, and study. However, how the brain works is still unknown, and neuromachine translation is still far from the goal of “faithfulness, expressiveness, and elegance.” Machine translation remains an important and challenging research area.This paper is an indepth analysis of existing sequence to sequence NN on the basis of machine translation. The subtitles are put forward according to the dataset for improving the model. The objective is to increase the amount of training data and the depth of the optimization of machine translation existing NN algorithm, to further improve for the depth of the short sequence of NMT effect.


## 2. Basic Process of Machine Translation

The whole process of machine translation can be divided into the following three parts: model training, parameter feature tuning, and translation decoding. Model training includes basic processing steps (processing data, establishing correspondence between words and reference translation, and extracting phrases), training language model, and sequencing model. At the stage of parameter feature optimization, the feature weights of the model are constantly updated through the existing bilingual corpus. The translation and decoding module learns the model and feature weights and other translation source statements through model training and parameter feature tuning. The model framework of the whole machine translation is shown in Figure1.Figure 1
Machine translation model framework.The model framework of machine translation is completely discussed. Basic processing processes are included in model training processing data, establishing correspondence between words and reference translation, and extracting phrases. Through model training and parameter feature adjustment, the translation and decoding module learns the model and feature weights, as well as other translation source statements.


### 2.1. Corpus Preprocessing

Before model training, the corpus needs to be transformed into a regular form that meets the input requirements of the model. For irregular corpus, a preprocessing process is required. Corpus preprocessing can be explained from the following three points: word segmentation, named entity recognition, and special vocabulary processing. Because this topic is the study of Chinese-English translation, here, we take Chinese-English bilingual as an example to introduce. Word segmentation is a basic processing process of natural language processing problems, but the quality of word segmentation directly affects the subsequent processing process and has a crucial influence on the final result.


#### 2.1.1. English Word Segmentation Algorithm

English segmentation is much simpler than Chinese segmentation. The basic units of English are single words, and word segmentation usually takes only a few simple steps. English has its inherent characteristics, which can be divided according to the space symbol through regular expression. However, the existence of the, A, and other high-frequency words will make the calculation formula based on word frequency produce a large error. Therefore, such words need to be filtered out. In addition, English also has its unique properties, such as a word in the third person, singular and plural, present continuous, and past tense deformation, and the original word is highly relevant and can be regarded as the same word. This requires a process of stem extraction to combine these different morphs. The process of stem extraction can also be commonly understood as the process of removing the anaphora at the end of English words. The process of stem extraction is relatively simple and can be realized based on rules or mapping tables. Porter stem algorithm is the most common and less complicated algorithm based on suffix stripping. In some natural language processing problems, word form reduction is added, which is the restoration of an arbitrary word form into the original word form with complete meaning. For example, “translated” is translated into “translate.” The commonly used restoration methods can be classified into four categories, among which the most typical restoration methods are those based on dictionaries. This method has high accuracy and the best effect and is widely used in various scenarios. However, it is limited by the size, language, and domain of the dictionary. Rule-based methods require a large variety of rule sets and high complexity and should be used in a small range and usually used together with other methods. The method based on machine learning is a supplement to the dictionary-based method and has a better processing effect for unknown words. Machine learn-based methods, which are free from dictionaries and language knowledge, have attracted more attention. Each method has its own advantages and disadvantages. Therefore, in order to improve accuracy and get more complete results, the hybrid method has obvious advantages and has begun to receive more and more attention [12].


#### 2.1.2. Chinese Word Segmentation Algorithm

The basic unit of the Chinese language is character, and different combinations of characters can end up with very different meanings. Chinese word segmentation is the segmentation of a series of Chinese characters into words that can express the complete meaning. There are three types of Chinese word segmentation methods commonly used at present, which are based on statistics, understanding, and string matching, respectively. The combination of Chinese characters is relatively deterministic. In a large number of corpora, the higher the frequency of some Chinese character combinations, the greater the probability of word formation. The probability or frequency of Chinese character combinations can better reflect the possibility of word formation. Therefore, the word segmentation method based on statistics came into being. The word segmentation method based on statistics calculates the cooccurrence frequency or probability of Chinese character combinations to find out the combinations whose probability or frequency is greater than the set threshold value and identifies them as words. In this way, only the cooccurrence information of the combination of Chinese characters is counted, so there is no need to rely on the dictionary. This kind of method can also be called the statistical word retrieval method. However, some Chinese character combinations judged as words, such as “Zhe yi” and “many of,” are not real words, so there are some limitations due to the poor accuracy of nondictionary segmentation. The actual application of statistical word segmentation is generally based on a high-frequency word dictionary, and on this basis, some new words are detected by the statistical method. This method combines the characteristics of fast and efficient word segmentation and automatic disambiguation without dictionary.


### 2.2. Word Alignment

After preprocessing, structured bilingual data is obtained, which leads to another basic process in natural language processing, word alignment. Word alignment is to establish word-level correspondence between bilingual texts that need to be translated into each other. It is the first step to solve the problem of translation and the basis of decoding. Word alignment is usually represented byi−>j, where the target word at position i corresponds to the source word at position j. Word alignment correspondence can be one-to-one, one-to-many, many-to-one, many-to-many, and more generally, empty and cross, as shown in Figure 2.Figure 2
General word alignment.There are different ways to align words for different prerequisites. If documented training data is available, a supervised approach can be used; if there is no well-marked data, training can be based on unsupervised learning. There are usually heuristic methods based on similarity function and statistical methods based on probabilistic model learning from corpus.Dictionary-based word alignment is based on the high quality of word translation information contained in the dictionary. The dictionary-based approach has good accuracy in the case of nonempty word alignment, but in the real translation context, the coverage of dictionary translation is relatively low due to the diversity of the context and the flexibility of translation. Therefore, if dictionary-based word alignment alone is not sufficient, other methods need to be introduced. There are some words that match up to a certain degree, but not 100 percent. Therefore, fuzzy mechanism can be used to match the whole.The calculation method of the fuzzy matching degree ofC1 and C2 is shown in the following formula:(1)SimC1,C2=2∗C1∩C2C1+C2.Among them,SimC1,C2=2∗C1∩C2/C1+C2 for the number of the intersections between words, C1 and C2 corresponding to the total count of both.If one of the two words is EnglishE and the other is Chinese C, the calculation method is as shown in the following formula:(2)DTSime,c=maxSimd,c+CountSimd,c>h1−1×0.1.In the formula, Count () represents the statistical function of times; DTe represents all translations ofe; h1 represents the threshold of similarity.


### 2.3. Weight Tuning

Machine translation is inseparable from decoding, which is the ultimate step of translation and is directly related to the generation of translation results. Therefore, the parameters related to decoding should be as optimal as possible. The decoding process is carried out by a number of separate modules working together. These modules include language models and sequencing models. Their contribution to decoding varies from large to small. How to determine the weight of these different modules is the core problem of weight tuning.Like many machine learning problems, weight tuning is the process of making parameters as optimal as possible. What is optimal requires a criterion; that is, an objective function is supplied, and the parameters are constantly improved so that the model’s output is as close to the actual outcome as possible. Without any restriction conditions, this objective function can be selected in a wide range, including maximum posterior probability estimation, distance maximization, and other methods. However, the training set and the test set are two different sets, and the model parameters perform well on the training set, but the effect on the test set cannot be guaranteed. Aiming at this problem, we can choose the algorithm of minimum error rate training to solve it.


### 2.4. The Decoding

No matter what translation model is adopted, decoding is an essential step in the process of translation. Decoding refers to traversing the set space of solution on the premise of source language sentence and parameter model, finding the most reasonable translation sentence from it, and taking it as the final translation result of the model. Traversal is a search process; since heuristic depth-first search has been applied to similar problems, decoding traversal can adopt this method. Search behavior is inseparable from the search space, search in the search space, and finding the optimal solution in the search space. The search behavior depends on the calculation results of different modules, and these modules take these results into consideration to determine the next direction; until the translation terminator is generated, the search process ends.Statistics-based machine translation computes a probability. Given a source statement to be input, the translation model produces more than one translation result, but each result has different rationality. The statistics-based model will score each sentence according to its rationality and select a translation result with the highest probability, namely, the highest score, as the output result of the final model.


## 2.1. Corpus Preprocessing

Before model training, the corpus needs to be transformed into a regular form that meets the input requirements of the model. For irregular corpus, a preprocessing process is required. Corpus preprocessing can be explained from the following three points: word segmentation, named entity recognition, and special vocabulary processing. Because this topic is the study of Chinese-English translation, here, we take Chinese-English bilingual as an example to introduce. Word segmentation is a basic processing process of natural language processing problems, but the quality of word segmentation directly affects the subsequent processing process and has a crucial influence on the final result.


### 2.1.1. English Word Segmentation Algorithm

English segmentation is much simpler than Chinese segmentation. The basic units of English are single words, and word segmentation usually takes only a few simple steps. English has its inherent characteristics, which can be divided according to the space symbol through regular expression. However, the existence of the, A, and other high-frequency words will make the calculation formula based on word frequency produce a large error. Therefore, such words need to be filtered out. In addition, English also has its unique properties, such as a word in the third person, singular and plural, present continuous, and past tense deformation, and the original word is highly relevant and can be regarded as the same word. This requires a process of stem extraction to combine these different morphs. The process of stem extraction can also be commonly understood as the process of removing the anaphora at the end of English words. The process of stem extraction is relatively simple and can be realized based on rules or mapping tables. Porter stem algorithm is the most common and less complicated algorithm based on suffix stripping. In some natural language processing problems, word form reduction is added, which is the restoration of an arbitrary word form into the original word form with complete meaning. For example, “translated” is translated into “translate.” The commonly used restoration methods can be classified into four categories, among which the most typical restoration methods are those based on dictionaries. This method has high accuracy and the best effect and is widely used in various scenarios. However, it is limited by the size, language, and domain of the dictionary. Rule-based methods require a large variety of rule sets and high complexity and should be used in a small range and usually used together with other methods. The method based on machine learning is a supplement to the dictionary-based method and has a better processing effect for unknown words. Machine learn-based methods, which are free from dictionaries and language knowledge, have attracted more attention. Each method has its own advantages and disadvantages. Therefore, in order to improve accuracy and get more complete results, the hybrid method has obvious advantages and has begun to receive more and more attention [12].


### 2.1.2. Chinese Word Segmentation Algorithm

The basic unit of the Chinese language is character, and different combinations of characters can end up with very different meanings. Chinese word segmentation is the segmentation of a series of Chinese characters into words that can express the complete meaning. There are three types of Chinese word segmentation methods commonly used at present, which are based on statistics, understanding, and string matching, respectively. The combination of Chinese characters is relatively deterministic. In a large number of corpora, the higher the frequency of some Chinese character combinations, the greater the probability of word formation. The probability or frequency of Chinese character combinations can better reflect the possibility of word formation. Therefore, the word segmentation method based on statistics came into being. The word segmentation method based on statistics calculates the cooccurrence frequency or probability of Chinese character combinations to find out the combinations whose probability or frequency is greater than the set threshold value and identifies them as words. In this way, only the cooccurrence information of the combination of Chinese characters is counted, so there is no need to rely on the dictionary. This kind of method can also be called the statistical word retrieval method. However, some Chinese character combinations judged as words, such as “Zhe yi” and “many of,” are not real words, so there are some limitations due to the poor accuracy of nondictionary segmentation. The actual application of statistical word segmentation is generally based on a high-frequency word dictionary, and on this basis, some new words are detected by the statistical method. This method combines the characteristics of fast and efficient word segmentation and automatic disambiguation without dictionary.


## 2.1.1. English Word Segmentation Algorithm

English segmentation is much simpler than Chinese segmentation. The basic units of English are single words, and word segmentation usually takes only a few simple steps. English has its inherent characteristics, which can be divided according to the space symbol through regular expression. However, the existence of the, A, and other high-frequency words will make the calculation formula based on word frequency produce a large error. Therefore, such words need to be filtered out. In addition, English also has its unique properties, such as a word in the third person, singular and plural, present continuous, and past tense deformation, and the original word is highly relevant and can be regarded as the same word. This requires a process of stem extraction to combine these different morphs. The process of stem extraction can also be commonly understood as the process of removing the anaphora at the end of English words. The process of stem extraction is relatively simple and can be realized based on rules or mapping tables. Porter stem algorithm is the most common and less complicated algorithm based on suffix stripping. In some natural language processing problems, word form reduction is added, which is the restoration of an arbitrary word form into the original word form with complete meaning. For example, “translated” is translated into “translate.” The commonly used restoration methods can be classified into four categories, among which the most typical restoration methods are those based on dictionaries. This method has high accuracy and the best effect and is widely used in various scenarios. However, it is limited by the size, language, and domain of the dictionary. Rule-based methods require a large variety of rule sets and high complexity and should be used in a small range and usually used together with other methods. The method based on machine learning is a supplement to the dictionary-based method and has a better processing effect for unknown words. Machine learn-based methods, which are free from dictionaries and language knowledge, have attracted more attention. Each method has its own advantages and disadvantages. Therefore, in order to improve accuracy and get more complete results, the hybrid method has obvious advantages and has begun to receive more and more attention [12].


## 2.1.2. Chinese Word Segmentation Algorithm

The basic unit of the Chinese language is character, and different combinations of characters can end up with very different meanings. Chinese word segmentation is the segmentation of a series of Chinese characters into words that can express the complete meaning. There are three types of Chinese word segmentation methods commonly used at present, which are based on statistics, understanding, and string matching, respectively. The combination of Chinese characters is relatively deterministic. In a large number of corpora, the higher the frequency of some Chinese character combinations, the greater the probability of word formation. The probability or frequency of Chinese character combinations can better reflect the possibility of word formation. Therefore, the word segmentation method based on statistics came into being. The word segmentation method based on statistics calculates the cooccurrence frequency or probability of Chinese character combinations to find out the combinations whose probability or frequency is greater than the set threshold value and identifies them as words. In this way, only the cooccurrence information of the combination of Chinese characters is counted, so there is no need to rely on the dictionary. This kind of method can also be called the statistical word retrieval method. However, some Chinese character combinations judged as words, such as “Zhe yi” and “many of,” are not real words, so there are some limitations due to the poor accuracy of nondictionary segmentation. The actual application of statistical word segmentation is generally based on a high-frequency word dictionary, and on this basis, some new words are detected by the statistical method. This method combines the characteristics of fast and efficient word segmentation and automatic disambiguation without dictionary.


## 2.2. Word Alignment

After preprocessing, structured bilingual data is obtained, which leads to another basic process in natural language processing, word alignment. Word alignment is to establish word-level correspondence between bilingual texts that need to be translated into each other. It is the first step to solve the problem of translation and the basis of decoding. Word alignment is usually represented byi−>j, where the target word at position i corresponds to the source word at position j. Word alignment correspondence can be one-to-one, one-to-many, many-to-one, many-to-many, and more generally, empty and cross, as shown in Figure 2.Figure 2
General word alignment.There are different ways to align words for different prerequisites. If documented training data is available, a supervised approach can be used; if there is no well-marked data, training can be based on unsupervised learning. There are usually heuristic methods based on similarity function and statistical methods based on probabilistic model learning from corpus.Dictionary-based word alignment is based on the high quality of word translation information contained in the dictionary. The dictionary-based approach has good accuracy in the case of nonempty word alignment, but in the real translation context, the coverage of dictionary translation is relatively low due to the diversity of the context and the flexibility of translation. Therefore, if dictionary-based word alignment alone is not sufficient, other methods need to be introduced. There are some words that match up to a certain degree, but not 100 percent. Therefore, fuzzy mechanism can be used to match the whole.The calculation method of the fuzzy matching degree ofC1 and C2 is shown in the following formula:(1)SimC1,C2=2∗C1∩C2C1+C2.Among them,SimC1,C2=2∗C1∩C2/C1+C2 for the number of the intersections between words, C1 and C2 corresponding to the total count of both.If one of the two words is EnglishE and the other is Chinese C, the calculation method is as shown in the following formula:(2)DTSime,c=maxSimd,c+CountSimd,c>h1−1×0.1.In the formula, Count () represents the statistical function of times; DTe represents all translations ofe; h1 represents the threshold of similarity.


## 2.3. Weight Tuning

Machine translation is inseparable from decoding, which is the ultimate step of translation and is directly related to the generation of translation results. Therefore, the parameters related to decoding should be as optimal as possible. The decoding process is carried out by a number of separate modules working together. These modules include language models and sequencing models. Their contribution to decoding varies from large to small. How to determine the weight of these different modules is the core problem of weight tuning.Like many machine learning problems, weight tuning is the process of making parameters as optimal as possible. What is optimal requires a criterion; that is, an objective function is supplied, and the parameters are constantly improved so that the model’s output is as close to the actual outcome as possible. Without any restriction conditions, this objective function can be selected in a wide range, including maximum posterior probability estimation, distance maximization, and other methods. However, the training set and the test set are two different sets, and the model parameters perform well on the training set, but the effect on the test set cannot be guaranteed. Aiming at this problem, we can choose the algorithm of minimum error rate training to solve it.


## 2.4. The Decoding

No matter what translation model is adopted, decoding is an essential step in the process of translation. Decoding refers to traversing the set space of solution on the premise of source language sentence and parameter model, finding the most reasonable translation sentence from it, and taking it as the final translation result of the model. Traversal is a search process; since heuristic depth-first search has been applied to similar problems, decoding traversal can adopt this method. Search behavior is inseparable from the search space, search in the search space, and finding the optimal solution in the search space. The search behavior depends on the calculation results of different modules, and these modules take these results into consideration to determine the next direction; until the translation terminator is generated, the search process ends.Statistics-based machine translation computes a probability. Given a source statement to be input, the translation model produces more than one translation result, but each result has different rationality. The statistics-based model will score each sentence according to its rationality and select a translation result with the highest probability, namely, the highest score, as the output result of the final model.


## 3. Encoder-Decoder Framework

Since 2013, the end-to-end framework pattern has gradually become popular in the field of NMT, which realizes the direct conversion from source language sequence to target language sequence. End-to-end NMT completely eliminates the process of manual design of translation processes. For example, there are no separate processing processes such as word alignment and syntax tree, but a joint NN is established to complete the processing of natural language tasks under the framework of encoder-decoder. The framework maps a given source language sentence to a dense and continuous vector through an encoder and then obtains a target language sentence from the source language vector through a decoder. This is a very bold new structure. It replaces the linear model in statistical machine translation with the nonlinear model, and there is no hidden structure pipeline, but a single complex NN. Time series network models can also capture historical information without site. Figure3 is the encoder-decoder frame diagram.Figure 3
Encoder-decoder framework.As shown in Figure3, assuming that a Chinese source statement “it is a fine day today” is input, the model first generates the word vector of each word and then generates the sentence vector in sequence through the time series network model. The last symbol represents the sentence terminator. The encoder is the part of the source language side that creates a vector representation of a dense and continuous sentence from the input. The target language end corresponds to the source language end, which has a similar structure to the encoder end and adopts the same RNN series network, but the purpose of the decoder end is to generate translation output. After getting the output statement “The weather today is good,” the decoding part is complete. Since the model is sequential, the output of the current moment depends on the output information of the previous moment.Compared with statistical machine translation, the encoder-decoder framework model has its unique advantages. Statistical machine translation requires experts in the field of natural language processing to design implicit structural features based on translation rules. However, compared with images, the logic of natural language is more abstract and more complex, so it is difficult to construct features to integrate all features information in the language. NMT can avoid this problem by automatically extracting features from corpus through NNs. In addition, the framework can capture long-distance information. Natural languages are highly complex, and expressions of a sentence with the same meaning in different languages may vary greatly. However, to prevent sparse statistics in machine translation, we restrict the search space, and during machine translation as the structure of the index space is often partially informative, this results in statistical machine translation that is correct for individual words but does not meet the criteria in terms of overall syntactic structure. Even so, NMT can improve the fluency and readability of the translation by using RNN networks capable of capturing long-distance dependencies.


### 3.1. CRNN-Embed Word Vector Generation

We need a way of character-specific lexical indication. Here, we make further improvement on the basis of the RNN-Embed model and adopt the combination of CNN and RNN. Firstly, CNN abstraction was used to enhance the characterization of the single character model, and information was supplemented by referring to Highway Network to solve the problem of blocked gradient backflow. On this basis, RNN was used to further enhance the character sequence characterization ability. The length of the filter is the number of upper and lower words that the convolution operation can cover. Although the size of the filter is variable, the window size is usually no more than 5 lines. The convolution process of the word vector is shown in Figure4.Figure 4
Convolution of word vectors.To further avoid the problem of zero vector in the language model, we compensate for the information before inputting the word vector into the language model. The corresponding formulas are shown in equations (3) to (9).(3)zi=σWzxi+Uzgi−1,(4)ri=σWrxi+Urgi−1,(5)gi=tanhWhxi+rtUhgi,(6)gi=1−zigi+zigi−1,(7)wi=σWwxi+Uwgi+Vwi−1,(8)ei−1x=1−wiUcxi+wigi−1,(9)gi=1−wigi.In the above word cutting method,wi is the word cutting switch, and exi-1 is the output of the word vector. wi is directly affected by gi and wi, and exi-1is restricted by wi, the word cutting switch at time i. The word cutting switch determines the closeness between the inputs at time i and the input at time i−1. To ensure that the last moment generates a meaningful vector, its corresponding word vector is forced to be identified as G end.We combine the deformed input and the original input information of the network as the output. The problem of zero vector gradient reflux obstruction can be effectively avoided by information supplement.


### 3.2. Decoder Language Model

Like the encoder, the decoder also has a corresponding language model. The language model in the encoder takes the output of the word vector generating module as the input, adopts the bidirectional GRU on the whole, and improves the structure of GRU by adding an output gate. Finally, the output sequence of the language model combined with the positive and negative outputs is generated. The decoder is similar but different. Although it also adopts the GRU structure, it is only a positive GRU and does not add an output gate. Different from the encoder side, which has determined the entire input sequence at the initial stage, the output of the decoder side is generated step by step in time order, with obvious timing and direction. Therefore, one-way GRU is adopted. In addition, the decoder side is connected to an output module behind the language model, so we can simplify the output gate of the improved GRU. However, if this is all, the encoder and decoder are still independent of each other, and a word alignment module is also required, which abstracts the correspondence between the source language and the target language.The calculation method of the decoder side language model is shown in formula (10) to formula (13).(10)zi=σWzei−1y+Uzsi−1y+Czci,(11)ri=σWrei−1y+Ursi−1y+Crci,(12)hiy=tanhWzei−1y+Uzsi−1y+Czci,(13)siy=1−zisi−1y+zihiy.The unidirectional GRU model is adopted in the decoder, which is mainly because the last output module of the decoder is output one by one in sequence. Each module refers to the previous information and cannot predict the subsequent sequence information at the current moment. Word vector and language model are common parts of encoder and decoder frameworks and also common parts of natural language processing problems. Now both academia and industry are actively exploring it.


## 3.1. CRNN-Embed Word Vector Generation

We need a way of character-specific lexical indication. Here, we make further improvement on the basis of the RNN-Embed model and adopt the combination of CNN and RNN. Firstly, CNN abstraction was used to enhance the characterization of the single character model, and information was supplemented by referring to Highway Network to solve the problem of blocked gradient backflow. On this basis, RNN was used to further enhance the character sequence characterization ability. The length of the filter is the number of upper and lower words that the convolution operation can cover. Although the size of the filter is variable, the window size is usually no more than 5 lines. The convolution process of the word vector is shown in Figure4.Figure 4
Convolution of word vectors.To further avoid the problem of zero vector in the language model, we compensate for the information before inputting the word vector into the language model. The corresponding formulas are shown in equations (3) to (9).(3)zi=σWzxi+Uzgi−1,(4)ri=σWrxi+Urgi−1,(5)gi=tanhWhxi+rtUhgi,(6)gi=1−zigi+zigi−1,(7)wi=σWwxi+Uwgi+Vwi−1,(8)ei−1x=1−wiUcxi+wigi−1,(9)gi=1−wigi.In the above word cutting method,wi is the word cutting switch, and exi-1 is the output of the word vector. wi is directly affected by gi and wi, and exi-1is restricted by wi, the word cutting switch at time i. The word cutting switch determines the closeness between the inputs at time i and the input at time i−1. To ensure that the last moment generates a meaningful vector, its corresponding word vector is forced to be identified as G end.We combine the deformed input and the original input information of the network as the output. The problem of zero vector gradient reflux obstruction can be effectively avoided by information supplement.


## 3.2. Decoder Language Model

Like the encoder, the decoder also has a corresponding language model. The language model in the encoder takes the output of the word vector generating module as the input, adopts the bidirectional GRU on the whole, and improves the structure of GRU by adding an output gate. Finally, the output sequence of the language model combined with the positive and negative outputs is generated. The decoder is similar but different. Although it also adopts the GRU structure, it is only a positive GRU and does not add an output gate. Different from the encoder side, which has determined the entire input sequence at the initial stage, the output of the decoder side is generated step by step in time order, with obvious timing and direction. Therefore, one-way GRU is adopted. In addition, the decoder side is connected to an output module behind the language model, so we can simplify the output gate of the improved GRU. However, if this is all, the encoder and decoder are still independent of each other, and a word alignment module is also required, which abstracts the correspondence between the source language and the target language.The calculation method of the decoder side language model is shown in formula (10) to formula (13).(10)zi=σWzei−1y+Uzsi−1y+Czci,(11)ri=σWrei−1y+Ursi−1y+Crci,(12)hiy=tanhWzei−1y+Uzsi−1y+Czci,(13)siy=1−zisi−1y+zihiy.The unidirectional GRU model is adopted in the decoder, which is mainly because the last output module of the decoder is output one by one in sequence. Each module refers to the previous information and cannot predict the subsequent sequence information at the current moment. Word vector and language model are common parts of encoder and decoder frameworks and also common parts of natural language processing problems. Now both academia and industry are actively exploring it.


## 4. Experimental Design

### 4.1. Experimental Environment

The experiment is based on Theano, a deep learning framework. Theano has a large number of convenient optimization libraries. Based on efficient multidimensional array, it makes full use of GPU for calculation, which can greatly improve the efficiency of operation. The specific configuration of the experimental platform is shown in Table1.Table 1
Hardware and software configuration in the experimental environment.
The operating systemCPUThe graphics cardMemoryThe hard diskUbuntu14.04 64 bits17-5930K CPUNVIDIA GeForce GTX TITAN x64 GB2.5 TB


### 4.2. Experimental Results and Analysis

In order to compare the model effect of word and character level input and verify the influence of data set size on model effect, two data sets were extracted in the experimental stage. The small-scale data set has a magnitude of 2 million, including 1.8 million bilingual pair training sets and 5000 test sets, while the large-scale data set contains 21 million bilingual training sets and 5000 test data. Sequence length increases exponentially due to the conversion of input form from words to characters.Table2 lists the translation results of different models. Moses [13] is a classic word-based statistical machine translation system, while GNMT [14] is a word and character-based NMT model launched by Google. However, RNN-Search [15], RNN-Embed [16], and the CRNN-Embed model proposed in this paper are all character-based.Table 2
Comparison of model effects.
ModelBLEUPlP2P3P4BPMoses23.5753.529.91911.90.961GNMT25.3950.831.620.212.91RNN-Search (char1)18.0137.622.214.110.91RNN-Search (char2)20.7340.825.216.510.91RNN-Embed (char1)17.4838.622.213.38.21RNN-Embed (char2)19.6940.324.315.59.91CRNN-Embed (char1)18.4438.322.614.49.21CRNN-Embed (char2)22.754327.318.312.51Table2 contains six indicators corresponding to different calculation results of the BLEU evaluation method. The first BLEU is the main evaluation index; BP is determined by length, while P1 to P4 represent the accuracy rate of monadic words to quadratic words, respectively. In addition, CHAR1 corresponds to the evaluation results of millions of data sets, while char2 corresponds to tens of millions of data sets. As the size of data sets increases, the effect of the model is significantly improved. Compared with RNN-Search and RNN-Embed model results, the method of this topic has certain advantages, but there is still a certain gap with the results of classic word-based Moses system and word-based and character-based GNMT model. However, they have a high starting point, which can be used as a benchmark for further efforts. It should be noted that the results of the Moses system model are obtained by training on the million-level data set. During the training of Moses system with ten million-level data set, memory overflow occurs in the training tuning stage.Input changes from words to characters, and the length of input increases exponentially. In order to prove the effect of sentence length on the classification effect, we graded the models according to sentence length. The result is shown in Figure5.Figure 5
Results are classified by sentence length.If the length is less than 20, the grade is higher. As sentence length increases, the score gradually decreases. Therefore, it is necessary to further strengthen the processing ability of long sentences.In order to further analyze and verify the influence of data set size growth on model effect, we trained data sets of 2 million, 8 million, 14 million, and 21 million, respectively, and evaluated the model. The specific results are shown in Figure6. With the increase of data set size, the accuracy of both BLEU value and N-word is constantly improving, and the model effect is significantly improved. Comparison of results of models with different data sizes is shown in Figure 6.Figure 6
Comparison of results of models with different data sizes.


## 4.1. Experimental Environment

The experiment is based on Theano, a deep learning framework. Theano has a large number of convenient optimization libraries. Based on efficient multidimensional array, it makes full use of GPU for calculation, which can greatly improve the efficiency of operation. The specific configuration of the experimental platform is shown in Table1.Table 1
Hardware and software configuration in the experimental environment.
The operating systemCPUThe graphics cardMemoryThe hard diskUbuntu14.04 64 bits17-5930K CPUNVIDIA GeForce GTX TITAN x64 GB2.5 TB


## 4.2. Experimental Results and Analysis

In order to compare the model effect of word and character level input and verify the influence of data set size on model effect, two data sets were extracted in the experimental stage. The small-scale data set has a magnitude of 2 million, including 1.8 million bilingual pair training sets and 5000 test sets, while the large-scale data set contains 21 million bilingual training sets and 5000 test data. Sequence length increases exponentially due to the conversion of input form from words to characters.Table2 lists the translation results of different models. Moses [13] is a classic word-based statistical machine translation system, while GNMT [14] is a word and character-based NMT model launched by Google. However, RNN-Search [15], RNN-Embed [16], and the CRNN-Embed model proposed in this paper are all character-based.Table 2
Comparison of model effects.
ModelBLEUPlP2P3P4BPMoses23.5753.529.91911.90.961GNMT25.3950.831.620.212.91RNN-Search (char1)18.0137.622.214.110.91RNN-Search (char2)20.7340.825.216.510.91RNN-Embed (char1)17.4838.622.213.38.21RNN-Embed (char2)19.6940.324.315.59.91CRNN-Embed (char1)18.4438.322.614.49.21CRNN-Embed (char2)22.754327.318.312.51Table2 contains six indicators corresponding to different calculation results of the BLEU evaluation method. The first BLEU is the main evaluation index; BP is determined by length, while P1 to P4 represent the accuracy rate of monadic words to quadratic words, respectively. In addition, CHAR1 corresponds to the evaluation results of millions of data sets, while char2 corresponds to tens of millions of data sets. As the size of data sets increases, the effect of the model is significantly improved. Compared with RNN-Search and RNN-Embed model results, the method of this topic has certain advantages, but there is still a certain gap with the results of classic word-based Moses system and word-based and character-based GNMT model. However, they have a high starting point, which can be used as a benchmark for further efforts. It should be noted that the results of the Moses system model are obtained by training on the million-level data set. During the training of Moses system with ten million-level data set, memory overflow occurs in the training tuning stage.Input changes from words to characters, and the length of input increases exponentially. In order to prove the effect of sentence length on the classification effect, we graded the models according to sentence length. The result is shown in Figure5.Figure 5
Results are classified by sentence length.If the length is less than 20, the grade is higher. As sentence length increases, the score gradually decreases. Therefore, it is necessary to further strengthen the processing ability of long sentences.In order to further analyze and verify the influence of data set size growth on model effect, we trained data sets of 2 million, 8 million, 14 million, and 21 million, respectively, and evaluated the model. The specific results are shown in Figure6. With the increase of data set size, the accuracy of both BLEU value and N-word is constantly improving, and the model effect is significantly improved. Comparison of results of models with different data sizes is shown in Figure 6.Figure 6
Comparison of results of models with different data sizes.


## 5. Conclusion

In this paper, we use character-based NMT to solve the problem of high semanticity and difficult rationality of text information, which is mainly word-based, but the processing is tedious, and there are word segment errors of unknown words in word-level input and poor translation results. Because of the transformation of the input form, feature extraction of the model becomes more difficult, so a CRNN-Embed word vector generation method is proposed to further strengthen the word vector generation module. The convolution operation is used to extract the original features, and the circular NN is used to formalize the input words to increase the abstract power of features. In addition, since the hard-cut word problem will increase the difficulty of model optimization, the word vector is supplemented before input into the language model to eliminate the zero vector, which improves the feature extraction ability of the model as a whole. In the experimental analysis part, the feasibility and effectiveness of this model were verified by comparing the CRNN-Embed model with RNN-Search and RNN-Embed model. Translation is a generalized cultural translation in which the cultural connotation of one language is transformed into another cultural form. Simultaneously, as people-to-people contacts between countries become more common, the precision of machine translation will enhance the external communication of various industries, particularly tourism cultures with national characteristics such as red tourism culture.



---

*Source: 6330814-2022-04-23.xml*