Machine Translated Text Detection Through Text Similarity with Round-Trip Translation (2021)

Abstract: Translated texts have been used for malicious purposes, i.e., plagiarism or fake reviews. Existing detectors have been built around a specific translator (e.g., Google) but fail to detect a translated text from a strange translator. If we use the same translator, the translated text is similar to its round-trip translation, which is when text is translated into another language and translated back into the original language. However, a round-trip translated text is significantly different from the original text or a translated text using a strange translator. Hence, we propose a detector using text similarity with round-trip translation (TSRT). TSRT achieves 86.9% accuracy in detecting a translated text from a strange translator. It outperforms existing detectors (77.9%) and human recognition (53.3%).

## introduction a reader may misunderstand the original meaning of a translated text 1 . for example, facebook translated "good morning" into "attack them," leading to an arrest 2 . adversaries can use a translator for malicious tasks such as round-trip translation used in plagiarism @xcite to avoid human recognition or in adversarial text @xcite to fool ai. existing work has investigated the detection of translated texts in various approaches. the parse tree approach @xcite ) exploits text structure. the ngram approach @xcite ) estimates text fluency. the text complexity approach uses complex words (nguyen1 when we mention a translated text, translation, translator, and google, all are related to machine translation systems 2 www.theguardian.com/technology/2017/oct/24/facebookpalestine-israel-translates-good-morning-attack-them-arrest @xcite and phrases @xcite . the text coherence approach is based on matching similar words on a paragraph level @xcite . a threelayer cnn @xcite ) is trained on either one-way or round-trip translated texts. our previous work @xcite combined round-trip translation with bleu scores. all these approaches fail to detect a text translated by another translator or from a different language. motivation the first translation round induces a low similarity between the translated and original texts, whereas the extent of similarity increases in later rounds @xcite . let us consider an example in fig. 1 . we randomly selected an english text t from an english-russian pair foot_0 ; the russian text was translated into english by google, called t ′ (go,@xmath0. we measured the similarity between a text and its roundtrip translation using the minimum edit distance (m ed) @xcite . the translated text t ′ is the result of using the translator once, and the similarity between t ′ and its round-trip translation t ′(go,@xmath1. otherwise, the similarity between the original text t with t (go,@xmath2. based on the difference in similarity, we can distinguish the original from the translated text. in reality, a translator's source language is often unknown. the similarity decreases when using another language. for example, the similarity between t ′(go,@xmath3, @xmath4. it is close to the similarity in the original pair 𝒕′ (𝑮𝒐,𝑹𝑼→𝑬𝑵) : the actions of the chief banker, their every word or hint suddenly take on tremendous significance. 𝒕 : their actions, their every word or wink, suddenly assumes immense importance. 𝑡′ (𝐺𝑜,𝑅𝑈→𝐸𝑁→𝑅𝑈→𝐸𝑁) ## text similarity with round-trip translation training phase first, we collect original texts t i and translated texts t ′ i , which are translated with a configuration tuple π @xmath5, translator τ i } (see fig. 2 ). second, we generate round-trip translations t π i i and t ′π i i for t i and t ′ i , respectively. finally, t i and t ′ i are combined with t π i i and t ′π i i to train a subclassifier χ π i by fine-tuning the bert model @xcite . we repeat the procedure with other subclassifiers. in fig. 1 , t, t ′ , t (go,ru ) , and t ′(go,ru ) belong to t , t ′ , t (go,ru ) , and t ′(go,ru ) , respectively, with π = (go, ru ). testing phase for a suspicious text s, we aim to determine if s is an original or a translated text. first, we generate round-trip translated texts s π i with all configuration tuples in the training phase. next, we calculate the similarity σ π i between t and all s π i using the minimum edit distance (m ed). finally, we process s with the subclassifier associated with the best similarity σ b corresponding to the lowest m ed. in the case of t ′ in fig. 1 , two round-trip translations t ′(go,ru ) and t ′(go,de) are generated with respect to σ (go,ru ) = 1 and σ (go,de) = 6. the subclassifier χ (go,ru ) associated with the lower m ed is chosen for classifying t ′ . ## conclusion this paper proposed a one-way and round-trip translation detection mechanism using text similarity with round-trip translation (tsrt), which is robust to language and translator changes. first, we trained subclassifiers on specific lan- in future work, we will exploit saturation after repeatedly using the same ai system to detect other artificial texts such as fake covid-19 news.