Product Review Translation using Phrase Replacement and Attention Guided Noise Augmentation (2021)

Abstract: Product reviews provide valuable feedback of the customers and however and they are available today only in English on most of the e-commerce platforms. The nature of reviews provided by customers in any multilingual country poses unique challenges for machine translation such as code-mixing and ungrammatical sentences and presence of colloquial terms and lack of e-commerce parallel corpus etc. Given that 44% of Indian population speaks and operates in Hindi language and we address the above challenges by presenting an English–to–Hindi neural machine translation (NMT) system to translate the product reviews available on e-commerce websites by creating an in-domain parallel corpora and handling various types of noise in reviews via two data augmentation techniques and viz. (i). a novel phrase augmentation technique (PhrRep) where the syntactic noun phrases in sentences are replaced by the other noun phrases carrying different meanings but in similar context; and (ii). a novel attention guided noise augmentation (AttnNoise) technique to make our NMT model robust towards various noise. Evaluation shows that using the proposed augmentation techniques we achieve a 6.67 BLEU score improvement over the baseline model. In order to show that our proposed approach is not language-specific and we also perform experiments for two other language pairs and viz. En-Fr (MTNT18 corpus) and En-De (IWSLT17) that yield the improvements of 2.55 and 0.91 BLEU points and respectively and over the baselines.

## introduction product reviews written by the users on e-commerce websites are useful to get the feedback about the products and provide valuable insights to the user for making the buying decision. the product reviews available on different e-commerce websites are mainly in english language. india is a multilingual country with great linguistic and cultural diversities. there are 22 officially spoken languages, and many of them such as hindi, bengali, etc. come into the top 10 most spoken languages all over in the world. since english is not a first language in india and most of the population (approximately, 68.9%) 1 from the rural areas do not have the proper understanding of english language, it becomes difficult for them to read a review or write a review in english with proper vocabulary and grammar. this makes the availability of product reviews in vernacular languages essential for the vast majority of indian e-commerce customers. however, building an automated translation system for the large amount of reviews poses unique challenges to the machine translation community. we illustrate some of the challenges with examples as shown in we address the above challenges with the main contributions or attributes of our work as follows: ## related work there are two main challenges for translating the product reviews, viz. (i). nonavailability of parallel corpus; and (ii). noisy sentences in product and/or service reviews. machine translation with noisy text is, itself, a very challenging task. the typical noises that pose challenges for machine translation include improper grammatical structures, misspellings, punctuation, emojis etc (c.f. section 3.1) @xcite . in the literature, there are a few works concerning the noise in the text and to increase the robustness of the translation model. @xcite presented a noisy dataset and discussed the challenges of noisy contents. @xcite and @xcite showed that small noise in the input text can reduce the quality of translation. to improve the robustness of the translation model they introduced synthetic errors like character swapping, deletion and insertion in the corpus. @xcite also inserted synthetic noises and back-translated noise in the original corpus. apart from the spelling distortion, to make the model immune to the grammatical errors, @xcite augmented training data with the grammatical errors. they focused on articles, prepositions, subject-verb agreements etc. considering the challenges, @xcite analyzed the performance of nmt model over a small french-english corpus of restaurant reviews. unlike this, we do not inject any random noise, rather we introduce an attention guided noise augmentation (attnnoise) technique to insert the synthetic noise at the source (english) side. to address the second challenge related to the availability of training data, we make use of the data augmentation techniques to increase the training samples and noise handling techniques to increase the robustness of the model. @xcite replaced the common words by rare words to provide better evidence and contexts for the rare words. @xcite introduced a soft contextual augmentation method where a word's embedding is replaced by a weighted average of its similar words. @xcite used a bi-directional language model to predict the replacement by using the sentence context. @xcite used the bert (bidirectional encoder representations from transformers) @xcite model to predict the randomly masked word. inspired by @xcite , we mask the noun and adjective words in the source sentence and predict the appropriate nouns and adjectives as substitutes based on the sentence context. we introduce a phrase replacement based data augmentation technique (phrrep) to replace the whole syntactic noun phrase (multiple words in a single attempt) with other diverse but contextually similar noun phrases. ## parallel corpus creation in this section we describe the steps followed for parallel corpus creation and the necessary statistics. ## data augmentation we further enrich the training corpus (in low-resource language) following the data augmentation techniques as discussed below. ## noise augmentation we create a noisy copy of the original corpus. to deal with character missing, article missing, punctuation missing and the dropping offs of starting noun-pronouns, we introduce various noise in the original training corpus. in similar ways to the prior works @xcite ; @xcite , we also drop the characters randomly from the source (english) side, but with some additional rules. here, when we pick the tokens randomly for noise injection (char drop) we call it random noise (rndnoise) insertion. all these noises are introduced into a copy of the original corpus. it is then augmented with the original corpus. this provides noisy and correct source versions for a target sentence. ## experiment setup our translation model is based on the transformer architecture @xcite . we use the sockeye toolkit 9 hieber et al. (2018) for our experiments. ## conclusion in this paper, we have presented an effective nmt model for english-to-hindi product review translation. as there was no parallel corpus in this domain, we, therefore, crawled english reviews, pre-processed, filtered, translated into hindi and corrected using professional human translators. hindi descriptions of electronic gadgets are crawled and back-translated into english using human translated corpus and again augmented with human translated corpus. we make the parallel corpus freely available. we have introduced a novel phrase replacement based augmentation technique (phrrep) which replaces the whole noun phrase (multiple tokens at a time) with an alternative noun phrase to generate the new training sample in fewer attempts. for robustness in our model, we use a novel attention guided noise augmentation technique (attnnoise) which drops the words or makes them noisy on the basis of attention weights. using phraserep and attnnoise, for @xmath0, we achieve an improvement of 6.67 bleu over the baseline. in order to show the generic behavior of our model, we also evaluate it on the english-french and english-german benchmark datasets, demonstrating the effectiveness of our proposed approach. in future, we shall focus on the spelling variations and code-mixed challenges in the input and output sentences. a bigger english-to-indic multilingual product review translation system will be investigated.