A Snapshot into the Possibility of Video Game Machine Translation (2022)

Abstract: In this article, we trained what we believe to be the first MT system adapted to video game translation and show that very limited in-domain data is enough to largely surpass publicly available systems, while also revealing interesting findings in the final translation. After introducing some of the challenges of video game translation, existing literature, as well as the systems and data sets used in this experiment, we provide and discuss the resulting translation as well as the potential benefits of such a system. We find that the model is able to learn typical rules and patterns of video game translations from English into French, indicating that the case of video game machine translation could prove useful given the encouraging results and the specific working conditions of translators this field. As with other use cases of MT in cultural sectors, however, we believe this is heavily dependent on the proper implementation of the tool, which we think could to stimulate creativity.

## introduction since the apparition of recurrent neural networks with attention mechanisms @xcite , neural machine translation (nmt) has improved to the point of becoming the default paradigm for this task. with new architectures such as the transformer @xcite and a similarly growing number of domain adaptation techniques @xcite , nmt has also started being used in increasingly more complex domains, and even tailored to the production of specific translators and companies. the video game market similarly seems to have been ever growing in the last decades, becoming one of the fastest growing and highest grossing industries today within the cultural and entertainment sectors. the recent global health crisis further reinforced this trend and showed that games have an important role to play beyond just entertainment, as can be seen in a recent eu report on the cultural and creative sectors (ccs) in europe @xcite : "few winners can be found in the ccs during the covid-19 global crisis. one of those, together with streaming platforms, is the video games sub-sector. with a turnover of eur 21.6 billion and a 3% year-on-year growth from 2018, the gaming sub-sector has proven to be strong also in hard economic times." as part of this success, it has now been expected for a while that video games be translated not just into the western e-figs (english to french, italian, german and spanish), but into at least eight or ten languages in order to be considered a profitable and successful enterprise @xcite . this creates, in turn, a huge demand for translation and a challenge in its own right, especially for smaller or newly formed studios. it should furthermore be noted that, nowadays, video game localization is mainly considered from a market perspective. indeed, the main purpose of this process is to make a given title available to a larger audience. to do so, several attempts have been made to produce game localization manifestos encouraging best practices @xcite . those practices -and scientific studies in the field of video game localization in general -are said to be market-driven @xcite . even though video games represent an exceptional place of intercultural contact, cultural traces are quasi systematically neutralized in an attempt to ensure sales, to the point where they would be considered as localization errors were they to remain in the target version @xcite , despite evidence from reception studies showing that current localization practices do not correspond to what players expect from a cultural point of view @xcite @xcite . on the other hand, the video game industry, as is the case with other digital productions 1 , is characterized by a particularly high level of fan-made content @xcite , with undertakings varying from patches 2 , mods 3 , or even (re)creations that far outgrow the original games themselves. it is also very common for fans and volunteers to look into the original translations, sometimes correcting minor mistakes in these very large works, sometimes coming up with entirely new localization projects if the title was never translated or if the translation was not done by professionals and its quality deemed too poor @xcite @xcite . indeed, translation has traditionally been seen in the market as a trivial and disposable process that could be targeted to cut costs, leading up to the numerous examples of bad video game translations that careless uses of machine translation (mt) definitely did not help with @xcite . this brings several issues, not only for the players, but for the companies as well. as bernal-merino (2008) explains: "one of the most common complaints we can read about in internet forums is the lack of translation quality, and how sometimes they have to go back to the original version to find out what to do and how. in many cases, even though the game might have been translated and it is playable, fans fail to be impressed because the poor quality of the localisation defeats its own purpose: to thrill and engage the gamer. [. . . ] language and culture are ever-present elements in us and the things we do, players cannot help but notice continuous serious mistakes in the game, and it will erode their trust in some developer and publisher brands." but what makes video game translation such an arduous task and how could machines play a role in it if it already proves challenging for humans? in addition to the common pitfalls of translation and creative works, translators in the field are typically presented with spreadsheets in which text segments are not ordered chronologically and do not offer any other kind of contextual information, concerning for instance the communication medium, the type of discourse, 1 as the web gave rise to the development of a "user-participatory culture" @xcite and "user-generated content" @xcite , the translation sector similarly witnessed the emergence of "user" or "amateur" crowdsourced translations @xcite @xcite . ## state of the art as we mentioned, the advances in nmt architectures and domain adaptation techniques allowed for better performances in various domains. in the cultural sector, machine translation has mostly been used in the process of film subtitling, but a growing number of studies has focused on the challenges of its development, for instance, with literary texts @xcite . concerning video games, it is hard to find scientific sources that go further than acknowledging the existence of mt within this sector, as in @xcite . this use of machine translation is further confirmed in various online posts and articles, but also in sessions dedicated to localization at the game developers conference @xcite . some online translation service providers also put it forward, alongside with the traditional arguments of cost and time savings that typically accompany mt, and it would appear that some studios and translators are sometimes forced to turn to this technology in order to meet the strict and tight deadlines of the industry. it remains nevertheless agreed that mt would never be suitable for the translation of the more creative in-game contents, such as dialogues (o'hagan and mangiron, 2013). we can once again find confirmation for this observation in online interviews @xcite , but also in the deployment and user feedback of open source tools like retroarch (libretro, 2019), that leverage google's translation service to make retro games accessible to a larger public. machine translation, however, might benefit the video game localization industry, whether it is to cope with the heavy requirements of a simultaneous multilingual game delivery, to help fans make their content available to more people, or to assist professionals during this long-term and demanding task. yet, we have found no evidence of mt tools being specifically adapted for the translation of video games, and the existing use of mt as well as the advances in nmt make it the perfect time to try and see how effective such a custom engine could be. to our knowledge, this is thus the first article to dive deeply into the possibility of video game machine translation, or, at the very least, that an mt system was specifically designed for such a task. ## methodology video game translation is an eminently difficult task that involves linguistic skills and cultural awareness from the two languages at hand, diegetic knowledge about the world in which the game takes place, as well as an understanding of how the game is to be played in real life and how the translation fits into its development. to have an idea of whether mt could be of help to professional translators, we built a first system tailored to the translation of video games for the english-french language pair. this system was trained with the 2nd edition of the opennmt framework @xcite on a custom-made corpus of video game translations. our bespoke mt engine was then evaluated on two role-playing games (rpg), namely the elder scrolls v: skyrim (bethesda game studios, 2011) and fallout 4 (bethesda game studios, 2015), taking place in a high fantasy and post-apocalyptic setting respectively. ## results for evaluation, we detokenized the output with the same moses script and computed a bleu score with sacrebleu @xcite . we first report in our first comparison indicated that adding data did not necessarily increase performance. in the first case, the added 3 million segments resulted in a negligible increase in bleu, whereas that score even decreased in the second case. for this reason, and since our aim was to evaluate systems trained on in-domain data, we discarded the tuned models. this decision was further motivated by our evaluation of the resulting translations, which showed that the indomain models seemed to learn inherent rules of the domain which the fine-tuned models did not, such as the fact that gendered variables tend to disappear when translating into french: 8 apart from in-house teams however, the use of tms remains relatively uncommon in the video game industry. 9 an experiment was also carried out with the dozens of millions of sentences from the wmt 2014 translation task @xcite , but the final score dropped systematically due to the quality and dissimilarity of these data sets. ## analysis as noted by @xcite , mt evaluation has become less comprehensive and reliable over the last years, but one way to overcome this pitfall is to support automatic evaluation with human analysis. for this reason, we present two sets of evaluations. the first conveys the broad trends observed in our analysis of the two games presented previously, which we cannot publish for copyright reasons. we will further illustrate these remarks with a second set offering concrete examples from the translation of a fan-made mod. we believe the best way to transparently convey and judge the result of an automatically translated text is to provide a full and continuous example of this translation, so we have made it available online and we report here the index of the quoted sentence where appropriate. 11 10 metric signatures for sacrebleu (publicly available systems tested on 23/12/2020): bleu #:1|c:mixed|e:no|tok:13a|s:exp|v:2.0.0 chrf2++ #:1|c:mixed|e:yes|nc:6|nw:2|s:no|v:2.0.0 ter #:1|c:lc|t:tercom|nr:no|pn:yes|a:no|v:2.0.0. 11 https://gitlab.uliege.be/dhansen/vgmt-article/ . we provide 323 segments, broken down into four illustrative excerpts. they are mainly made up of the dialogues that are often raised as criticism against mt and that were chosen because they formed mostly coherent pieces of conversation, with the exception of disrupting segments that are typical of video game files. the first excerpt is singular, as it contains segments from the 2002 game, which therefore also appear in our original training data. we included it nonetheless, ## discussion as we have established in previous sections, we were able to achieve very encouraging results on a video game translation experiment by training an mt system on a surprisingly small in-domain data set. this was both expected and unexpected, in the sense that using relevant training data would logically boost performance and allow the system to assimilate specific terms or previously translated phrases, but we had not foreseen that this custom engine would learn abstract translation strategies that are particular to this domain, such as neutralizing variables when translating from a neutral into a gender-inflected language. we therefore think that, if implemented the right way, mt might be susceptible of helping with terminology and formal or standardized expressions, to offer relevant suggestions and maybe speed up the process, becoming a useful tool for either professional translators or amateurs working on fan translations. this requires nevertheless that studios consider the translation process and resources as an integral part of the game development, which is not often the case and a source of problems even with human translation. indeed, localization should be planned early in the development by rigorously following internationalization steps to accommodate translations into any language @xcite . the use of mt further emphasizes this need, which could mean ensuring that texts are not hard-coded and mixed with lines of code, using tags that could be useful to human and machine to alleviate the ambiguities and lack of context, or refraining from imposing formal constraints to account for differences with languages other than english. on the other hand, the video game industry benefits from a significant advantage, which is that all translations are already aligned and easily convertible into language resources. yet, our study also highlighted the parallel between the decontextualized working conditions of the translator and the machine translation system. this lack of contextual information and resources is pointed as one of the main challenges translators have to face in the field @xcite , and one might wonder in these circumstances why translators so rarely have access to some kind of translation memory, or at least an official glossary. this alignment incidentally makes it easy to train mt systems tailored to the particular video game genre, or even specific works, which our study has shown to be able to effectively reproduce the expected terminology and offer relevant suggestions. this might even come as a welcome change, if we consider that some professionals in the field (around 12%) already use mt in their workflow @xcite . with the convenience of a customized engine, we could even expect this number to rise, as such a tool help deliver faster translations, provide otherwise lacking information to translators and, most important of all, boost creativity. this last point might be the biggest asset of human translators in this domain, given that it is required to overcome the numerous technical and linguistic challenges that are typical of video game translation @xcite , without mentioning the very creative nature of the content itself. to this end, we plan to delve into a closer reading of these ad hoc mt solutions from the perspective of inter-semiotic dynamics @xcite . in this respect, mt could free up time for creative thinking and particularly challenging segments or simply offer relevant suggestions, especially if it is used in combination with other tools and material such as tms, reference corpora or termbases. however, we think this would happen if mt is used not as a first draft that may constrain the translation, but as a suggested translation akin to a tm match that would help correct points of interpretation in the source text, maintain stylistic and lexical cohesion, and even spot eventual mistakes in the target text. on the amateur side, this technology could prove even more useful to fans for the translation of their user-generated content, thus expanding the success and replayability of their favourite games, or allowing games that were never intended to be translated to reach a wider audience. this, in turn, could promote cultural exchanges and sensitivity through a medium that otherwise tends to erase such influences against players' own expectations @xcite . all of these positive aspects of mt conversely depend on various ethical issues, the main one being the reason for its introduction. relying on machine translation only for the sake of productivity, without human intervention, or blindly forcing its deployment even when it is not appropriate is sure to have a drastic impact on quality, creativity and the overall appeal of the game. such habits evidently tie into much broader issues, some of which mt might even reinforce as in the case of amateur translations being used by companies seeking to cut costs. finally, we should not forget that all of the resources used to train these systems depends on the quality of the material that is provided by human translators, whose rights seem to become less and less apparent as translation technologies progress @xcite . we therefore hope that this exploratory article and its conclusions will hopefully start some discussions in the video game, or the cultural sector as a whole.