Audiovisual Translation through NMT and Subtitling in the Netflix Series ‘Cable Girls’ (2021)

Abstract: In recent years, the emergence of streaming platforms such as Netflix, HBO or Amazon Prime Video has reshaped the field of entertainment, which increasingly relies on subtitling, dubbing or voice-over modes. However, little is known about audiovisual translation when dealing with Neural Machine Translation (NMT) engines. This work-in-progress paper seeks to examine the English subtitles of the first episode of the popular Spanish Netflix series Cable Girls and the translated version generated by Google Translate and DeepL. Such analysis will help us determine whether there are significant linguistic differences that could lead to miscomprehension or cultural shocks. To this end, the corpus compiled consists of the Spanish script, the English subtitles available in Netflix and the translated version of the script. For the analysis of the data, errors have been classified following the DQF/MQM Error typology and have been evaluated with the automatic BLEU metric. Results show that NMT engines offer good-quality translations, which in turn may benefit translators working with audiovisual entertainment resources.

## introduction over the past few years, the rise of netflix, hbo, amazon prime video and other streaming platforms has made it necessary to rethink entertainment media @xcite . accessibility to their catalogues not only offers the audience the opportunity to choose among a variety of films, series, documentaries and other audiovisual resources but also to make use of subtitling and dubbing options @xcite . however, even though these audiovisual translation practices are meant to meet the needs of different markets and users @xcite , the quality of the translation may be affected by errors when translators are given tight deadlines. machine translation (mt) is widely used in the translation industry, especially in technical fields because the texts tend to be repetitive, and studies have shown that it increases translators' productivity @xcite by post-editing the mt output. however, despite the fact that platforms like netflix announced that they are using mtpe in their subtitling workflows three years ago, research on this topic is still scarce in creative fields, such as literary or audiovisual texts. it might be assumed that nmt will not work when dealing with audiovisual translation due to its time and character constraints, and especially in media entertainment where cultural aspects are prevalent. given that the quality of nmt, although still not at the human level @xcite , is improving every day, some issues need to be considered. would it be beneficial for audiovisual translators to use mt and post-edit the texts? or is scratch translation still the best solution for this field? in any commercial deployment of mt in a subtitling workflow, a bespoke engine would be used. in fact, there are already subtitling specialised mt systems available in the market like apptek, omniscien and xl8. however, growing volumes of audiovisual content, short turnaround times or lack of access to this type of engines are some of the challenges novice translators need to overcome. these issues have been addressed in previous studies where mt may serve as a possible solution @xcite . numerous publications arised from the sumat project, a large-scale eu-funded project that inspected the creation of high-quality parallel corpora of subtitles through mt @xcite . matusov et al. @xcite , for example, analysed improvement in productivity after integrating mt in audiovisual translation. this ongoing project aims to ascertain the quality of google translate and deepl translations (i.e. open mt resources) when compared to the subtitling of tv series in the source language. on this account, the current study draws from the following research questions: rq(1) how do english subtitling and translations from nmt differ from the source text in spanish? what types of errors can be found? and rq(2) does the integration of mt on the audiovisual translation workflow benefit translators? the section below delves into the methodological procedure followed in this study. later on, the discussion of the preliminary results as well as the conclusions and next steps of this ongoing project will be provided. ## methodology the corpus under study revolves around the spanish netflix original series called cable girls. this drama, premiered in 2017 and set in the 1920s, tells the story of four women working as operators for the national telephone company at a time of social changes. the first season consists of eight episodes, with a length of 47 to 64 minutes. for the compilation of this small corpus, the focus has been on the first 10 minutes of the first episode of the first season released in spanish. the spanish script and the official english subtitles incorporated in the streaming platform have been transcribed from netflix @xcite and examined for the purpose of this preliminary study. in addition, the spanish transcript was translated with google translate @xcite and deepl @xcite to analyse the quality of these nmt engines. google translate is an mt engine that provides the translation of texts and files into more than 100 languages, including english, spanish, greek, belarusian, afrikáans or chinese @xcite . the fact that google offers these services has caught the attention of scholars who have been concerned with error analysis on mt output. evidence may be found in trzaskawka's study @xcite , which explored the accuracy of this tool in the translation of contracts in english and polish. issues related to the quality of the translation output have also been explored in specialised areas, such as literature @xcite and scientific writing @xcite . however, research on entertainment media seems to be scarce, with studies delving into the dubbing and subtitling of tv series @xcite and documentaries @xcite . deepl translator @xcite is an nmt software developed in 2016 with the aim of producing high-quality translated texts. at the moment, deepl works with more than 20 languages and also offers a formal/informal register for their translations. deepl has also caught the attention of researchers and several studies compare its quality to other mt engines like google translate, yandex or microsoft translator @xcite @xcite . the quality of the machine-translated texts has been assessed manually following the dqf/mqm error typology @xcite the integration of dqf (dynamic quality framework) @xcite and mqm (multidimensional quality metrics) @xcite paying attention to the categories labelled as accuracy and fluency. for this manual evaluation, 153 segments containing 7 words on average were examined by two annotators with experience in translation (i.e. post-editing) and linguistics. the translated texts were then analysed automatically with the bleu metric @xcite , using the original subtitles as the human translation and the nmt output from google translate and deepl. ## conclusions and further research the research questions attempted to determine the quality of the translations provided by google translate and deepl when dealing with audiovisual media. hence, the cable girls series script in the source language was compared with the mt outputs from google translate and deepl. as to rq(1), the findings suggest that the most common errors occur at fluency and accuracy levels. in addition, the results show that deepl outperforms google translate in both manual and automatic evaluation. with regard to rq(2), the next steps of the project will delve into translators' postediting efforts: is it useful to use mt for audiovisual texts? in this vein, technical, temporal, and cognitive variables will be considered to prove whether these efforts are higher or lower when integrating mt tools. accordingly, an eye-tracking device and a keystroke logging tool will be employed. limitations in this study should be acknowledged. the small size of the corpus compiled for this preliminary study may affect the validity of the generalisations presented here. nonetheless, it should be noted that the corpus will be expanded in the near future. moreover, the mt engines that were used are not trained on subtitling and may contain an enormous amount of noise. deepl and google translate were used to emulate the experience of freelance translators using general mt. notwithstanding, the use of these mt engines could have a negative impact on translation quality as the length of the segments, a relevant feature in subtitling, is not taken into consideration. further research could also focus on other audiovisual resources, including documentaries or realities. such examination would prove the efficiency of google translate in specialised and non-specialised contexts or the quality of other machine translation software like deepl in audiovisual domains. other lines of the proposal presented here could involve the role of mt in the translation of humour and cultural aspects, which are prolific in entertainment media.