Controlling Extra-Textual Attributes about Dialogue Participants: A Case Study of E nglish-to- P olish Neural Machine Translation (2022)

Abstract: Unlike English, morphologically rich languages can reveal characteristics of speakers or their conversational partners, such as gender and number, via pronouns, morphological endings of words and syntax. When translating from English to such languages, a machine translation model needs to opt for a certain interpretation of textual context, which may lead to serious translation errors if extra-textual information is unavailable. We investigate this challenge in the English-to-Polish language direction. We focus on the underresearched problem of utilising external metadata in automatic translation of TV dialogue, proposing a case study where a wide range of approaches for controlling attributes in translation is employed in a multi-attribute scenario. The best model achieves an improvement of +5.81 chrF++/+6.03 BLEU, with other models achieving competitive performance. We additionally contribute a novel attribute-annotated dataset of Polish TV dialogue and a morphological analysis script used to evaluate attribute control in models.

## introduction in some languages, dialogue explicitly expresses certain information about the interlocutors: for example, while in english words describing the speaker "i" and the interlocutor "you" are ambiguous w.r.t. their gender, number and formality, languages such as polish, german or spanish will mark for one or more of these attributes. in industrial settings such as dubbing and speech translation, there is an abundance of available metadata about the interlocutors, such as their genders, that could be used to help resolve these ambiguities. field value source "are you blind?" spoken by (=speaker) "anne" speaker's gender "feminine" spoken to (=interlocutor(s)) @xcite interlocutor(s)' gender "masculine" formality "informal" in the context of the gender of the speaker and interlocutor, prior research has explored two ways in which such information influences a text @xcite . firstly, naturally occurring texts satisfy grammatical agreement between the gender of the speaker and interlocutor and the utterances which describe them. how this agreement is expressed in speech varies among different languages @xcite . polish is a grammatical gender language: every noun is assigned a gender, and grammatical forms must agree with that noun. in contrast, english is a natural gender language, with "no grammatical markings of sex" @xcite . secondly, gender can be seen as a demographic factor that influences the way people express themselves (e.g. word choice). hereinafter we refer to the former as grammatical agreement and the latter as behavioural agreement. in this work, we seek to build machine translation (mt) models that satisfy grammatical agreement. given an english sentence and a set of attributes (e.g. the gender of the speaker and number of interlocutors), an mt system must translate this sentence into polish with a correct grammatical agreement to all attributes but introduce no markings of behavioural agreement. we explore the agreement to one speaker attribute: the gender of the speaker (spgender), and three interlocutor attributes: the gender(s) and number of interlocutor(s) (ilgender, ilnumber), as well as the desired formality of addressing the interlocutor(s). figure 1 exemplifies the extent of ambiguity these attributes introduce in english-to-polish translation. the main contributions of our work are: (1) a novel english-polish parallel corpus of tv dialogue annotated for spgender, ilgender, ilnumber and formality; (2) a tool for analysing attributes expressed in polish utterances; the paper is structured as follows. section 2 discusses previous work. section 3 presents the problem definition, focusing on polish as the target language. the creation of the parallel english-polish corpus of dialogue utterances that mark subsets of the investigated attributes is presented in section 4.1. how the mt models are trained to control the four extra-textual attributes is discussed in section 4.3, whilst the results are presented in section 4.2. finally, we describe conclusions and potential directions for future work in section 6. ## related work the state-of-the-art in mt is currently represented by neural mt (nmt) @xcite implemented via the transformer architecture @xcite . despite their unparalleled performance, these models are limited by ignoring the extra-textual context (e.g. speaker's gender). consequently, much recent work aims to control nmt with various attributes. in particular, attention has been paid to tasks such as multilingual nmt @xcite , by specifying the target language in the input; formality or politeness transfer (e.g. @xcite ); controlling the gender of the speaker and/or interlocutor @xcite @xcite ; length and verbosity @xcite ; or constraining the vocabulary @xcite . attribute control in nmt is most commonly facilitated with a tagging (or side constraints) approach, whereby a set of terms is added to the vocabulary, each embedding a certain type. these are trained alongside token embeddings and used in various ways during inference. controlling multiple attributes with this approach has not been excessively studied @xcite , but can be facilitated by simply concatenating the tags @xcite . however, for a set of equally important attributes, their ordering should not matter, but a tagging approach by design requires tags to be ordered in a specific way. combining attributes by averaging their embeddings has also been explored in previous work (cf. lample et al. (2019), schioppa et al. (2021) ), where authors incorporated the resulting vectors either into the input of the encoder or the decoder or directly into the model @xcite . typically, attribute-controlling neural models are fully supervised, requiring annotated training data. such annotations can be obtained directly, e.g. from metadata @xcite ; although most available corpora are unannotated. @xcite and @xcite automatically annotate the data using morphosyntactic parsers based on rules, validating agreement to the attribute in question in target-side sentences. to verify that the rules capture the attribute completely, a precision/recall score is computed against a manually labelled test set. ## problem specification recognising the small number of studies within machine translation research on the english-to-polish language direction, as well as our capacity (thanks to the available parsers and native speakers to validate their performance), we decide to focus the study on this language pair. polish is a west slavic language spoken by over 50m people over the world @xcite . it uses an expanded version of the latin alphabet and is characterised by a complex inflectional morphology @xcite . it is a grammatical gender language (koniuszaniec and błaszkowska, 2003) meaning all forms dependent on pronouns must agree to their gender and number. it uses a west slavic system of honorifics pani, pan, panie, panowie, państwo (henceforth pan+) @xcite . being a nullsubject language @xcite , it does not require that pronouns signifying the speaker or the interlocutor are explicit, unless they belong to the pan+ group @xcite . english lacks a grammatical gender or a system of honorifics, and the pronoun "you" is used for both plural and singular second person addressees. it is therefore ambiguous w.r.t. some expressions describing the speaker or the interlocutor, which we capture into four attributes, as follows (the attributes are summarised in ## results we report quantitative results in ## conclusions and future work in this work, we have highlighted the problem of grammatical agreement in translation of tv dia-logue in the english-to-polish language direction. we have created and described a dataset annotated for four speaker and interlocutor attributes that directly influence grammar in dialogue: speaker's gender, interlocutor's gender and number and formality relations between them. we have presented a selection of models capable of controlling these attributes in translation, yielding a performance gain of up to +5.81chrf++/+6.03bleu over the baseline (non-controlling) model. finally, we have produced a tool that produces an accuracy score for agreement to each type. considering all criteria of evaluation, we have identified tagenc as the best performing approach, with embenc, and embsos also achieving competitive performance. tagenc may be more attractive in scenarios where interventions in the model architecture are impossible as it can be implemented via data preprocessing alone, but the other two have a more scalable design (cf. §2). finally, contrary to some previous work, we found no advantages stemming from including the contextual information in the decoder as well as the encoder. future work nmt research should strive to move beyond seeing gender as a dichotomous phenomenon @xcite . within this paper we did not consider the scenarios with non-binary interlocutors due to i) lack of available data and ii) lack of consensus regarding non-binary gender expression in the polish language @xcite . however, our work can be applied to non-binary expression once data and more studies are avail-able. furthermore, the influence in nmt of other extra-textual attributes (e.g. multimodal ones, like spatial information, or emergent ones, such as personal attributes) is yet to be explored. it remains an open question whether such attributes should all be considered individually, or whether there is a way of identifying and/or using them implicitly.