Introduction
No translation can simultaneously satisfy all possible goals, and translation is therefore an art of navigating competing objectives Darwish ( 2008 ) . Many objectives are discussed in the literature, but two in particular seem especially fundamental. The first is accuracy (also known as fidelity or adequacy), or the goal of preserving the information in the source text (ST). The second is fluency, or the goal of producing target text (TT) that respects the norms of the target language (TL) and is easy for the recipient to process Kunilovskaya ( 2023 ) .

Here we study the relationship between accuracy and fluency and work with two operationalizations of these notions. The first relies on human judgments of accuracy and fluency collected in prior work on translation quality estimation Castilho et al. ( 2018 ) . The second relies on probabilities estimated using neural machine translation (NMT) models. Given a source-translation pair (\boldsymbol{x},\boldsymbol{y}) , p(\boldsymbol{x|y}) corresponds to accuracy, and p(\boldsymbol{y}) corresponds to fluency Teich et al. ( 2020 ) . p(\boldsymbol{x|y}) will be low if \boldsymbol{y} fails to preserve all of the information in \boldsymbol{x} , and p(\boldsymbol{y}) will be low if \boldsymbol{y} violates the norms of the target language. To highlight that model estimates p(\boldsymbol{x|y}) and p(\boldsymbol{y}) are related to but distinct from human ratings of accuracy and fluency, we refer to p(\boldsymbol{x|y}) as \text{accuracy}_{M} and p(\boldsymbol{y}) as \text{fluency}_{M} .

Some parts of the literature argue that accuracy trades off with fluency. In Figure 1 a, the blue dots are translations of the same source segment, and Table 1 shows three translations that illustrate the same kind of tradeoff. A translator choosing between these alternatives cannot simultaneously maximize accuracy and fluency, because the most accurate translations are not the most fluent, and vice versa. Teich et al. ( 2020 ) argues that \text{accuracy}_{M} and \text{fluency}_{M} should trade off in this way, and the same view is implicitly captured by noisy-channel models of translation Brown et al. ( 1993 ) , which aim to generate translations \boldsymbol{y} that maximize p(\boldsymbol{y}|\boldsymbol{x})\propto p(\boldsymbol{x}|\boldsymbol{y})p(\boldsymbol{y}) . Typically these models include weights for the two components p(\boldsymbol{x|y}) and p(\boldsymbol{y}) that can be interpreted as the extent to which \text{accuracy}_{M} is prioritized over \text{fluency}_{M} , or vice versa Yu et al. ( 2016 ); Yee et al. ( 2019 ); Yu et al. ( 2020 ); Müller et al. ( 2020 ) .

An opposing view of the relationship between accuracy and fluency emerges from the literature on quality estimation. Here the common wisdom is that accuracy and fluency are highly correlated and practically indistinguishable to human annotators (Callison-Burch et al., 2007 ; Banchs et al., 2015 ; Mathur, 2021 , but see Djiako 2019 ; Sulem et al. 2020 ) . As a result, accuracy and fluency are conflated as a single assessment score in
recent WMT General Machine Translation Tasks, with more emphasis given to accuracy than fluency Farhad et al. ( 2021 ); Kocmi et al. ( 2022 , 2023 ) .

We argue that the conflict between these views is an instance of Simpson’s paradox Yuan et al. ( 2021 ) , which occurs when a relationship at one level of analysis (e.g. the corpus level) is reversed at a different level (e.g. the segment or sentence level). Figure 1 shows how the correlation between accuracy and fluency can be positive over a miniature corpus including translations of three source segments even though the correlation for each individual source segment is negative. Of the two levels of analysis, the segment level is the appropriate level for understanding how humans and machine translation systems should choose among possible translations of a source segment.
The central goal of our work is therefore to establish that the correlation between accuracy and fluency is negative at the level of individual source segments. 1 1 1 Code will be made available upon publication.