Pilot testing gender-inclusive translations and machine translations for G erman quadball referee certification test takers (2024)

Abstract: Gender-inclusive translations are the default at the International Quadball Association, yet translators make different choices for the (timed) referee certification tests to improve readability. However, the actual impact of a strategy on readability and performance has not been tested. This pilot study explores the impact of translation strategy (masculine generic, gender-inclusive, and machine translation) on the speed, performance and perceptions of quadball referee test takers in German. It shows promise for inclusive over masculine strategies, and suggests limited usefulness of MT in this context.

## introduction while the inherent importance of gender-inclusive language is clear @xcite , a commonly heard argument against the use of genderinclusive language strategies is that they negatively impact readability and comprehensibility. with some notable exceptions @xcite , however, this impact has not been empirically tested. at the international quadball association (iqa), gender-inclusive language is of critical importance, given the sports' commitment to gender inclusivity. while iqa translators aim to produce gender-inclusive translations, the desire for readability can outweigh the desire for inclusivity, particularly in the context of timed assessment for the referee certification tests @xcite foot_0 . the german iqa translation team currently uses the colon as the non-binary marker in most translations, which also seems to be the strategy preferred by professional translators @xcite . this pilot study was conducted to answer the following research questions about referee certification test takers in german: ## methodology a survey was created in qualtrics (qualtrics, provo, ut) and distributed in april and may 2024. the main test block was randomised so as to evenly present the three conditions -genderinclusive, generic masculine, and machine translation -to participants. it consisted of 14 multiplechoice questions taken from the official referee tests. only questions with multiple references to people were selected to guarantee differences between the conditions. questions were translated by iqa translators into the generic masculine and the gender-inclusive variant. for mt, each question and answer was translated using deepl (translations generated in april 2024). at the end of the survey, participants were asked how strongly they agreed with the following statements: "i could understand the questions and answers", "i found it easy to read the questions and answers", "i answered the questions as fast as i would in a real referee test", and "i answered most questions correctly" (likert scale of 1 = "not at all" to 5 = "completely"). twenty-four valid survey responses were collected (eight for each condition). all participants were native german-speaking quadball players, and all but one were currently or soon to be certified referees. statistical analyses were conducted using microsoft excel (anova) and rstudio (linear mixed effects models), but no statistical differences were found between conditions so only descriptive numbers are presented here. ## results speed: surprisingly, test takers were fastest in the inclusive condition, despite the text being 8% longer than the masculine condition ( perceptions: mt scores worst overall (figure 2 ). the inclusive condition scored highest on understandability and perceived correctness. ## conclusion & future work results suggest that (contrary to oft-heard criticism) speed and test scores are actually highest for the inclusive condition, showing its potential going forward. based on the pilot study findings and participant feedback, mt is not currently seen as a viable strategy for referee test translation. given the small sample size, statistically significant differences could not be identified, so for future work, we will expand this work by creating two variants (gender-inclusive and generic masculine) of the official iqa referee tests, in order to collect data from the entire population of referee test takers in a real-life setting.