Computer, enhence: POS -tagging improvements for nonbinary pronoun use in S wedish (2023)

Abstract: Part of Speech (POS) taggers for Swedish routinely fail for the third person gender-neutral pronoun “hen”, despite the fact that it has been a well-established part of the Swedish language since at least 2014. In addition to simply being a form of gender bias, this failure can have negative effects on other tasks relying on POS information. We demonstrate the usefulness of semi-synthetic augmented datasets in a case study, retraining a POS tagger to correctly recognize “hen” as a personal pronoun. We evaluate our retrained models for both tag accuracy and on a downstream task (dependency parsing) in a classicial NLP pipeline. Our results show that adding such data works to correct for the disparity in performance. The accuracy rate for identifying “hen” as a pronoun can be brought up to acceptable levels with only minor adjustments to the tagger’s vocabulary files. Performance parity to gendered pronouns can be reached after retraining with only a few hundred examples. This increase in POS tag accuracy also results in improvements for dependency parsing sentences containing hen.

## introduction the gender-neutral third person singular pronoun hen (subject/object form: hen; possessive form: hens) was added to the swedish academy's glossary in 2015 (saol, 2015), following at least occasional use since the mid-20th century @xcite . the use and acceptance of hen has since increased @xcite , although it remains much less common in media than hon ('she') or han ('he') @xcite . @xcite provides a detailed study of the use of hen in blog posts from the years 2001-2017. ## background since pronouns are a much smaller class than other parts of speech such as nouns or verbs, more-or-less perfect accuracy should be expected from taggers. indeed, we find that for the swedish gendered pronouns hon and han, 100% accuracy is achieved for both taggers investigated ( §3.1). stockholm-umeå corpus. the stockholm-umeå corpus foot_2 (suc) is an annotated corpus of texts from the 1990s @xcite . it contains about a million annotated words and is freely available for research purposes from språkbanken (after signing a license agreement). the latest version (v3) was released in 2012. ## discussion our initial findings showed that two common pos taggers for swedish either cannot identify hen as a pronoun at all, or identify it at notably lower rates than other pronouns. this likely has downstream consequences on performance of language technologies relying on these taggers, and on the level of the taggers themselves is a problem for gender equality. it also demonstrates a weakness of such taggers, namely their ability to be flexible in light of language shift. in our initial tests with swewinogender, kb-bert performed nearly-perfectly for hen: it only missed the two instances where hens was the first word of a sentence (and thus capitalized). however, swewinogender is a very regularized test set (as it is designed for challenging coreference systems, not pos taggers), and kb-bert's performance for hen drops to less than 95% when tested on the more complex, realistic 'hen' suc test dataset. this makes it plausible that having only unlabeled data might not be sufficient to learn, e.g., pronouns that have recently come into use and are underrepresented in the data. as the kb-bert model was originally fine-tuned for pos tagging on suc, it seems reasonable that fine-tuning on the enhenced suc data could mitigate this weakness. another reason for keeping tools such as efselab around is that at present is that the kb-bert model does not provide more complex morphological information, which is desirable in some cases. training existing architectures on augmented data containing even a small number of sentences containing the pronoun hen can effectively correct for this disparity. we reach complete parity to binary-gendered pronouns, at 100% accuracy, with a representative sample (hen2), and see no real improvement when adding more sentences containing hen (hen10 and hen80). this suggests that an up to date annotated dataset, based on contemporary swedish usage, would be enough to obtain inclusive results, without the need for synthetic data, at least for the case of hen. in terms of effect on downstream tasks, this improvement carries over to label accuracy for dependency parsing on hen tokens, with no loss of to las over all tokens. as nouns and pronouns often occupy similar grammatical roles, it is somewhat unsurprising that there is not also an effect on head accuracy (as measured by uas).