First WMT Shared Task on Sign Language Translation ( WMT - SLT 22) (2023)

Abstract: This paper is a brief summary of the First WMT Shared Task on Sign Language Translation (WMT-SLT22), a project partly funded by EAMT. The focus of this shared task is automatic translation between signed and spoken languages. Details can be found on our website ( https://www.wmt-slt.com/ ) or in the findings paper (Müller et al., 2022).

## objectives the project envisioned that there would be benefits both for deaf sign language users and for the research community. for deaf communities, the shared task aimed for better access to linguistic tools, including mt, in their native languages and also to improve recognition for sign languages. for the mt research community, our goal was to include sign languages in wmt shared tasks as a way of informing researchers about sign languages and boosting research on sign language translation. more concretely, we were looking to produce public benchmark data for mt systems, translations by many state-of-the-art systems and judgements of translation quality by humans. for sign languages, such resources did not exist before the shared task. main outcome seven teams (including one from the university of zurich whose submission we consider a baseline) participated in our task. all of them submitted to the dsgs-to-german track, while there were no submissions for the second translation direction, presumably because this direction is more challenging. seven teams is a high turnout, considering that other comparable efforts (such as a shared task on taiwanese sign language translation co-located with loresmt 2021 @xcite or the workshop on sign language recognition, translation and production (slrtp) 2022 foot_2 ) had fewer participants. we presented the final results at wmt 2022 in abu dhabi in december 2022 foot_3 . the shared task was well received and sparked considerable interest in the machine translation community. other important artifacts besides a system ranking and system papers describing state-of-theart techniques, our shared task made the following scientific contributions: novel corpora, reproducible baseline systems and new protocols and software for human evaluation. finally, the task also resulted in the first publicly available set of system outputs and human evaluation scores for sign language translation. ## funding agencies this shared task was funded by eamt (through the call "sponsorship of activities") and by microsoft ai for accessibility. we are grateful for their support which enabled us to provide test data, human evaluation and interpretation in international sign during the wmt conference. the organizing committee further acknowledge funding from the following projects: the eu horizon 2020 projects easier (grant agreement number 101016982) and signon (101017255), the swiss innovation agency (innosuisse) flagship iict (pffs-21-47) and the german ministry of education and research through the project so-cialwear (01iw20002).