Including Signed Languages in Natural Language Processing (2021)

Abstract: Signed languages are the primary means of communication for many deaf and hard of hearing individuals. Since signed languages exhibit all the fundamental linguistic properties of natural language, we believe that tools and theories of Natural Language Processing (NLP) are crucial towards its modeling. However, existing research in Sign Language Processing (SLP) seldom attempt to explore and leverage the linguistic organization of signed languages. This position paper calls on the NLP community to include signed languages as a research area with high social and scientific impact. We first discuss the linguistic properties of signed languages to consider during their modeling. Then, we review the limitations of current SLP models and identify the open challenges to extend NLP to signed languages. Finally, we urge (1) the adoption of an efficient tokenization method; (2) the development of linguistically-informed models; (3) the collection of real-world signed language data; (4) the inclusion of local signed language communities as an active and leading voice in the direction of research.

## sign language lingusitics signed languages consist of phonological, morphological, syntactic, and semantic levels of structure that fulfill the same social, cognitive, and communicative purposes as other natural languages. while spoken languages primarily channel the oralauditory modality, signed languages use the visualgestural modality, relying on the face, hands, body of the signer, and the space around them to create distinctions in meaning. we present the linguistic features of signed languages foot_1 that must be taken into account during their modeling. phonology signs are composed of minimal units that combine manual features such as hand configuration, palm orientation, placement, contact, path movement, local movement, as well as non-manual features including eye aperture, head movement, and torso positioning 4 (liddell and johnson, 1989; simultaneity though an asl sign takes about twice as long to produce than an english word, the rates of transmission of information between the two languages are similar @xcite . one way signed languages compensate for the slower production rate of signs is through simultaneity: signed languages make use of multiple visual cues to convey different information simultaneously @xcite . for example, the signer may produce the sign for 'cup' on one hand while simultaneously pointing to the actual cup with the other to express "that cup". similarly to tone in spoken languages, the face and torso can convey additional affective information @xcite . facial expressions can modify adjectives, adverbs, and verbs; a head shake can negate a phrase or sentence; eye direction can help indicate referents. ## current state of slp in this section, we present the existing methods, resources, and tasks in slp, and discuss their limitations to lay the ground for future research. ## towards including signed languages in natural language processing the limitations in the design of current slp models often stem from the lack of exploring the linguistic possibilities of signed languages. we therefore invite the nlp community to collaborate with the cv community, for their expertise in visual processing, and signing communities and sign linguists, for their expertise in signed languages and the lived experiences of signers, in researching slp. we believe that first, the development of known tasks in the standard nlp pipeline to signed languages will help us better understand how to model them, as well as provide valuable tools for higher-level applications. although these tasks have been thoroughly researched for spoken languages, they pose interesting new challenges in a different modality. we also emphasize the need for real-world data to develop such methods, and a close collaboration with signing communities to have an accurate understanding of how signed language technologies can benefit signers, all the while respecting the deaf community's ownership of signed languages. ## conclusions we urge the inclusion of signed languages in nlp. we believe that the nlp community is wellpositioned, especially with the plethora of successful spoken language processing methods coupled with the recent advent of computer vision tools for videos, to bring the linguistic insight needed for better signed language models. we hope to see an increase in both the interests and efforts in collecting signed language resources and developing signed language tools while building a strong collaboration with signing communities.