E nglish To I ndian S ign L anguage:Rule-Based Translation System Along With Multi-Word Expressions and Synonym Substitution (2022)

Abstract: The hearing challenged communities all over the world face difficulties to communicate with others. Machine translation has been one of the prominent technologies to facilitate communication with the deaf and hard of hearing community worldwide. We have explored and formulated the fundamental rules of Indian Sign Language(ISL) and implemented them as a translation mechanism of English Text to Indian Sign Language glosses. According to the formulated rules and sub-rules, the source text structure is identified and transferred to the target ISL gloss. This target language is such that it can be easily converted to videos using the Indian Sign Language dictionary. This research work also mentions the intermediate phases of the transfer process and innovations in the process such as Multi-Word Expression detection and synonym substitution to handle the limited vocabulary size of Indian Sign Language while producing semantically accurate translations.

## introduction there are more than 300 sign languages all over the world, depending upon the region of the world (uni). nearly 4 million deaf people and more than 10 million hard of hearing people in india @xcite . out of such a big number, approximately 1 million deaf adults and around 0.5 million deaf children in india use indian sign language. rest of nearly 2.5 million deaf and hard of hearing people do not use any sign language to communicate. thus nearly 80% of hearing-impaired individuals have very limited or no access to education and other information. hearing-impaired people use sign language using handshapes, fingers, facial expressions, gestures, and other body parts. it is a visual-spatial language as the signer often uses the 3d space around his body to describe an event. sign languages, until the 1960s, were not viewed as bona fide languages but just collections of gestures and mime. dr stokoe's research on american sign language proved that it is a full-fledged language with its own grammar, syntax, and other linguistic attributes @xcite . there are some efforts to prove the same for other sign languages, including indian sign language. the sign language used in india is indian sign language (isl). a study by @xcite specifies that the isl used in different parts of india is almost identical in its structure, with differentiation in signs. it is a social need to encourage the hearing impaired individuals of the indian subcontinent with a tool that can translate the english text to isl. isl has all the properties of a natural language and is also considered a natural language like british sign language (bsl), american sign language (asl), and australian sign language (auslan). the idea of automatic machine translation in the area of sign language translation has been developing very fast in the last two decades as the technology needs to be used for the hearing impaired people all over the world. most of the researchers experimented with the rule-based machine translation methodology to translate a spoken or written language to sign language as this method relies upon the dictionary and the grammar of the source and target languages. rule-based machine translation is mostly used these days as indian sign language is a low resource communication system and there is a lack of content published in indian sign language. the current state of the art system is described in @xcite which is also a rule based system. we have tried to improve upon their work. videos. up till now, most research has focused on the grammar rules. @xcite and (mishra, 2019) contain extensive lists of grammar transfer rules but this method for a large part has been restricted to very simple sentences. we try to tackle more complicated sentences here which are simple in structure but some of the meanings conveyed by them are complicated and the previously used methods fall short in some regards. we have therefore used approaches such as multi-word expression detection and synonym substitution to handle hidden meanings within the sentence structure. not much work has been done in these areas in this field. @xcite have used synonyms before but the methods used by them for synonym substitutions are limited by the size of their manually created dictionary of synonyms for certain words. we have used existing english wordnet described by @xcite for finding synonyms. ## methodology we have broken down the steps into 3 phases: pre-processing, grammar transfer rules and post-processing. we are using stanza by @xcite , which is a collection of accurate and efficient tools for the linguistic analysis of many human languages. it is created by the stanford nlp group. apart from this, we are also using mwe tokenizer from nltk by @xcite ) for mwe tokenization. ## results and discussion as there was no previously established standard for this task, we selected the simple-wiki-dataset. we ran our model on 741 sentences with lengths of about 5-9 words per sentence. the results were manually evaluated. ## conclusion and future work in this paper we have tried to tackle semantically complex sentences and we see that the previously used methods fall short in some regards. therefore, we have used approaches such as multi-word expression detection and synonym substitution to handle hidden meanings within the sentence structure. we needed simple english sentences to test out our algorithm, as the rules worked best for simple sentences. in future, we intend to use machine learning approaches to do the translations on a large dataset to convert the standard english sentences. the methods mentioned in the paper could be a way to convert a large dataset of sentences on which deep learning models such as sequence2sequence and transformers can be trained.