Automatic Translation of Complex E nglish Sentences to I ndian S ign L anguage Synthetic Video Animations (2020)

Abstract: Sign Language is the natural way of expressing thoughts and feelings for the deaf community. Sign language is a diagrammatic and non-verbal language used by the impaired community to communicate their feeling to their lookalike one. Today we live in the era of technological development, owing to which instant communication is quite easy but even then, a lot of work needs to be done in the field of Sign language automation to improve the quality of life among the deaf community. The traditional approaches used for representing the signs are in the form of videos or text that are expensive, time-consuming, and are not easy to use. In this research work, an attempt is made for the conversion of Complex and Compound English sentences to Indian Sign Language (ISL) using synthetic video animations. The translation architecture includes a parsing module that parses the input complex or compound English sentences to their simplified versions by using complex to simple and compound to simple English grammar rules respectively. The simplified sentence is then forwarded to the conversion segment that rearranges the words of the English language into its corresponding ISL using the devised grammar rules. The next segment constitutes the removal of unwanted words or stop words. This segment gets an input sentence generated by ISL grammar rules. Unwanted or unnecessary words are eliminated by this segment. This removal is important because ISL needs only a meaningful sentence rather than unnecessary usage of linking verbs, helping verbs, and so on. After parsing through the eliminator segment, the sentence is sent to the concordance segment. This segment checks each word in the sentence and translates them into their respective lemma. Lemma is the basic requiring node of each word because sign language makes use of basic words irrespective of other languages that make use of gerund, suffixes, three forms of verbs, different kinds of nouns, adjectives, pronouns in their sentence theory. All the words of the sentence are checked in the lexicon which contains the English word with its HamNoSys notation and the words that are not in the lexicon are replaced by their synonym. The words of the sentence are replaced by their counter HamNoSys code. In case the word is not present in the lexicon, the HamNoSys code will be taken for each alphabet of the word in sequence. The HamNoSys code is converted into the SiGML tags (a form of XML tags) and these SiGML tags are then sent to the animation module which converts the SiGML code into the synthetic animation using avatar (computer-generated animation character).

## sign language sign language is the mother language for hard of hearing people, it is a physical movement-based language. sign language is recognized as the first language for the hard of hearing people. in sign language especially hands and head are used to express their thoughts with others. it is not a universal language; it has its syntax and grammar. each nation has its sign language. sign language varies from place to place. sign language used in the usa is called american sign language (asl), in british they use british sign language(bsl). similarly, in india, we use the indian sign language (isl). it is very difficult for the hard of hearing people understand any kind of information because normal people cannot understand their special language and the availability of an interpreter most of the time is not possible. so, these problems make it hard of hearing people isolated from the rest of the world. ## structure of the english sentence the english phrase follows the order of svo (subject-verb-object). english sentences can be classified into three different forms. these are simple sentences, compound sentences, and complex sentences. ## proposed approaches the system consists of seven modules: ## results for evaluating the quality of the english-isl translator, a set of test sentences is required. the selection of complex and compound sentences for testing the translator has been taken from various english books and internet sources. a total of 2810 sentences have been tested to evaluate the performance of the translator. to evaluate the translator for english text to isl synthetic animations, we have a qualitative system. in the case of our translator (english text to isl synthetic animations), the dictionary of isl is limited to 5370 words including their synonyms. if there is any word in english sentence whose sigml is not available, then that word is fingerspelled. analysis of sentence error rate (ser) sentence error rate (ser) is the ratio of the number of sentences of mt output which does not match with the reference sentences to the total number of reference sentences. out of 2810 sentences, 471 sentences are considered as incorrect, so ser is 0.16.