Towards a Linguistically Motivated Irish Sign Language Conversational Avatar

Avatars are life-like characters that exist in a virtual world on our computer monitors. They are synthetic actors that have, in more recent times, received a significant amount of investigation and development. This is primarily due to leverage gained from advances in computing power and 3D animation technologies. Since the release of the movie “Avatar” last year, there is also a broader awareness and interest in avatars in the public domain. Ishizuka and Prendinger (2004) describe how researchers, while endeavouring to develop a creature that is believable and capable of intelligible communication, use a wide variety of terms to describe their work: avatars, anthropomorphic agents, creatures, synthetic actors, non-player characters, embodied conversational agents, bots, intelligent agents. While most of these terms are inspired from the character specific applications, some intend to draw attention to a particular aspect of the life-like character. To date it seems that there is no universal agreement with regard to terminology. The term avatar can be used to refer to the visual representation of a human being within a virtual environment whereas the term embodied conversational agent refers to a character that visually incorporates knowledge with regard to the conversational process. For the purpose of this research, the term embodied conversational agent is deemed an appropriate descriptor for the synthetic agent undergoing development. The value that RRG contributes to this is that it is a theory of grammar that is concerned with the interaction of syntax, semantics and pragmatics across grammatical systems. RRG can be characterised as a descriptive framework for the analysis of languages and also an explanatory framework for the analysis of language acquisition (Van Valin, 2008). As a lexicalist theory of grammar, RRG can be described as being well motivated cross-linguistically. The grammar model links the syntactic structure of a sentence to the semantic structure by means of a linking algorithm, which is bi-directional in nature. With respect to cognitive issues, RRG adopts the criterion of psychological adequacy formulated in Dik (1991), which states that a theory should be compatible with the results of psycholinguistic research on the acquisition, processing, production, interpretation and memorisation of linguistic expressions. It also accepts the criterion put forward in Bresnan and Kaplan (1982), that theories of linguistic structure should be directly relatable to testable theories of language production and comprehension. RRG incorporates many of the viewpoints of current functional grammar theories. RRG takes language to be a system of communicative social action, and accordingly, analysing the communicative functions of grammatical structures plays a vital role in grammatical description and theory from this perspective. The view of the lexicon in RRG is such that lexical entries for verbs should contain unique information only, while as much information as possible should be derived from general lexical rules. It is envisaged that the RRG parser/generator described in this paper will later be used as a component in the development of a computational framework for the embodied conversational agent for ISL. This poses significant technical and theoretical difficulties within both RRG and for software (Nolan and Salem 2009, Salem, Hensman and Nolan 2009). As ISL is a visual gestural language without any aural or written form, like all other sign languages, the challenge is to extend the RRG view of the lexicon and the layered structure of the word, indeed the model itself, to accommodate sign languages. In particular, the morphology of sign languages is concerned with manual and non-manual features, handshapes across the dominant and non-dominant hand in simultaneous signed constructions, head, eyebrows and mouth shape. These are the morphemes and lexemes of sign language. How can these fit into the RRG lexicon and what are the difficulties this presents for RRG at the semantic-morphosyntax interface? This paper will discuss this research as a work in progress to date. It is envisaged that the embodied conversational agent undergoing development in this research will later be employed for real-time sign language visualisation for Irish Sign Language (ISL).