A rabic NLG Language Functions (2020)

Abstract: The Arabic language has very limited supports from NLG researchers. In this paper, we explain the challenges of the core grammar, provide a lexical resource, and implement the first language functions for the Arabic language. We did a human evaluation to evaluate our functions in generating sentences from the NADA Corpus.

## introduction the arabic language is a member of the semitic language family, the fifth most spoken language in the world. around 330 million people speak this language as their mother tongue language distributed in 25 countries. it is mainly classified into two types; classical and modern standard arabic (msa) @xcite , in addition to regional variances (dialects). as far as we know, there are not any pipeline nlg systems or libraries that support an arabic surface realiser at the time of writing this paper. two factors might be the reasons: arabic language complexity and limited resources. an example that reflects the second cause is that even the google n-gram viewer does not support the arabic language @xcite . therefore, in this paper, we are presenting six simple morphological msa natural language functions @xcite . these functions can be considered as a good start to build an arabic surface realiser, which can be used in a pipeline nlg system as they can be used to correctly generate and inflect different parts of speech including the generation of a noun phrase. we implemented these functions from scratch using a rule-based approach. furthermore, a lexicon containing around 9000 lemmata were collected. an open-source versions of lexicon and code with a web interface are provided and available online on github. foot_0 . ## related works as there is no nlg system done for arabic, we investigated similar approaches that targets the language challenges. @xcite presented a method to cover words agreements to generate correct words inflections in a translating task using a class-based agreement model. @xcite dealt with un-vocalisation and agglutination in arabic. they built a statistical model to extract grammatical rules from a treebank corpus to use these rules for parsing. furthermore, another member of the semitic group, which is the hebrew language, also lacks nlg resources. one attempt to generate a noun compounds in hebrew goes back to 1998 @xcite , they used a rule-based approach and listed the difficulties between english and hebrew in terms of the syntactic, semantic and lexical constraints for generating a noun compounds, including a special feature of hebrew called "smixut". for other languages, researchers used different realisers' technologies such as handcrafted grammars rule-based @xcite , statistical approaches @xcite and human-generated template-based @xcite . however, simplenlg is still considered the most popular method since it was presented by @xcite , it is a rulebased realiser that uses three main steps syntactic, morphological and orthographic realisations to produce a correctly inflected and high-quality output. it was adapted to many languages includ-ing (in chronological order) french (vaudry and lapalme, 2013), italian @xcite , spanish @xcite , galician @xcite , dutch (de jong and theune, 2018), mandarin @xcite and german @xcite . another popular approach is to provide grammars' functions to be used in a template-based nlg system. such as rosaenlg foot_1 ,which is an open-source, template-based, nlg library that gives simpler functions than simplenlg and had been used for english, french, german, italian and spanish languages. similar approaches with fewer options can be found in core-nlg foot_2 and xspin foot_3 . our approach is more similar to rosaenlg than to simplenlg. as we will provide language functions and a noun phrase function to give correct grammatical inflections and agreements. due to the huge morphological difference between the semitic and non-semitic languages, we could not adopt simplenlg nor rosaenlg. therefore, we found it convenient as a start to building our functions from scratch following the same approach of rosaenlg. ## arabic language challenges the arabic language consists of 28 alphabet letters, written from right to left and there is no letters capitalisation. it has a very rich morphology measured in the number of different forms for its nouns, verbs and particles @xcite . ## design and implementation of the arabic language functions in this section, we describe the arabic lexicon, list the arabic language functions proposed and what they cover from the challenges listed in section 3. type 1st person 2nd person 3rd person m-singularpossessive (ii)my (ka)your (hu)his m-plural-object (naa) -our (kum) -your (hum) -their m-singularsubject (ana) -i (anta) -you (huwa) -he ## evaluation these functions are the first arabic nlg functions and there are no other similar arabic approaches to compare with. we evaluated our approach using the same two methods used in @xcite . the robustness and correctness of the implementation of the arabic language functions were measured by examining 250 test units consist of different parts of speech (nouns, adjectives, verbs, numerals and pronouns). the results from the functions were all correct, whether it was inflected using a collections of rules or by using the lexical resource. to measure how much these functions can cover from the arabic language, nada corpus is used @xcite . the corpus categorised the data into ten categories. an average of 10 sentences extracted from each of the 10 categories provided in nada and tried to generate them with our language functions. this was successful in 85% of cases. most of the sentences that the system failed to reproduce were because of the use of other pronouns types or wrong verb inflections when a verb is used in the sentence that is not a form (i) type and not in the lexicon. such as this sentence " " (karejo silva scored the game's only goal in the second minute), the verb " " (scored) is a singular, past, not form (i) and does not exist in wiktionary. another failed sentence is " " (the match ended with the victory of sunderland, who halted liverpool's balance of 15 points), the word " " (who) is a relative pronoun which can take different inflections. therefore, this sentence considered as failed to reproduce, because relative pronouns are not yet covered by our functions. as mentioned before, as far as we know this is the first nlg system for arabic. therefore, we could not compare it with others. however, for a similar approach with different system architecture and german language, @xcite achieved 84%, which means our results are acceptable. in terms of scalability, these functions can be adapted easily to use in classical arabic and all arabic dialects (more than 20 dialects) abdul-mageed et al. @xcite . ## future work we plan to enhance the performance of the system by covering other verbs forms, clauses and other types of pronouns (demonstrative and relative pronouns). furthermore, we intend to increase the number of lemmata covered by the lexicon to get better accuracy, especially with weak verbs and broken plural cases. after that, we can create an arabic version of simplenlg. albert gatt and emiel krahmer. 2018. survey of the state of the art in natural language generation: core tasks, applications and evaluation. journal of artificial intelligence research, 61:65-170. albert gatt and ehud reiter. 2009. simplenlg: a realisation engine for practical applications. in proceedings of the 12th european workshop on natural language generation (enlg 2009), pages 90-93. spence green and john denero. 2012. a class-based agreement model for generating accurately inflected translations. in proceedings of the 50th annual meeting of the association for computational linguistics (volume 1: long papers), pages 146-155. ali idrissi, eiman mustafawi, tariq khwaileh, and r muralikrishnan. 2019. a neurophysiological study of noun-adjective agreement in arabic: the impact of animacy and diglossia on the dynamics of language processing. biorxiv, page 729855. zainab kadim igaab and intisar raham tarrad. 2019. pronouns in english and arabic: a contrastive study. engl. lang. lit. stud, 9(1). ruud de jong and mariët theune. 2018. going dutch: creating simplenlg-nl. in proceedings of the 11th international conference on natural language generation, pages 73-78. nabil khoufi, chafik aloulou, and lamia hadrich belguith. 2016. parsing arabic using induced probabilistic context free grammar. international journal of speech technology, 19(2):313-323. alessandro mazzei, cristina battaglino, and cristina bosco. 2016. simplenlg-it: adapting simplenlg to italian. in proceedings of the 9th international natural language generation conference, pages 184-192. yael netzer and michael elhadad. 1998. generation of noun compounds in hebrew: can syntactic knowledge be fully encapsulated? in natural language generation. alejandro ramos-soto, julio janeiro-gallardo, and alberto bugarín diz. 2017. adapting simplenlg to spanish. in proceedings of the 10th international conference on natural language generation, pages 144-148. karin c ryding. 2005. a reference grammar of modern standard arabic. cambridge university press. khaled shaalan, marwa magdy, and aly fahmy. 2015. analysis and feedback of erroneous arabic verbs. natural language engineering, 21(2):271. muayad abdul-halim ahmad shamsan and abdulmajeed attayib. 2015. inflectional morphology in arabic and english: a contrastive study. international journal of english linguistics, 5(2):139. pierre-luc vaudry and guy lapalme. 2013. adapting simplenlg for bilingual english-french realisation. in proceedings of the 14th european workshop on natural language generation, pages 183-187. a appendix 1 -definitions and parameters of the arabic language functions 1.1 ar_inflectverb returns the inflected form of the input verb taking into consideration five aspects (number, gender, person, voice and mood/tense). parameter type description word string (mandatory) the input verb. ideally, the verb root should be provided, however, if the word did not match any root in the db, then, all other forms will be checked. if the word did not match again, this time the verb will be inflected manually using the known rules of the verb inflections number number or string (optional) (default "plural" )the grammatical number of the verb. either singular, dual, plural or a number (not: dual or number 2 cannot be used with the 1st person) gender number or string (optional)