A rabic W ord N et: New Content and New Applications (2016)

Abstract: The Arabic WordNet project has provided the Arabic Natural Language Processing (NLP) community with the first WordNet-compliant resource. It allowed new possibilities in terms of building sophisticated NLP applications related to this Semitic language. In this paper, we present the new content added to this resource, using semi-automatic techniques, and validated by Arabic native-speaker lexicographers. We also present how this content helps in the implementation of new Arabic NLP applications, especially for Question Answering (QA) systems. The obtained results show the contribution of the added content. The resource, fully transformed into the standard Lexical Markup Framework (LMF), is made available for the community.

## introduction wordnets are important as lexical resources containing not only words of the targeted language but also synsets and semantic relations between them such as synonymy, meronymy and antonymy. synsets are groups of words that each can substitute others in a sentence without changing its general meaning. therefore, in natural language processing (nlp), various applications used this information, especially query expansion (qe), information retrieval (ir) and question answering (qa) systems. thus, the development of new wordnets targeting new languages and dialects and/or enriching existing ones, witnessed regular experiences and research works. arabic, as a semitic language spoken by around 300 million people worldwide, is concerned by these experiences as well as the use of arabic wordnet (awn) @xcite in recent nlp applications such as information retrieval @xcite elearning of arabic @xcite , semantic-based applications @xcite , conceptual search (al-zoghby and shaalan 2015), etc. after the first release of awn, there were many attempts to enrich its content (al khalifa @xcite @xcite . nevertheless, the gap between awn and the arabic language as well as other similar wns remained one of the limitations for its use. also, some particularities of the arabic language, including broken plurals (bp), has not been sufficiently addressed. indeed, the morphological analysis of bp is not an easy task in nlp since they are irregular forms of plurals, and cannot be identified using patterns. making these bp forms in a resource such as awn is much helpful for the developers of nlp applications. in previous research @xcite , we presented a new content that has been added to the awn in order to cover more words and synsets and, therefore, enhance the usability of this resource for arabic nlp applications. this paper keeps on the track of this previous research by presenting the last experiments con-ducted using the new content of awn and the different refinements brought by manual validation made by lexicographers. this paper also presents the transformation of the new content of awn into the lexical markup framework (lmf) format in order to make this resource available for the community in the context of the open multilingual wordnet project @xcite , providing free access to wordnets in several languages in a common format. the paper is organized as follows: section 2 recalls some existing works around the awn enrichment and its use in nlp applications. section 3 draws a synthesis of the main techniques that we used to enrich awn. section 4 presents the conducted experiments to show the usability of the new content as well as to validate this content. section 5 addresses the transformation of the enriched awn into the lmf format. finally, section 6 provides a conclusion of this research and highlights the future works. ## related works the awn project followed the development of wordnets for other languages, including euro wordnet @xcite by focusing, first, on the most common concepts and word-senses in pwn 2.0 @xcite . the first release was available on 2007 foot_0 @xcite . the first awn release contains 9,698 synsets, corresponding to 21,813 modern standard arabic (msa) words, and 6 different relation types @xcite . a later version of awn, is also available and contains 11,269 synsets, corresponding to 23,841 words, including new named entities @xcite . 2 this content is smaller than the pwn 2.0 and much smaller than what is expected by arabic nlp applications. indeed, although various research experiences used awn in many arabic nlp applications, the common limitation reported in these experiences was the shortcomings of this resource in terms of the coverage of the arabic language. in a previous work @xcite , we made a comparison between the size of awn and a dictionary for msa on one side, and between awn and english and spanish wns on another side. this comparison allowed us to measure the gap to be filled in to improve the quality and usability of this resource. the previously mentioned experiences in using awn @xcite @xcite @xcite @xcite show the importance of the target community once this resource is enriched with a content that suits the size of msa and the expectations by arabic nlp applications. in this direction, it is worth mentioning that the experiences reported in other languages (english, spanish, etc.) show the opportunity to use wordnets in more sophisticated and complex applications, such as humour detection @xcite . nevertheless, awn remains one of the few and important resources that can support the development of arabic nlp applications regarding the following findings that we reported in @xcite : despite the above advantages of the awn project, there were just a few attempts to enrich its content. these attempts relied on existing tools and resources. del @xcite proposed an enhancement of arabic wordnet content using the pwn and aramorph bilingual dictionary as bilingual resource. 3 this attempt resulted in adding new words and synonyms. there was also another attempt to build a wordnet for an arabic dialect based on the content of awn and a parallel dictionary @xcite . @xcite proposed a linguistic method based on two steps to enrich awn. the first step defines morpho-lexical patterns and the second step enriches semantic relations using these patterns. the wikipedia resource is also used in both steps. in comparison with those attempts, our approach is particular in that: (i) it uses techniques and resources with higher confidence, (ii) it is followed by a significant validation by lexicographers, and (iii) the usability of the enriched awn resource is proven through experiments in the context of real-world application, i.e., arabic qa. ## new experiments using the enriched awn in @xcite @xcite , experiments were conducted in the field of arabic qa systems using the awn that we enriched following the techniques described above. let us recall that the experimental process relied on a three-level approach for arabic qa as follows: the effectiveness of the new content added in awn was proven by means of different experiments previously reported in @xcite : in this paper, we reconducted experiment #1 using the 2,264 clef and trec questions @xcite @xcite @xcite @xcite @xcite , considering also the semanticbased level. this is to assess the ability of the enriched awn to improve the three levels of the arabic qa approach regardless the considered test-set and passage collection (the qa4mre test-set has local collection, the clef and trec are assigned a web collection). the obtained results show a significant improvement in terms of c@1. indeed, the threelevel approach reaches a 0.51 c@1 which is higher than the 0.21 c@1 registered by the participating arabic qa systems in qa4mre track 2012, including the idraaq system @xcite ) that we built. according to results of the manual validation of the new content as well as the promising results obtained with this content in a challenging task like arabic qa, we decided to make available this content for the community of arabic nlp. the next section shows the steps followed to achieve this goal., ## conclusion and future works as a conclusion, the awn project is important for arabic nlp as witnessed by various attempts and research having used this resource. however, its content needs much extension and refinement. our research tries to fill in a part of the gap registered between the current coverage of awn and the expected one. we presented in this paper the last experiments and manual validation of the awn enrichment proposed in previous research @xcite . this enrichment was based on semi-automatic techniques and used external resources, thus the added content required refinement brought by lexicographers. also, the experiments conducted in the context of this paper shows a significant improvement of arabic qa after using the enriched content of awn. the new experiments together with the results previously presented in @xcite show that regardless the considered test-set of questions, the new awn content allowed an improvement of arabic qa performance through various measures, especially the c@1 (from 0.21 to 0.51 after using the enriched awn as a lexical resource in the keyword-based level and as a support resource for the semantic-based level). thus, the new release of awn, manually validated by lexicographers and experimentally tested in the context of arabic qa, is now available for the community in its lmf format 7 . the process of transformation into this format was described in section 5 of this paper. as future works, we can mention the requirement to add new relation types such as mer7 the awn v2 can be downloaded from the open multilingual wordnet project web site. the resource is available at: http://compling.hss.ntu.edu.sg/omw/ onymy and antonymy that currently are slightly present in awn. in addition, new techniques and resources could be investigated for this enrichment.