Introduction
Computational Linguistics (CL) is concerned with building accurate linguistic computational models. Natural Language Processing (NLP) is concerned with automating the understanding of natural language. CL and NLP tasks range from simple ones such as spell checking and typing error correction to more complex tasks including named entity recognition (NER), cross-document analysis, machine translation, and relational entity extraction BIBREF0 , BIBREF1 . Entities are elements of text that are of interest to an NLP task. Relational entities are elements that connect entities. Annotations relate chunks of text to labels denoting semantic values such as entities or relational entities. We refer to annotations and labels as tags and tag types, respectively, in the sequel.
Supervised and unsupervised empirical learning techniques tackle NLP and CL tasks. They employ machine learning without the need to manually encode the requisite knowledge BIBREF2 . Supervised learning techniques require training corpora annotated with correct tags to learn a computational model. Supervised and unsupervised techniques require annotated reference corpora to evaluate the accuracy of the technique using metrics such as precision and recall BIBREF3 , BIBREF4 , BIBREF5 .
Researchers build training and reference corpora either manually, incrementally using learning techniques, or using knowledge-based annotation techniques that recognize and extract entities and relational entities from text. Knowledge-based techniques use linguistic and rhetorical domain specific knowledge encoded into sets of rules to extract entities and relational entities BIBREF2 . While existing annotation, entity, and relational entity extraction tools exist BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , most of them lack Arabic language support, and almost all of them lack Arabic morphological analysis support BIBREF12 . Fassieh BIBREF13 is a commercial Arabic annotation tool with morphological analysis support and text factorization. However, this tool lacks support for entity and relational entity extraction.
Figure illustrates the target of MERF using the directions to Dubai Mall example . The figure also presents a transliteration and an English translation of the Arabic text. The framed words in the text are entities referring to names of people ( INLINEFORM0 ), names of places ( INLINEFORM1 ), relative positions ( INLINEFORM2 ), and numerical terms ( INLINEFORM3 ). We would like to extract those entities, and then extract the relational entities forming the graph in Figure where vertices express entities, and edges represent the relational entities.
In this paper, we present MERF, a morphology-based entity and relational entity extraction framework for Arabic text. MERF provides a user-friendly interface where the user defines tag types and associates them with regular expressions over Boolean formulae. A Boolean formula is defined by a term, negation of a term, or disjunction of terms. Terms are matches to Arabic morphological features including prefix, stem, suffix, part of speech (POS) tags, gloss tags, extended synonym tags, and semantic categories. For example, entity INLINEFORM0 in Figure has a “place” semantic category. MERF regular expressions support operators such as concatenation, zero or one, zero or more, one or more, up to INLINEFORM1 repetitions where INLINEFORM2 is a non-zero positive integer, and logical conjunction and disjunction. For example, the sequence between INLINEFORM3 and INLINEFORM4 matches a regular expression INLINEFORM5 that requires two semantic place categories with a place-preposition POS tag ( INLINEFORM6 ) in between.
An editor allows the user to associate an action with each subexpression. The user specifies the action with C++ code and uses an API to access information related to the matches such as text, position, length, morphological features, and numerical value. Each regular expression is associated with a named identifier to form a local grammar like structure BIBREF14 . A relation definition GUI allows the user to provide relational tuples where each tuple has a source, a destination and an edge label. The user uses the regular expression identifiers to define the relational tuple elements. For example, the relation between INLINEFORM0 and INLINEFORM1 shown in Figure is a match of a relational tuple over the components of INLINEFORM2 . We refer to regular expressions and Boolean formulae as expressions and formulae, respectively. We also refer to expressions as rules when used in a grammar context; e.g. when used with an identifier.
MERF takes an Arabic text and the local grammar defined by the Boolean formulae and the regular expressions. MERF computes the morphological solutions of the input text then computes matches to the Boolean formulae therein. MERF then generates a non-deterministic finite state automata (NDFSA) for each expression and simulates it with the sequence of Boolean formulae matches to compute the regular expression matches. MERF generates executable code for the actions associated with the regular expressions, compiles, links, and executes the generated code as shared object libraries. Finally, MERF constructs the semantic relations and cross-reference between entities. MERF also provides visualization tools to present the matches, and estimate their accuracy with respect to reference tags.
This work significantly extends JaZaMatar BIBREF15 that allows for manual, and morphology annotation. MERF enables a user to incrementally create complex annotations for Arabic based on automatic extraction of morphological tags through a user-friendly interactive interface. MERF has the following advantages.
In MERF, we make the following contributions.
The rest of the paper is structured as follows. Section SECREF2 introduces Arabic morphological analysis and its important role in Arabic NLP. Section SECREF3 explains the methodology of MERF. Section SECREF4 presents MERF components. Section SECREF5 presents MERF GUI. Section SECREF6 presents and discusses related work. Section SECREF7 presents the evaluation results. Finally, we conclude and discuss future work in Section SECREF8 .
Background: Morphological Analyzer
Morphological analysis is key to Arabic NLP due to the exceptional degree of ambiguity in writing, the rich morphology, and the complex word derivation system BIBREF16 , BIBREF17 , BIBREF18 . Short vowels, also known as diacritics, are typically omitted in Arabic text and inferred by readers BIBREF12 . For example, the word bncan be interpreted as bon (“coffee”) with a damma diacritic on the letter utf8بـstandard or bin (“son of”) with a kasra diacritic on the letter utf8بـstandard.
Morphological analysis is required even for tokenization of Arabic text. The position of an Arabic letter in a word (beginning, middle, end, and standalone) changes its visual form. Some letters have non-connecting end forms which allows visual word separation without the need of a white space separator. For example, the word utf8ياسمينstandard can be interpreted as the “Jasmine” flower, as well as utf8ياstandard (the calling word) followed by the word utf8سمينstandard (obese). Consider the sentence AlmdrsTdhb alwald-ilA dhb alwald-ilA almdrsT (“the kid went to school”). The letters utf8دstandard and utf8ىstandard have non-connecting end of word forms and the words utf8الولدstandard,utf8الىstandard, andutf8المدرسةstandard are visually separable, yet there is no space character in between. Newspaper articles with text justification requirements, SMS messages, and automatically digitized documents are examples where such problems occur.
MERF is integrated with Sarf, an in-house open source Arabic morphological analyzer based on finite state transducers BIBREF19 . Given an Arabic word, Sarf returns a set of morphological solutions. A word might have more than one solution due to multiple possible segmentations and multiple tags associated with each word. A morphological solution is the internal structure of the word composed of several morphemes including affixes (prefixes and suffixes), and a stem, where each morpheme is associated with tags such as POS, gloss, and category tags BIBREF16 , BIBREF20 .
Prefixes attach before the stem and a word can have multiple prefixes. Suffixes attach after the stem and a word can have multiple suffixes. Infixes are inserted inside the stem to form a new stem. In this work we consider a set of stems that includes infix morphological changes. The part-of-speech tag, referred to as POS, assigns a morpho-syntactic tag for a morpheme. The gloss is a brief semantic notation of morpheme in English. A morpheme might have multiple glosses as it could stand for multiple meanings. The category is a custom tag that we assign to multiple morphemes. For example, we define the Name of Person category to include proper names.
We denote by INLINEFORM0 , INLINEFORM1 , INLINEFORM2 , INLINEFORM3 , INLINEFORM4 , and INLINEFORM5 , the set of all stems, prefixes, suffixes, POS, gloss, and user defined category tags, respectively. Let INLINEFORM6 be a set of Arabic words denoting the text documents. MERF uses Sarf to compute a set of morphological solutions INLINEFORM7 for each word INLINEFORM8 . Each morphological solution INLINEFORM9 is a tuple of the form INLINEFORM10 where INLINEFORM11 , INLINEFORM12 , INLINEFORM13 , INLINEFORM14 , and INLINEFORM15 . INLINEFORM16 and INLINEFORM17 are the POS, gloss and category tags of prefix INLINEFORM18 . INLINEFORM19 and INLINEFORM20 are the POS, gloss and category tags of suffix INLINEFORM21 . INLINEFORM22 and INLINEFORM23 are the POS, gloss and category tags of stem INLINEFORM24 . Intuitively, INLINEFORM25 and INLINEFORM26 are concatenations of prefix, suffix, POS, gloss and category values, respectively.
Table TABREF9 shows the morphological analysis of the word utf8فَسَيَأْكُلهاstandard. The word is composed of the prefix morphemes utf8فَstandard, utf8سَstandard, and utf8يَstandard, followed by the stem utf8أْكُلstandard, and then followed by the suffix morpheme utf8هاstandard. Each morpheme is associated with a number of morphological features. The CONJ, FUT, IV3MS VERB_IMPERFECT, and IVSUFF_DO:3FS POS tags indicate conjunction, future, third person masculine singular subject pronoun, an imperfect verb, and a third person feminine singular object pronoun, respectively. The POS and gloss notations follow the Buckwalter notation BIBREF21 .
MERF Methodology
Figure illustrates the four processes involved in MERF methodology. The first process takes Arabic text and provides the user with a morphology-based Boolean (MB) formulae GUI. The user interactively composes MB-formulae using the GUI and the output of the simulator and the INLINEFORM0 detector. The simulator and the detector apply the formulae over the morphological solutions of the Arabic text and produce the MB-formulae tags.
The second process takes the MB-formulae tags and the Arabic text and provides the user with a morphology-based grammar rule GUI. The user interactively composes MB-grammar rules using the GUI and the output of the MB-grammar rule simulator. The grammar rule simulator applies the rules over the MB-formulae tags and produces the MB-grammar rule tags.
The third process takes the MB-grammar rule tags and provides the user with a relation and action GUI. The user interactively provides (1) the relation definitions and (2) the actions in terms of identifiers from the MB-grammar rules. The relation extraction produces the target entities and relational entities. The action execution enriches the entities and the relational entities with powerful semantics. For example, users can utilize actions to compute statistical features, store intermediate results, or apply intelligent entity inference techniques as we show later in the numerical extraction example of Subsection SECREF41 . Finally, in the fourth process the user compares the results with golden reference chunks and visualizes the difference. This allows the user to refine the formulae, rules, relations and actions.
After relation extraction, we are interested to relate entities that express the same concept. MERF provides the extended synonym feature of second order as a default cross-reference relation ( INLINEFORM0 ). In Figure , triggering this feature creates the edge labeled with isSyn between the nodes Khalifa Tower and The building.
The user may refine the defined formulae, rules and relations and the corresponding formulae tags, rule tags, entities and relational entities either using the GUI or directly through readable output files. The files are in the javascript object notation (JSON) BIBREF22 format that is intuitive to read and modify. MERF separates the user defined formulae, rules, actions and relations in a MERF tag type file and the matching tags in a tags files. The separation serves the user to apply the tag types to multiple case studies and to obtain a separate file of resulting tags for each.
The extended synonymy feature Syn k Syn^k
Up to our knowledge, INLINEFORM0 provides the first light Arabic WordNet based on the lexicon of Sarf. The sets INLINEFORM1 and INLINEFORM2 denote all English words, Arabic words, and Arabic lexicon words, respectively. Recall that INLINEFORM3 and INLINEFORM4 denote the set of glosses and stems in the morphological analyzer, respectively. We have INLINEFORM5 and INLINEFORM6 . Function INLINEFORM7 maps Arabic stems to subsets of related English glosses, where INLINEFORM8 denotes the power set of INLINEFORM9 which is the set of all subsets of INLINEFORM10 . Function INLINEFORM11 maps Arabic lexicon words to subsets of relevant Arabic stems.
Given a word INLINEFORM0 , INLINEFORM1 is the set of Arabic stems directly related to INLINEFORM2 through the gloss map. Let INLINEFORM3 denote stems related to INLINEFORM4 using the gloss map of order INLINEFORM5 recursively such that INLINEFORM6 and INLINEFORM7 . Formally, INLINEFORM8 for INLINEFORM9 . The example in Figure FIGREF11 illustrates the computation. Let INLINEFORM10 denote an input Arabic word mA', which has the gloss water, i.e. water INLINEFORM11 . INLINEFORM12 shares this gloss with the stem n.d.h, denoted INLINEFORM13 , i.e. INLINEFORM14 . Next, the stem rss, denoted INLINEFORM15 , shares the gloss spray with INLINEFORM16 , i.e. INLINEFORM17 . Therefore, INLINEFORM18 relates the words mA' and rss.
MRE: Morphology-based regular expressions
Let INLINEFORM0 be the set of atomic term predicates, where INLINEFORM1 and INLINEFORM2 denote exact match and containment, respectively. Also, let INLINEFORM3 be the set of morphological features where each morphological feature INLINEFORM4 is in turn a set of morphological feature values. Given a word INLINEFORM5 , a user defined constant feature value INLINEFORM6 , and an integer INLINEFORM7 , the following are morphology-based atomic terms (MAT), terms for short.
INLINEFORM0 where INLINEFORM1 , INLINEFORM2 , and INLINEFORM3 . Informally, a solution vector of INLINEFORM4 exists with a feature containing or exactly matching the user-chosen feature value INLINEFORM5 .
INLINEFORM0 . Informally, this checks if INLINEFORM1 is an extended synonym of a stem INLINEFORM2 . We limit INLINEFORM3 to a maximum of 7 since we practically noticed that (1) values above 7 introduce significant semantic noise and (2) the computation is expensive without a bound.
A morphology-based Boolean formula (MBF) is of the following form.
INLINEFORM0 and INLINEFORM1 are MBF formulae where INLINEFORM2 is a MAT and INLINEFORM3 is the negation operator.
INLINEFORM0 is an MBF where INLINEFORM1 and INLINEFORM2 are MBF formulae, and INLINEFORM3 is the disjunction (union) operator.
Moreover, MERF provides INLINEFORM0 to be a default Boolean formula that tags all other words in the text that do not match a user defined formula. We also refer to those words as null words.
Consider the task we discussed in the introduction (Figure ) and recall that we are interested in identifying names of people, names of places, relative positions, and numerical terms. Table presents the defined formulae. The user denotes the “name of person” entities with formula INLINEFORM0 which requires the category feature in the morphological solution of a word to be Name_of_Person. The entities INLINEFORM1 , INLINEFORM2 , and INLINEFORM3 are matches of the formula INLINEFORM4 in the text. Similarly, the user specifies formula INLINEFORM5 to denote “name of place” entities. The user specifies formula INLINEFORM6 to denote “relative position” entities, and defines it as a disjunction of terms that check for solutions matching stems such as qrb (“near”) and fy (“in”). Similarly, INLINEFORM7 denotes numerical terms and is a disjunction of constraints requiring the stem feature to belong to a set of stems such as 'wl(“first”), tAny(“second”), ...`Asr(“tenth”).
Next, we define a morphology-based regular expression (MRE) as follows.
INLINEFORM0 is an MRE where INLINEFORM1 is an MBF.
INLINEFORM0 is an MRE where INLINEFORM1 and INLINEFORM2 are both MRE expressions. A match of INLINEFORM3 followed by a match of INLINEFORM4 satisfies this concatenation operation.
INLINEFORM0 ⌃ INLINEFORM1 and INLINEFORM2 are MRE where INLINEFORM3 is an MRE, and are satisfied by zero or more, one or more, up to INLINEFORM4 matches, and an optional single match of INLINEFORM5 , respectively.
INLINEFORM0 (conjunction) and INLINEFORM1 (disjunction) are MRE where INLINEFORM2 and INLINEFORM3 are MRE, and are satisfied by the intersection of INLINEFORM4 and INLINEFORM5 matches, and the union of the INLINEFORM6 and INLINEFORM7 matches, respectively.
We denote by INLINEFORM0 the set of matches of an MRE INLINEFORM1 .
Back to the example in Figure . We use the formulae defined in Table to construct an MRE such as INLINEFORM0 where INLINEFORM1 and INLINEFORM2 denote disjunction, one or more, zero or one, and up to INLINEFORM3 matches, respectively. The expression specifies a sequence of places or names of persons, optionally followed by a null word, followed by one relative position, followed by up to two possible null words, followed by one or more match of name of place, name of person, or numerical term. INLINEFORM4 and INLINEFORM5 are used in the expression to allow for flexible matches.
The matching parse trees in Figure illustrate two matches of the expression computed by MERF. The first tree refers to the text brj hlyfT bAlqrb mn AltqA.t` Al-'wl(“Khalifa Tower next to the first intersection”). The second tree refers to the text dby mwl `lA mqrbT mn hdA AlmbnA(“Dubai Mall is located near this building”). The leaf nodes of the trees are matches to formulae and the internal nodes represent roots to subexpression matches. For instance, brj hlyfT in match 1 tree corresponds to the subexpression INLINEFORM0 .
User-defined relations and actions
A relation is defined by the user as a tuple INLINEFORM0 where INLINEFORM1 and INLINEFORM2 are identifiers associated with subexpressions of an MRE INLINEFORM3 . Matches of the relation are a set of labeled binary edges where matches of INLINEFORM4 and INLINEFORM5 are the source and destination nodes and matches of INLINEFORM6 are the edge labels. We denote INLINEFORM7 to be the set of matches of the corresponding relation, and we refer to them as relational entities.
We are interested in constructing the relational entity graph in Figure . Let INLINEFORM0 , INLINEFORM1 , INLINEFORM2 , INLINEFORM3 , and INLINEFORM4 be identifiers to the subexpressions INLINEFORM5 , INLINEFORM6 , INLINEFORM7 , INLINEFORM8 , and INLINEFORM9 , respectively. The matches to INLINEFORM10 , INLINEFORM11 , INLINEFORM12 , and INLINEFORM13 in match 1 (Fig. ) are brj hlyfT (“Khalifa Tower”), bAlqrb (“next”), mn (“to”), and AltqA.t` Al-'wl (“first intersection”). Note that there is no match to the optional INLINEFORM14 formula in match 1. Similarly, the matches to INLINEFORM15 , INLINEFORM16 , INLINEFORM17 , INLINEFORM18 , and INLINEFORM19 in the second matching tree are dby mwl (“Dubai Mall”), `lA (“is located”), mqrbT (“near”), mn hdA (“this”), and AlmbnA (“building”), respectively.
We define the semantic relations INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 . Relation INLINEFORM3 creates the edge labeled next to between Khalifa tower and intersection 1 nodes from match 1, and the edge labeled near between Dubai Mall and the building nodes from match 2. Relation INLINEFORM4 creates the edge labeled prep between Dubai Mall and near nodes from match 2. Relation INLINEFORM5 creates the edge labeled from between intersection 1 and next to nodes in match 1, and the edge labeled from this between near and the building nodes in match 2.
Moreover, MERF allows advanced users to write C++ code snippets to process matches of subexpressions. Each subexpression can be associated with two computational actions: pre-match and on-match. MERF provides an API that enriches the actions with detailed access to all solution features of an expression or a formula match including text, position, length, equivalent numerical value when applicable, and morphological features. The API follows a decorator pattern in that it incrementally adds the action results to the matching entities. Once MERF computes all matching parse trees, it traverses each tree to execute the user defined pre-match actions in pre-order manner and the on-match actions in post-order manner. This follows an observer pattern that notifies listeners with each produced match.
MERF simulators
The set of tag types INLINEFORM0 contains tuples of the form INLINEFORM1 where INLINEFORM2 is a text label with a descriptive name, INLINEFORM3 is an MRE, and INLINEFORM4 is a visualization legend with font and color information. For the example of Figure , INLINEFORM5 is “direction”, INLINEFORM6 is INLINEFORM7 , and INLINEFORM8 is italic.
For each word INLINEFORM0 . MERF computes a Boolean value for all MBFs. For example, brj matches MBF INLINEFORM1 . Then, it computes the set of MBF tags INLINEFORM2 which tags a word INLINEFORM3 with INLINEFORM4 iff the MBF INLINEFORM5 associated with tag type INLINEFORM6 is true for INLINEFORM7 . The MBF evaluation results in a sequence of tag sets INLINEFORM8 . If a word INLINEFORM9 has no tag type match, its tag set INLINEFORM10 is by default the singleton INLINEFORM11 . For example, the tag sets for the text in Figure follows INLINEFORM12 brj INLINEFORM13 xlyfT INLINEFORM14 .
utf8
standard
For each MRE, MERF generates its equivalent non-deterministic finite automaton (NFA) in the typical manner BIBREF23 . We support the upto operation ( INLINEFORM0 INLINEFORM1 ), which is not directly supported in sipser2012introduction BIBREF23 , by expanding it into a regular expression form; for example INLINEFORM2 3 is equivalent to INLINEFORM3 . Consider the example of Figure and the corresponding expression INLINEFORM4 . Figure FIGREF23 shows part of the corresponding NFA where INLINEFORM5 represent NFA states, and edges are transitions based on MBF tags such as INLINEFORM6 and INLINEFORM7 . Edges labeled with the empty string INLINEFORM8 are non-deterministic.
MERF simulates the generated NFA over the sequence of tag sets matching the MBF formulae. A simulation match INLINEFORM0 of an expression INLINEFORM1 is a parse tree where the root spans the expression, the internal nodes are roots to subexpressions of INLINEFORM2 , and the leaves are matches of the MBF formulae of INLINEFORM3 , e.g. Figure . The sequence of leaf matches forms a vector of tags INLINEFORM4 corresponding to the text sequence INLINEFORM5 where INLINEFORM6 . If we have more than one match for an expression, MERF returns the longest.
Finally, MERF computes the relational entities corresponding to each user defined relation INLINEFORM0 INLINEFORM1 .
MERF GUI
MERF provides a user friendly interface to specify the atomic terms, the MBFs, the MREs, the tag types, and the legends. The GUI also allows the user to modify and correct the tag set INLINEFORM0 . The GUI allows the user also to compute accuracy results that compare different tag sets and that can serve well as inter annotation agreement results when the tag sets come from two human annotators, or as evaluation results when comparing with reference tag sets.
Tag type Boolean formula editor
The user writes MBF tag types with the tag type editor introduced in JaZaMatar BIBREF15 . First the user specifies atomic terms by selecting a feature from INLINEFORM0 . The user can also choose whether to require an exact match using the isA predicate, or a substring match using the contains predicate option.
The user can add and remove feature values to the atomic terms using push buttons. A check box in the “Feature” column allows negating the term, and the “Relation” column switches the predicate between isA and contains. The list of feature and value pairs is interpreted as a disjunction to form the MBF. A right pane shows a description of the tag type and a set of legend descriptors. When the stem or gloss features are selected, the user has the option to use the INLINEFORM0 feature.
In the direction extraction task example, the user specifies four MBF-based tag types with labels INLINEFORM0 , INLINEFORM1 , INLINEFORM2 , and INLINEFORM3 with “name of person”, “name of place”, “relative position”, and “numerical term” descriptions, respectively. For each MBF, the user selects the morphological features, specifies the constant value INLINEFORM4 , and adds it to the Boolean formula editor.
MBF match visualization
The MBF match visualizer shows color sensitive text view, the tag list view, and the tag description view. The tag description view presents the details of the selected tag along with the relevant tag type information. The user can edit the tags using a context sensitive menus. MERF GUI also allows manual tag types and corresponding tags that are not based on morphological features. This enables building reference corpora without help from the morphological analyzer.
Tag type regular expression editor
After interacting with the MBF editor, the user moves to specify the regular expressions. The MRE editor of Figure FIGREF26 allows the definition of an MRE tag type in a user-friendly manner. The user first adds the required MBF formulae by selecting a label from INLINEFORM0 under MBFs. The Boolean formula of a highlighted tag type is shown in the table on the lower left pane. Each selected MBF is associated with an automatic name. The user can nest the MRE expression using a tree view of the MRE operations. The tree features the name, MBF, and operation for each subexpression.
To specify a binary operation the user selects two subexpressions and clicks the corresponding operation button. The operations include disjunction, conjunction, zero or one, sequence, zero or more, one or more, and up to a user defined constant. The right pane shows a description of the tag type and a set of legend descriptors.
MRE match visualization
While specifying an MRE, the user can interact with the visualization and editor views to make sure the MRE expresses the intent. The color-sensitive text view in Figure FIGREF29 shows the highlighted tag matches after the user called the MRE simulator using the Tagtypes menu.
The matching parse tree view shows the selected match in a graph view. Figure FIGREF29 shows the matching parse tree of the direction task dby mwl `lA mqrbT mn hdA AlmbnA(“Dubai Mall is located near this building”).
User defined relation editor
After the user is satisfied with the MRE matches, the user moves to define relations and code actions. The relation editor allows the user to define relations by specifying INLINEFORM0 tuples, where INLINEFORM1 and INLINEFORM2 denote source and destination entities, and INLINEFORM3 denotes the label. The editor shows the MRE tree and allows the user to select the subexpressions and select features of the matches of the subexpressions to define the three components of the relation.
A snapshot of the GUI in Figure FIGREF29 shows in an interactive graph view the entity-relation graph of the match of the user defined relation extracted from the matching parse tree of the MRE. In the computational action editor, an advanced user can enter C++ code and use the MERF API to program and process subexpression matches.
Analysis
In the analysis view, the user provides two tag sets INLINEFORM0 and INLINEFORM1 and two tag type sets INLINEFORM2 and INLINEFORM3 as input. The tag type difference view shows the text annotated in three panes: (i) the common tag types INLINEFORM4 , (ii) the tag types in INLINEFORM5 but not in INLINEFORM6 , and (iii) the tag types in INLINEFORM7 and not in INLINEFORM8 . Similarly, the tag difference view shows INLINEFORM9 , INLINEFORM10 and INLINEFORM11 in addition to precision, recall and F-measure values. The user selects a predicate to compute the metrics from the following predicates: (1) “Intersection”: a tag from INLINEFORM12 intersects in text with a tag in INLINEFORM13 , (2) “Exact”: a tag from INLINEFORM14 exactly matches a tag in INLINEFORM15 , (3) “A includes B”: a tag from INLINEFORM16 contains a tag from INLINEFORM17 , and (4) “B includes A”: a tag from INLINEFORM18 contains a tag from INLINEFORM19 .
Related Work
In this section we review the literature on entity and relation IE and on automatic and manual annotation techniques and compare to MERF.
Information Extraction. The common pattern specification language (CPSL) targets system independent IE specifications BIBREF24 . MERF extends CPSL with Arabic morphological features, code actions, and user defined relations. SystemT BIBREF6 aims to overcome the limitations of CPSL. It is based on an algebraic approach to declarative information extraction, uses the declarative annotation query language (AQL), and uses an optimizer to generate high performance execution plans for the AQL rules. MERF supports multiple tags per word, and supports the MRE conjunction operator which overcomes the overlapping annotation problem discussed in SystemT.
TEXTMARKER is a semi-automatic rule-based IE system for structured data acquisition BIBREF7 . Both TEXTMARKER and MERF provide the user with GUI editor and result visualizer.
The work in urbain2012user BIBREF8 presents a user-driven relational model and targets entity and relation extraction. The user enters a natural language query, and uses the OpenNLP toolkit to extract tags and relations from the query. Similar to MERF, the system constructs entities and relations.
QARAB is an Arabic question answering system that takes an Arabic natural language query and provides short answers for it BIBREF25 . QARAB uses traditional information retrieval techniques and an outdated Arabic NLP analyzer with limited features of Arabic words compared to the morphological analysis of MERF.
Table TABREF32 summarizes the comparison between MERF and other systems. MERF differs in that it provides code actions, user defined relations, and an interactive graph visualization of the relational entities. It also differs in that it fully supports Arabic morphological analysis while only QARAB supports Arabic linguistic features using a parser, and the work in urbain2012user BIBREF8 uses OpenNLP that currently lacks full support for Arabic morphological features. Similar to TEXTMARKER, MERF has the advantage of providing a user-friendly interactive interface to edit the entity and relational specifications and visualize the results.
DUALIST is an annotation system for building classifiers for text processing tasks using machine learning techniques BIBREF9 . MERF doesn't support classification tasks. However, MERF provides an interactive GUI where the user can edit MBF and MRE tags. This interactive environment contributes to the regular expression extraction and semantic relation construction which increases the overall accuracy.
Another track in the literature targets specific tasks such as NER using statistical and machine-learning techniques such as maximum entropy, optimized feature sets and conditional random fields BIBREF26 , BIBREF27 , BIBREF28 , BIBREF29 . Knowledge-based techniques such as zaghouani2010adapting BIBREF30 and traboulsi2009arabic BIBREF14 propose local grammars with morphological stemming. ZaMaHaCicling2012Entity BIBREF31 extract entities and events, and relations among them, from Arabic text using a hierarchy of manually built finite state machines driven by morphological features, and graph transformation algorithms. Such techniques require advanced linguistic and programming expertise.
WordNet is a lexical reference system that mimics human lexical memory and relates words based on their semantic values and their functional categories: nouns, verbs, adjectives, adverbs, and function words BIBREF32 . The INLINEFORM0 feature in MERF is inspired by WordNet.
Annotation tools. MMAX2 is a manual multi-level linguistic annotation tool with an XML based data model BIBREF10 . BRAT BIBREF11 and WordFreak BIBREF33 are manual multi-lingual user-friendly web-based annotators that allow the construction of entity and relation annotation corpora. Knowtator BIBREF34 is a general purpose incremental text annotation tool implemented as a Protégé BIBREF35 plug-in. Protégé is an open-source platform with a suite of tools to construct domain models and knowledge-based applications with ontology. However, it doesn't support the Arabic language.
MERF differs from MMAX2, BRAT, WordFreak, and Knowtator in that it is an automatic annotator that allows manual corrections and sophisticated tag type and relation specifications over Arabic morphological features.
kholidy2010towards BIBREF36 present an overview of annotation tools and concludes with a set of rules and guidelines needed in an Arabic annotation alignment tool. The work in dukes2013supervised BIBREF37 presents a collaborative effort towards morphological and syntactic annotation of the Quran. dorr2010interlingual BIBREF38 present a framework for interlingual annotation of parallel text corpora with multi-level representations. kulick2010consistent BIBREF39 presents the integration of the Standard Arabic Morphological Analyzer (SAMA) into the workflow of the Arabic Treebank.
The work in smrz2004morphotrees BIBREF40 presents a customizable general purpose tree editor, with the Arabic MorphoTrees annotations. The MorphoTrees present the morphological analyses in a hierarchical organization based on common features.
Task specific annotation tools such as alrahabi2006semantic BIBREF41 use enunciation semantic maps to automatically annotate directly reported Arabic and French speech. AraTation is another task specific tool for semantic annotation of Arabic news using web ontology based semantic maps BIBREF42 . We differ in that MERF is general, and not task specific, and it uses morphology-based features as atomic terms. Fassieh is a commercial Arabic text annotation tool that enables the production of large Arabic text corpora BIBREF13 . The tool supports Arabic text factorization including morphological analysis, POS tagging, full phonetic transcription, and lexical semantics analysis in an automatic mode. Fassieh is not directly accessible to the research community and requires commercial licensing. MERF is open source and differs in that it allows the user to build tag types and extract entities and relations from text.
Results
In this section we evaluate MERF with four case studies. We perform a survey-like evaluation where developers manually built task specific information extraction tools for the case studies and other developers built equivalent MERF tools. The aim of the comparison is to showcase that MERF enables fast development of linguistic applications with similar accuracy and a reasonable affordable overhead in computational time. We report development time, size of developed code versus size of grammar, running time, and precision-recall as metrics of cost, complexity, overhead, and accuracy, respectively.
We survey three case studies from the literature: (1) narrator chain, (2) temporal entity, and (3) genealogy entity extraction tasks, and we use the reported development time for the task specific techniques proposed in ANGE BIBREF43 , ATEEMA BIBREF44 , and GENTREE BIBREF31 , respectively. We also compare a MERF number normalization task to a task specific implementation.
We evaluated ANGE with Musnad Ahmad, a hadith book, where we constructed an annotated golden reference containing 1,865 words. We evaluated ATEEMA with articles from issues of the Lebanese Al-Akhbar newspaper where we constructed an annotated golden reference containing 1,677 words. For the genealogical tree extraction we used an extract from the Genesis biblical text with 1,227 words. Finally, we used an annotated article from the Lebanese Assafir newspaper with 1,399 words to evaluate the NUMNORM case study. In the online appendix, we report on eight additional MERF case studies. Manual annotators inspected the outcome and provided corrections where tools made mistakes. The corrections form the manual gold annotation that we compared against.
Table TABREF35 reports the development time, extraction runtime, recall and precision of the output MRE tags, the size of the task in lines of code or in number of MERF rules, for both the standalone task specific and the MERF implementations. The development time measures the time required for developing the case study. For instance, ANGE BIBREF43 required two months of development by a research assistant with 6 and 14 hours of course work and teaching duties, respectively. Recall refers to the fraction of the entities correctly detected against the total number of entities. Precision refers to the fraction of correctly detected entities against the total number of extracted entities.
Table TABREF35 provides runtime results of MERF compared to the task specific implementations while running MBF and MRE simulations jointly. This is a rough estimate of the complexity of the MERF simulator. The complexity of the MBF simulation is the total number of morphological solutions for all the words multiplied by the number of user-defined MBFs. We do not provide a limit on the number of user defined formulae. In practice, we did not encounter more than ten formulae per case study. As for the complexity of MRE simulation, converting the rules into non-deterministic finite state machines (NDFSM) is done once. Simulating an NDFSM over the MBF tags is potentially exponential. In practice, all our case studies terminated within a predetermined time bound of less than 30 minutes. MERF required reasonably more runtime than the task specific implementations and reported acceptable and slightly less precision metrics with around the same recall.
Table TABREF35 shows that MERF has a clear advantage over task specific techniques in the effort required to develop the application at a reasonable cost in terms of accuracy and run time. Developers needed three hours, three hours, four hours, and one hour to develop the narrator chain, temporal entity, genealogy, and number normalization case studies using MERF, respectively. However, the developers of ANGE, ATEEMA, GENTREE, and NUMNORM needed two months, one and a half months, three weeks, and one week, respectively. MERF needed eight MBFs and four MREs for narrator chain, three MBFs and two MREs for temporal entity, three MBFs and three MREs for genealogy, and three MBFs, one MRE, and 57 lines of code actions for the number normalization tasks. However, ANGE, ATEEMA, GENTREE, and NUMNORM required 3,000+, 1,000+, 3,000+, and 500 lines of code, respectively.
utf8
standard
Narrator chain case study
A narrator chain is a sequence of narrators referencing each other. The chain includes proper nouns, paternal entities, and referencing entities. ANGE uses Arabic morphological analysis, finite state machines, and graph transformations to extract entities and relations including narrator chains BIBREF43 .
Table TABREF36 presents the MREs for the narrator chain case study. MBF PN checks the abstract category Name of Person. MBF FAM denotes “family connector” and checks the stem gloss “son”. MBF TOLD denotes referencing between narrators and checks the disjunction of the stems .hdt(“spoke to”), `n(“about”), sm`(“heard”), 'hbr(“told”), and 'nb-'(“inform”). MBF MEAN checks the stem `ny(“mean”). MBFs BLESS, GOD, UPONHIM, and GREET check the stems .sllA, Al-llah, `ly, and sllm, respectively.
MRE name is one or more PN tags optionally followed with a MEAN tag. MRE nar denotes narrator which is a complex Arabic name composed as a sequence of Arabic names (name) connected with family indicators (FAM). The NONE tags in nar allow for unexpected words that can occur between names. MRE pbuh denotes a praise phrase often associated with the end of a hadith (“peace be upon him”), and is the satisfied by the sequence of BLESS, GOD, UPONHIM, and GREET tags. MRE nchain denotes narrator chain, and is a sequence of narrators (nar) separated with TOLD tags, and optionally followed by a pbuh tag.
The first row in Table TABREF36 is an example narrator chain, the second is the transliteration, the third shows the MBF tags. Rows 4, 5, and 6 show the matches for name, nar, and nchain, respectively. MERF assigns the symbols INLINEFORM0 and INLINEFORM1 for the MRE subexpressions TOLD and nar, respectively. We define the relation INLINEFORM2 to relate sequences of narrators with edges labeled by the tags of TOLD where INLINEFORM3 denotes the next match of nar in the one or more MRE subexpression. Table TABREF38 shows that MERF detected almost all the MBF matches with 99% recall and 85% precision and extracted user-defined relations with 98% recall and 99% precision.
Temporal entity extraction
Temporal entities are text chunks that express temporal information. Some represent absolute time such as AlhAms mn 'Ab 2010 INLINEFORM0 . Others represent relative time such as b`d hmsT 'ayAm INLINEFORM1 , and quantities such as 14 ywmA INLINEFORM2 . ATEEMA presents a temporal entity detection technique for the Arabic language using morphological analysis and finite state transducers BIBREF44 . Table TABREF38 shows that MERF detected almost all the MBF matches with 99% recall, however it shows low precision (52%). As for the semantic relation construction, MERF presents a 98% recall and 89% precision.
Genealogy tree
Biblical genealogical lists trace key biblical figures such as Israelite kings and prophets with family relations. The family relations include wife and parenthood. A sample genealogical chunk of text is w wld hArAn lw.tA meaning “and Haran became the father of Lot”. GENTREE BIBREF31 automatically extracts the genealogical family trees using morphology, finite state machines, and graph transformations. Table TABREF38 shows that MERF detected MBF matches with 99% recall, and 75% precision, and extracted relations with 81% recall and 96% precision.

Question: What existing techniques do the authors compare against?
Answer: ANGE, ATEEMA, GENTREE, and NUMNORM

ANGE, ATEEMA, GENTREE