Incorporating Dropped Pronouns into Coreference Resolution: The case for T urkish (2023)

Abstract: Representation of coreferential relations is a challenging and actively studied topic for pro-drop and morphologically rich languages (PD-MRLs) due to dropped pronouns (e.g., null subjects and omitted possessive pronouns). These phenomena require a representation scheme at the morphology level and enhanced evaluation methods. In this paper, we propose a representation & evaluation scheme to incorporate dropped pronouns into coreference resolution and validate it on the Turkish language. Using the scheme, we extend the annotations on the only existing Turkish coreference dataset, which originally did not contain annotations for dropped pronouns. We provide publicly available pre and post processors to enhance the prominent CoNLL coreference scorer also to cover coreferential relations arising from dropped pronouns. As a final step, the paper reports the first neural Turkish coreference resolution results in the literature. Although validated on Turkish, the proposed scheme is language-independent and may be used for other PD-MRLs.

## introduction coreference resolution (cr) is a semantic-level natural language processing (nlp) task and aims to determine sets of mentions which describe the same real-world entity (e.g., a person, a place, a thing, an event). an end-to-end cr system has two sequential steps: mention detection and mention clustering. the mention detection stage focuses on identifying all possible coreferential mentions referring to a real-world entity within a text. in the next step, the mention clustering stage collects mentions referring to the same real-world entity under the same cluster, resolving which extracted mentions are coreferential. although cr is an nlp subject that has been studied for quite a long time @xcite , studies on pd-mrls are still in their infancy, and turkish is one of them. in mrls, words may appear under different surface forms taking different types of affixes. in some languages, the richness level may be very high so that most syntactic information is carried at the morphological level leading to the possibility of dropping some functional words and pronouns. an example from the turkish language (a highly rich mrl) is provided below 1 , where verbal agreement and possessive suffixes 2 allow the drop of personal and possessive pronouns. morphemes emphasized with bold font describe the dropped pronouns: '-im' holds for the pronoun 'benim' (me) and '-n' holds for 'sen' (you). however, the sentence is naturally made as exemplified below in the second line without personal and possessive pronouns. sen benim geldigimi gördün mü? sen benim geldigimi gördün mü? you i came see did did you see that i came? the pro-drop nature of such languages reveals the need for mention annotation on other tokens (i.e., artificially inserted @xcite or existing tokens @xcite other than the dropped pronouns, such as verbs carrying personal suffixes). the morphological richness in these languages may reveal the appearance of multiple coreference relations on a single token which is illustrated below. the word 'annemin' (of my mother) in the below example carries multiple coreferential relations foot_0 to different people: me and my mother. 1 color codes are used to indicate mentions referring to the same entity. unfortunately, existing coreference evaluators @xcite , originally developed for nonprodrop languages (e.g., english), do not support multiple coreferential relations on a single token. on the other hand, representations relying on artificially inserted tokens have their deficiencies, although eliminating the multiple coreference issue: in this paper, we propose a representation & evaluation scheme using existing tokens to incorporate dropped pronouns into coreference resolution and validate it on turkish. using this scheme, we extend the annotations on the only existing turkish coreference dataset @xcite , which originally did not contain annotations for dropped pronouns. we provide publicly available pre and post processors foot_1 to enhance the prominent conll coreference scorer foot_2 @xcite to also cover multiple coreferential relations arising from dropped pronouns. as a final step, the paper reports the first neural turkish coreference resolution results in the literature providing a strong baseline for future studies in this field. the preliminary results are reported on a neural coreference resolution model with a mention-ranking approach @xcite , which was introduced for slovene, another pd-mrl. since the coreference information is coded at the morphology level, we investigate the impact of different word embeddings @xcite , neural language models @xcite @xcite , and the inclusion of hand-crafted features used in previous studies @xcite to analyze their representation power for morphological richness. although validated on turkish, the proposed scheme is language-independent and may be used for other pd-mrls. the paper is structured as follows: section 2 gives the related work, section 3 introduces the representation of dropped pronouns on existing data sets in the literature, section 4 presents the proposed representation & evaluation scheme for dropped pronouns, section 5 presents the experimental setup and results, and section 6 gives the conclusion. ## related work machine learning methods requiring hand-crafted features have been used in the cr literature for a long time. generally, learning-based cr models are collected under three main categories: mentionpair @xcite @xcite @xcite , entitymention @xcite @xcite , and ranking mechanisms @xcite @xcite . deep neural networks have been frequently used in recent studies: mention-pair @xcite , entitymention @xcite , mentionranking @xcite @xcite @xcite . recently, several neural end-to-end systems which focus on determining the mentions automatically before or in line with the coreference resolution stage have been also introduced @xcite @xcite . besides the above-mentioned studies on english, the cr studies focusing on pro-drop languages have been increasing. @xcite improved the cr performance reported in @xcite ) by exploiting zero-pronouns (i.e. elided pronouns) on chinese with traditional machine learning methods. @xcite enhanced the available approach @xcite with a richer feature set and also incorporated the dropped pronouns as a referential mention. neural cr architectures were also employed in the chinese cr studies @xcite . for korean, @xcite proposed a neural architecture using pointer networks to reduce the computational complexity of an available end-to-end model @xcite . @xcite used electra @xcite on the neural structure @xcite for italian. @xcite proposed a neural cr model focusing on only mention clustering stage for slovene. evaluation of cr systems is a challenging topic which resulted with several evaluation metrics in the literature: muc @xcite , b-cubed @xcite , mention-based & entity-based ceaf @xcite , blanc (recasens and hovy, 2011), the averaged conll score @xcite . each metric evaluates a cr system from different perspectives and has pros and cons. a widely used evaluator (from now on referred to as the conll scorer ( pradhan et al., 2014) ) outputs cr performances via all these metrics. previous works on turkish cr are based on traditional machine learning algorithms @xcite @xcite @xcite . the most recent turkish coreference dataset (mtcc -marmara turkish coreference corpus) is from @xcite , and consists of a document subset extracted from metu turkish corpus (mtc) @xcite . the dataset had been later extended by morpho-syntactic features by @xcite using an automated turkish nlp pipeline @xcite . this dataset does not contain annotations for dropped pronouns. ## representation of dropped pronouns on existing data sets the cr literature has annotated datasets supporting various languages: muc @xcite @xcite , @xcite , se-meval2010 @xcite , ontonotes @xcite . the muc covers coreference relation only for english which is not a pro-drop language. the ace focuses on only seven pre-defined type entities, therefore, dropped pronouns were excluded in the annotation process for arabic (a pro-drop language). although the se-meval2010 includes pro-drop languages (e.g. catalan, spanish @xcite ), dropped pronouns were not covered during the annotation. compared with these datasets, the ontonotes is more comprehensive and contains gold-standard coreferential relations of dropped pronouns for chinese and arabic. in the ontonotes, dropped pronouns are represented by a unique artificial token: ("*pro*" for chinese and "*" for arabic), which is inserted into the correct position where the subject or object is omitted in a sentence during the annotation. this token indicates that a pronoun has been dropped from this location in the sentence. example 1 shows how a dropped pronoun is represented for chinese. example 1: annotation of dropped pronouns with artificial (*pro*) token in chinese @xcite chinese and english translations of the same sentence are shown in the example 1. in chinese, *pro* is inserted for an omitted subject pronoun, which is referential with another pronoun: (我们) @xcite . a recent study @xcite proposed a similar representation scheme as in the ontonotes, built on top of the conll-ud framework @xcite , called the corefud. dropped pronouns are represented by inserted tokens, called empty nodes (i.e. zeros), and they are related to their syntactic heads (hereinafter referred to as 'owner') by dependency relations. the corefud introduces how the inserted tokens should be represented (with a sub-indexed token number i.e. <tokenid>.<@xmath0; however, there is no standard on where to add them across different languages. in hungarian, they are added immediately after their owners in the sentence (with some minor exceptions due to punctuations). in czech, spanish and catalan, there is no strict rule about their positions except that empty nodes are almost always placed before their owners. the decisions about their positions seem to be affected by the fact that they will have an influence on the dependency trees of the related language. besides the explained representation above, @xcite introduced another approach and applied it on the italian cr dataset. italian is a partial pd-mrl, allowing only omitted subjects, called null-subjects. in this approach, instead of an artificially inserting token, dropped subject pronouns are directly annotated on the verbs. example 2 shows how a dropped subject pronoun is represented for italian. example 2: annotation of dropped subject pronoun in italian @xcite italian and english translations of the same sentence are shown in example 2. in this approach, each verb is considered as a potential coreferential mention. mentions existing before this verbal mention in a text are considered antecedents of the verb. in the example, the predicate of the second italian sentence, vide, has a coreference relation with pahor in the first sentence. since english is not a pro-drop language, the subject of the second sentence, he, is explicitly defined, and the coreference relation are made between he and pahor. slovenian cr dataset @xcite used the same representation approach for null-subjects as in italian. both the ontonotes and corefud approaches propose inserting new tokens to represent dropped pronouns, but from different perspectives. in the ontonotes, all type of dropped pronouns are represented with the same artificial token which could be easily adapted to various languages. however, each dropped pronoun is represented with the same surface form creating ambiguity for automated cr systems. on the other hand, the corefud proposes inserting an empty token according to pronominal information at the morphology level, not a unique token for all dropped pronouns; however, it requires extra coding of the already available information easily deducible from the owner's morphology information. these approaches harm the original sentence flow, reduce human readability and also cause an extra burden to the annotation process from the perspective of determining the most accurate and natural position of the these newly inserted tokens in the sentence. however, they both allow direct use of existing evaluation tools, which may be considered as an advantage of these approaches. moreover, the universal dependencies (ud) 6 @xcite initiative suggests to re-6 ud aims to create a common framework for annotation duce the use of additional artificial tokens (i.e., inflectional groups) even in case of derivational suffixes/cases requiring a new sub-token group. however, the above-mentioned approaches propose inserting extra nodes based on morphological suffixes, which may be treated as contradictory. using existing tokens to represent coreferential relations of dropped pronouns overcomes these drawbacks. however, in extreme pd-mrls, dropping may occur in cases other than null-subjects; e.g., dropped possessive pronouns. the morphological richness in these languages may reveal the appearance of multiple coreference relations on a single token; e.g., a nominal as exemplified in the introduction section. this is a barrier in front of using existing evaluation tools for such kind of representations. ## the proposed scheme this section introduces our representation and evaluation scheme and its validation. ## experiments & results this section introduces the first neural turkish coreference resolution results which provides a strong baseline for future studies in the field. ## conclusion the paper proposed a language-independent representation and evaluation scheme to incorporate dropped pronouns into coreference resolution for pro-drop and morphologically rich languages. pre and post-processors to enhance available cr evaluators to cover dropped pronouns (i.e., multiple annotations over a single word) are developed. the scheme was validated on the turkish language. the study revealed the first turkish cr dataset including the annotations for dropped pronouns and the first neural cr results for this language as a strong baseline for future studies. the impact of interaction between different text encodings and linguistic features were investigated on this task. the best performance was achieved by using fasttext embeddings together with hand-crafted linguistic features with 50.2% conll f-score, which provides a 5 percentage points improvement over a bert baseline.