Weight Derivation for Salience Algorithms in Pronominal Anaphora Resolution ( working paper )

Work is presented which describes research carried out over a ten-week period. Whilst still ongoing, the paper here describes the steps taken towards creating an effective pronoun resolution system, based on that of Lappin and Leass. Existing implementation, difficulties and proposed future additions to this project are described in full. This paper also sets out to describe briefly some of the theory behind pronominal anaphora resolution and previous work in the field. Introduction An important problem in natural language processing is the resolution of pronouns to their intended referents. Consider the sentence: (1) John gave Tom the ball and when John nodded his head, Tom kicked it. This sentence demonstrates an anaphor, where the pronoun ‘it’ refers back to a referent. Intuitively, ‘it’ refers to ‘the ball’. The task for a natural language system is to work out that ‘it’ does indeed refer back to ‘the ball’ and not to ‘[John’s] head’. (It should be noted that ‘his’ is also anaphoric, but it is “it” which shall be of interest here). One such algorithm for the resolution of pronouns was proposed by Lappin and Leass (1994). This algorithm is based on the fact that pronouns are more likely to refer to entities mentioned recently in the discourse (usually no further than one or two sentences back). The algorithm involves calculating salience values for each new entity that is encountered in a noun phrase. These salience values are calculated by summing the weights assigned to various factors. Such factors might include sentence recency, indirect object and oblique complement emphasis, head noun emphasis (referents embedded in long noun phrases) and nonadverbial emphasis. Higher weights are assigned to factors that are deemed to be more “useful” when deciding how to resolve a pronoun. Once salience values have been calculated for each referent, the algorithm can be applied to resolve the pronouns. The entity with the highest salience value is declared to be the most likely referent. Where two or more entities are awarded the same salience value, the most recent one is used. The weights used in the salience algorithm are ad hoc. This research was concerned with implementing a pronominal resolution system and adjusting these weights empirically using machine-learning techniques based on real corpus data. The remainder of this paper will give the theoretical details of anaphora; provide an overview of Lappin and Leass’ (ibid) anaphora resolution algorithm; discuss other work carried out with refined salience weighting; describe the work already carried out and future development by this author and finally, briefly look at similar research. Anaphora Consider again example (1). The entity referred back to (in this case ‘the ball’) is called the ‘referent’ or ‘antecedent’. ‘It’ is called the ‘referring expression’ or ‘anaphor’; that is, the expression used to perform reference. Reference to an entity that has been previously introduced into this discourse is called anaphora. Determining the antecedent of an anaphor is called anaphora resolution. When a referent is initially mentioned, a representation for it is evoked. Subsequent mentions can be accessed from the discourse model. Cohesion is not a structural relation, so is unrestricted by sentence boundaries. However, pronouns refer to entities evoked no further than two sentences back (whereas definite noun phrases, for example, can refer further back). Sentences where the antecedent is in the same sentence as the anaphor are called intrasentential, whereas intersentential refers to antecedents that are in a different sentence to the anaphor. When performing anaphora resolution, all noun phrases are typically treated as potential candidates for antecedents. The scope is usually limited to the current and preceding sentences and all candidate antecedents within that scope are considered. In the English language, all antecedents and anaphor agree in gender and number. It is eliminating factors (or constraints) such as these that can be used in algorithms such as Lappin and Leass’ (ibid). Other constraints include person and case agreement; syntactic constraints, for instance “Mark bought him a present” [him≠Mark] and selectional restrictions. For example: (2) John parked his Acura in the garage. He had driven it around for hours. Whilst “it” could be “his Acura” or “the garage”, the verb “drive” requires an object that can be driven. However, selectional constraints are complicated by metaphor such as: (3) John bought a new Acura. It drinks petrol like you would not believe. Obviously, cars cannot literally drink petrol, but the metaphor allows “it” to refer to the “new Acura”. The Algorithm This section attempts to summarise Lappin and Leass’ algorithm (ibid). The algorithm exploits constraints such as those mentioned in the previous sentence, and also uses rules about other preferences that can be used when trying to perform pronominal resolution. These 1 Adapted from Jurafsky & Martin (2000) p681 salience factors and their associated weights are sentence recency (100); subject emphasis (80); existential emphasis (70); accusative emphasis (50); indirect object and oblique complement emphasis (40); non-adverbial emphasis (50) and head noun emphasis (80). When a new entity is evoked, these weights can be used to calculate its salience value. The salience value is simply the sum of the associated weights. If there are no pronouns to be resolved in a sentence, the next sentence is processed and the weights that contribute to an entity’s salience are halved (to account for sentence recency). It is possible that the same referent is mentioned in multiple sentences (each with different salience values) and so the contributions of each need to be considered. As such, if a potential referent is mentioned in the current sentence and a previous one, then the weight for each will be taken into account (though, if the referent appears more than once in the same sentence, then the weight will only be considered once). Consequently, the idea of repeated mention is taken care of. This theory says that entities that have been the focus previously, are more likely to be referred to by pronouns in subsequent discourse. When a pronoun is encountered, the referents that do not abide by the constraints already indicated are removed. At this stage, two new salience weights are introduced. They are role parallelism (35) and cataphora (-175). Finally, of the referents that remain, the one with the highest salience value is deemed to be the result. Weight Derivation The work initiated as outlined in this paper is being continued and extended as part of the author’s final year project. The author hopes to resolve the problems with the grammar that is being used with the chart parser quickly and then complete the weight summation and anaphora resolution algorithm. Once an implementation of Lappin and Leass’ algorithm (ibid) is functional and has 2 Both these factors require a pronoun in order to be