Towards Handling Verb Phrase Ellipsis in E nglish- H indi Machine Translation (2019)

Abstract: English-Hindi machine translation systems have difficulty interpreting verb phrase ellipsis (VPE) in English, and commit errors in translating sentences with VPE. We present a solution and theoretical backing for the treatment of English VPE, with the specific scope of enabling English-Hindi MT, based on an understanding of the syntactical phenomenon of verb-stranding verb phrase ellipsis in Hindi (VVPE). We implement a rule-based system to perform the following sub-tasks: 1) Verb ellipsis identification in the English source sentence, 2) Elided verb phrase head identification 3) Identification of verb segment which needs to be induced at the site of ellipsis 4) Modify input sentence; i.e. resolving VPE and inducing the required verb segment. This system obtains 94.83 percent precision and 83.04 percent recall on subtask (1), tested on 3900 sentences from the BNC corpus. This is competitive with state-of-the-art results. We measure accuracy of subtasks (2) and (3) together, and obtain a 91 percent accuracy on 200 sentences taken from the WSJ corpus. Finally, in order to indicate the relevance of ellipsis handling to MT, we carried out a manual analysis of the English-Hindi MT outputs of 100 sentences after passing it through our system. We set up a basic metric (1-5) for this evaluation, where 5 indicates drastic improvement, and obtained an average of 3.55. As far as we know, this is the first attempt to target ellipsis resolution in the context of improving English-Hindi machine translation.

## introduction verb phrase ellipsis is a particularly frequent form of ellipsis, both in speech and in text. english vpe is the elimination of a non-finite verb phrase, introduced by an auxiliary or the particle 'to' @xcite . we observe that state-of-the-art mt systems often cannot correctly interpret and translate sentences with vpe. for example, (elliptical phrase and site of ellipsis are in bold and italics respectively): ram cooked the food quickly, but shyam did @xmath0, cook-pt, lekin but shyam shyam-erg ne nahi not diya. give-pt (created example) in our initial analysis, we find that google translate could translate only 6/50 vpe sentences, taken from the wsj corpus, correctly from english to hindi. this motivated us to identify and resolve vpe in english and give this modified sentence to the machine translation system to check whether it improves the quality of translation. we used our mt system anusaaraka for this purpose. 1 we present a rule-based approach that performs three sub-tasks in order to solve this problem: 1) ellipsis identification 2) identification of the antecedent head verb 3) addition of necessary auxiliaries and/or complements to the head verb. finally, it modifies the input sentence by inducing the identified verb segment at the site of ellipsis. we tailor this algorithm for the purpose of english-hindi mt, and therefore provide a special treatment to compound verbs (including phrasal verbs), serial verbs, and verb complements in english. our solution is based on an understanding of verb phrase ellipsis in hindi. hindi does not exhibit vpe in the same manner as english; however, it exhibits a phenomenon called verb-stranding verb phrase ellipsis (vvpe) @xcite . this means that a verbal phrase, including objects and other arguments, may be completed elided, stranding the head verb, which then appears at the site of ellipsis. we propose that english vpe can be transferred into hindi vvpe in order to improve mt results on the modified sentence. for this, we claim that it is enough to identify the head verb antecedent in the english sentence, and perform an analysis to support the claim. finally, we want to see that mt systems indeed perform better on statements with ellipsis after our treatment. since we want to test the performance of the mt on a very specific facet: i.e. the elliptical clause, we do not utilize standard evaluation metrics but set up and define our own scale, and perform a manual analysis of 100 sentences. this evaluation dataset is meant to be merely indicative of the benefits of ellipsis handling for mt. the results are explained below. ## background and our contribution there have been several previous works addressing the problem of antecedent head resolution for vpe in english. cheung et al adapt the margin-infused-relaxed algorithm (mira) for target detection and antecedent resolution and obtain an accuracy of 65 percent (kenyon-dean et al., 2016). nielson, 2005, re-implements daniel hardt's vpe-res algorithm on the penn treebank to obtain a highest head overlap success of 85.87 percent and a lower exact match success, about 78 percent on the brown corpus @xcite ) @xcite . @xcite experiment with various joint modelling techniques, and obtain a recall of 83.46 percent for boundary identification @xcite . earlier works include daniel hardt's linguistically motivated rule-based system, that eliminates impossible antecedents by looking at be-do conflicts, contained antecedents and assigned scores based on co-reference of the noun subjects and clausal relationships, such as 'as' constructions. @xcite the necessity for tools to deal with vpe is widely recognized in literature, for the purposes of information extraction, finding event co-occurrence, etc @xcite . in the context of mt, a possible solution is to identify the antecedent, or the source verb phrase, and induce it at the site of ellipsis to gain a legitimate, simplified sentence. this solution functions by reiterating the antecedent at the site of ellipsis. by breaking the link between the ellipsis and the antecedent, we give the mt two independent clauses, which it can translate without error. indeed, the previous works listed above are aimed at identifying the antecedent verb phrase, which includes compulsorily a head verb, and optionally its arguments and adverbials. while it is important to pick up arguments and adverbials of the head verb for the purpose of comprehension, there are some problems that it introduces: namely, it must often disambiguate by context and therefore can be a great source of error, and in the context of mt, it might make the output sentence clumsy and unnatural. since we are looking at antecedent head resolution from a particular angle i.e. transforming the english sentence containing vpe in order to align with hindi vvpe and therefore help the mt, our problem does not require us to find the antecedent boundary of the head verb, while we do provide an additional treatment of certain verb constructions in english according to the manner in which they would be translated into hindi. as far as we are aware, this is the first attempt to tailor english vpe resolution in the context of english-hindi mt. ## ellipsis in english verb phrase ellipsis in english is introduced by an auxiliary. we consider five classes of el-lipsis depending upon the auxiliary at the site of ellipsis: 1) to_be, 2) to_have, 3) to_do, 4) modals, and 5) to_particle ellipsis. cheung et al include a sixth class: the do-so anaphora, while acknowledging that modals and do-x anaphora are not technically auxiliaries @xcite . we have chosen to identify ellipsis introduced by modals, however, because 1. the behaviour of the former is identical with ellipsis by true auxiliaries, 2. likewise, we observe that it poses a problem to state-of-the-art mt english-hindi systems. we do not identify do-x anaphora in this system, however, since this is simply a pronominalization of the antecedent rather than eliding. these have a different treatment than vpe -for example, they may be directly pronominalized in hindi as well, rather than transferred into vvpe. indeed, mt systems are able to do this: although mr. azoff won't produce films at first, it is possible that he could do so later, the sources @xmath1, although, shree mr. azoph azoff pahalee first baar time philmon film-pl-poss ka nirmaan production nahin not karenge, do-f.m, lekin but yah this sambhav possible hai is ki that vah he baad later-post mein aisa this kar do sake can (taken from wsj corpus) there are certain constraints on the antecedent, depending upon the auxiliary at the site of ellipsis. for example, 1. auxiliaries of the form to_be require antecedents with to_be auxiliaries. we know that state-of-the-art machines cannot interpret vpe. @xcite . translation to hindi from english requires a prediction of the elided english verb. an elided vp consists of a head verb, optionally along with its object arguments and adverbials. for example, it is clear that the adverbial is interpreted as part of the vp in the following sentence: i could walk quickly, at that age, across traffic-filled roads, but now i cannot. (created example) one solution to eliminate errors due to vpe, is to identify the antecedent and induce it at the site of ellipsis entirely, as a preliminary step before translation, thus eliminating the ellipsis entirely. however, this naive approach has the following issues: identifying the boundaries of the elided verb phrase is a much harder task than identifying the antecedent head verb. we note, for example, the consistent drop in exact match accuracy in neilson's study of hardt's algorithm over different corpora @xcite -sometimes dipping as much as 30 percent lower. the matter is not as simple as picking up the entire verb sub-tree -always importing all arguments and adjuncts is not permissible, since the correct interpretation often depends on surrounding knowledge. for example, in ram would have liked to eat out with you on sunday afternoon, but he can't. (created example) there are more than one interpretations of what ram can't do: eat out, eat out with you, or eat out with you on sunday afternoon. a native speaker selects one of these in context. if we are seeking to eliminate ellipsis with our system, we must select one of these interpretations to paste at the site of ellipsis, or the whole verb phrase. we may also not adopt intermediate policies such as importing noun objects but not adjuncts, because this risks placing undue emphasis on certain arguments, and may result in an incorrect semantics. the second problem with this strategy is that it may render sentences, after pasting, clumsy, unnatural or nonsensical, as illustrated, respectively, by the following example, taken from the wsj corpus @xcite . the volokhs were afraid that they'd end up like a friend of theirs who'd applied for a visa and waited for 10 years, having been demoted from his profession of theoretical mathematician to shipping clerk. they didn't (end up like…shipping clerk) 5 hindi vvpe verb-stranding verb phrase ellipsis is the phenomenon wherein a verb is stranded at a site of ellipsis, and its internal arguments are elided. manetta establishes, by various diagnostics, that hindi-urdu do exhibit vvpe. for example, 1. ram-ne ram-erg chomsky-ka chomsky-gen naya new lekh writing do two baar time parha. read-pfv.m.sg ram read the new paper by chomsky twice. a (original sentence) ram would have liked to eat out with you on sunday afternoon, but he can't. b (with induced head verb) ram would have liked to eat out with you on sunday afternoon, but he can't eat. c ram ram-erg ko aapke you-erg saath with sunday sunday dopahar afternoon-erg ko baahar out khana to-eat accha good lagta feel tha, would-have, lekin but vah he nahi not kha eat sakta. can-m.sg resolving the ellipsis in (b) we get an english sentence with 'eat' at the previous site of ellipsis, that means that ram is incapable of eating. however, (c) in hindi, that similarly has 'eat' at previous site of english vpe, is a perfectly acceptable translation of the original sentence, exhibiting vvpe. in (c) the stranded verb 'kha' (eat) has access to the internal arguments 'aapke saath', (with you), 'sunday dopahar' (sunday afternoon), etc., and therefore it carries all the interpretations of the original. this is the core idea of the system. ## dealing with multi-word verbs in english while we do not require to identify the boundaries of the head verb, we do need to identify all the components of the verb. this may be required in several cases, such as phrasal verbs, idioms, or serial verbs. ## algorithm to identify ellipsis and head verb of antecedent we are using a dependency tree of the input sentence for all tasks. ## results and error analysis for antecedent head resolution in part 2 of our testing, we tested this system on 200 sentences from the wsj corpus and got an accuracy of 91 percent on antecedent head resolution. here are some errors, and their analysis: mr. dinkins also has failed to allay jewish voters' fears about his association with the rev. jesse jackson, despite the fact that few local non-jewish politicians have been as vocal for jewish causes in the past 20 years as mr. dinkins has. here, the algorithm identifies "failed" instead of "been". it awards "failed" for common noun subject, common auxiliary and being the first verb upon backtracking, whereas "been" is only awarded for common auxiliary. but sony also says in its filing that the warner contract "doesn't require that guber and peters take any affirmative steps to produce motion pictures; it simply rewards them when they do and prohibits them from producing for another entertainment company." here the algorithm identifies "rewards" as the source verb instead of "take". it awards "take" for noun subject number, "require" for common auxiliary, and "rewards" by backtracking. finally, it resolves the tie by choosing "rewards" which is the closest. now they know who you mean and you know who you mean -but no one else does. the algorithm gives "mean" instead of "know". there are errors introduced by the pos tagger; for example, in: a good half-hour into breakfast at the palmer house, mr. o'brien looks up from his plate after mr. straszheim says something about people who believe interest rates are about to nosedive -"i'm one of them who hope they will, with 6 billion in debt on the books. "nosedive" is not recognized as a verb, similar to, in another examples, "program-trade", and "move". there are errors introduced by the parser: the text by patrick o'connor is a tough read, but the pictures make her magnetism clear and help explain why ernest hemmingway called baker, "the most sensational women anybody ever saw -or ever will." here, the algorithm wrongly identifies "called" as the antecedent main verb, instead of "saw". the clause "or ever will" is labelled a conjugate dependent of the noun baker, instead of a conjunct of the verb "saw". if this had been so, "saw" would have got scores for a common subject ("anybody") and being the verb obtained upon backtracking. the algorithm identifies "assure". this would be avoided if the dependency marked "i will not" as a complement clause of "assure" -however the dependency misses this relation. if "assure" was given a penalty on this grounds, the next highest candidate is indeed the correct one: "give". ## evaluation of effect on mt outputs we now show that inducing the head elliptical verb makes the input sentence easier for the mt system to comprehend. we perform a manual analysis of 100 sentences with ellipsis, taken from the wsj corpus: we create a "before" and "after" translation pair for each, and compare to identify improvements. this was done by two fluent speakers of hindi and english. we define a scale (1-5) to quantify this improvement: we add some flags to further nuance this scale: we also mark the translations for fluidity, i.e. if the translation while rendered better is still not fluent (f) or if the translation while not making the meaning clearer is rendered more fluent (f), and for the overall meaning of the entire sentence -i.e. beyond the clauses of the antecedent and the ellipsis. these markers, however, are only for exceptional cases, since most of the translations we got, both "before" and "after" were not fluent. the average score over 100 sentences was found to be 3.55, with 18 cases of correct nonfluent sentences, and 5 cases of incorrect sentences with especial improvement in fluency. these are the large-scale sources of lack of improvement that we found: an example of the first instance is: the system induces the elliptical verb complex "report". both anusaaraka and google output incorrect translations for the original sentence, although google shows errors only due to the ellipsis. therefore, it is able to improve its translation after we induce the elliptical verb: american entarapraij enterprise insteetyoot institute-pos ke vidvaan scholar norman normal orsteen ornstin-erg ne teevee t.v. gaid guide mein in "kaangres "congress-erg ke baare about-in mein kya what teevee t.v. riport report nahin not hai does -aur and riport report karanee does chaahie": should" (score: 5) whereas anusaarak outputs: (3) american american enterprise enterprise sansthaan institute vidvaan scholar vastushaili norman ornstein ornstein par on t.v. tv guide guide mein: in "tv "tv samaachaar news vat what congress congress about about report report nahi not does.. does… aur and haal recently likhta writes-msg hai chahiye" should-inf." the original gives a similarly incorrect output, though not quite as ungrammatical. this output, even after the elliptical verb has been added, conveys little to no meaning in hindi. here are some micro-level sources of nonimprovement: an example of both of the above is: during the takeover, mr. hahn said he would put his account up for review if wpp's bid were successful, but he didn't. the system induces the elliptical verb complex "put up". both anusaaraka and google output incorrect translations for the original sentence, although google interprets the idiomatic meaning of "put up" correctly. therefore, it is able to improve its translation after we induce the elliptical verb: anusaaraka makes error type 1 (literal interpretation of verb): ## future work the concept and the system that we have introduced above, for handling ellipsis in a targeted manner to improve english-hindi mt, are still in their nascent stages. there are three primary entry points for future work: firstly, the conceptual negotiation of the phenomenon in english and hindi. we have decided, as we explain, only to induce the main verb. while this gives satisfactory results most of the times, it also fails in some cases: it might help, for example, to make a decision to induce object arguments in these cases. secondly, the identification of vpe in english, and the antecedent resolution. we are already dealing with complex verbs, verbal complements etc. in a certain manner, but this treatment invites further and more rigorous work, both in terms in nuance with the treatment, and how exhaustive we are with the lists that we have drawn up, introducing, for example, treatment of idioms. thirdly, the application of our system as over the input, before it is passed through the mt. there are certain problems that may be solved by pipelining this ellipsis handling differently into the mt system: we leave this to future investigation.