Question: What is their sequence labeling task?
Evidence: Zero anaphora is a discourse phenomenon, where pronouns can be omitted when they are pragmatically or grammatically inferable from intra- and inter-sentential context BIBREF0 . However, translating such implicit information (i.e. zero pronoun, ZP) poses various difficulties for machine translation (MT) in terms of completeness and correctness. Although neural models are getting better at learning representations, it is still difficult to implicitly learn complex ZPs in a general model. Actually, ZP prediction and translation need to not only understand the semantics or intentions of a single sentence, but also utilize its discourse-level context.
Two technological advances in the field of ZP and MT, have seen vast progress over the last decades, but they have been developed very much in isolation. Early studies BIBREF1 , BIBREF2 , BIBREF3 fed MT systems with the results of ZP prediction models, which are trained on a small-scale and non-homologous data compared to MT models. To narrow the data-level gap, Wang:2016:NAACL proposed an automatic method to annotate ZPs by utilizing the parallel corpus of MT. The homologous data for both ZP prediction and translation leads to significant improvements on translation performances for both statistical MT BIBREF4 and neural MT models BIBREF5 . However, such approaches still require external ZP prediction models, which have a low accuracy of 66%. The numerous errors of ZP prediction errors will be propagated to translation models, which leads to new translation problems. In addition, relying on external ZP prediction models in decoding makes these approaches unwieldy in practice, due to introducing more computation cost and pipeline complexity.
In this work, we try to further bridge the model-level gap by jointly modeling ZP prediction and translation. Joint learning has proven highly effective on alleviating the error propagation problem, such as joint parsing and translation BIBREF6 , as well as joint tokenization and translation BIBREF7 . Similarly, we expect that ZP prediction and translation could interact with each other: prediction offers more ZP information beyond 1-best result to translation and translation helps prediction resolve ambiguity. Specifically, we first cast ZP prediction as a sequence labeling task with a neural model, which is trained jointly with a standard neural machine translation (NMT) model in an end-to-end manner. We leverage the auto-annotated ZPs to supervise the learning of ZP prediction component, which releases the reliance on external ZP knowledge in decoding phase.
In addition, previous studies revealed that discourse-level information can better tackle ZP resolution, because around 23% of ZPs appear two or more sentences away from their antecedents BIBREF8 , BIBREF9 . Inspired by these findings, we exploit inter-sentential context to further improve ZP prediction and thus translation. Concretely, we employ hierarchical neural networks BIBREF10 , BIBREF11 to summarize the context of previous sentences in a text, which is integrated to the joint model for ZP prediction.
We validate the proposed approach on the widely-used data for ZP translation BIBREF5 , which consist of 2.15M Chinese–English sentence pairs. Experimental results show that the joint model indeed improves performances on both ZP prediction and translation. Incorporating discourse-level context further improves performances, and outperforms ther external ZP prediction model BIBREF5 by +2.29 BLEU points in translation and +11% in prediction accuracy. Experimental results on a further Japanese–English translation task show that our model consistently outperforms both the baseline and the external ZP prediction model, demonstrating the universality of the proposed approach.
The key contributions of this paper are:
Specifically, we first cast ZP prediction as a sequence labeling task with a neural model, which is trained jointly with a standard neural machine translation (NMT) model in an end-to-end manner.
Answer: ZP prediction.