Existing approaches for Chinese zero pronoun resolution typically utilize only syntactical and lexical features while ignoring semantic information . The fundamental reason is that zero pronouns have no descriptive information , which brings difficulty in explicitly capturing their semantic similarities with antecedents . Meanwhile , representing zero pronouns is challenging since they are merely gaps that convey no actual content . In this paper , we address this issue by building a deep memory network that is capable of encoding zero pronouns into vector representations with information obtained from their contexts and potential antecedents . Consequently , our resolver takes advantage of semantic information by using these continuous distributed representations . Experiments on the OntoNotes 5.0 dataset show that the proposed memory network could substantially outperform the state-of-the-art systems in various experimental settings . A zero pronoun ( ZP ) is a gap in a sentence , which refers to an entity that supplies the necessary information for interpreting the gap ( Zhao and Ng , 2007 ) . A ZP can be either anaphoric if it corefers to one or more preceding noun phrases ( antecedents ) in the associated text , or non-anaphoric if there are no such noun phrases . Below is an example of ZPs and their antecedents , where " φ " denotes the ZP . [ 警方 ] 表示 他们 自杀 的 可能性 很高 ， 不 过 φ 1 也 不 排除 φ 2 有 他杀 的 可能 。 * Email corresponding . ( [ The police ] said that they are more likely to commit suicide , but φ 1 could not rule out φ 2 the possibility of homicide . ) In this example , the ZP " φ 1 " is an anaphoric ZP that refers to the antecedent " 警方 / The police " while the ZP " φ 2 " is non-anaphoric . Unlike overt pronouns , ZPs lack grammatical attributes such as gender and number that have been proven to be essential in pronoun resolution ( Chen and Ng , 2014a ) , which makes ZP resolution a more challenging task than overt pronoun resolution . Automatic Chinese ZP resolution is typically composed of two steps , i.e. , anaphoric zero pronoun ( AZP ) identification that identifies whether a ZP is anaphoric ; and AZP resolution , which determines antecedents for AZPs . For AZP identification , state-of-the-art resolvers use machine learning algorithms to build AZP classifiers in a supervised manner ( Chen and Ng , 2013 , 2016 ) . For AZP resolution , literature approaches include unsupervised methods ( Chen and Ng , 2014b , 2015 ) , feature-based supervised models ( Zhao and Ng , 2007 ; Kong and Zhou , 2010 ) , and neural network models ( Chen and Ng , 2016 ) . Neural network models for AZP resolution are of growing interest for their capacity to learn task-specific representations without extensive feature engineering and to effectively exploit lexical information for ZPs and their candidate antecedents in a more scalable manner than feature-based models . Despite these advantages , existing supervised approaches ( Zhao and Ng , 2007 ; Chen and Ng , 2013 , 2016 ) for AZP resolution typically utilize only syntactical and lexical information through features . They overlook semantic information that is regarded as an important factor in the resolution of common noun phrases ( Ng , 2007 ) . The fundamental reason is that ZPs have no descriptive information , which results in difficulty in calculating semantic similarities and relatedness scores between the ZPs and their antecedents . Therefore , the proper representations of ZPs are required so as to take advantage of semantic information when resolving ZPs . However , representing ZPs is challenging because they are merely gaps that convey no actual content . One straightforward method to address this issue is to represent ZPs with supplemental information provided by some available components , such as contexts and candidate antecedents . Motivated by Chen and Ng ( 2016 ) who encode a ZP 's lexical contexts by utilizing its preceding word and governing verb , we notice that a ZP 's context can help to describe the ZP itself . As an example of its usefulness , given the sentence " φ taste spicy " , people may resolve the ZP " φ " to the candidate antecedent " red peppers " , but can hardly regard " my shoes " as its antecedent , because they naturally look at the ZP 's context " taste spicy " to resolve it ( " my shoes " can not " taste spicy " ) . Meanwhile , considering that the antecedents of a ZP provide the necessary information for interpreting the gap ( ZP ) , it is a natural way to express a ZP by its potential antecedents . However , only some subsets of candidate antecedents are needed to represent a ZP1 . To achieve this goal , a desirable solution should be capable of explicitly capturing the importance of each candidate antecedent and using them to build up the representation for the ZP . In this paper , inspired by the recent success of computational models with attention mechanism and explicit memory ( Sukhbaatar et al . , 2015 ; Tang et al . , 2016 ; Kumar et al . , 2015 ) , we focus on AZP resolution , proposing the zero pronounspecific memory network ( ZPMN ) that is competent for representing a ZP with information obtained from its contexts and candidate antecedents . These representations provide our system with an ability to take advantage of semantic information when resolving ZPs . Our ZPMN consists of multiple computational layers with shared parameters . With the underlying intuition that not all candidate antecedents are equally relevant for representing the ZP , we develop each computational layer as an attention-based model , which first learns the importance of each candidate antecedent and then utilizes this information to calculate the continu-ous distributed representation of the ZP . The attention weights over candidate antecedents with respect to the ZP 's representation obtained by the last layer are regarded as the ZP coreference classification result . Given that every component is differentiable , the entire model could be efficiently trained end-to-end with gradient descent . We evaluate our method on the Chinese portions of the OntoNotes 5.0 corpus by comparing with the baseline systems in different experimental settings . Results show that our approach significantly outperforms the baseline algorithms and achieves state-of-the-art performance . In this study , we propose a novel zero pronounspecific memory network that is capable of encoding zero pronouns into the vector representations with supplemental information obtained from their contexts and candidate antecedents . Consequently , these continuous distributed vectors provide our model with an ability to take advantage of the semantic information when resolving zero pronouns . We evaluate our method on the Chinese portion of OntoNotes 5.0 dataset and report substantial improvements over the state-ofthe-art systems in various experimental settings .

Challenge: Existing Chinese zero pronoun resolution methods ignore semantic information because zero pronouns have no descriptive information to explicitly capture their semantic similarities with antecedents.
Approach: They propose an attention-based deep memory network which encodes zero pronouns into vector representations of information obtained from their contexts and potential antecedents.
Outcome: The proposed model outperforms the state-of-the-art and baseline models in various experimental settings on the Chinese portion of the OntoNotes 5.0 dataset.