In this paper , we propose a neural network model for graph-based dependency parsing which utilizes Bidirectional LSTM ( BLSTM ) to capture richer contextual information instead of using high-order factorization , and enable our model to use much fewer features than previous work . In addition , we propose an effective way to learn sentence segment embedding on sentence-level based on an extra forward LSTM network . Although our model uses only first-order factorization , experiments on English Peen Treebank and Chinese Penn Treebank show that our model could be competitive with previous higher-order graph-based dependency parsing models and state-of-the-art models . Dependency parsing is a fundamental task for language processing which has been investigated for decades . It has been applied in a wide range of applications such as information extraction and machine translation . Among a variety of dependency parsing models , graph-based models are attractive for their ability of scoring the parsing decisions on a whole-tree basis . Typical graph-based models factor the dependency tree into subgraphs , including single arcs ( McDonald et al . , 2005 ) , sibling or grandparent arcs ( McDonald and Pereira , 2006 ; Carreras , 2007 ) or higher-order substructures ( Koo and Collins , 2010 ; Ma and Zhao , 2012 ) and then score the whole tree by summing scores of the subgraphs . In these models , subgraphs are usually represented as high-dimensional feature vectors which are then fed into a linear model to learn the feature weights . However , conventional graph-based models heavily rely on feature engineering and their performance is restricted by the design of features . In addition , standard decoding algorithm ( Eisner , 2000 ) only works for the first-order model which limits the scope of feature selection . To incorporate high-order features , Eisner algorithm must be somehow extended or modified , which is usually done at high cost in terms of efficiency . The fourth-order graph-based model ( Ma and Zhao , 2012 ) , which seems the highest-order model so far to our knowledge , requires O(n 5 ) time and O(n 4 ) space . Due to the high computational cost , highorder models are normally restricted to producing only unlabeled parses to avoid extra cost introduced by inclusion of arc-labels into the parse trees . To alleviate the burden of feature engineering , Pei et al . ( 2015 ) presented an effective neural network model for graph-based dependency parsing . They only use atomic features such as word unigrams and POS tag unigrams and leave the model to automatically learn the feature combinations . However , their model requires many atomic features and still relies on high-order factorization strategy to further improve the accuracy . Different from previous work , we propose an LSTM-based dependency parsing model in this paper and aim to use LSTM network to capture richer contextual information to support parsing decisions , instead of adopting a high-order factorization . The main advantages of our model are as follows : • By introducing Bidirectional LSTM , our model shows strong ability to capture potential long range contextual information and exhibits improved accuracy in recovering long distance dependencies . It is different to previous work in which a similar effect is usually achieved by high-order factorization . More-over , our model also eliminates the need for setting feature selection windows and reduces the number of features to a minimum level . • We propose an LSTM-based sentence segment embedding method named LSTM-Minus , in which distributed representation of sentence segment is learned by using subtraction between LSTM hidden vectors . Experiment shows this further enhances our model 's ability to access to sentence-level information . • Last but important , our model is a first-order model using standard Eisner algorithm for decoding , the computational cost remains at the lowest level among graph-based models . Our model does not trade-off efficiency for accuracy . We evaluate our model on the English Penn Treebank and Chinese Penn Treebank , experiments show that our model achieves competitive parsing accuracy compared with conventional high-order models , however , with a much lower computational cost . In this paper , we propose an LSTM-based neural network model for graph-based dependency parsing . Utilizing Bidirectional LSTM and segment embeddings learned by LSTM-Minus allows our model access to sentence-level information , making our model more accurate in recovering longdistance dependencies with only first-order factorization . Experiments on PTB and CTB show that our model could be competitive with conventional high-order models with a faster speed .

Challenge: Neural graph-based dependency parsers can alleviate feature engineering but require atomic features and use a high-order factorization strategy which is incompatible with standard decoding algorithms.
Approach: They propose to use Bidirectional LSTM to capture contextual information instead of high-order factorization and learn sentence segment embeddings on the sentence-level by another LSTM.
Outcome: While the proposed model uses first-order factorization, it performs competitively with state-of-the-art models on English and Chinese Penn Treebank by capturing long range contextual information.