Almost all current dependency parsers classify based on millions of sparse indicator features . Not only do these features generalize poorly , but the cost of feature computation restricts parsing speed significantly . In this work , we propose a novel way of learning a neural network classifier for use in a greedy , transition-based dependency parser . Because this classifier learns and uses just a small number of dense features , it can work very fast , while achieving an about 2 % improvement in unlabeled and labeled attachment scores on both English and Chinese datasets . Concretely , our parser is able to parse more than 1000 sentences per second at 92.2 % unlabeled attachment score on the English Penn Treebank . In recent years , enormous parsing success has been achieved by the use of feature-based discriminative dependency parsers ( Kübler et al . , 2009 ) . In particular , for practical applications , the speed of the subclass of transition-based dependency parsers has been very appealing . However , these parsers are not perfect . First , from a statistical perspective , these parsers suffer from the use of millions of mainly poorly estimated feature weights . While in aggregate both lexicalized features and higher-order interaction term features are very important in improving the performance of these systems , nevertheless , there is insufficient data to correctly weight most such features . For this reason , techniques for introducing higher-support features such as word class features have also been very successful in improving parsing performance ( Koo et al . , 2008 ) . Second , almost all existing parsers rely on a manually designed set of feature templates , which require a lot of expertise and are usually incomplete . Third , the use of many feature templates cause a less studied problem : in modern dependency parsers , most of the runtime is consumed not by the core parsing algorithm but in the feature extraction step ( He et al . , 2013 ) . For instance , Bohnet ( 2010 ) reports that his baseline parser spends 99 % of its time doing feature extraction , despite that being done in standard efficient ways . In this work , we address all of these problems by using dense features in place of the sparse indicator features . This is inspired by the recent success of distributed word representations in many NLP tasks , e.g. , POS tagging ( Collobert et al . , 2011 ) , machine translation ( Devlin et al . , 2014 ) , and constituency parsing ( Socher et al . , 2013 ) . Low-dimensional , dense word embeddings can effectively alleviate sparsity by sharing statistical strength between similar words , and can provide us a good starting point to construct features of words and their interactions . Nevertheless , there remain challenging problems of how to encode all the available information from the configuration and how to model higher-order features based on the dense representations . In this paper , we train a neural network classifier to make parsing decisions within a transition-based dependency parser . The neural network learns compact dense vector representations of words , part-of-speech ( POS ) tags , and dependency labels . This results in a fast , compact classifier , which uses only 200 learned dense features while yielding good gains in parsing accuracy and speed on two languages ( English and Chinese ) and two different dependency representations ( CoNLL and Stanford dependencies ) . The main contributions of this work are : ( i ) showing the usefulness of dense representations that are learned within the parsing task , ( ii ) developing a neural network architecture that gives good accuracy and speed , and ( iii ) introducing a novel acti-vation function for the neural network that better captures higher-order interaction features . We have presented a novel dependency parser using neural networks . Experimental evaluations show that our parser outperforms other greedy parsers using sparse indicator features in both accuracy and speed . This is achieved by representing all words , POS tags and arc labels as dense vectors , and modeling their interactions through a novel cube activation function . Our model only relies on dense features , and is able to automatically learn the most useful feature conjunctions for making predictions . An interesting line of future work is to combine our neural network based classifier with searchbased models to further improve accuracy . Also , there is still room for improvement in our architecture , such as better capturing word conjunctions , or adding richer features ( e.g. , distance , valency ) .

Challenge: Existing dependency parsers are based on millions of parse indicator features which have poor generalization ability and restrictions on parsing speed.
Approach: They propose to represent words, part-of-speech tags, and dependency labels and train a neural network classifier with an activation function which performs transition-based dependency parsing.
Outcome: The proposed parser can parse more than 1000 sentences per second on the English Penn Treebank and also achieves both speed and accuracy in Chinese.