Accurate scoring of syntactic structures such as head-modifier arcs in dependency parsing typically requires rich , highdimensional feature representations . A small subset of such features is often selected manually . This is problematic when features lack clear linguistic meaning as in embeddings or when the information is blended across features . In this paper , we use tensors to map high-dimensional feature vectors into low dimensional representations . We explicitly maintain the parameters as a low-rank tensor to obtain low dimensional representations of words in their syntactic roles , and to leverage modularity in the tensor for easy training with online algorithms . Our parser consistently outperforms the Turbo and MST parsers across 14 different languages . We also obtain the best published UAS results on 5 languages . 1 Finding an expressive representation of input sentences is crucial for accurate parsing . Syntactic relations manifest themselves in a broad range of surface indicators , ranging from morphological to lexical , including positional and part-of-speech ( POS ) tagging features . Traditionally , parsing research has focused on modeling the direct connection between the features and the predicted syntactic relations such as head-modifier ( arc ) relations in dependency parsing . Even in the case of firstorder parsers , this results in a high-dimensional vector representation of each arc . Discrete features , and their cross products , can be further complemented with auxiliary information about words participating in an arc , such as continuous vector representations of words . The exploding dimensionality of rich feature vectors must then be balanced with the difficulty of effectively learning the associated parameters from limited training data . A predominant way to counter the high dimensionality of features is to manually design or select a meaningful set of feature templates , which are used to generate different types of features ( Mc-Donald et al . , 2005a ; Koo and Collins , 2010 ; Martins et al . , 2013 ) . Direct manual selection may be problematic for two reasons . First , features may lack clear linguistic interpretation as in distributional features or continuous vector embeddings of words . Second , designing a small subset of templates ( and features ) is challenging when the relevant linguistic information is distributed across the features . For instance , morphological properties are closely tied to part-of-speech tags , which in turn relate to positional features . These features are not redundant . Therefore , we may suffer a performance loss if we select only a small subset of the features . On the other hand , by including all the rich features , we face over-fitting problems . We depart from this view and leverage highdimensional feature vectors by mapping them into low dimensional representations . We begin by representing high-dimensional feature vectors as multi-way cross-products of smaller feature vectors that represent words and their syntactic relations ( arcs ) . The associated parameters are viewed as a tensor ( multi-way array ) of low rank , and optimized for parsing performance . By explicitly representing the tensor in a low-rank form , we have direct control over the effective dimensionality of the set of parameters . We obtain role-dependent low-dimensional representations for words ( head , modifier ) that are specifically tailored for parsing accuracy , and use standard online algorithms for optimizing the low-rank tensor components . The overall approach has clear linguistic and computational advantages : • Our low dimensional embeddings are tailored to the syntactic context of words ( head , modifier ) . This low dimensional syntactic abstraction can be thought of as a proxy to manually constructed POS tags . • By automatically selecting a small number of dimensions useful for parsing , we can leverage a wide array of ( correlated ) features . Unlike parsers such as MST , we can easily benefit from auxiliary information ( e.g. , word vectors ) appended as features . We implement the low-rank factorization model in the context of first-and third-order dependency parsing . The model was evaluated on 14 languages , using dependency data from CoNLL 2008 and CoNLL 2006 . We compare our results against the MST ( McDonald et al . , 2005a ) and Turbo ( Martins et al . , 2013 ) parsers . The low-rank parser achieves average performance of 89.08 % across 14 languages , compared to 88.73 % for the Turbo parser , and 87.19 % for MST . The power of the low-rank model becomes evident in the absence of any part-of-speech tags . For instance , on the English dataset , the low-rank model trained without POS tags achieves 90.49 % on first-order parsing , while the baseline gets 86.70 % if trained under the same conditions , and 90.58 % if trained with 12 core POS tags . Finally , we demonstrate that the model can successfully leverage word vector representations , in contrast to the baselines . Accurate scoring of syntactic structures such as head-modifier arcs in dependency parsing typically requires rich , high-dimensional feature representations . We introduce a low-rank factorization method that enables to map high dimensional feature vectors into low dimensional representations . Our method maintains the parameters as a low-rank tensor to obtain low dimensional representations of words in their syntactic roles , and to leverage modularity in the tensor for easy training with online algorithms . We implement the approach on first-order to third-order dependency parsing . Our parser outperforms the Turbo and MST parsers across 14 languages . Future work involves extending the tensor component to capture higher-order structures . In particular , we would consider second-order structures such as grandparent-head-modifier by increasing the dimensionality of the tensor . This tensor will accordingly be a four or five-way array . The online update algorithm remains applicable since each dimension is optimized in an alternating fashion .

Challenge: To obtain a rich representation for parsing, manually designed templates have been used which can cause a lack of clear linguistic interpretation or design difficulties.
Approach: They propose to use tensors to map high-dimensional feature vectors into low-dimensional representations which can also represent syntactic relations and use them for online training.
Outcome: The proposed approach outperforms the Turbo and MST parsers on 14 languages CoNLL 2008 and 2006, and it performs especially well without part-of-speech tags.