Venue: EMNLP 2017

# Learning Language Representations for Typology Prediction

 Chaitanya Malaviya & Graham Neubig & Patrick Littell

Language Technologies Institute

Carnegie Mellon University

{cmalaviy,gneubig,pw1}@cs.cmu.edu

###### Abstract

One central mystery of neural NLP is what neural models "know" about their subject matter. When a neural machine translation system learns to translate from one language to another, does it learn the syntax or semantics of the languages? Can this knowledge be extracted from the system to fill holes in human scientific knowledge? Existing typological databases contain relatively full feature specifications for only a few hundred languages. Exploiting the existence of parallel texts in more than a thousand languages, we build a massive many-to-one neural machine translation (NMT) system from 1017 languages into English, and use this to predict information missing from typological databases. Experiments show that the proposed method is able to infer not only syntactic, but also phonological and phonetic inventory features, and improves over a baseline that has access to information about the languages' geographic and phylogenetic neighbors.1

Footnote 1: Code and learned vectors are available at [http://github.com/chaitanyamalaviya/lang-reps](http://github.com/chaitanyamalaviya/lang-reps)

## 1 Introduction

Linguistic typology is the classification of human languages according to syntactic, phonological, and other classes of features, and the investigation of the relationships and correlations between these classes/features. This study has been a scientific pursuit in its own right since the 19th century Greenberg (1963); Comrie (1989); Nichols (1992), but recently typology has borne practical fruit within various subfields of NLP, particularly on problems involving lower-resource languages.

Typological information from sources like the World Atlas of Language Structures (WALS) Dryer and Haspelmath (2013), has proven useful in many NLP tasks O'Horan et al. (2016), such as multilingual dependency parsing Ammar et al. (2016), generative parsing in low-resource settings Naseem et al. (2012); Tackstrom et al. (2013), phonological language modeling and loanword prediction Tsvetkov et al. (2016), POStagging Zhang et al. (2012), and machine translation Daiber et al. (2016).

However, the needs of NLP tasks differ in many ways from the needs of scientific typology, and topological databases are often only sparsely populated, by necessity or by design.2 In NLP, on the other hand, what is important is having a relatively full set of features for the particular group of languages you are working on. This mismatch of needs has motivated various proposals to reconstruct missing entries, in WALS and other databases, from known entries Daume III and Campbell (2007); Daume III (2009); Coke et al. (2016); Littell et al. (2017).

Footnote 2: For example, each chapter of WALS aims to provide a statistically balanced set of languages over language families and geographical areas, and so many languages are left out in order to maintain balance.

In this study, we examine whether we can

Figure 1: Learning representations from multilingual neural MT for typology classification. (Model MTBoth)

tackle the problem of inferring linguistic typology from parallel corpora, specifically by training a massively multi-lingual neural machine translation (NMT) system and using the learned representations to infer topological features for each language. This is motivated both by prior work in linguistics [14; 2] demonstrating strong links between translation studies and tools for contrastive linguistic analysis, work in inferring typology from bilingual data [30] and English as Second Language texts [5], as well as work in NLP [68; 79] showing that syntactic knowledge can be extracted from neural nets on the word-by-word or sentence-by-sentence level. This work presents a more holistic analysis of whether we can discover what neural networks learn about the linguistic concepts of an entire language by aggregating their representations over a large number of the sentences in the language.

We examine several methods for discovering feature vectors for typology prediction, including those learning a language vector specifying the language while training multilingual neural language models [31] or neural machine translation [15] systems. We further propose a novel method for aggregating the values of the latent state of the encoder neural network to a single vector representing the entire language. We calculate these feature vectors using an NMT model trained on 1017 languages, and use them for typlogy prediction both on their own and in composite with feature vectors from previous work based on the genetic and geographic distance between languages [10]. Results show that the extracted representations do in fact allow us to learn about the typology of languages, with particular gains for syntactic features like word order and the presence of case markers.

## 2 Dataset and Experimental Setup

Typology Database:To perform our analysis, we use the URIEL language typology database [10], which is a collection of binary features extracted from multiple typological, phylogenetic, and geographical databases such as WALS (World Atlas of Language Structures) [32], PHOIBLE [12], Ethnologue [13], and Glottolog [14]. These features are divided into separate classes regarding syntax (e.g. whether a language has prepositions or postpositions), phonology (e.g. whether a language has complex syllabic onset clusters), and phonetic inventory (e.g. whether a language has interdental fricatives). There are 103 syntactical features, 28 phonology features and 158 phonetic inventory features in the database.

Baseline Feature Vectors:Several previous methods take advantage of typological implicature, the fact that some typological traits correlate strongly with others, to use known features of a language to help infer other unknown features of the language [15, 16, 17]. As an alternative that does not necessarily require pre-existing knowledge of the typological features in the language at hand, littell2017language have proposed a method for inferring typological features directly from the language's \(k\) nearest neighbors (\(k\)-NN) according to geodesic distance (distance on the Earth's surface) and genetic distance (distance according to a phylogenetic family tree). In our experiments, our baseline uses this method by taking the 3-NN for each language according to normalized geodesic+genetic distance, and calculating an average feature vector of these three neighbors.

Typology Prediction:To perform prediction, we trained a logistic regression classifier3 with the baseline \(k\)-NN feature vectors described above and the proposed NMT feature vectors described in the next section. We train individual classifiers for predicting each typological feature in a class (syntax etc). We performed 10-fold cross-validation over the URIEL database, where we train on 9/10 of the languages to predict 1/10 of the languages for 10 folds over the data.

Footnote 3: We experimented with a non-linear classifier as well, but the logistic regression classifier performed better.

## 3 Learning Representations for Typology Prediction

In this section we describe three methods for learning representations for typology prediction with multilingual neural models.

LM Language VectorSeveral methods have been proposed to learn multilingual language 

models (LMs) that utilize vector representations of languages Tsvetkov et al. (2016); Ostling and Tiedemann (2017). Specifically, these models train a recurrent neural network LM (RNNLM; Mikolov et al. (2010)) using long short-term memory (LSTM; Hochreiter and Schmidhuber (1997)) with an additional vector representing the current language as an input. The expectation is that this vector will be able to capture the features of the language and improve LM accuracy. Ostling and Tiedemann (2017) noted that, intriguingly, agglomerative clustering of these language vectors results in something that looks roughly like a phylogenetic tree, but stopped short of performing typological inference. We train this vector by appending a special token representing the source language (e.g. "\({}^{\prime}\)fra)" for French to the beginning of the source sentence, as shown in Fig. 1, then using the word representation learned for this token as a representation of the language. We will call this first set of feature vectors LMVec, and examine their utility for typology prediction.

NMT Language VectorIn our second set of feature vectors, MTVec, we similarly use a language embedding vector, but instead learn a multilingual neural MT model trained to translate from many languages to English, in a similar fashion to Johnson et al. (2016); Ha et al. (2016). In contrast to LMVec, we hypothesize that the alignments to an identical sentence in English, the model will have a stronger signal allowing it to more accurately learn vectors that reflect the syntactic, phonetic, or semantic consistencies of various languages. This has been demonstrated to some extent in previous work that has used specifically engineered alignment-based models Lewis and Xia (2008); Ostling (2015); Coke et al. (2016), and we examine whether these results apply to neural network feature extractors and expand beyond word order and syntax to other types of typology as well.

NMT Encoder Mean Cell StatesFinally, we propose a new vector representation of a language (MTCell) that has not been investigated in previous work: the average hidden cell state of the encoder LSTM for all sentences in the language. Inspired by previous work that has noted that the hidden cells of LSTMs can automatically capture salient and interpretable information such as syntax Karpathy et al. (2015); Shi et al. (2016) or sentiment Radford et al. (2017), we expect that the cell states will represent features that may be linked to the typology of the language. To create vectors for each language using LSTM hidden states, we obtain the mean of cell states (\(c\) in the standard LSTM equations) for all time steps of all sentences in each language.4

Footnote 4: We also tried using the mean of final hidden cell states of the encoder LSTM, but the mean cell state over all words in the sentence gave improved performance. Additionally, we tried using the hidden states \(\mathbf{h}\), but we found that these had significantly less information and lesser variance, due to being modulated by the output gate at each time step.

## 4 Experiments

### Multilingual Data and Training Regimen

To train a multilingual neural machine translation system, we used a corpus of Bible translations that was obtained by scraping a massive online Bible database at bible.com.5 This corpus contains data for 1017 languages. After preprocessing the corpus, we obtained a training set of 20.6 million sentences over all languages.

Footnote 5: A possible concern is that Bible translations may use archaic language not representative of modern usage. However, an inspection of the data did not turn up such archasisms, likely because the bulk of world Bible translation was done in the late 19th and 20th centuries. In addition, languages that do have antique Bibles are also those with many other Bible translations, so the effect of the archaisms is likely limited.

The implementation of both the LM and NMT models described in SS3 was done in the DyNet toolkit Neubig et al. (2017). In order to obtain a manageable shared vocabulary for all languages, we divided the data into subwords using joint byte-pair encoding of all languages Sennrich et al. (2016) with 32K merge operations. We

\begin{table}
\begin{tabular}{|l|r r|r r r|} \hline  & \multicolumn{2}{c|}{Syntax} & \multicolumn{2}{c|}{Phonology} & Inventory \\ \cline{2-6}  & -Aux & +Aux & -Aux & +Aux & -Aux & +Aux \\ \hline None & 69.91 & 83.07 & 77.92 & 86.59 & 85.17 & 90.68 \\ LMVec & 71.32 & 82.94 & 80.80 & 86.74 & 87.51 & 89.94 \\ MTVec & 74.90 & 83.31 & 82.41 & 87.64 & 89.62 & 90.94 \\ MTcell & 75.91 & 85.14 & 84.33 & 88.80 & 90.01 & 90.85 \\ MTBoth & **77.11** & **86.33** & **85.77** & **89.04** & **90.06** & **91.03** \\ \hline \end{tabular}
\end{table}
Table 1: Accuracy of syntactic, phonological, and inventory features using LM language vectors (LMVec), MT language vectors (MTVec), MT encoder cell averages (MTCell) or both MT feature vectors (MTBoth). Aux indicates auxiliary information of geodesic/genetic nearest neighbors; “None -Aux” is the majority class chance rate, while “None +Aux” is a 3-NN classification.

 

used LSTM cells in a single recurrent layer with 512-dimensional hidden state and input embedding size. The Adam optimizer was used with a learning rate of 0.001 and a dropout of 0.5 was enforced during training.

### Results and Discussion

The results of the experiments can be found in Tab. 1. First, focusing on the "-Aux" results, we can see that all feature vectors obtained by the neural models improve over the chance rate, demonstrating that indeed it is possible to extract information about linguistic typology from unsupervised neural models. Comparing LMVec to MTVec, we can see a convincing improvement of 2-3% across the board, indicating that the use of bilingual information does indeed provide a stronger signal, allowing the network to extract more salient features. Next, we can see that MTCell further outperforms MTVec, indicating that the proposed method of investigating the hidden cell dynamics is more effective than using a statically learned language vector. Finally, combining both feature vectors as MTBoth leads to further improvements. To measure statistical significance of the results, we performed a paired bootstrap test to measure the gain between None+Aux and MTBoth+Aux and found that the gains for syntax and inventory were significant (p=0.05), but phonology was not, perhaps because the number of phonological features was fewer than the other classes (only 28).

When further using the geodesic/genetic distance neighbor feature vectors, we can see that these trends largely hold although gains are much smaller, indicating that the proposed method is still useful in the case where we have a-priori knowledge about the environment in which the language exists. It should be noted, however, that the gains of LMVec evaporate, indicating that access to aligned data may be essential when inferring the typology of a new language. We also noted that the accuracies of certain features decreased from None-Aux to MTBoth-Aux, particularly gender markers, case suffix and negative affix, but these decreases were to a lesser extent in magnitude than the improvements.

Interestingly, and in contrast to previous methods for inferring typology from raw text, which have been specifically designed for inducing word order or other syntactic features Lewis and Xia (2008); Ostling (2015); Coke et al. (2016), our proposed method is also able to infer information about phonological or phonetic inventory features. This may seem surprising or even counterintuitive, but a look at the most-improved phonology/inventory features (Tab. 2) shows a number of features in which languages with the "non-default" option (e.g. having uvular consonants or initial velar nasals, _not_ having lateral consonants, etc.) are concentrated in particular geographical regions. For example, uvular consonants are not common world-wide, but are common in particular geographic regions like the North American Pacific Northwest and the Caucasus Maddieson (2013b), while initial velar nasals are common in Southeast Asia Anderson (2013), and lateral consonants are _uncommon_ in the Amazon Basin Maddieson (2013a). Since these are also regions with a particular and sometimes distinct syntactic character, we think the model may be find

\begin{table}
\begin{tabular}{|c|c c c|} \hline  & **None** & **MT Gain** \\ \hline S.NUMERAL_APTERNON & 37.40 & 81.26 & 43.86 \\ S.NUMERAL_BEFORE_NOUN & 46.93 & 83.26 & 36.73 \\ S.POESSESSOR_AFHERNON & 42.05 & 75.60 & 33.55 \\ S.OBJECT_BEREOKE_VERNB & 50.97 & 80.89 & 29.92 \\ S.ADPOSITION_AFHER_NOUN & 52.41 & 79.10 & 26.69 \\ P.JUULAR CONTINNATS & 77.57 & 97.37 & 19.80 \\ P.LETERIALs & 67.30 & 86.48 & 19.18 \\ P.LATERALL & 64.05 & 78.16 & 14.10 \\ P.LABMI_VELARS & 82.16 & 95.93 & 13.76 \\ P.VELEA NASAL_INITIAL & 72.14 & 85.82 & 13.68 \\ \hline LVEERANASAL & 39.89 & 62.08 & 22.20 \\ LAVEDAL_AKLTERAN-Approximant & 60.92 & 79.32 & 18.40 \\ LAVEOLAR_NASAL & 81.49 & 92.98 & 11.48 \\ I.VOICED LASBOTLATRICATIVE & 65.75 & 77.10 & 11.36 \\ I.VOICEDLESS_PALALAL_PRACTIVE & 82.41 & 93.66 & 11.25 \\ \hline \end{tabular}
\end{table}
Table 2: Top 5 improvements from “None –Aux” to “MTBoth -Aux” in the syntax (“S..”), phonology (“P..”), and inventory (“I ̇”) classes.

Figure 2: Cell trajectories for sentences in languages where S_OBJ_BEFORE_VERB is either active or inactive.

 

ing regional clusters through syntax, and seeing an improvement in regionally-distinctive phonology/inventory features as a side effect.

Finally, given that MTCell uses the feature vectors of the latent cell state to predict typology, it is of interest to observe how these latent cells behave for typologically different languages. In Fig. 2 we examine the node that contributed most to the prediction of "S_OBJ_BEFORE_VERB" (the node with maximum weight in the classifier) for German and Korean, where the feature is active, and Portuguese and Catalan, where the feature is inactive. We can see that the node trajectories closely track each other (particularly at the beginning of the sentence) for Portuguese and Catalan, and in general the languages where objects precede verbs have higher average values, which would be expressed by our mean cell state features. The similar trends for languages that share the value for a topological feature (S_OBJ_BEFORE_VERB) indicate that information stored in the selected hidden node is consistent across languages with similar structures.

## 5 Conclusion and Future Work

Through this study, we have shown that neural models can learn a range of linguistic concepts, and may be used to impute missing features in typological databases. In particular, we have demonstrated the utility of learning representations with parallel text, and results hinted at the importance of modeling the dynamics of the representation as models process sentences. We hope that this study will encourage additional use of typological features in downstream NLP tasks, and inspire further techniques for missing knowledge prediction in under-documented languages.

## Acknowledgments

We thank Lori Levin and David Mortensen for their useful comments and also thank the reviewers for their feedback about this work.

## References

* Ammar et al. (2016) Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah Smith. 2016. Many languages, one parser. _Transactions of the Association for Computational Linguistics_, 4:431-444.
* Anderson (2013) Gregory D.S. Anderson. 2013. The velar nasal. In Matthew S. Dryer and Martin Haspelmath, editors, _The World Atlas of Language Structures Online_. Max Planck Institute for Evolutionary Anthropology, Leipzig.
* Belinkov et al. (2017) Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_.
* Berzak et al. (2014) Yevgeni Berzak, Roi Reichart, and Boris Katz. 2014. Reconstructing native language typology from foreign language usage. In _Eighteenth Conference on Computational Natural Language Learning (CoNLL)_.
* Bugarski (1991) Ranko Bugarski. 1991. Contrastive analysis of terminology and the terminology of contrastive analysis. _Languages in Contact und Contrast. Essays in Contact Laguistics/Edited by Vladimir Ivir and Damir Kalogiera.-Berlin_, pages 73-82.
* Coke et al. (2016) Reed Coke, Ben King, and Dragomir Radev. 2016. Classifying syntactic regularities for hundreds of languages. _arXiv preprint arXiv:1603.08016_.
* Collins and Kayne (2011) Chris Collins and Richard Kayne. 2011. _Symatecic Structures of the World's Languages_. New York University, New York.
* Comrie (1989) Bernard Comrie. 1989. _Language Universals and Linguistic Typology: Syntax and Morphology_. Blackwell, Oxford.
* Daiber et al. (2016) Joachim Daiber, Milos Stanojevic, and Khalil Sima'an. 2016. Universal reordering via linguistic typology. In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_, pages 3167-3176, Osaka, Japan. The COLING 2016 Organizing Committee.
* Daume III (2009) Hal Daume III. 2009. Non-parametric Bayesian areal linguistics. In _Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics_, pages 593-601, Boulder, Colorado. Association for Computational Linguistics.
* Dumme III and Campbell (2007) Hal Daume III and Lyle Campbell. 2007. A Bayesian model for discovering typological implications. In _Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics_, pages 65-72, Prague, Czech Republic. Association for Computational Linguistics.
* Dryer and Haspelmath (2013) Matthew S. Dryer and Martin Haspelmath, editors. 2013. _WALS Online_. Max Planck Institute for Evolutionary Anthropology, Leipzig.
* Garcia (2002) Noelia Ramon Garcia. 2002. Contrastive linguistics and translation studies interconnected: The corpus-based approach. _Linguistic Antwerpiensis, New Series-Themes in Translation Studies_, (1) 

Joseph Greenberg. 1963. Some universals of grammar with particular reference to the order of meaningful elements. In Joseph Greenberg, editor, _Universals of Language_, pages 110-113. MIT Press, London.
* Ha et al. (2016) Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016. Toward multilingual neural machine translation with universal encoder and decoder. _arXiv preprint arXiv:1611.04798_.
* Hammarstrom et al. (2015) Harald Hammarstrom, Robert Forkel, Martin Haspellmath, and Sebastian Bank. 2015. _Glottolog 2.6_. Max Planck Institute for the Science of Human History, Jena.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural computation_, 9(8):1735-1780.
* Johnson et al. (2016) Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viegas, Martin Wattenberg, Greg Corrado, et al. 2016. Google's multilingual neural machine translation system: Enabling zero-shot translation. _arXiv preprint arXiv:1611.04558_.
* Karpathy et al. (2015) Andrej Karpathy, Justin Johnson, and Li Fei-Fei. 2015. Visualizing and understanding recurrent networks. _arXiv preprint arXiv:1506.02078_.
* Kuncoro et al. (2017) Adhiguna Kuncoro, Miguel Ballesteros, Lingpeng Kong, Chris Dyer, Graham Neubig, and Noah A. Smith. 2017. What do recurrent neural network grammars learn about syntax? In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_, pages 1249-1258, Valencia, Spain. Association for Computational Linguistics.
* Lewis et al. (2015) M. Paul Lewis, Gary F. Simons, and Charles D. Fennig. 2015. _Ethnologue: Languages of the World, Eighteenth edition_. SIL International, Dallas, Texas.
* Lewis and Xia (2008) William D Lewis and Fei Xia. 2008. Automatically identifying computationally relevant typological features. In _Proceedings of the Third International Joint Conference on Natural Language Processing, Volume II_, pages 685-690.
* Littell et al. (2017) Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlise Turner, and Lori Levin. 2017. Uriel and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, pages 8-14, Valencia, Spain. Association for Computational Linguistics.
* Maddieson (2013a) Ian Maddieson. 2013a. Lateral consonants. In Matthew S. Dryer and Martin Haspelmath, editors, _The World Atlas of Language Structures Online_. Max Planck Institute for Evolutionary Anthropology, Leipzig.
* Maddieson (2013b) Ian Maddieson. 2013b. UVular consonants. In Matthew S. Dryer and Martin Haspelmath, editors, _The World Atlas of Language Structures Online_. Max Planck Institute for Evolutionary Anthropology, Leipzig.
* Mikolov et al. (2010) Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In _Interspeech_, volume 2, page 3.
* Moran et al. (2014) Steven Moran, Daniel McCloy, and Richard Wright. 2014. _PHOIBLE Online_. Max Planck Institute for Evolutionary Anthropology, Leipzig.
* Naseem et al. (2012) Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. Selective sharing for multilingual dependency parsing. In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1_, pages 629-637. Association for Computational Linguistics.
* Neubig et al. (2017) Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, et al. 2017. Dynet: The dynamic neural network toolkit. _arXiv preprint arXiv:1701.03980_.
* Nichols (1992) Joanna Nichols. 1992. _Linguistic Diversity in Space and Time_. University of Chicago Press, Chicago.
* O'Horan et al. (2016) Helen O'Horan, Yevgen Berzak, Ivan Vulic, Roi Reichart, and Anna Korhonen. 2016. Survey on the use of typological information in natural language processing. In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_, pages 1297-1308, Osaka, Japan. The COLING 2016 Organizing Committee.
* Ostling (2015) Robert Ostling. 2015. Word order typology through multilingual word alignment. In _The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing_, pages 205-211.
* Ostling and Tiedemann (2017) Robert Ostling and Jorg Tiedemann. 2017. Continuous multilinguality with language vectors. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, pages 644-649, Valencia, Spain. Association for Computational Linguistics.
* Radford et al. (2017) Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. _arXiv preprint arXiv:1704.01444_.
* Sennrich et al. (2016) Rico Sennrich, Barry Hadlow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1715-1725, Berlin, Germany. Association for Computational Linguistics 

Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does string-based neural MT learn source syntax? In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 1526-1534, Austin, Texas. Association for Computational Linguistics.
* Tackstrom et al. (2013) Oscar Tackstrom, Ryan McDonald, and Joakim Nivre. 2013. Target language adaptation of discriminative transfer parsers. In _Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1061-1071, Atlanta, Georgia. Association for Computational Linguistics.
* Takamura et al. (2016) Hiroya Takamura, Ryo Nagata, and Yoshifumi Kawasaki. 2016. Discriminative analysis of linguistic features for typological study. In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)_, Paris, France. European Language Resources Association (ELRA).
* Tsvetkov et al. (2016) Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui, Guillaume Lample, Patrick Littell, David Mortensen, Alan W Black, Lori Levin, and Chris Dyer. 2016. Polyglot neural language models: A case study in cross-lingual phonetic representation learning. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1357-1366, San Diego, California. Association for Computational Linguistics.
* Zhang et al. (2012) Yuan Zhang, Roi Reichart, Regina Barzilay, and Amir Globerson. 2012. Learning to map into a universal POS tagset. In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_, pages 1368-1378. Association for Computational Linguistics.