SlowGuess/ACL_2020_OCR huggingface dataset:
Our probing results suggest that although BERT has captured a certain amount of syntax, there are still substantial disagreements between the syntax BERT learns and those designed by linguists. For instance, our constituency probe on PTB23 significantly outperforms most baselines, but it only roughly agree with the PTB formalism (41.2% F1). However, BERT has already demonstrated its superiority in many downstream tasks. An interesting question is whether BERT is learning an empirically useful or even better structure of a language.
To answer this question, we turn to neural networks that adopt dependency parsing trees as the explicit structure prior to improve downstream
4For reference, a supervised graph-based parser (Li et al., 2014) achieves an UAS of 57.6 on SciDTB
tasks. We replace the ground-truth dependency trees those networks used with ones induced from BERT and approximate the effectiveness of different trees by the improvements they introduced.
We conduct experiments on the Aspect Based Sentiment Classification (ABSC) task (Pontiki et al., 2014). ABSC is a fine-grained sentiment classification task aiming at identifying the sentiment expressed towards each aspect of a given target entity. As an example, in the following comment of a restaurant, "I hated their fajitas, but their salads were great", the sentiment polarities for aspect fajitas is negative and that of salads is positive. It has been shown in Zhang et al. (2019) that injecting syntactic knowledge into neural networks can improve ABSC accuracy. Intuitively, given an aspect, a syntactically closer context word should play a more important role in predicting that aspect's sentiment. They integrate the distances between context words and the aspect on a dependency tree into a convolution network and build a Proximity-Weighted Convolution Network (PwCN). As a naive baseline, they compare with network weighted by relative position between aspect and context words.
Setup. We experimented on two datasets from SemEval 2014 (Pontiki et al., 2014), which consist of reviews and comments from two categories: LAPTOP and RESTAURANT. We adopt the standard evaluation metrics: Accuracy and Macro-Averaged F1. We follow the instructions of Zhang et al. (2019) to run the experiments 5 times with random initialization and report the averaged performance. We denote the original PwCN with relative position information as PwCN-Pos, and that utilizes dependency trees constructed by  $\mathrm{SpaCy}^5$  as PwCN-Dep. SpaCy has reported an UAS of 94.5 on English PTB and so it can serve as a good reference for human-designed dependency schema. We also compare our model against two trivial trees (left-chain and right-chain trees). For our model, we feed the corpus into BERT and extract dependency trees with the best performing setting: Eisner+Dist. For parsing, we introduce an inductive bias to favor short dependencies (Eisner and Smith, 2010). To ensure a fair comparison, we induce the root word from the impact matrix  $\mathcal{F}$  instead of using the gold root. Specifically, we select the root word  $x_{k}$  based on the simple heuristic  $\arg \max_i\sum_{j = 1}^T f(x_i,x_j)$ .
Model	Laptop	Restaurant
Acc	Macro-F1	Acc	Macro-F1
LSTM	69.63	63.51	77.99	66.91
PWCN
+Pos	75.23	71.71	81.12	71.81
+Dep	76.08	72.02	80.98	72.28
+Eisner	75.99	72.01	81.21	73.00
+right-chain	75.64	71.53	81.07	72.51
+left-chain	74.39	70.78	80.82	72.71
Table 5: Experimental results of aspect based sentiment classification.
Results. Table 5 presents the performance of different models. We observe that the trees induced from BERT is either on-par (LAPTOP) or marginally better (RESTAURANT) in terms of downstream task's performance when comparing with trees produced by SpaCy. LAPTOP is considerably more difficult than RESTAURANT due to the fact that the sentences are generally longer, which makes inducing dependency trees more challenging. We also see that the Eisner trees generally perform better than the right-/left- chain baselines. It is also worth noting that the right-chain baseline also outperforms PwCN+Dep on RESTAURANT, which leads to an exciting future work that investigates how encoding structural knowledge can help ABSC.
Our results suggest that although the tree structures BERT learns can disagree with parser-provided-linguistically-motivated ones to a large extent, they are also empirically useful to downstream tasks, at least to ABSC. As future work, we plan to extend our analysis to more downstream tasks and models, like those reported in Shi (2018).
There has been substantial research investigating what pre-trained language models have learned about languages' structures.
One rising line of research uses probing classifiers to investigate the different syntactic properties captured by the model. They are generally referred to as "probing task" (Conneau et al., 2018), "diagnostic classifier" (Giulianelli et al., 2018), and "auxiliary prediction tasks" (Adi et al., 2017). The syntactic properties investigated range from basic ones like sentence length (Shi et al., 2016; Jawahar et al., 2019), syntactic tree depth (Jawahar et al., 2019), and segmentation (Liu et al., 2019)
$^{5}$ https://spacy.io/
to challenging ones like syntactic labeling (Tenney et al., 2019a,b), dependency parsing (Hewitt and Manning, 2019; Clark et al., 2019), and constituency parsing (Peters et al., 2018a). However, when a probe achieves high accuracy, it's difficult to differentiate if it is the representation that encodes targeted syntactic information, or it is the probe that just learns the task (Hewitt and Liang, 2019).
In line with our work, recent studies seek to find correspondences between parts of the neural network and certain linguistic properties, without explicit supervision.
Most of them focus on analyzing attention mechanism, by extracting syntactic tree for each attention head and layer individually (Raganato and Tiedemann, 2018; Clark et al., 2019). Their goal is to check if the attention heads of a given pre-trained model can track syntactic relations better than chance or baselines. In particular, Raganato and Tiedemann (2018) analyze a machine translation model's encoder by extracting dependency trees from its self-attention weights, using Chu-Liu/Edmonds algorithm. Clark et al. (2019) conduct a similar investigation on BERT, but the simple head selection strategy they used does not guarantee a valid dependency tree. Marecek and Rosa (2018) propose heuristic methods to convert attention weights to syntactic trees. However, they do not quantitatively evaluate their approach. In their later study (Marecek and Rosa, 2019), they propose a bottom-up algorithm to extract constituent trees from transformer-based NMT encoders and evaluate their results on three languages. Htut et al. (2019) reassess these works but find that there are no generalist heads that can do holistic parsing. Hence, analyzing attention weights directly may not reveal much of the syntactic knowledge that a model has learned. Recent dispute about attention as explanation (Jain and Wallace, 2019; Serrano and Smith, 2019; Wegreiffe and Pinter, 2019) also suggests that the attention's behavior does not necessarily represent that of the original model.
Another group of research examine the outputs of language models on carefully chosen input sentences (Goldberg, 2019; Bacon and Regier, 2019). They extend previous works (Linzen et al., 2016; Gulordava et al., 2018; Marvin and Linzen, 2018) on subject-verb agreement test (generating the correct number of a verb far away from its subject) to
provide a measure of the model's syntactic ability. Their results show that the BERT model captures syntax-sensitive agreement patterns well in general. However, subject-verb agreement cannot provide more nuanced tests of other complex structures (e.g., dependency structure, constituency structure), which are the interest of our work.
Two recent works also perturb the input sequence for model interpretability (Rosa and Mareček, 2019; Li et al., 2019b). However, these works only perturb the sequence once. Rosa and Mareček (2019) utilize the original MLM objective to estimate each word's "reducibility" and import simple heuristics into a right-chain baseline to construct dependency trees. Li et al. (2019b) focus on evaluating word alignment in NMT, but unlike our two-step masking strategy, they only replace the token of interest with a zero embedding or a randomly sampled word in the vocabulary.
8 Discussion & Conclusion
One concern shared by our reviewers is that performance of our probes are underwhelming: the induced trees are barely closer to linguist-defined trees than simple baselines (e.g., rightbranching) and are even worse in the case of discourse parsing. However, this does not mean that supervised probes are wrong or that BERT captures less syntax than we thought. In fact, there is actually no guarantee that our probe will find a strong correlation with human-designed syntax, since we do not introduce the human-designed syntax as supervision. What we found is the "natural" syntax inherent in BERT, which is acquired from self-supervised learning on plain text. We would rather say our probe complements the supervised probing findings in two ways. First, it provides a lower-bound (on the unsupervised syntactic parsing ability of BERT). By improving this lower-bound, we could uncover more "accurate" information to support supervised probes' findings. Second, we show that when combined with a down-stream application (sec 6), the syntax learned by BERT might be empirically helpful despite not totally identical to the human design.
In summary, we propose a parameter-free probing technique to complement current line of work on interpreting BERT through probes. With carefully designed two-stage perturbation, we obtain impact matrices from BERT. This matrix mirrors
the function of attention mechanism that captures inter-word correlations, except that it emerges through the output of BERT model, instead of from intermediate representations. We devise algorithms to extract syntactic trees from this matrix. Our results reinforce those of (Hewitt and Manning, 2019; Liu et al., 2019; Jawahar et al., 2019; Tenney et al., 2019b,a) who demonstrated that BERT encodes rich syntactic properties. We also extend our method to probe document structure, which sheds lights on BERT's effectiveness in modeling long sequences. Finally, we find that feeding the empirically induced dependency structures into a downstream system (Zhang et al., 2019) can further improve its accuracy. The improvement is compatible with or even superior to a human-designed dependency schema. This offers an insight into BERT's success in downstream tasks. We leave it for future work to use our technique to test other linguistic properties (e.g., coreference) and to extend our study to more downstream tasks and systems.
We would like to thank Dr.Lingpeng Kong from DeepMind for his constructive feedback of the paper. This research is supported by Hong Kong Research Grant Council GRF grants 17254016.
Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks. In ICLR '17.
Geoff Bacon and Terry Regier. 2019. Does BERT agree? Evaluating knowledge of structure dependence through agreement relations. ArXiv.
Lynn Carlson, Daniel Marcu, and Mary Ellen Okuowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and new directions in discourse and dialogue, pages 85-112. Springer.
Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Scientia Sinica, 14:1396-1400.
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What Does BERT Look at? An Analysis of BERT's Attention. In BlackboxNLP@ACL, pages 276-286.
Michael Collins. 1999. HEAD-DRIVEN STATISTICAL MODELS FOR NATURAL LANGUAGE PARSING. Ph.D. thesis, University of Pennsylvania.
Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2126-2136.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv, abs/1810.04805.
Chris Dyer, Gábor Melis, and Phil Blunsom. 2019. A critical analysis of biased parsers in unsupervised parsing. arXiv preprint arXiv:1909.09428.
Jack Edmonds. 1967. Optimum branchings. Journal of Research of the national Bureau of Standards B, 71(4):233-240.
Jason Eisner and Noah A Smith. 2010. Favor short dependencies: Parsing with soft and hard constraints on dependency length. In *Trends in Parsing Technology*, pages 121-150. Springer.
Jason M Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th conference on Computational linguistics-Volume 1, pages 340-345. Association for Computational Linguistics.
Mario Giulianielli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 240-248.
Yoav Goldberg. 2019. Assessing BERT's Syntactic Abilities. ArXiv, pages 2-5.
Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless Green Recurrent Networks Dream Hierarchically. In NAACL-HLT, pages 1195-1205.
John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733-2743.
John Hewitt and Christopher D. Manning. 2019. A Structural Probe for Finding Syntax in Word Representations. In Naacl, pages 4129-4138.
Phu Mon Htut, Jason Phang, Shikha Bordia, and Samuel R. Bowman. 2019. Do attention heads in bert track syntactic dependencies?
Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543-3556.
Ganesh Jawahar, Benoit Sagot, and Djamé Seddah. 2019. What Does BERT Learn about the Structure of Language? In ACL, pages 3651-3657.
Dan Klein and Christopher D Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 478. Association for Computational Linguistics.
Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency parsing. Synthesis Lectures on Human Language Technologies, 1(1):1-127.
Bowen Li, Lili Mou, and Frank Keller. 2019a. An imitation learning approach to unsupervised parsing. arXiv preprint arXiv:1906.02276.
Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014. Text-level discourse dependency parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25-35.
Xintong Li, Guanlin Li, Lemao Liu, Max Meng, and Shuming Shi. 2019b. On the Word Alignment from Neural Machine Translation. In ACL, pages 1293-1303.
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies. TACL, 4(1990):521-535.
Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew E Peters, and Noah A Smith. 2019. Linguistic knowledge and transferability of contextual representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1073-1094.
Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55-60.
Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank.
David Mareček and Rudolf Rosa. 2018. Extracting Syntactic Trees from Transformer Encoder Self-Attentions. In BlackboxNLP@EMNLP, pages 347-349.
David Mareček and Rudolf Rosa. 2019. From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions. In Black-boxNLP@ACL, pages 263-275.
Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192-1202.
Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018a. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499-1509.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke S. Zettlemoyer. 2018b. Deep contextualized word representations. ArXiv, abs/1802.05365.
Livia Polanyi. 1988. A formal model of the structure of discourse. Journal of pragmatics, 12(5-6):601-638.
Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27-35, Dublin, Ireland.
Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. The penn discourse treebank 2.0. Citeseer.
Alessandro Raganato and Jörg Tiedemann. 2018. An Analysis of Encoder Representations in Transformer-Based Machine Translation. In BlackboxNLP@EMNLP, pages 287-297.
Rudolf Rosa and David Mareček. 2019. Inducing Syntactic Trees from BERT Representations. In Black-boxNLP@ACL.
Roy Schwartz, Omri Abend, Roi Reichart, and Ari Rappoport. 2011. Neutralizing linguistically problematic annotations in unsupervised dependency parsing evaluation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 663-672. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715-1725.
Sofia Serrano and Noah A. Smith. 2019. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
Yikang Shen, Zhouhan Lin, Chin wei Huang, and Aaron Courville. 2018. Neural language modeling by jointly learning syntax and lexicon. In International Conference on Learning Representations.
Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron Courville. 2019. Ordered neurons: Integrating tree structures into recurrent neural networks.
Haoyue Shi, Hao Zhou, Jiaze Chen, and Lei Li. 2018. On tree-based neural sentence modeling. In International Conference on Learning Representations.
Wei Shi and Vera Demberg. 2019. Next sentence prediction helps implicit discourse relation classification within and across domains. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5794-5800.
Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does String-Based Neural MT Learn Source Syntax? In EMNLP, Table 2, pages 1526-1534.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019a. Bert rediscovers the classical nlp pipeline. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics.
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanian Das, and Ellie Pavlick. 2019b. What do you learn from context? Probing for sentence structure in contextualized word representations. In ICLR, pages 1-17.
Reut Tsarfaty, Joakim Nivre, and Evelina Ndersson. 2011. Evaluating dependency parsing: robust and heuristics-free cross-annotation evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 385-396. Association for Computational Linguistics.
Sarah Wegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11-20.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771.
Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the Eighth International Conference on Parsing Technologies, pages 195-206.
An Yang and Sujian Li. 2018. Scidtb: Discourse dependency treebank for scientific abstracts. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 444-449.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. ArXiv, abs/1906.08237.
Daniel Zeman, Martin Popel, Milan Straka, Jan Hajic, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampa Pyysalo, Slav Petrov, Martin Potthast, et al. 2017. Conll 2017 shared task: Multilingual parsing from raw text to universal dependencies. In CoNLL 2017 Shared Task, pages 1-19. Association for Computational Linguistics.
Chen Zhang, Qiuchi Li, and Dawei Song. 2019. Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1145-1148. ACM.