Venue: NAACL 2021

# A Non-Linear Structural Probe

Jennifer C. White\({}^{\text{\textcircledS}}\) Tiago Pimentel\({}^{\text{\textcircledS}}\) Naomi Saphra\({}^{\text{\textcircledRyan Cotterell,${}^{\textcircledS}$}}\)

\({}^{\textcircledS}\)University of Cambridge, \({}^{\textcircledS}\)University of Edinburgh, \({}^{\textcircledS}\)ETH Zurich

jw2088@cam.ac.uk, tp472@cam.ac.uk n.saphra@ed.ac.uk, ryan.cotterell@inf.ethz.ch

###### Abstract

Probes are models devised to investigate the encoding of knowledge--e.g. syntactic structure--in contextual representations. Probes are often designed for simplicity, which has led to restrictions on probe design that may not allow for the full exploitation of the structure of encoded information; one such restriction is linearity. We examine the case of a structural probe Hewitt and Manning (2019), which aims to investigate the encoding of syntactic structure in contextual representations through learning only linear transformations. By observing that the structural probe learns a metric, we are able to kernelize it and develop a novel non-linear variant with an identical number of parameters. We test on 6 languages and find that the radial-basis function (RBF) kernel, in conjunction with regularization, achieves a statistically significant improvement over the baseline in all languages--implying that at least part of the syntactic knowledge is encoded non-linearly. We conclude by discussing how the RBF kernel resembles BERT's self-attention layers and speculate that this resemblance leads to the RBF-based probe's stronger performance.

## 1 Introduction

Probing has been widely used in an effort to better understand what linguistic knowledge may be encoded in contextual word representations such as BERT Devlin et al. (2019) and \(\operatorname{ELMo}\)Peters et al. (2018). These probes tend to be designed with simplicity in mind and with the intent of revealing what linguistic structure is encoded in an embedding, rather than simply learning to perform an NLP task Hewitt and Liang (2019); Zhang and Bowman (2018); Voita and Titov (2020) This preference for simplicity has often led researchers to place restrictions on probe designs that may not allow them to fully exploit the structure in which information is encoded Saphra and Lopez (2019); Pimentel et al. (2020b, a). This preference has led many researchers to advocate the use of linear probes over non-linear ones Alain and Bengio (2017).

This paper treats and expands upon the structural probe of Hewitt and Manning (2019), who crafted a custom probe with the aim of investigating the encoding of sentence syntax in contextual representations. They treat probing for syntax as a distance learning problem: they learn a linear transformation that warps the space such that two words that are syntactically close to one another (in terms of distance in a dependency tree) should have contextual representations whose Euclidean distance is small. This linear approach performs well, but the restriction to learning only linear transformations seems arbitrary. Why should it be the case that this information would be encoded linearly within the representations?

In this paper, we recast Hewitt and Manning (2019)'s structural probing framework as a general metric learning problem. This reduction allows us to take advantage of a wide variety of non-linear extensions--based on kernelization--proposed in the metric learning literature Kulis (2013). These extensions lead to probes with the _same_ number of parameters, but with an increased expressivity.

By exploiting a kernelized extension, we are able to directly test whether a structural probe that is capable of learning non-linear transformations improves performance. Empirically, we do find that non-linearity helps--a structural probe based on a radial-basis function (RBF) kernel improves performance significantly in all 6 languages tested over a linear structural probe. We then perform an analysis of \(\operatorname{BERT}\)'s attention, asserting it is a rough approximation to an RBF kernel. As such, it is not surprising that the syntactic information in \(\operatorname{BERT}\) representations is more accessible with this specific _non-linear_ transformation. We conclude that kernelization is a useful tool for analyzing contextual representations--enabling us to run controlled

experiments and investigate the structure in which information is encoded.

## 2 The Structural Probe

Hewitt and Manning (2019) introduce the **structural probe**, a novel model designed to probe for syntax in contextual word representations. We review their formulation here and build upon it in SS4. A sentence \({\bf w}\) lives in a space \(V^{*}\), defined here as the Kleene closure of a (potentially open) vocabulary \(V\). The syntactic distance \(\Delta_{ij}\) between any two words in a sentence \({\bf w}\) is the number of steps needed to go from one word to the other while walking in the sentence's syntactic tree. More formally, if we have a dependency tree \({\bf t}\) (a tree on \(n+1\) nodes) of a sentence \({\bf w}\) of length \(n\), we define \(\Delta_{ij}\) as the length of the shortest path in \({\bf t}\) between \(w_{i}\) and \(w_{j}\); this may be computed, for example, by Floyd-Warshall. Contextual representations of a sentence \({\bf w}\) are a sequence of vectors \({\bf h}_{i}\in\mathbb{R}^{d_{1}}\) that encode some linguistic knowledge about a sequence. In the case of \({\rm BERT}\), we have

\[{\bf h}_{i}={\rm BERT}({\bf w})_{i}\in\mathbb{R}^{d_{1}} \tag{1}\]

Here, the goal of probing is to evaluate whether the contextual representations capture the syntax in a sentence. In the case of the structural probe, the goal is to see whether the syntactic distance between any two words can be approximated by a learned, linear distance function:

\[d_{\mathbf{B}}({\bf h}_{i},{\bf h}_{j})=||{\bf B}{\bf h}_{i}-{\bf B}{\bf h}_{ j}||_{2} \tag{2}\]

where \({\bf B}\in\mathbb{R}^{d_{2}\times d_{1}}\) is a linear projection matrix. That is to say, they seek a linear transformation such that the transformed contextual representations relate to one another roughly as their corresponding words do in the dependency tree. To learn this probe, Hewitt and Manning minimize the following per-sentence objective with respect to \({\bf B}\) through stochastic gradient descent

\[\frac{1}{|{\bf w}|^{2}}\sum_{i=1}^{|{\bf w}|}\sum_{j=i+1}^{|{\bf w}|}|\Delta_{ ij}-d_{\mathbf{B}}({\bf h}_{i},{\bf h}_{j})| \tag{3}\]

This is simply minimizing the difference between the syntactic distances obtained from the dependency tree and the distance between the two vectors under our learned transformation. From the pairwise distances predicted by the probe, Prim's (1957) algorithm can be used to recover the one-best undirected dependency tree.

## 3 Kernelized Metric Learning

The restriction to a linear transformation may hinder us from uncovering some of the syntactic structure encoded in the contextual representations. Indeed, there is no reason _a-priori_ to expect that \({\rm BERT}\) encodes its knowledge in a fashion that is specifically accessible to a _linear_ model. However, if we were to introduce non-linearity by using a neural probe, for example, we would have to pit a model with very few parameters (the linear model) against one with very many (the neural network); this comparison is not fair and also goes against the spirit of designing simple probes. To preclude the need for a neural probe, we instead turn to a kernelized probe.

The key insight is that the structural probe reduces the problem of probing for linguistic structure to that of **metric learning**Kulis (2013). This can be clearly seen in eq. (3), where the probe learns a distance metric between two representations in such a way that it matches the syntactic one. Recognizing this relationship allows us to take advantage of established techniques from the metric learning literature to improve the performance of the probe without increasing its complexity, e.g. through kernelization.

### The "Kernel Trick" for Distances

Many algorithms in machine learning, e.g. support vector machines and \(k\)-means, can be **kernelized**Scholkopf and Smola (2002), thus allowing for linear models to be adapted into non-linear ones. Expanding on a classic result Schoenberg (1938), Scholkopf (2001) show that any positive semi-definite (PSD) kernel can be used to construct a distance in a Hilbert space \(\mathcal{H}\). Formally, their result states that for any PSD kernel \(\kappa:\mathcal{X}\times\mathcal{X}\rightarrow\mathbb{R}_{\geq 0}\), there exists a feature map \(\phi:\mathcal{X}\rightarrow\mathcal{H}\) such that

\[||\phi(\mathbf{x})-\phi(\mathbf{y})||_{2}=\\ \sqrt{\kappa(\mathbf{x},\mathbf{x})-2\kappa(\mathbf{x},\mathbf{ y})+\kappa(\mathbf{y},\mathbf{y})} \tag{4}\]

This generalizes eq. (2) to yield a new, non-linear distance metric. This means that we can achieve the effects of using some non-linear feature mapping \(\phi\) without having to specify it: we need only specify a kernel function and perform calculations using this kernelized distance metric. Importantly, as opposed to deep neural probes, this learnable metric has an _identical_ number of parameters to the original.1 



### Common Kernels

In this section we introduce the kernels to be used. These kernels were chosen as they represent a comprehensive selection of commonly-used kernels in the metric learning literature (Kulis, 2013). The original work of Hewitt and Manning (2019) makes use of the **linear kernel**:

\[\kappa_{\mathrm{linear}}(\mathbf{h}_{i},\mathbf{h}_{j})=(\mathbf{B}\mathbf{h}_ {i})^{\top}(\mathbf{B}\mathbf{h}_{j}) \tag{5}\]

The first non-linear kernel we consider is the **polynomial kernel**, defined as

\[\kappa_{\mathrm{poly}}(\mathbf{h}_{i},\mathbf{h}_{j})=\left((\mathbf{B} \mathbf{h}_{i})^{\top}(\mathbf{B}\mathbf{h}_{j})+c\right)^{d} \tag{6}\]

where \(d\in\mathbb{Z}_{+}\) and \(c\in\mathbb{R}_{\geq 0}\). A polynomial kernel of degree \(d\) allows for \(d\)-order interactions between the terms. When working with \(\mathrm{BERT}\), this means that we may construct \(d\)-order conjunctions of the dimensions of the contextual representations input into the probe. Next, we consider the **radial-basis function kernel** (RBF). This kernel is also called the Gaussian kernel and is defined as

\[\kappa_{\mathrm{rbf}}(\mathbf{h}_{i},\mathbf{h}_{j})=\exp\left(-\frac{|| \mathbf{B}\mathbf{h}_{i}-\mathbf{B}\mathbf{h}_{j}||^{2}}{2\sigma^{2}}\right) \tag{7}\]

This kernel has an alternative interpretation as a similarity measure between both vectors, being at its maximum value of \(1\) when \(\mathbf{h}_{i}=\mathbf{h}_{j}\). In contrast to the polynomial kernel, the Gaussian kernel implies a feature map in an infinite dimensional Hilbert space. When the RBF kernel is used in our probe, we may rewrite eq. (2) as follows:

\[d_{\kappa_{\mathrm{rbf}}}(\mathbf{h}_{i},\mathbf{h}_{j})^{2} \tag{8}\] \[=\kappa_{\mathrm{rbf}}(\mathbf{h}_{i},\mathbf{h}_{i})-2\kappa_{ \mathrm{rbf}}(\mathbf{h}_{i},\mathbf{h}_{j})+\kappa_{\mathrm{rbf}}(\mathbf{ h}_{j},\mathbf{h}_{j})\] \[=2-2\kappa_{\mathrm{rbf}}(\mathbf{h}_{i},\mathbf{h}_{j})\] \[=2-2\exp\left(-\frac{||\mathbf{B}\mathbf{h}_{i}-\mathbf{B} \mathbf{h}_{j}||^{2}}{2\sigma^{2}}\right)\]

Which is similar to the original linear case in eq. (2), but with a scaling term \(-\frac{1}{2\sigma^{2}}\) and a non-linearity \(\exp(\cdot)\). Finally, we consider, the **sigmoid kernel**, which is defined as 2

Footnote 2: Lin and Lin (2003) observe that it is difficult to effectively tune \(a\) and \(b\) in the sigmoid kernel. They also note that although this kernel is not in fact PSD, it is PSD when \(a\) and \(b\) are both positive, which we enforce in this work.

\[\kappa_{\mathrm{sig}}(\mathbf{h}_{i},\mathbf{h}_{j})=\tanh\left(a(\mathbf{B} \mathbf{h}_{i})^{\top}(\mathbf{B}\mathbf{h}_{j})+b\right) \tag{9}\]

## 4 Regularized Metric Learning

We also take advantage of two common regularization techniques employed in the metric learning literature to further improve the transformations learned; both act on the matrix \(\mathbf{A}=\mathbf{B}^{\top}\mathbf{B}\) and are added to the objective specified in eq. (3). The **Frobenius norm regularizer** takes the form

\[r(\mathbf{A})=||\mathbf{A}||_{\mathrm{F}}^{2}=\operatorname{tr}\left(\mathbf{ A}^{\top}\mathbf{A}\right) \tag{10}\]

This is the matrix analogue of the \(L_{2}\) squared regularizer. Minimizing the Frobenius norm of the learned matrix has the effect of keeping the values in the matrix small. It has been a popular choice for regularization in metric learning with adaptations to a variety of problems (Schultz and Joachims, 2004; Kwok and Tsang, 2003). We also consider the **trace norm regularizer**, which is of the form

\[r(\mathbf{A})=\operatorname{tr}(\mathbf{A}) \tag{11}\]

The trace norm regularizer is the matrix analogue of the \(L_{1}\) regularizer and it encourages the matrix \(\mathbf{A}\) to be low rank. As Jain et al. (2010) point out, using a low-rank transformation in conjunction with a kernel corresponds to a supervised kernel dimensionality reduction method.

## 5 Experiments

We experiment with Hewitt and Manning's (2019) probe on 6 typologically diverse languages, following the experimental design of Hall Maudslay et al. (2020). Our data comes from the Universal Dependency 2.4 Treebank (Nivre et al., 2019), providing sentences and their dependency trees, annotated using the Universal Dependencies annotation scheme.3 For each sentence we calculate contextual representations using multilingual \(\mathrm{BERT}\). For all languages, we took the first 12,000 sentences (or the maximum number thereof) in the train portion of the treebank and created new 80-10-10 train-test-dev splits.4

We present the results from our comparison of a re-implementation of Hewitt and Manning's (2019) linear structural probe and the non-linear kernelized probes in Table 1. The two evaluation metrics shown are unlabeled undirected attachment score (UUAS) and the Spearman rank-order correlation (DSpr) between predicted distances and gold standard pairwise distances. UUAS is a standard parsing metric expressing the percentage of correct attachments in the dependency tree, while DSpr is a measure of how accurately the probe predicts the overall ordering of distances between words. We can see that the use of an RBF kernel results in a statistically significant improvement in performance, as measured by UUAS, in all 6 of the languages tested.5 For some languages this improvement is quite substantial, with Tamil seeing an improvement of 8.44 UUAS from the baseline probe to the RBF kernel probe.

Footnote 5: Significance was established using paired permutation tests with 10,000 samples, to the level of \(p<0.05\).

## 6 The RBF Kernel and Self-Attention

The RBF kernel produces improvements across all analyzed languages. This suggests that it is indeed the case that syntactic structure is encoded nonlinearly in \({\rm BERT}\). As such, analyzing this specific kernel may yield insights into what this structure is. Indeed, none of the other kernels systematically improve over the linear baseline, implying this is not just an effect of the non-linearity introduced through use of a kernel--the specific structure of the RBF kernel must be responsible. In this section, we argue that the reason that the RBF kernel serves as such a boon to probing is that it resembles \({\rm BERT}\)'s attention mechanism; recall that \({\rm BERT}\)'s attention mechanism is defined as

\[\mathrm{att}(\mathbf{h}_{i},\mathbf{h}_{j})\propto\exp\left(\frac{(\mathbf{K} \mathbf{h}_{i})^{\top}(\mathbf{Q}\mathbf{h}_{j})}{\sqrt{d_{2}}}\right) \tag{12}\]

where \(\mathbf{K}\) and \(\mathbf{Q}\) are linear transformations and \(d_{2}\) is the dimension vectors are projected into. \(\mathbf{K}\) projects vector \(\mathbf{h}_{i}\) into a key vector, while \(\mathbf{Q}\) projects \(\mathbf{h}_{j}\) into a query one. When the key and query vectors are similar (i.e. have a high dot product), the value of this equation is large and word \(j\) attends to word \(i\).

This bears a striking resemblance to the Gaussian kernel. Indeed, if we assume the linearly transformed representations have unit norm, i.e.

\[||\mathbf{B}\mathbf{h}_{i}||^{2}=||\mathbf{B}\mathbf{h}_{j}||^{2}=1 \tag{13}\]

then we have

\[\exp\left(\frac{-1}{2\sqrt{d_{2}}}||\mathbf{B}\mathbf{h}_{i}- \mathbf{B}\mathbf{h}_{j}||^{2}\right) \tag{14}\] \[\qquad=\exp\left(\frac{-1}{\sqrt{d_{2}}}+\frac{(\mathbf{B} \mathbf{h}_{i})^{\top}(\mathbf{B}\mathbf{h}_{j})}{\sqrt{d_{2}}}\right)\] \[\qquad\propto\exp\left(\frac{(\mathbf{B}\mathbf{h}_{i})^{\top}( \mathbf{B}\mathbf{h}_{j})}{\sqrt{d_{2}}}\right)\]

where we take \(\sigma^{2}=\sqrt{d_{2}}\). The similarity between eqs. (12) and (14) suggests the attention mechanism in \({\rm BERT}\) is, up to a multiplicative factor, roughly equivalent to an RBF kernel--as such, it is not surprising that the RBF kernel produces the strongest results.

The resemblance between these equations, taken together with the significant improvements in capturing syntactic distance, suggest that this encoded information indeed lives in an RBF-like space in \({\rm BERT}\). Such information can then be used in its self-attention mechanism; allowing \({\rm BERT}\) to pay attention to syntactically close words when solving the cloze language modeling task. Being attentive to syntactically close words would also be supported by recent linguistic research, since words sharing syntactic dependencies have higher mutual information on average (Futrell et al., 2019).

The representations we analyze, though, are taken from \({\rm BERT}\)'s final layer; as such, they are not trained to be used in any self-attention layer--so why should such a resemblance be relevant? \({\rm BERT}\)'s architecture is based on the Transformer

\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline  & \multicolumn{2}{c}{Basque} & \multicolumn{2}{c}{English} & \multicolumn{2}{c}{Finish} & \multicolumn{2}{c}{Korean} & \multicolumn{2}{c}{Tamil} & \multicolumn{2}{c}{Turkish} \\ \cline{2-17} Kernel & UUAS & DSpr & UUAS & DSpr & UUAS & DSpr & UUAS & DSpr & UUAS & DSpr & UUAS & DSpr \\ \hline None & 58.39 & 0.6737 & 57.96 & 0.7382 & 59.90 & **0.7560** & 68.63 & **0.7026** & 48.52 & 0.5116 & 58.87 & 0.6784 \\ Polynomial & 50.10 & 0.5751 & 59.67 & **0.7635** & 57.12 & 0.7401 & 67.58 & 0.6966 & 54.43 & **0.5776** & 55.29 & 0.6421 \\ Sigmoid & 43.14 & 0.4500 & 41.62 & 0.6152 & 53.14 & 0.6201 & 44.48 & 0.3734 & 44.30 & 0.3836 & 51.77 & 0.5557 \\ RBF & **60.99** & **0.6937** & **62.77** & 0.7213 & **63.08** & 0.7382 & **71.87** & 0.6918 & **56.96** & 0.5379 & **61.67** & **0.6841** \\ \hline \end{tabular}
\end{table}
Table 1: Results of probes using various kernels, in terms of UUAS and DSpr 

(Vaswani et al., 2017), and uses skip connections between each self-attention layer. Such skip connections create an incentive for residual learning, i.e. only learning residual differences in each layer, while propagating the bulk of the information (He et al., 2016). As such, \({\rm BERT}\)'s final hidden representations should roughly live in the same manifold as its internal ones.

It is interesting to note that the RBF kernel achieves the best performance in terms of UUAS in all languages, but it only twice achieves the best performance in terms of DSpr. This may be due to the fact that, as we can see by examination of eq.8, the distance returned by the RBF kernel will not exceed 2, whereas syntactic distances in the tree will. Further, the gradient of the RBF kernel contains an exponential term which will cause it to go to zero as distance increases (while an examination of the unkernelized loss function reveals the opposite behavior). This means that it will be less sensitive to the distances between syntactically distant words and focus more on words with small distances. This may partially explain its better performance on UUAS, and comparably worse performance as measured by correlation (which counts pairwise differences between _all_ words, not just those which are directly attached in the tree). Furthermore, our probe's focus on nearby words resembles the general attentional bias towards syntactically close words (Voita et al., 2019).

The direct resemblance between self-attention mechanisms and our proposed probe metric poses a new way of understanding results from more complex probes. While Reif et al. (2019) understood the Euclidean-squared distance of Hewitt and Manning as an isometric tree embedding, their geometric interpretation did not factor in the rest of BERT's architecture. Such simplified contextless probes cannot tell us how linguistic properties are processed by a sequence of learned modules (Saphra and Lopez, 2019). However, we consider representations _in the context_ of the model which is expected to employ them. From this perspective, simpler metrics may be rough approximations to our RBF kernel space, which is actually capable of measuring linguistic properties that can be easily extracted by an attention-based architecture.

## 7 Conclusion

We find that the linear structural probe (Hewitt and Manning, 2019) used to investigate the encoding of syntactic structure in contextual representations can be improved through kernelization, yielding a non-linear model. This kernelization does not introduce additional parameters and thus does not increase the complexity of the probe--at least if one treats the number of parameters as a good proxy for model complexity. At the same time, the RBF kernel improves probe performance in all languages under consideration. This suggests that syntactic information may be encoded non-linearly in the representations produced by \({\rm BERT}\). We hypothesize that this is true due to the similarity of the RBF kernel and \({\rm BERT}\)'s self-attention layers.

## Ethical Considerations

The authors foresee no ethical concerns with the research presented in this paper.

## References

* Alain and Bengio (2017) Guillaume Alain and Yoshua Bengio. 2017. Understanding intermediate layers using linear classifier probes. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota. External Links: Link Cited by: SS3.4.
* R. W. Floyd (1962)Algorithm 97: shortest path. Communications of the ACM5 (6), pp. 345. Cited by: SS1.
* R. Futrell, P. Qian, E. Gibson, E. Fedorenko, and I. Blank (2019)Syntactic dependencies correspond to word pairs with high mutual information. In Proceedings of the Fifth International Conference on Dependency Linguistics (Depling, SyntaxFest 2019), Paris, France, pp. 3-13. External Links: ISBN 978-3-319-3861-1, Document Cited by: SS1.
* R. H. Maudslay, J. Valvoda, T. Pimentel, A. Williams, and R. Cotterell (2020)A tale of a probe and a parser. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. . Cited by: SS1.
* K. He, X. Zhang, S. Ren, and J. Sun (2016)Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778.

 

John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2733-2743, Hong Kong, China. Association for Computational Linguistics.
* Hewitt and Manning [2019] John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4129-4138, Minneapolis, Minnesota. Association for Computational Linguistics.
* Jain et al. [2010] Prateek Jain, Brian Kulis, and Inderjit Dhillon. 2010. Inductive regularized learning of kernel functions. In _Advances in Neural Information Processing Systems_, volume 23.
* Kulis [2013] Brian Kulis. 2013. _Metric Learning: A Survey_, volume 5 of _Foundations and Trends\(\circled{@}\) in Machine Learning_. Now Publishers, Inc.
* Kuznetsov and Gurevych [2020] Ilia Kuznetsov and Iryna Gurevych. 2020. A matter of framing: The impact of linguistic formalism on probing results. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 171-182, Online. Association for Computational Linguistics.
* Kwok and Tsang [2003] James T. Kwok and Ivor W. Tsang. 2003. Learning with idealized kernels. In _Proceedings of the 20th International Conference on Machine Learning_, pages 400-407.
* Lin and Lin [2003] Hsuan-Tien Lin and Chih-Jen Lin. 2003. A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods. _Neural Computation_, 3:1-32.
* Nivre et al. [2016] Joakim Nivre, Mitchell Abrams, Zeljko Agic, Lars Ahrenberg, Gabriele Aleksandravicinte, Lene Antonsen, Katya Aplonova, Maria Jesus Aranzabe, Gashaw Arutie, Masayuki Asahara, Luma Ateyah, Mohammed Attia, Aitziber Autuxa, Liesbeth Augustinus, Elena Badmaeva, Miguel Ballesteros, Esha Banerjee, Sebastian Bank, Verginia Barbu Mititelu, Victoria Basmov, John Bauer, Sandra Bellato, Kepa Bengoetxea, Yevgeni Berzak, Irshad Ahmad Bhat, Riyaz Ahmad Bhat, Erica Biagetti, Eckhard Bick, Agne Beilinskiene, Rogier Blokland, Victoria Bobicev, Loic Boizou, Emanuel Borges Volker, Carlis Bronstell, Cristina Bosco, Gosse Bouma, Sam Bowman, Adriane Boyd, Kristina Brokaite, Alijoscha Burchardt, Marie Candito, Bernard Caron, Gauthier Caron, Gulsen Cebiroglu Eryigit, Flavio Massimilation Cecchini, Giuseppe G. A. Celano, Slavomir Ceplo, Savas Cetin, Fabricio Chalub, Jinho Choi, Yongsoeh Cho, Jayed Chun, Silvie Cinkova, Avelie Collomb, Cagn Coltekin, Miram Connor, Marine Courtin, Elizabeth Davidson, Marie-Catherine de Marmeffe, Valeria de Paiva, Arantza Diaz de Harraza, Carlo Dickerson, Bamba Dione, Peter Dirix, Kaiaj Dobrovolz, Timothy Dozat, Kira Droganova, Puneu Dwivedt, Hanne Eckhoff, Mariaba Eli, Ali Elkahky, Binyam Ephrem, Tomaz Erjuec, Aline Gienne, Richard Farkas, Hector Fernandez Allacle, Jennifer Foster, Claudia Freitas, Kazunori Fujita, Katarina Gajdosova, Daniel Galbraith, Marcos Garcia, Moa Gardenfors, Sebastian Garza, Kim Gerdes, Filip Girinet, Lakes Goenaga, Kaldo Gojeenda, Memdu Gokrmak, Yoaro Goldberg, Xavier Gomez Guinovart, Bernta Gonzalez Saavedra, Matias Grioni, Normunds Guruztis, Bruno Guillaume, Celine Guillot-Barbance, Nizar Habash, Jan Hajic jr., Linh Ha My, Na-Rae Han, Kim Harris, Dag Haug, Johannes Heinecke, Felix Hennig, Barbara Halda, Jaroslava Hlavacova, Florieni Hocung, Petter Holhe, Jena Hwang, Takumi Ikeda, Radu Ion, Elena Irmia, Qljide Ishola, Tomas Jelinek, Anders Johannsen, Fredrik Jorgensen, Hunner Kasikara, Andre Kaassen, Sylvain Kahane, Hiroshi Kanayama, Jenna Kanerva, Boris Katz, Tolga Kayadelen, Jessica Kenney, Vaclava Kettenrova, Jesse Kirchner, Arne Kohn, Kamil Kopacewicz, Natalia Kotsyba, Jolanta Kovalevskait, Simon Krek, Sookoyoung Kwaki, Veroniska Liappala, Lorenzo Lambertino, Lucia Lam, Tetiana Lando, Septian Dian Larasaki, Alexei Lavrentiev, John Lee, Phutong Le Hong, Alessandro Lenci, Saran Lertpradit, Herman Leung, Cheuk Ying Li, Josie Li, Keying Li, Kyung-Tae Lim, Yuan Li, Nikola Ljubesic, Olga Loginova, Olga Lyasheskaya, Teresa Lynn, Vivien Mackentanz, Aibek Makzhanov, Michael Mandl, Christopher Manning, Ruli Manurung, Catalina Marinduc, David Marecek, Katrin Marineenecke, Hector Martinez Alonso, Andre Martins, Jan Masek, Yuji Matsumoto, Ryan McDonald, Sarah McGuinness, Gustavo Mendonca, Niko Miekka, Margaritis Misripasabaveg, Anna Missila, Catalan Mititelu, Yusuke Miyao, Simometta Montemigni, Amir More, Laura Moreno Romero, Keiko Sophie Mori, Tomohiko Morioka, Shinsuke Mori, Shigeki Moro, Bjatur Mortensen, Bohdan Moskalovskyi, Kadri Muischnek, Yugo Murawaki, Kaili Muurisep, Pinkey Nainwani, Juan Ignacio Navarro Horniacek, Anna Nedoluzhko, Gunta Negore-Berzkalne, Lotng Nguyen Thi, Huyen Nguyu Pin Thi Minh, Yoshishiro Nikaido, Vitaly Nikolaev, Rattimina Nitarioj, Hamna Nurmi, Stina Ojala, Addedyo Oloku, Mai Omura, Petya Osenova, Robert Ostling, Lilja Ovrelid, Niko Partanen, Elena Pascual, Marco Passarotti, Agnieszka Palejuk, Guilherme Paulino-Passos, Angelika Peljak-Lapinska, Siyao Peng, Cenel-Augusto Perez, Guy Perrier, Daria Petrova, Slav Petrov, Jussi Piitulainen, Tommi A Pirinen, Emily Pitler, Barbara Plank, Thierry Poibeau, Martin Popel, Laumn Prektalning, Sophie Prevost, Prokopis Prokopidis, Adam Przepiorkowski, Tiina Puolakainen, Sampo Pyysalo, Andriela Raabis, Alexandre Rademaker, Loganathan Ramasamy, Taraka Rama, Carlos Ramisch, Vinit Ravishankar, Livy Real, Siva Reddy, Georg Rehm, Michael Riessler, Erika Rimkute, Larissa Rinaldi, Laura Rituma, Luisa 

Rocha, Mykhailo Romanenko, Rudolf Rosa, Davide Rovati, Valentin Rosca, Olga Rudina, Jack Rueter, Shoval Sadde, Benoti Sagot, Shaid Saleh, Alessio Salomoni, Tanja Samardzic, Stephanie Samson, Manuela Sanguinetti, Dage Sarg, Baiba Saulite, Yanin Sawanakunnanon, Nathan Schneider, Sebastian Schuster, Djame Seddah, Wolfgang Seeker, Mojgan Seraji, Mo Shen, Atsuko Shimada, Hiroyuki Shirasu, Muh Shohibussiri, Drmitry Sichinava, Natalia Silveira, Maria Simi, Radu Simonescu, Katalin Simko, Maria Sinkova, Kirli Simov, Aaron Smith, Isabela Soares-Bastos, Carolyn Spadine, Antonio Stella, Milan Straka, Jana Strnadova, Alane Suhr, Umut Sulubacak, Shingo Suzuki, Zsolt Szanto, Dima Taiji, Yuta Takahashi, Fabio Tamburini, Takaaki Tanaka, Isabelle Tellier, Guillaume Thomas, Liisi Torga, Trond Trosterud, Anna Trukhnia, Reut Tsarfaty, Francis Tyers, Sumire Uematsu, Zdenka Ureseova, Larraitz Uria, Hans Uszkeoriet, Sowmya Vajjala, Daniel van Niekerk, Gertjan van Noord, Viktor Varga, Eric Villemonte de la Clergerie, Veronika Vincze, Lars Wallin, Abigail Walsh, Jing Xian Wang, Jonathan North Washington, Maximilian Wendt, Seyi Williams, Mats Wuren, Christian Wittern, Tsegay Woldamariam, Tak-sum Wong, Alain Wyobelewska, Mary Yako, Naoki Yamazaki, Chunxiao Yan, Koichi Yasuoka, Marat M. Yavrumyan, Zhuoran Yu, Zdenek Zabokrtsky, Amir Zeldes, Daniel Zeman, Manying Zhang, and Hanzhi Zhu. 2019. Universal Dependencies 2.4. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (UFAL), Faculty of Mathematics and Physics, Charles University.
* Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
* Pimentel et al. (2020a) Tiago Pimentel, Naomi Saphra, Adina Williams, and Ryan Cotterell. 2020a. Pareto probing: Trading off accuracy for complexity. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 3138-3153, Online. Association for Computational Linguistics.
* Pimentel et al. (2020b)Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020b. Information-theoretic probing for linguistic structure. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
* Prim (1957) Robert Clay Prim. 1957. Shortest connection networks and some generalizations. _The Bell System Technical Journal_, 36(6):1389-1401.
* Reif et al. (2019) Emily Reif, Ann Yuan, Martin Wattenberg, Fernanda B Viegas, Andy Coenen, Adam Pearce, and Been Kim. 2019. Visualizing and measuring the geometry of bert. In _Advances in Neural Information Processing Systems_, volume 32.
* Saphra and Lopez (2019) Naomi Saphra and Adam Lopez. 2019. Understanding learning dynamics of language models with SVCCA. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 3257-3267, Minneapolis, Minnesota. Association for Computational Linguistics.
* Schoenberg (1938) I. J. Schoenberg. 1938. Metric spaces and positive definite functions. _Transactions of the American Mathematical Society_, 44(3):522-536.
* Scholkopf (2001) Bernhard Scholkopf. 2001. The kernel trick for distances. In _Advances in Neural Information Processing Systems_, pages 301-307.
* Scholkopf and Smola (2002) Bernhard Scholkopf and Alexander J. Smola. 2002. _Learning with Kernels_. MIT Press.
* Schultz and Joachims (2004) Matthew Schultz and Thorsten Joachims. 2004. Learning a distance metric from relative comparisons. In _Advances in Neural Information Processing Systems_, pages 41-48.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in Neural Information Processing Systems_, pages 5998-6008.
* Voita et al. (2019) Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 5797-5808, Florence, Italy. Association for Computational Linguistics.
* Voita and Titov (2020) Elena Voita and Ivan Titov. 2020. Information-theoretic probing with minimum description length. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 183-196, Online. Association for Computational Linguistics.
* Warshall (1962) Stephen Warshall. 1962. A theorem on boolean matrices. _Journal of the ACM_, 9(1):11-12.
* Zhang and Bowman (2018) Kelly Zhang and Samuel Bowman. 2018. Language modeling teaches you more than translation does: Lessons learned through auxiliary syntactic task analysis. In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_, pages 359-361, Brussels, Belgium. Association for Computational Linguistics