Tracking the progress of Language Models by extracting their underlying Knowledge Graphs

1 INTRODUCTION . Natural language processing ( NLP ) encompasses a wide variety of applications such as summarization ( Kovaleva et al. , 2019 ) , information retrieval ( Zhan et al. , 2020 ) , and machine translation ( Tang et al. , 2018 ) , among others . Currently , the use of pre-trained language models has become the de facto starting point to tackle most of these applications . The usual pipeline consists of finetuning a pre-trained language model by using a discriminative learning objective to adapt the model to the requirements of each specific task . As key ingredients , these models are pre-trained using massive amounts of unlabeled data that can include millions of documents , and may include billions of parameters . Massive data and parameters are supplemented with a suitable learning architecture , resulting in a highly powerful but also complex model , whose internal operation is hard to analyze . The success of pre-trained language models has driven the interest to understand how they manage to solve NLP tasks . As an example , in the case of BERT ( Devlin et al. , 2019 ) , one of the most popular pre-trained models based on a Transformer architecture ( Vaswani et al. , 2017 ) , several studies have attempted to access the knowledge encoded in its layers and attention heads ( Tenney et al. , 2019b ; Devlin et al. , 2019 ; Hewitt & Manning , 2019 ) . In particular , ( Jawahar et al. , 2019 ) shows that BERT can solve tasks at a syntactic level by using Transformer blocks to encode a soft hierarchy of features at different levels of abstraction . Similarly , ( Hewitt & Manning , 2019 ) shows that BERT is capable of encoding structural information from text . In particular , using a structural probe , they show that syntax trees are embedded in a linear transformation of the encodings provided by BERT . In general , previous efforts have provided strong evidence indicating that current pre-trained language models encode complex syntactic rules , however , relevant evidence about their abilities to capture semantic information remains still elusive . As an example , a recent study ( Si et al. , 2019 ) attempts to locate the encoding of semantic information as part of the top layers of Transformer architectures , however , results provide contradictory evidence . Similarly , ( Kovaleva et al. , 2019 ) focuses on studying knowledge encoded by self-attention weights , however , results provide evidence for over-parameterization but not about language understanding capabilities . In this work , we study to which extent pre-trained language models encode semantic information . As a key source of semantic knowledge , we focus on studying how precisely pre-trained language models encode the concept relations embedded in the conceptual taxonomy of WordNet1 ( Miller , 1995 ) . The ability to understand , organize , and correctly use concepts is one of the most remarkable capabilities of human intelligence ( Lake et al. , 2017 ) . Therefore , a quantification of the ability that a pre-trained language model can exhibit to encode the conceptual organization behind WordNet is highly valuable . In particular , it can provide useful insights about the inner mechanisms that these models use to encode semantic information . Furthermore , an analysis of concepts and associations that result difficult to these models can provide relevant insights about how to improve them . In contrast to most previous works , we do not focus on a particular model , but target a large list of the most popular pre-trained language and text-embedding models . In this sense , one of our goals is to provide a comparative analysis of the capacities of different types of approaches . Following Hewitt & Manning ( 2019 ) , we study semantic performance by defining a probing classifier based on concept relatedness according to WordNet . Using this tool , we analyze the different models , enlightening how and where semantic knowledge is encoded . Furthermore , we explore how these models encode suitable information to recreate the structure of WordNet . Among our main results , we show that the different pre-training strategies and architectures lead to different model biases . In particular , we show that contextualized word embeddings , such as BERT , encode high-level concepts and hierarchical relationships among them , creating a taxonomy . This finding corroborates previous work results ( Reif et al. , 2019 ) that claim that BERT vectors are stored in sub-spaces that have correspondence with semantic knowledge . Our study also shows evidence about the limitations of current pre-trained language models , demonstrating that they all have difficulties encoding specific concepts . As an example , all the models struggle with concepts related to “ taxonomical groups ” , performing worse than chance in some cases . Our results also reveal that models have very distinctive patterns in terms of where they encode most of the semantic information . These patterns are dependant on architecture and not on model sizes . 2 RELATED WORK . The success of deep learning architectures in various NLP tasks has fueled a growing interest from the community to improve understanding of what these models encode . Several works have studied these models ’ impact on downstream tasks at the syntactic or semantic level . Some studies ( Tenney et al. , 2019b ) claim that success in a specific task helps understand what type of information the model encodes . Other studies have improved the understanding of what and where these models encode information , by analyzing correlations between input-targets and specific architecture blocks , such as layers ( Jawahar et al. , 2019 ) , encoded hidden states ( Tang et al. , 2018 ; Saphra & Lopez , 2019 ) , and attention heads ( Michel et al. , 2019 ) . Evidence of syntactic information : Using probing classifiers , Clark et al . ( 2019 ) claims that some specific BERT ’ s attention heads show correspondence with syntactic tasks . Goldberg ( 2019 ) illustrates the capabilities that BERT has to solve syntactic tasks , such as subject-verb agreement . BERT ’ s success in these tasks fuels the belief that BERT can code the syntax of a language . Hewitt & Manning ( 2019 ) proposes a structural probe that evaluates whether syntax trees are encoded in a linear transformation of BERT embeddings . The study shows that such transformation exists in BERT , providing evidence that syntax trees are implicitly embedded in BERT ’ s vector geometry . Reif et al . ( 2019 ) has found evidence of syntactic representation in BERT ’ s attention matrices , with specific directions in space representing particular dependency relations . Evidence of semantic information : Reif et al . ( 2019 ) suggests that BERT ’ s internal geometry may be broken into multiple linear subspaces , with separate spaces for different syntactic and semantic information . Despite this , previous work has not yet reached consensus about this topic . While some studies show satisfactory results in tasks such as entity types ( Tenney et al. , 2019a ) , semantic roles ( Rogers et al. , 2020 ) , and sentence completion ( Ettinger , 2020 ) , other studies show less favorable results in coreference ( Tenney et al. , 2019b ) and Multiple-Choice Reading Comprehension ( Si et al. , 2019 ) , claiming that BERT ’ s performance may not reflect the model ’ s true ability of language understanding and reasoning . Some works have studied which blocks of BERT are used to solve 1WordNet is a human-generated graph , where each one of its 117000 nodes ( also called synsets ) represent a concept . In this work we use the hyponymy relations , which represent if a concept is a subclass of another . tasks at the semantic level . Tenney et al . ( 2019b ) proposes a set of edge probing tasks to test the encoded sentential structure of contextualized word embeddings . The study shows evidence that the improvements that BERT and GPT-2 offer over non contextualized embeddings as GloVe is only significant in syntactic-level tasks . Regarding static word embeddings , Yaghoobzadeh et al . ( 2019 ) shows that senses are well represented in single-vector embeddings if they are frequent , and that this does not have a negative impact on NLP tasks whose performance depends on frequent senses . Layer-wise or head-wise information : Tenney et al . ( 2019a ) shows that the first layers of BERT focus on encoding short dependency relationships at the syntactic level ( e.g. , subject-verb agreement ) while top layers focus on encoding long-range dependencies ( e.g. , subject-object dependencies ) . Peters et al . ( 2018a ) supports similar declarations for Convolutional , LSTM , and self-attention architectures . While these studies also support that the top layers appear to encode semantic information , the evidence to support this claim is not conclusive or contradictory with other works . As an example , Jawahar et al . ( 2019 ) could only identify one SentEval semantic task that topped at the last layer . In terms of information flow , Voita et al . ( 2019a ) reports that information about the past in left-to-right language models gets vanished as the information flows from bottom to top BERT ’ s layers . Hao et al . ( 2019 ) shows that the lower layers of BERT change less during finetuning , suggesting that layers close to inputs learn more transferable language representations . In terms of architecture design , Press et al . ( 2020 ) provides evidence that increasing self-attention at the bottom and increasing feed-forward sub-layers at the top improves results in language modeling tasks using BERT . Other studies have focused on understanding how self-attention heads contribute to solving specific tasks ( Vig , 2019 ) . Kovaleva et al . ( 2019 ) shows a set of attention patterns that is repeated across different heads when trying to solve GLUE tasks ( Wang et al. , 2018 ) . Furthermore , Michel et al . ( 2019 ) and Voita et al . ( 2019b ) show that several heads can be removed from trained Transformer models without degradation in downstream tasks . In summary , on the one hand , related work shows results that provide strong evidence concluding that BERT and other Transformer-based models can encode information at the syntactic level . Furthermore , BERT uses this information to solve various benchmark NLP tasks . Working with ambiguous words has allowed BERT to perform well on Machine Translation tasks ( MT ) , and outperforming other architectures in word sense disambiguation ( WSD ) . On the other hand , while some studies claim that top layers are helpful to solve semantic tasks , the results are not conclusive . Efforts to locate which blocks of the Transformer architecture operate at the semantic level have been unsuccessful . In this context , our work helps to fill the gap between the deeper understanding that we have now about how word embeddings and language models encode and work with syntax , and the still shallow comprehension of their abilities to encode and work with semantics . 3 STUDY METHODOLOGY . Probing methods consist of using the representation of a frozen pre-trained model to train a classifier to achieve a particular task . If the probing classifier succeeds in this setting but fails using an alternative model , it means that the source model encodes the knowledge needed to solve the task . Furthermore , the performance of the classifier can be used to measure how well the model captures this knowledge ( Conneau et al. , 2018 ) . Following Hewitt & Manning ( 2019 ) , we use a similar probing method at the semantic level and apply it to the nine models presented in Section 3.1 . Probing techniques have shown useful for NLP tasks ( Saphra & Lopez , 2019 ) , however , they have also been questioned ( Ravichander et al. , 2020 ) , as probe success does not guarantee that the model relies on that information to solve a target task . Consequently , our study limits to shed light on whether the models under evaluation encode relevant knowledge to solve the task of predicting concept relatedness in a semantic ontology such as Wordnet . To study how precisely the models encode semantic information , we measure correctness in predicted relations among concepts at two levels : ( a ) pair-wise-level by studying performance across sampled pairs of related or unrelated concepts , and ( b ) graph-level by using pair-wise predictions to reconstruct the actual graph . We describe these two approaches in Sections 3.2 and 3.3 , respectively . They essentially study the same phenomenon as both share the exact same model predictions . They only differ in the way they use these predictions .

Summary:  The authors conduct a study investigating how different language models incorporate semantic information in their respective learned representations. Investigating language models on their performance in concept-level tasks is motivated by the importance of the ability to organize and understand concepts in human intelligence. Another motivation is that other studies on the semantics in language models are not conclusive according to the authors, especially in determining where the semantic knowledge lies within the language models.