Probing the representations of named entities in Transformer-based Language Models (2022)

Abstract: In this work we analyze the named entity representations learned by Transformer-based language models. We investigate the role entities play in two tasks: a language modeling task, and a sequence classification task. For this purpose we collect a novel news topic classification dataset with 12 topics called RefNews-12. We perform two complementary methods of analysis. First, we use diagnostic models allowing us to quantify to what degree entity information is present in the hidden representations. Second, we perform entity mention substitution to measure how substitute-entities with different properties impact model performance. By controlling for model uncertainty we are able to show that entities are identified, and depending on the task, play a measurable role in the model’s predictions. Additionally, we show that the entities’ types alone are not enough to account for this. Finally, we find that the the frequency with which entities occur are important for the masked language modeling task, and that the entities’ distributions over topics are important for topic classification.

## introduction the probability a language model should assign to a sequence depends not only on what is being said, but also on the context, i.e. who is saying it, where, when, and why? some types of context such as (cultural) background knowledge may already be represented to some degree within pre-trained language models. however, recent work shows that when it comes to world knowledge language models and knowledge bases are complementary, and that various forms of integration are beneficial @xcite . being able to condition on the context explicitly would be particularly useful when we consider, for example, more specific cultural knowledge or interpersonal knowledge, which are unlikely to be contained in pre-training corpora. in order to integrate language models and knowledge bases effectively it is important to know pre-cisely how these two sources of information complement each other. entities are specifically interesting as they occur within characteristic contexts learnable by language models and at the same time provide access to knowledge graphs. in this work we investigate what information is present in the entity representations of transformerbased @xcite language models. we also study whether some entities' representations contain more information than others and why. finally, we show how much these aspects change after a pre-trained language model is fine-tuned. for our investigation we choose news articles as our primary source of data. news articles often describe events that involve numerous entities. which is the primary reason they have been used in the past for entity-related tasks such as nerc @xcite , nel @xcite , and coreference resolution @xcite . we fine-tune and evaluate our models for news topic classification and masked language modeling. for this purpose, we collect a new dataset of english news articles by following links cited on wikipedia pages covering many newsworthy incidents @xcite . our data collection method allows for this dataset to easily be expanded with additional topics and languages in the future. furthermore, the set of entities linked to from wikipedia pages covering such incidents can also provide us with a 'shortlist' of entities likely to be referenced in the news articles themselves. these entity links will allow integration with wikidata, an avenue we wish to explore in future work. our analysis includes two complementary methods: diagnostic models @xcite @xcite and a novel method of analysis we call entity mention substitution. diagnostic models are trained on a relevant task (in our case entity recognition) with hidden representations of another model as input. the di-agnostic model is kept as simple as possible such that its performance can be attributed to the information being available in the hidden representation. entity mention substitution measures the impact of various kinds of substitutions on the prediction of the model. if the impact is high we interpret this as evidence that the entity was important for the prediction. by manipulating which entities we choose as substitutes and by comparing the results to those of the diagnostic models, we answer the following research questions: rq1: when entities are mentioned in the input text, are they identified and used by transformer-based language models? rq2: does a transformer-based language model either partially, or fully represent entities by their type? rq3: do the answers to rq1 & rq2 depend on: (a) the frequency with which an entitymention occurs in the data; and (b) the distribution of that entity across the news topics? we make two important contributions. first, we collect a novel news classification dataset we call refnews-12, which consists of 106,167 articles which cover 9,878 incidents grouped by 12 topics. second, we analyze what information is present in entity representations in two ways. one concerns training and evaluating diagnostic models on entity recognition using only the model's hidden representations for entities as input. the other involves corrupting entity mentions in the data in various ways, showing how the model relies on the entities to make predictions. we find that entities are identified in pre-trained models both before and after fine-tuning. entities are also used to perform the task for which the model is trained or fine-tuned, even if they cannot be identified clearly by their representation. we also find that entities are represented by more than their type. finally, our experiments suggest that the importance of the frequency with which entities occur and entities' distribution over topics is taskspecific. ## related work the use of news articles to study the interaction of entities and topics is not new. @xcite mention that "news articles are ideal because they have the primary purpose of conveying information about who, what, when and where." we use them for the same reason, but focus on studying the entity representations in recent language models. previous investigations into the representation of entities in language models have come from various directions. @xcite investigates entity knowledge in pre-trained bert through entity linking. they frame entity linking as a token classification problem over the entire vocabulary of 700k entities, thereby solving mention detection, candidate generation, and entity disambiguation simultaneously. when trained with bert's weights frozen this method still obtains decent f1 scores (67.8 versus sota of 85.8 by @xcite ) on the aida benchmark @xcite . this indicates that bert already assigns representations that are sufficiently distinct for entity linking to a lot of the entity tokens. we expect that this distinctness does not necessarily imply that entities are really treated by the model as individuals. thus, we directly investigate the degree to which these entity representations are interchangeable. @xcite study whether pre-trained language models capture information helpful with the resolution of pronominal anaphora. they hypothesize that the model will learn helpful grammatical properties, but not semantic-referential information. to test this hypothesis they train diagnostic models and analyze how their performance varies as the variables of interest change. their evidence suggests that language models do in fact learn some referential aspects, but that they are still much better at grammar. we also investigate the presence of semantic properties in representations of entities, but do so with different methods and include models that have been fine-tuned. @xcite use entity embeddings obtained from various language models to classify entities as one of 14 types. interestingly, bert embeddings obtain the lowest accuracy of the models tested. where @xcite embeds only the name of the entity, our work studies representations of entities that appear in context. a number of other works do not investigate the representations of entities specifically, but test to what extent language models are able to reproduce relational world knowledge, which involves numerous facts about entities as well @xcite . for a recent survey of this type of research see @xcite . ## methodology the following section describes the method by which we collect our data (3.1), and the two methods we used to perform our analysis (3.2, 3.3). ## experiments this section details our experimental setup. ## conclusion we have presented refnews-12, a novel news topic classification dataset. this dataset was collected by scraping wikipedia articles for links. this collection method allows it to be expanded with additional topics and languages in the future. because the wikipedia pages also link to the pages of entities relevant to the incident, the dataset can be bridged easily to knowledge from wikidata. we have investigated entity representations in transformer-based language models. we find that after having been fine-tuned for news topic classification these models do identify and use the entities to accomplish the task at hand. although, whether they are used also depends on the task for which the model is trained. our results also show that on average these language models do not represent entities only by their type. entities are used by the model as distinctly different even within the same type. finally, we have shown that the frequency with which an entity occurs in the data does not play a significant role in models performing topic classification. nor does the topic distribution play a significant role in masked language modeling. we obtained our results by altering the inputs of a model and measuring the change in performance. crucially, to allow us to draw conclusions from these results we also control for model uncertainty. we believe this general methodology can be used to probe for many kinds of properties. as such it provides an additional probing technique which can be used to strengthen existing experimental evidence in the future.