Representations of Meaning in Neural Networks for NLP : a Thesis Proposal (2021)

Abstract: Neural networks are the state-of-the-art method of machine learning for many problems in NLP. Their success in machine translation and other NLP tasks is phenomenal, but their interpretability is challenging. We want to find out how neural networks represent meaning. In order to do this, we propose to examine the distribution of meaning in the vector space representation of words in neural networks trained for NLP tasks. Furthermore, we propose to consider various theories of meaning in the philosophy of language and to find a methodology that would enable us to connect these areas.

## nlp, language, and meaning language has been one of the central topics of artificial intelligence (ai) research ever since @xcite considered the question "can machines think?" and proposed to replace it with the "imitation game", based purely on textual communication. even though language is still one of the hardest problems in ai, there has been a tremendous development in recent years in nlp. machine translation systems achieve super-human performance (at least in a competition setting) @xcite . voice assistants are getting better and better. some text generation models are so powerful that their authors consider them to pose a danger to society @xcite . artificial neural networks are behind a lot of these achievements. the models that are used in nlp can have billions of parameters. the same architecture is often used for various tasks. consequently, neural networks are often regarded as black boxes, and interpretation of the trained models presents a major scientific challenge @xcite . certain specific questions, such as whether a layer of a particular model contains information about part of speech (pos) can be answered with various methods. other, more general questions, are proving more difficult. how do neural machine translation (nmt) systems achieve the level of translation quality comparable to humans? are there any fundamental limitations in language understanding for artificial neural networks? do neural networks represent meaning and if they do, then how? it is the last question we are interested in. the nature of meaning is itself a subject of debate in the philosophy of language. this presents a challenging methodological problem: on the one hand, we need a definition of meaning for the question to make sense; on the other hand, we do not want to restrict our research to a predefined concept of meaning, because then we are in danger of assuming the conclusion and presenting a circular argument. the solution would be to refine the sought-after concept of meaning gradually, based on careful justification supported by empirical observations. the focus of this work is on lexical semantics in the embedding layer the neural network. we believe that this is a good place to start, as it is the interface between the input text and the network. furthermore, there are interesting models for obtaining words embeddings without any hidden layers. ## nlp and philosophy of language there is no agreed-upon general definition of 'meaning' (or 'sense', 'semantics', . . . ; see e. g. stokhof (2013), bender and koller (2020) ). to be able to talk about representations of meaning, we will have to review different conceptualizations of meaning and find one that is useful for describing the phenomena we encounter when we examine how neural networks work in nlp. we will contrast meaning representations in neural language models with representations in other applications, with emphasis on nmt. there is very little related work that connects nlp with the philosophy of language. honkela (2007) links neural language models, selforganizing maps and quine's semantic holism. the works of @xcite are discussed in section 2.5. ## properties of word embeddings in this section, we present methods for analysis of words embeddings and provide examples of results obtained with these methods. we will concentrate on embeddings from unsupervised learning algorithms, language models and nmt. unsupervised learning algorithms for obtaining word representations, such as word2vec @xcite , glove @xcite or fasttext @xcite , have the advantage of being simple, both conceptually and regarding computational costs. language modelling is the most general language task. pretrained models, such as masked lms from the bert family @xcite , can be finetuned for many nlp tasks. large generative lms can even be used for various tasks with little or no fine-tuning @xcite . nmt is a mature nlp application and the task itself is closely connected to the concept of meaning. by comparing embeddings from different models, we hope to distinguish between properties of the specific model and general properties of continuous word representations. we will also investigate contextual representations in current transformer models (e.g. @xcite . it is possible to reduce contextual embedding to static embeddings @xcite and apply the methods for analyzing static embeddings. in this section we present methods for analyzing word embeddings and their results. related work on examining vector representations in nlp was surveyed by @xcite . further information can also be found in the overview of methods for analyzing deep learning models for nlp by @xcite . for more on interpretation in general and unsupervised methods in examining word embeddings, see @xcite . probing is the most common approach for examining linguistic properties in neural network components @xcite . it is the method of using a supervised classifier to predict these properties from activations of the neural network. the methodology may present problems with train/test overlap @xcite . probing is most useful when there are high quality annotated data for the property that is being probed. even though we plan to occasionally use probing in such cases, we will generally emphasize unsupervised methods of interpretation, because we do not want to bias the results by restricting the possible outcome by probing for specific features. component analysis is an unsupervised method for factoring the vector space of embeddings into meaningful components. pca is a generally well known example. it is often used for dimensionality reduction. the resulting components are ordered by their importance and they maximize variance of the data given all the previous components. ica @xcite @xcite is an algorithm originally developed for finding separate sources in a mixed signal, such as a recording of multiple people in the same room speaking at the same time. it was used, for example, to extract features from distribution representations of the words @xcite . the ica algorithm consists of: optional dimension reduction, usually with pca; centering the data and whitening them (setting variance of each component to 1); iteratively finding directions in the data that are the most non-gaussian. the last step is based on the assumption of the central limit theorem: the mixed signal is a sum of independent variables, therefore it should be closer to the normal distribution, than the variables themselves. clustering is another unsupervised method for examining embeddings. the t-sne clustering algorithm is often used for visualizing embeddings (e.g. @xcite . word embeddings are clustered according to meaning in t-sne @xcite . we show elsewhere @xcite ) that clusters of embeddings of derivational relations mostly match manually annotated semantic categories of these relations (e.g. the relation 'bakebaker' belongs to the category 'actor', and a correct clustering puts it into the same cluster as 'governgovernor'). unsupervised mapping there are unsupervised methods for finding a mapping between two embedding spaces that can be used for simple word-forword translation, as a starting point for creating an unsupervised nmt system @xcite . mapping of embedding spaces from different corpora of the same language can lead to interesting insights, as demonstrated by @xcite , who show polarization in us political comments by highlighting different use of specific words or phrases by supporters of different political parties. we have found that a neural translation model divides words into pos classes @xcite . it also distinguishes between proper names and gen-eral nouns. the structure of representation varies between the encoder and the decoder of the nmt system. the structure of the representation of the same data in the word2vec model is different, for example, in that it distinguishes infinitive forms of verbs or modal verbs. a completely different structure is found in the space of representations of words in the neural model for sentiment analysis. all of these facts can be shown without annotated data and thus without deciding beforehand what we will look for in the space of representations. for this reason, we find these results more convincing than if they had been obtained through probing. ## conclusion and future work interpretability is an important challenge for neural networks in nlp. there is a limited amount of findings about linguistic phenomena that we are able to predict from embeddings. much less is known about the semantic properties of the embedding space. the proposed approach to finding a description of the process of representing meaning in neural networks for nlp both from the technological and philosophical perspective would contribute to our understanding of the technology and of the concept of meaning. future work could also address the relation between neural networks for natural language inference and the philosophy of inferentialism @xcite . this proposal leaves out important methodological questions: we are using machine learning methods to run experiments on the results of other machine learning methods. it may be a challenging task to interpret experiments correctly and attribute the discovered properties to the original model or to the model we are using to examine it. the question of how to incorporate results of machine learning into the scientific workflow is starting to come up in other sciences as well, e.g. biology @xcite . this question is perhaps too broad and general to be solved as a part of this thesis. however, we hope to at least formulate in detail the challenges that we are facing when performing this kind of research, as we encounter them while completing the work proposed in the previous sections.