Automatic Word Association Norms ( AWAN ) (2020)

Abstract: Word Association Norms (WAN) are collections that present stimuli words and the set of their associated responses. The corpus is widely used in diverse areas of expertise. In order to reduce the effort to have a good quality resource that can be reproduced in many languages with minimum sources, a methodology to build Automatic Word Association Norms is proposed (AWAN). The methodology has an input of two simple elements: a) dictionary, and b) pre-processed Word Embeddings. This new kind of WAN is evaluated in two ways: i) learning word embeddings based on the node2vec algorithm and comparing them with human annotated benchmarks, and ii) performing a lexical search for a reverse dictionary. Both evaluations are done in a weighted graph with the AWAN lexical elements. The results showed that the methodology produces good quality AWANs.

## introduction word associations is a technique that helps researchers to learn how words are connected by their meanings and the relationships among them in the human mind. although vocabulary diversity and lexicon size depend on a variety of social elements among individuals, the final result is a kind of word distribution in the population. the method is used in psychology and linguistics to discover how the human mind structures knowledge @xcite . this type of resources reflect both semantic and episodic contents @xcite . in free association tests, a person is asked to say the first word that comes to mind in response to a given stimulus word. the set of lexical relations obtained with these experiments is called word association norms @xcite . the development of technological tools that will help gather these kinds of resources is starting to draw attention, mostly taking advantage of distributed technologies like the internet. small world of words 1 is a clear example of that. we believe that this way of collaborative construction could bring a variety of problems, biasing the final results. on the other hand, the classic methodologies of wan's construction are very time-consuming. just to mention some disadvantages, many people are needed to compile the data. furthermore, good control of the environment conditions of the experiments is important, as well as carefully selecting a set of metadata that must be annotated: age, education years, gender, etc. in the end, the complete wan could take years to be polished and shared with the scientific community. nevertheless, this effort is worthwhile, as wan could help diverse areas of study: psychologists, linguists, neuroscientists and others, to test new theories about how we represent and process language. in this paper, a methodology to build automatic wan is presented. we called the resource generated automatic word associaton norms (awan). the language used to prove our methodology is spanish, more specifically mexican spanish. the only wan corpus for mexican spanish is the normas de asociacion de palabras para el español de méxico @xcite (from here, this corpus will be referred to as mexican spanish wan; i.e., mswan), which was built using a classic methodology. the awan methodology presented here pretends to infer semantic relatedness between stimuli and their responses. the main reason is that word association has been of great interest as a tool to research mechanisms related to semantic memory @xcite . the main relations shown in mswan are: metonymy, meronymy, functionality, cohyponymy, qualification, hyponymy, 'made of' and synonymy @xcite . our objective is to capture the semantic relatedness but not the types of relation. ## related work in linguistics and psycholinguistics, semantic networks (sowa, 1992) are defined as graphs relating words @xcite . their use is not exclusive to learn the organization of the vocabulary, but also to draw the structure of knowledge. wan are a special kind of semantic networks, and they are available in many languages. the creation of wan is not new. the first example is @xcite , and two very well-known resources are the edinburgh associative thesaurus foot_2 (eat) @xcite and the collection of the university of south florida (usf) @xcite 4 . thanks to the internet and new technologies, wan lists have been more efficiently compiled in the last years, with the help of a large number of volunteers. some examples are: jeux de mots foot_4 , in french (lafourcade, 2007) and the multilingual dataset small world of words foot_5 . for spanish, there exists several corpora of word associations. @xcite integrate 16,000 words, including statistical analyses of the results. @xcite build norms for 58 words in children, and @xcite work with 247 lexical items that correspond to spanish @xcite . as stated above, the only resource designed and compiled for mexican spanish is the mswan. reyes-magaña et al. (2019a) introduced a method for lexical search based on that compilation that worked from clue words or definitions to the concept, i.e., from the responses to the stimuli. in some cases, authors create this type of corpus from scratch and in other cases, they extend the available wan to learn more responses to the stimuli. in recent years, bel-enguix et al. (2014) used techniques of graph analysis to calculate associations from large collections of texts. additionally, @xcite published a model of word associations that was sensitive to the demographic context. this was based on a neural network architecture with n-skip-grams and improved the performance of the generic techniques, which do not take into account the demography of the participant. @xcite showed that word association thesaurus (wat) is comparable to balanced text corpora and can replace them in case of absence of a corpus. the authors presented a methodological framework for building and extending semantic networks with wat, including a comparison of quality and information provided by wat vs. other language resources. borge-holthoefer and arenas (2009) used free association information for extracting semantic similarity relations with a random inheritance model (rim). the obtained vectors were compared with lsa-based vector representations and the was (word association space) model. their results indicate that rim can successfully extract word feature vectors from a free association network. in the work by de deyne et al. ( 2016 ), the authors introduced a method for learning word vectors from wans using a spreading activation approach in order to encode a semantic structure from wan. the authors used part of the small world of words network. the word-association-based model was compared with a word embeddings model (word2vec) using relatedness and similarity judgments from humans, obtaining an average of 13% of improvement over the word2vec model. in the recent work by @xcite , the authors used two wan in english, eat and usf to produce word embeddings that were tested against human-annotated benchmarks and some external tasks, showing that this kind of learning method produces good quality vectors without a training corpus based on billions of words. wans are proved to be good in a reverse dictionary task since they suitably represent the connections between words and the way concepts are linked in the human mind. the whole scenario of onomasiological searches changed with the universalization of the internet and language technologies that allowed to build online resources powered by the huge corpus the world wide web provides. in the last two decades, several online dictionaries have been designed that allow natural language searches. users enter their own definition in natural language and the engine looks for the words that match such definition. one of the first online dictionaries allowing this type of search was the one created for french by @xcite . @xcite designed a dictionary for japanese where the users can freely enter their definitions. it has an algorithm that calculates the similarity between concepts comparing the words. el-kahlou & oflazer (el-kahlout and oflazer, 2004) built a similar resource for turkish. they took into account some synonymy relations between words, as well as the similarity of definitions by means of a counter of similar words in the same order and in subsets of such words. for english, there exists an online onomasiological dictionary, onelook reverse dictionary, foot_6 that retrieves acceptable results. one of the main works in spanish is the one by @xcite . debo is an onomasiological dictionary that works with user queries given in natural language and a search engine, which was later improved; the database structure was also optimized @xcite . finally, the use of wans to build a reverse dictionary in spanish is presented by reyes-magaña et al. @xcite . the authors used the corpus mswan and graph-based techniques, specifically a measure of betweenness centrality, to perform searches in the knowledge graph. the results of the search model overcome the information retrieval systems it was compared to. the same methodology is successfully applied to english in @xcite . in the latter work, another graph algorithm was presented additionally to perform the search, the pagerank. nevertheless, the results show that betweenneess centrality is more suitable for the reverse dictionary task. ## methodology of automatic word association norms the aim of this work is to present a general methodology that could serve as a model to build wan for any language. the main process consists in parsing the entire dictionary, working with the entries and their definitions. we consider that all the entries become the stimuli words, and each one of the words that define the entries become the associate responses to them. the process also involves the inference of a numeric value that measures the relationship between words, allowing us to obtain the weight the classic wans have. algorithm 1 presents the overall schema of our model. the dictionary, diccionario del español de méxico (dem, 2010), is the result of a set of investigations of the vocabulary used in mexico since 1921. the investigations have been carried out since 1973 at the center for linguistic and literary studies of el colegio de méxico. the mexican dictionary of spanish is a comprehensive dictionary of spanish in algorithm 1: automatic word association norms data: dictionary, word embeddings result: awan pre-process(dictionary) for each entry do for each word in definition do @xmath0,word); @xmath1; ordering(words) its mexican variety, prepared on the basis of an extensive study of the corpus of contemporary mexican spanish and a set of data after that last date to the present. sometimes, the definitions of each one of the entries bring examples of use. all of this additional information was removed because we consider that this kind of data could contaminate the final wans. then, in order the prepare the definitions, we performed some preprocessing steps, as described. later, with the remaining words, we calculate the cosine similarity between the entry and each word corresponding to its definition. for this purpose, we use pretrainned word embeddings foot_7 for spanish language. we called this weight approximation strength (w as ). the final step is to order from high to low the weights of all the associated responses (words in a definition) to the entries (stimuli). the corpus of the awans is available in github 9 . we generate four different collections, one for each embedding used to calculate the cosine similarity. ## awan evaluation to measure the quality of the awans, two types of experiments were performed. the first one allows us to know the representativeness of the embeddings that were trained using awan and node2vec, trying to describe similarity against human-annotated benchmarks. the second experiment is about the lexical search model used in the reverse dictionary. this evaluation is done because the wans prove to be well-performing lexical searches using this kind of corpus as input @xcite . each one of the experiments will be compared with the results of these tasks using mswan. we select these outcomes as the gold standard because the wan corpus used to perform the experiments corresponds to mexican spanish, same as our awans. ## conclusions and future work we introduced a method for learning word association norms in mexican spanish from a dictionary. although we could use a general spanish dictionary like the real academia de la lengua (rae), the experiments did not yield good results, mainly because the test corpus is based on definitions made by people that use mexican spanish as their mother tongue. nevertheless, the methodology we provide in this paper can be applied to any kind of dictionary. to evaluate the awans, we used two types of test. the first one is the intrinsic test which uses the node2vec algorithm to learn word vectors on the graph built with the awan corpus. the results determine that these vectors overcome the spearman correlation presented with the mswan corpus. the second one is the extrinsic test which presents a more realistic use of this type of corpus; the lexical search model shows that even if we did not outperformed the results of the mswan, it is competitive enough to outperform classic information retrieval systems. we employ a weighting function on the graph edges considering the inverse approximation strength because all the tests use the shortest paths. we consider that the methodology proposed is a helpful tool for the construction of word association norms. the input elements to produce awans are somehow easy to get, and consist mainly of a dictionary, and the pretrained word embeddings. we also believe that the mswan collected and processed by humans, will bring more accurate results depending on the task that will be used. nevertheless, in some cases where time and availability of wan is urgent or simply impossible to collect in the classic way, the creation of awan is a reliable and fast solution. in a more advanced stage, the success of the technique can make unnecessary the effort and resources that are currently dedicated to collect wans. besides, as a parallel result, we provide the word embeddings foot_12 that we trained using the node2vec algorithm, having as the most important feature that these vectors are based on mexican spanish. we claim that this methodology can be used to produce embeddings for specific variants of a language without a huge amount of data. as future work, we plan to do some additional experiments to increase the precision in lexical search in order to apply some additional filters in the response words, like having only nouns, verbs, and/or adjectives, with all the possible combinations a pos tagging can produce. @xcite showed that the wans can be enriched using diverse types of corpora in addition to a dictionary. hence, as future work, we plan to add some encyclopedic and co-ocurrence corpora resources in order to improve the tasks performance on wans. also, it is possible to have the incorporation of multi-terms is possible to have, adding the vector representation of each word in the multi-term. this could be done by applying the same methodology.