Commonsense Knowledge in Word Associations and C oncept N et (2021)

Abstract: Humans use countless basic, shared facts about the world to efficiently navigate in their environment. This commonsense knowledge is rarely communicated explicitly, however, understanding how commonsense knowledge is represented in different paradigms is important for (a) a deeper understanding of human cognition and (b) augmenting automatic reasoning systems. This paper presents an in-depth comparison of two large-scale resources of general knowledge: ConceptNet, an engineered relational database, and SWOW, a knowledge graph derived from crowd-sourced word associations. We examine the structure, overlap and differences between the two graphs, as well as the extent of situational commonsense knowledge present in the two resources. We finally show empirically that both resources improve downstream task performance on commonsense reasoning benchmarks over text-only baselines, suggesting that large-scale word association data, which have been obtained for several languages through crowd-sourcing, can be a valuable complement to curated knowledge graphs.

## introduction humans understand and navigate everyday situations with great efficiency using a wealth of shared, basic facts about their social and physical environment -a resource often called commonsense knowledge @xcite . advances in artificial intelligence in general, and natural language processing in particular, have led to a surge of interest in its nature: what constitutes commonsense knowledge? and despite this knowledge being rarely explicitly stated in text @xcite , how can we equip machines with commonsense to enable more general inference (davis and marcus, 2015)? recently, large language models @xcite pretrained on massive text corpora achieved promising results on commonsense reasoning benchmarks with fine-tuning, however, the lack of interpretability remains a problem. augmenting them with commonsense knowledge can provide complementary knowledge @xcite and thus make models more robust and explainable @xcite . attempts to capture general human knowledge include inventories of machine-readable logical rules @xcite and large, curated databases which have been collected with the specific purpose to reflect either domain general (e.g., conceptnet; @xcite ) or domain-specific (e.g., atomic; @xcite ) commonsense knowledge. word association norms are a third resource of explicit, basic human knowledge: in a typical study, human participants are presented with a cue word (e.g., 'camping') and asked to produce one or more words that spontaneously come to mind (e.g., 'hike', @xcite . the resulting data sets of word associations have been used to explore the mental representations and connections underlying human language and reasoning @xcite , and have been shown to contain information beyond that present in text corpora @xcite . word association studies have been scaled to thousands of cue words, tens of thousands of participants, and several languages, and hence provide a way of collecting diverse and unbiased representations. however, the extent to which they capture commonsense knowledge, and their utility for downstream applications have not yet been examined. prior work has probed the extent to which word associations contain lexical knowledge @xcite , has systematically compared the relational coverage and overlap of curated knowledge bases @xcite (not including word associations data), and has investigated how much relational commonsense knowledge is present in language models (a. @xcite @xcite . we contribute to this line of research with an in-depth comparison of a dedicated commonsense knowledge base, and resources derived from spontaneous human-generated word associations. we systematically compare the most comprehensive, domain general, curated commonsense knowledge base (conceptnet) with the largest data set of english word associations (the "small world of words"; swow; @xcite ). we compare the two resources both in their structure and content, and apply them in downstream reasoning tasks. our contribution is important for three reasons. first, comparing explicitly engineered with spontaneously produced knowledge graphs can advance our fundamental understanding of the similarities and differences between the paradigms, and suggest ways to combine them. second, recent progress in automatic commonsense reasoning largely focused on english and relies heavily on the availability of very large language models. these are, however, infeasible to train for all but a few high-resource languages. finally, recent work has shown that the competitive performance of large language models on commonsense reasoning tasks is at least partially due to spurious correlations in language rather than genuine reasoning abilities; and that they perform close to random once evaluation controls for such confounds @xcite . explicit representations of commonsense knowledge bases are thus have the potential to promote robust and inclusive natural language reasoning models. in summary, our contributions are: ## experiment 1: intrinsic comparisons a knowledge graph consists of a set of nodes n and edges e, comprising triples e, r, e to denote a directed edge from head node e to tail node e labelled with relation @xmath0. we denote e(e) as the incoming and outgoing edge set for node e, e(r) as the set of edges with relation r, and | • | as the size of a set. here, we consider the specific knowledge graphs conceptnet and swow, and begin by comparing the intrinsic properties: their typology and content encoded. ## experiment 2: coverage of commonsense knowledge this section probes conceptnet and swow specifically for (situational) commonsense knowledge. daily activities such as doing the laundry or visiting the doctor involve a wealth of general knowledge touching on causal, temporal, physical, or social knowledge which is rarely explicitly stated @xcite @xcite . we leverage the mcscript2.0 data set @xcite , a large collection of explicit descriptions of everyday scenarios, and investigate whether conceptnet and swow encode the commonsense knowledge underlying these situations. in ## conclusions we presented an in-depth analysis of the general and commonsense knowledge encoded in human word association norms (swow), versus a traditional curated commonsense knowledge graph (conceptnet). we showed that the two knowledge resources differ systematically in their structure and content. we also showed that swow brings comparable gains to conceptnet when applied to three commonsense question answering benchmarks, which is important as word associations are simpler than structured relations, and accordingly can be created more cheaply via crowdsourcing and without the need for experts. finally, we showed that swow encodes situational commonsense knowledge as encoded in the humancreated mcscript2.0 narratives more directly than conceptnet; and that both kgs impose meaningful additional relations between concepts that were left implicit in the descriptions. there are several directions for future work, most notably extending our framework for characterizing commonsense knowledge in the two kgs, exploring swow relation types, developing means of consolidating the two knowledge graphs, and exploring downstream utility of kgs in less well-resourced languages than english, where the portability of the word association annotation methodology confers a substantial advantage.