Learning Prototypical Functions for Physical Artifacts (2021)

Abstract: Humans create things for a reason. Ancient people created spears for hunting, knives for cutting meat, pots for preparing food, etc. The prototypical function of a physical artifact is a kind of commonsense knowledge that we rely on to understand natural language. For example, if someone says “She borrowed the book” then you would assume that she intends to read the book, or if someone asks “Can I use your knife?” then you would assume that they need to cut something. In this paper, we introduce a new NLP task of learning the prototypical uses for human-made physical objects. We use frames from FrameNet to represent a set of common functions for objects, and describe a manually annotated data set of physical objects labeled with their prototypical function. We also present experimental results for this task, including BERT-based models that use predictions from masked patterns as well as artifact sense definitions from WordNet and frame definitions from FrameNet.

## introduction humans are a creative species. new objects are invented by people every day, and most are created for a reason. knives were created for cutting, bicycles were created for transportation, and telephones were created for communication. some objects can perform multiple functions (e.g., smart phones) and humans are also creative at finding secondary uses for objects (e.g., heavy objects are often used as makeshift paperweights). but when we mention physical objects in conversation or in writing, people generally infer that the object will be used in the most prototypical way, unless they are told otherwise. the prototypical function of human-made physical artifacts is a kind of commonsense knowledge that often plays a role in natural language understanding. consider the following examples of inferences that arise from physical artifacts. example 1 a) he killed the mayor with a gun. b) he killed the mayor with a knife. c) he killed the mayor with a bomb. example 1 describes a killing with three different types of instruments. most readers would assume that a) describes a shooting, b) describes a stabbing, and c) describes an explosion. but exactly how each instrument was used is implicit. we make different inferences about how they were used based on our knowledge of the objects. example 2 illustrates how we infer different actions based on the object when the main action is elided (i.e., "finished" means that some action has ended but the action itself is implicit). most people would assume that the cigarette was smoked, the puzzle was solved, and the movie was watched. example 3 a) she put the cake in the box. b) she put the cake in the oven. c) she put the cake in the refrigerator. example 3 illustrates second-order inferences that can follow from a sentence. the verb "put" means that the cake was placed somewhere, but the object of "in" leads to different inferences about intention. putting a cake in an oven implies that it will be baked, but putting a cake in a refrigerator implies that it will be cooled. example 4 a) he ordered a taxi. b) he ordered a pizza. c) he ordered a t-shirt. example 4 reveals inferences about motivations and future plans. if someone orders a taxi then we infer that they need transportation, if they order a pizza then we expect they will eat it, and if they order a t-shirt then we assume it will be worn. we believe that it is essential for nlp systems to "read between the lines" and make the same types of inferences that people do when reading these sentences. the goal of our research is to explore methods for learning the prototypical functions of human-made physical artifacts so that future nlp systems can benefit from this knowledge. first, we define a new nlp task to associate physical objects with frames from framenet as a canonical representation for their prototypical function. we introduce a gold standard data set of 938 physical artifacts that have each been labeled with a frame that represents its prototypical function based on human judgements. second, we evaluate baseline models to assess how well existing resources and simple methods perform on this task. third, we present transformer-based models for this task that exploit both masked sentence patterns and the definitions of physical artifacts and frames. experiments show that our best model yields substantially better results than the baseline methods. ## related work researchers have known for a long time that commonsense knowledge is essential for natural language understanding @xcite . some of this work specifically argued that commonsense knowledge about physical objects, including functional knowledge, plays an important role in narrative text understanding @xcite . these observations have led to considerable work toward constructing commonsense knowledge repositories. the cyc project @xcite built a large ontology of commonsense concepts and facts over many years. more recently, con-ceptnet @xcite captures commonsense knowledge in the form of predefined relations expressed in natural language words and phrases. it was built from open mind common sense, a crowd-sourced knowledge project @xcite , and later enhanced with other sources such as wiktionary and wordnet @xcite . within the nlp community, a variety of recent projects have focused on trying to acquire different types of commonsense knowledge, such as forbes and choi (2017); @xcite ; @xcite . @xcite presented a crowd-sourced commonsense reasoning data set called atomic that focuses on inferential knowledge related to events, which is organized as if-then relations. @xcite later proposed comet, a transformer-based framework for automatic construction of commonsense knowledge bases that was trained from atomic and conceptnet. both conceptnet and comet include a usedfor relation that is relevant to our task, and we evaluate their performance on our data set in section 6. of relevance to our work, jiang and riloff (2018) learned the prototypical "functions" of locations by identifying activities that represent a prototypical reason why people go to a location. for example, people go to restaurants to eat, airports to catch a flight, and churches to pray. they referred to the associated activity as a prototypical goal activity and presented a semi-supervised method to iteratively learn the goal activities. our work is also related to frame semantics, which studies how we associate words and phrases with conceptual structures called frames @xcite , which characterize an abstract scene or situation. the berkeley framenet project @xcite provides an online lexical database for frame semantics and a corpus of annotated documents. there has been substantial work on frame semantic parsing (e.g., @xcite , which is the task of automatically extracting frame structures from sentences. several efforts have enhanced framenet by mapping it to other lexicons, such as wordnet, propbank and verbnet @xcite @xcite . @xcite increased the lexical coverage of framenet through automatic paraphrasing and manual verification. @xcite introduced situation recognition, which is the problem of producing a concise summary of the situation that an image depicts. similar to our work, they selected a subset of frames from framenet to represent possible situations depicted in an image. our work uses a subset of frames from framenet to represent the prototypical functions for human-made physical artifacts. ## motivation our work was motivated by observing sentences that mention physical objects and realizing that we often infer a richer meaning for these sentences than what they explicitly state. we came to appreciate that the prototypical function of an object was the basis for many of our inferences, but we also recognized that not all objects have a prototypical function. in particular, naturally occurring objects rarely have a prototypical function (e.g., rock, snake). in contrast, human-made physical objects usually do have a prototypical function because they were created for a purpose. consequently, we limited the scope of our work to human-made artifacts. of course, some objects are commonly used for multiple purposes, but in most cases there seems to be one use that is dominant, so for the sake of tractability we decided to assign a single (most) prototypical function to each artifact for this research. we had initially planned to include food items, but many foods are also naturally occurring plants or animals (e.g., watermelon, shrimp) so we omitted them. it may be worth re-examining these limitations in future work. another key decision that we had to make was how to represent the prototypical functions. some recent work on commonsense knowledge acquisition has opted to generate words and phrases as expressions of a relation, such as conceptnet @xcite and atomic @xcite . as an example, conceptnet includes a relation called usedfor that lists the following phrases as uses for a knife: stabbing, butter, cutting food, carving wood, slicing, boning. we chose to adopt a different approach. first, we wanted a canonical representation for each type of function that represents a general concept, rather than a list of phrases. this approach naturally captures clusters of objects (i.e., those assigned to the same frame) and avoids evaluation issues arising from differing phrases that may be learned for similar objects (e.g., cut vs. carve vs. slice). second, we did not want to reinvent the wheel and develop a new taxonomy of action types ourselves. for these reasons, we chose to use the semantic frames in framenet as a canonical representation for our prototypical functions. although framenet is not perfect nor complete, it contains many of the actions that we needed. overall, it serves as an appropriate platform for our work. this approach also opens up new avenues for research down the road. although it is beyond the scope of this paper, we can imagine that sentences could trigger frames based on inferences originating from physical objects during semantic parsing. for example, "she used a pencil" should arguably be represented as a writing (text creation) event. however we leave that challenge for future work. this paper focuses on the specific task of learning the prototypical functions for human-made physical artifacts using a subset of framenet frames as the set of function types. ## methods we explored several approaches for learning the prototypical functions of human-made physical artifacts. to assess the difficulty of this task, we first present baseline models that 1) exploit information extracted from existing knowledge bases and 2) use co-occurrence information extracted from a text corpus. next, we explore methods that use large neural language models. we describe a method that uses masked pattern predictions, and then present models that also incorporate artifact sense definitions and frame definitions. ## evaluation 6.1 experiment settings our gold standard data set contains 938 artifacts that are each paired with one frame that represents its most prototypical use. we set aside 20% (188) of the data as a development set and used 80% (750) as the test set. we evaluated all of the learning models by performing 5-fold cross validation on the test set. we use the pre-trained uncased bert-base model with the same settings as @xcite and fine-tuned bert on the training data. we set the max sequence length as 200, batch size as 1, learning rate started at 2e-5, and train for 10 epochs. all reported results are averaged over 3 runs. we report overall accuracy as well as precision, recall and f1 scores macro-averaged over the 43 class labels (42 frames + none). ## conclusion we introduced the new task of learning prototypical functions for human-made physical artifacts, and 7 in fact, snowplow can also refer to a skiing action, although wordnet does not contain that word sense. used a subset of frames from framenet to represent the set of common functions. we also presented a manually annotated data set of 938 physical artifacts for this task. our experiments showed that a transformer-based model using both artifact and frame definitions as well as masked pattern predictions outperforms several baseline methods. in future work, we hope to show the value of functional knowledge about objects for sentence-level understanding tasks as well as narrative document understanding.