SlowGuess/ACL_2020_OCR huggingface dataset:
We focus on comprehension (similar processes apply in production), and assume some mechanism for proposing interpretations consisting of a set of conceptual structures and associated compatibility constraints. Compatibility constraints are analogous to various kinds of binding constraints proposed in the literature (variable binding, role-filler bindings, unification bindings, and the like): they are indicators that two structures should be conceptualized as a single unit. But compatibility is softer and more permissive than identity or type-compatibility, in that it can also be satisfied with the help of construal operations. Some operations effect relatively subtle shifts in meaning; others have more dramatic effects, including changes to truth-conditional aspects of meaning.
Below we illustrate how some example linguistic
phenomena fit into the sketch just presented and mention connections to prior lines of work.
Count/mass coercion. English nouns are flexible in their count/mass status (see §3.4). Atypical marking for number or definiteness can cause a shift, or coercion, in boundedness: plural or indefinite marking on mass nouns (a lemonade, two lemonades) yields a bounded interpretation (cups or bottles of lemonade). Conversely, count nouns with no determiner are coerced to an undifferentiated mass, via a phenomenon known as grinding ("there was mosquito all over the windshield") (Pelletier and Schubert, 1989, 2003; Copestake and Briscoe, 1995). Here we see evidence of the outsize influence of tiny grammatical markers on manipulating lexical defaults in the construal process.
Aspectual composition. Aspect is a prime arena for studying how multiple factors conspire to shape event construal. Verbs are associated with default aspectual classes that can be coerced under pressure from conflicting cues, where details of event structure systematically constrain possible coerctions and their inferential consequences (Moens and Steedman, 1988; Talmy, 1988).
In fact, aspectual coercion can be reanalyzed in terms of construal dimensions. For example, durative modifiers (e.g. for an hour) prefer to combine with atelic processes (lacking a defined endpoint, as in 3a) on which to impose a bound (analogous to count/mass coercion) and duration. Combination with any other aspectual class triggers different operations to satisfy that preference:
(3) a. He  $\{\mathrm{sleep} / \mathrm{ran}\}$  for an hour.
b. He sneezed for an hour.
c. He read the book for an hour.
d. He left for an hour.
A single sneeze, being a discrete event unlikely to last an hour, undergoes ITERATION into a series of sneezes (3b), illustrating a change in plexity ( $\S 3.4$ ); while the book-reading in (3c) is simply viewed as unfinished (cf. "He read the book"). The departure in (3d) is a discrete event, but unlike sneezing, it also results in a state change that is reversible and therefore boundable (cf. the iterative reading of "He broke the glass for an hour", the non-permanent reading of 2c). Its coercion thus features multiple operations: a PROMINENCE shift to profile the result state of being gone; and then a BOUNDING that also reverses state, implying a return (Chang et al., 1998).
Constructional coercion. The flagship example cited in the construction grammar literature (4a) has also been analyzed as a kind of coercion, serving to resolve conflicts between lexical and grammatical meaning (Goldberg, 1995, 2019):
(4) a. She sneezed the napkin off the table.
b. She {pushed / blew / sneezed / ?slept} the napkin off the table.
Here, the verb sneeze, though not typically transitive or causal, appears in a Caused Motion argument structure construction, which pairs oblique-transitive syntax with a caused motion scene. The resulting conflict between its conventional meaning and its putative causal role is resolvable, however, by a commonsense inference that sneezing expels air, which can plausibly cause the napkin's motion (cf. Forbes and Choi, 2017).
This coercion, also described as role fusion, differs from the previous examples in manipulating the PROMINENCE of a latent component of meaning. Coercion doesn't always succeed, however: presumably sneezing could only move a boulder with contextual support, and sleeping has a less plausibly forceful reading. In fact, construal depends on the interaction of many factors, including degree of conventionality (where push and blow are prototypical caused motion verbs), embodied and world knowledge (the relative forces of sneeze and sleep to napkin weight), and context.[5]
There is extensive psycholinguistic evidence of constructional coercion and the many factors influencing ease of construal (see Goldberg (2003, 2019) for reviews). Some of these phenomena have been analyzed within computational implementations of construction grammar (Bergen and Chang, 2005; Bryant, 2008; Bergen and Chang, 2013; Dodge and Petruck, 2014; Steels, 2017; Steels and Feldman, 2017; Matos et al., 2017), and have also been incorporated in corpus annotation schemes (Bonial et al., 2011; Hwang et al., 2014; Lyngfelt et al., 2018).
Metonymy and metaphor. Metonymy and metaphor are associated with semantic mismatches
that trigger construal operations. A possible analysis of tiny iron grip from (1) illustrates both.
First, the modifiers tiny and iron expect a physical entity, but grip is a (nominalized) action. This conflict triggers a profile shift (PROMINENCE) to the grip's effector (a hand), effectively licensing a metonymy. A further conflict arises between the hand and its description as iron (unlikely to be literal unless the protagonist is of robotic lineage). A structural alignment (METAPHOR) then maps the iron's strength to the grip's force, which in turn maps to the degree of dictatorial control.[6]
We observe that multiple construal operations can occur in sequence; that a conceptual or linguistic element may afford more than one construal within the same analysis (grip as both a hand and metaphorical control); and that aspects of common sense, world knowledge, and culture (though not the focus of the present work) inevitably constrain construal options.
5 Case studies
We turn to a few illustrations of how the pervasive effects of construal can arise in applied settings.
5.1 Case study 1: Conversational assistants
Even simple tasks like rescheduling a meeting pose many challenges to dialogue systems, in both understanding users' intents and formulating natural responses. Consider the following exchange:
U-1: When is my 1-1 with Chuck?
A-2: 4 PM today, in 15 minutes.
U-3: Is there another slot soon?
A-4: Not today, should I check tomorrow?
U-5: Let's push it to his tomorrow evening.
A-6: Rescheduled 1-1 with Chuck for 2 PM tomorrow, 6 PM in Brazil.
The agent's first response (A-2) demonstrates sensitivity to PERSPECTIVE by providing a relative time. Interpreting "another slot soon" in the user's follow-up (U-3) requires both understanding that another is implicitly defined in contrast to the existing slot (relying on PROMINENCE) and then inferring the appropriate RESOLUTION meant by soon (on the scale of hours, rather than minutes or seconds). The agent's succinct response in (A-4) exploits PROMINENCE yet again, both by eliding reference to the sought-after open meeting slot with
Alternatively, iron grip could be treated as an entrenched idiom with a readily accessible construal that tiny can modify.
5 A related theory is Dowty's (1991) semantic proto-roles account, which links the grammatical subject/object asymmetry to two clusters of semantic features that are more agent-like (e.g., animacy) or patient-like (e.g., affectedness), respectively; associations between these proto-roles and grammatical subjects and objects are attested in comprehension (Kako, 2006; Pyykkönen et al., 2010) and have been investigated computationally (Reisinger et al., 2015; Rudinger et al., 2018).
Chuck, and by using "tomorrow" (the direct object of "check") as a metonymic shorthand for the joint constraints of the user's and Chuck's calendars.
The next user turn (U-5) employs METAPHOR in its construal of an event as a physical object, capable of being pushed. The metaphorical destination ("his tomorrow evening") requires consideration of differing time zones (PERSPECTIVE), as made explicit in the final agent turn (A-6).
Interactions between situational context and the kinds of compatibility constraints discussed in §4 can also affect a dialogue system's best response. A user asking a fitness tracking app "How long have I been running?" while panting around a track may be referring to the current run, but the same question asked while sitting at home is more likely wondering how long they've been habitually running. A successful response requires the integration of the constraints from (at least): the verb running, whose progressive marking is associated with ongoing processes, but ambiguous between a single run and a series of runs (CONFIGURATION); the present-perfect have been V-ing, which implies an internal view (PERSPECTIVE); and the situational context (is the user currently running?).
5.2 Case study 2: Human-robot interaction
Situated interactions between humans and robots require the integration of language with other modalities (e.g., visual or haptic). Clearly, any spatially grounded referring expressions must be tailored to the interlocutors' PERSPECTIVE (whether shared or not) (Kunze et al., 2017).
Focus of attention (PROMINENCE) is especially important for systems that must interpret procedural language. Recipes, for example, are notoriously telegraphic, with rampant omissions of information that a human cook could easily infer in context (Ruppenhofer and Michaelis, 2010; Malmaud et al., 2014). Consider (5):
(5) In a medium bowl, cream together the sugar and butter. Beat in the eggs, one at a time, then stir in the vanilla.
The italicized words provide crucial constraints that would help a cook (human or robot) track the evolving spatial relations. The first in establishes
the bowl as the reference point for the creaming action, whose result—the mixture of sugar and butter together—becomes the implicit landmark for the subsequent beating in of eggs and vanilla.
Systems following instructions also require a means of segmenting continuous sensorimotor data and linking it to discrete linguistic categories (Regneri et al., 2013; Yagcioglu et al., 2018) (cf. the symbol grounding problem (Harnad, 1990)). This mapping may depend on flexibly adjusting RESOLUTION and CONFIGURATION based on linguistic cues (e.g., cut/dice/slice/sliver the apple).
5.3 Case study 3: Paraphrase generation
Despite many advances, paraphrase generation systems remain far from human performance. One vexing issue is the lack of evaluation metrics that correlate with human judgments for tasks like paraphrase, image captioning, and textual entailment (see, e.g., Bhagat and Hovy, 2013; Pavlick and Kwiatkowski, 2019; Wang et al., 2019b).
In particular, it is unclear how closely a good paraphrase should hew to all aspects of the source sentence. For example, should active/passive descriptions of the same scene, or the sets of sentences in (2), be considered meaning-equivalent? Or take the putative paraphrase below:
(6) a. The teacher sat on the student's left.
b. Next to the children was a mammal.
These could plausibly describe the same scene; should their differences across multiple dimensions (PERSPECTIVE, PROMINENCE, RESOLUTION) be rewarded or penalized for this diversity?
A first step out of this quandary is to recognize construal dimensions and operations as a source of linguistic variability. Paraphrase generation and other semantically oriented tasks could incorporate these into system design and evaluation in task-specific ways.
Throughout this paper, we have emphasized the flexible and multivalent nature of linguistic meaning, as evidenced by the construal phenomena described here. The effects of construal are ubiquitous: from conventional to creative language use, through morphemes and metaphors. Indeed, even the smallest forms can, like tiny tyrants, exert a transformative force on their surroundings, inducing anything from a subtle shift in emphasis to a
Indeed, the needs of human-robot interaction have motivated extensions to Abstract Meaning Representation (Banarescu et al., 2013) beyond predicate-argument structure and entities to capture tense and aspect, spatial information, and speech acts (Bonial et al., 2019).
5177
radical reconceptualization.
As illustrated in §5, this flexibility of language use poses a challenge for NLP practitioners. Yet crucially—and fortunately—construal is not random: variations in linguistic form correspond systematically to differences in construal. The dimensions of construal and their associated operations (§3 and §4) offer principled constraints that render the search for coherence more tractable.
How, then, should we proceed? Our goal is for construal dimensions such as those highlighted in §3 to be incorporated into any research program aspiring to human-level linguistic behavior. Below, we describe several concrete recommendations for how to do this.
More meaningful metrics. Taking construal seriously means rethinking how NLP tasks are designed and evaluated. Construal dimensions can provide a rubric for assessing tasks, datasets, and meaning representations (Abend and Rappoport, 2017) for which meaningful distinctions they make or require. (E.g.: Does it capture the level of RESOLUTION at which entities and events are described? Does it represent METAPHOR? Is it sensitive to the PROMINENCE of different event participants?)
Such questions might also help guard against unintended biases like those recently found in NLP evaluations and systems (e.g., Caliskan et al., 2017; Gururangan et al., 2018). Popular NLU benchmarks (like SuperGLUE; Wang et al., 2019a) should be critically examined for potential construal biases, and contrasts should be introduced deliberately to probe whether systems are modeling lexical choices, grammatical choices, and meaning in the desired way (Naik et al., 2018; Kaushik et al., 2020; McCoy et al., 2019; Gardner et al., 2020).
As a broader suggestion, datasets should move away from a one-size-fits-all attitude based on gold annotations. Ideally, evaluation metrics should take into account not only partial structure matches, but also similarity to alternate construals.
Cognitive connections. The many connections between construal and the rest of cognition highlight the need for further interdisciplinary engagements in the study of construal.
The psycholinguistics literature is a particularly rich source of construal-related data and human language benchmarks. Psycholinguistic data could also be used to probe neural language models (Futrell et al., 2018; Linzen and Leonard, 2018; van Schijndel and Linzen, 2018; Ettinger, 2020).
How well do such models capture the phenomena reviewed in  $\S 3$ , and where do they fall short?
A fuller account of the constellation of factors involved in construal should also take seriously the grounded, situated nature of language use (Harnad, 1990; Kiros et al., 2018; Bender and Koller, 2020; Bisk et al., 2020). Frameworks motivated by the linguistic insights mentioned in  $\S 2$  (such as the work on computational construction grammar referenced in  $\S 4$ ) and by growing evidence of embodied simulations as the basis for meaning (Narayanan, 1999; Bergen and Chang, 2005; Feldman, 2006; Bergen, 2012; Tamari et al., 2020) are especially relevant lines of inquiry.
Much work remains to flesh out the construal dimensions, operations and phenomena preliminarily identified in §3 and §4, especially in connecting to typological, sociolinguistic, developmental, and neural constraints on conceptualization. We believe a concerted effort across the language sciences would provide valuable guidance for developing better NL systems and resources.
As the saying goes, the camera doesn't lie—but it may tell us only a version of the truth. The same goes for language.
Some of the phenomena we have described may seem, at first glance, either too subtle to bother with or too daunting to tackle. But we believe it is both timely and necessary, as language technologies grow in scope and prominence, to seek a more robust treatment of meaning. We hope that a deeper appreciation of the role of construal in language use will spur progress toward systems that more closely approximate human linguistic intelligence.
We are grateful to Lucia Donatelli, Nick Hay, Aurelie Herbelot, Jena Hwang, Jakob Prange, Susanne Riehemann, Hannah Rohde, Rachel Rudinger, and anonymous reviewers for many helpful suggestions; and to the ACL 2020 organizers for planning a special theme, Taking Stock of Where We've Been and Where We're Going. Special thanks to Nora Chang-Hay for finally relaxing her tiny iron grip.
This research was supported in part by NSF award IIS-1812778. The FrameNet Brasil Lab is funded by CAPES grants 88887.125411/2016-00 and 88887.144043/2017-00.
Omri Abend and Ari Rappoport. 2017. The state of the art in semantic representation. In Proc. of ACL, pages 77-89, Vancouver, Canada.
Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffith, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract Meaning Representation for sembanking. In Proc. of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178-186, Sofia, Bulgaria.
Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proc. of ACL.
Benjamin Bergen and Nancy Chang. 2013. Embodied Construction Grammar. In Thomas Hoffmann and Graeme Trousdale, editors, The Oxford Handbook of Construction Grammar, pages 168-190. Oxford University Press, New York.
Benjamin Bergen and Kathryn Wheeler. 2010. Grammatical aspect and mental simulation. *Brain and Language*, 112(3):150-158.
Benjamin K. Bergen. 2012. Louder Than Words: The New Science of How the Mind Makes Meaning. Perseus Books Group, New York.
Benjamin K. Bergen and Nancy Chang. 2005. Embodied Construction Grammar in simulation-based language understanding. In Jan-Ola Östman and Mirjam Fried, editors, Construction grammars: cognitive grounding and theoretical extensions, pages 147-190. John Benjamins, Amsterdam.
Rahul Bhagat and Eduard Hovy. 2013. What is a paraphrase? Computational Linguistics, 39(3):463-472.
Dorrit Billman and Meredithy Krych. 1998. Path and manner verbs in action: Effects of "skipping" or "exiting" on event memory. In Proc. of CogSci, volume 20, pages 156-161, Madison, Wisconsin.
Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020. Experience grounds language. arXiv:2004.10151 [cs].
Claire Bonial, Susan Windisch Brown, Jena D. Hwang, Christopher Parisien, Martha Palmer, and Suzanne Stevenson. 2011. Incorporating coercive constructions into a verb lexicon. In Proc. of the ACL 2011 Workshop on Relational Models of Semantics, pages 72-80, Portland, Oregon, USA.
Claire Bonial, Lucia Donatelli, Stephanie M. Lukin, Stephen Tratz, Ron Artstein, David Traum, and Clare Voss. 2019. Augmenting Abstract Meaning Representation for human-robot dialogue. In Proc. of the First International Workshop on Designing
Meaning Representations, pages 199-210, Florence, Italy.
Lera Boroditsky. 2000. Metaphoric structuring: Understanding time through spatial metaphors. Cognition, 75(1):1-28.
Lera Boroditsky. 2001. Does language shape thought?: Mandarin and English speakers' conceptions of time. Cognitive Psychology, 43(1):1-22.
Joan Bresnan, Anna Cueni, Tatiana Nikitina, and R. Harald Baayen. 2007. Predicting the dative alternation. In Gerlof Bouma, Irene Kraemer, and Joost Zwarts, editors, Cognitive foundations of interpretation, pages 69-94. KNAw, Amsterdam.
Penelope Brown and Stephen C. Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge University Press.
Tad T. Brunyé, Tali Ditman, Caroline R. Mahoney, Jason S. Augustyn, and Holly A. Taylor. 2009. When you and I share perspectives: Pronouns modulate perspective taking during narrative comprehension. Psychological Science, 20(1):27-32.
John Bryant. 2008. Best-fit constructiona! analysis. Ph.D. dissertation, University of California, Berkeley, Berkeley, California.
Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183-186.
Eugene Casad. 1995. Seeing it in more than one way. In John R. Taylor, editor, Language and the Cognitive Construal of the World, pages 23-49. Mouton de Gruyter, Berlin.
Wallace L. Chafe and Johanna Nichols. 1986. Evidentiality: The linguistic coding of epistemology, volume 20. Ablex Publishing Corporation, Norwood, NJ.
Nancy Chang, Daniel Gildea, and Srini Narayanan. 1998. A dynamic model of aspectual composition. In Proc. of CogSci, pages 226-231, Madison, WI, USA.
Herbert H. Clark. 1979. Responding to indirect speech acts. Cognitive Psychology, 11(4):430-477.
Bernard Comrie. 1976. Aspect: An introduction to the study of verbal aspect and related problems, volume 2. Cambridge University Press, New York.
Ann Copestake and Ted Briscoe. 1995. Semi-productive polysemy and sense extension. Journal of Semantics, 12(1):15-67.
William Croft. 2012. Verbs: Aspect and Causal Structure. Oxford University Press, Oxford, UK.
William Croft and D. Alan Cruse. 2004. Conceptualization and construal operations. In Cognitive Linguistics, chapter 3. Cambridge University Press.
William Croft and Esther J. Wood. 2000. Construal operations in linguistics and artificial intelligence. In Liliana Albertazzi, editor, Meaning and Cognition: A multidisciplinary approach, pages 51-78. John Benjamins, Amsterdam.
Oana David and Barbara Dancygier. 2017. Computational approaches to metaphor: the case of MetaNet. In Barbara Dancygier, editor, The Cambridge Handbook of Cognitive Linguistics, pages 574-589. Cambridge University Press, Cambridge.
Sandra Devin and Rachid Alami. 2016. An implemented theory of mind to improve human-robot shared plans execution. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 319-326. IEEE.
Tali Ditman, Tad T. Brunyé, Caroline R. Mahoney, and Holly A. Taylor. 2010. Simulating an enactment effect: Pronouns guide action simulation during narrative comprehension. Cognition, 115(1):172-178.
Ellen Dodge, Jisup Hong, and Elise Stickles. 2015. MetaNet: Deep semantic automatic metaphor analysis. In Proc. of the Third Workshop on Metaphor in NLP, pages 40-49, Denver, Colorado, USA.
Ellen K. Dodge and Miriam R. L. Petruck. 2014. Representing caused motion in Embodied Construction Grammar. In Proc. of the ACL 2014 Workshop on Semantic Parsing, pages 39-44, Baltimore, MD.
Lucia Donatelli, Michael Regan, William Croft, and Nathan Schneider. 2018. Annotation of tense and aspect semantics for sentential AMR. In Proc. of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 96-108, Santa Fe, New Mexico, USA.
David R. Dowty. 1991. Thematic proto-roles and argument selection. Language, 67(3):547-619.
Allyson Ettinger. 2020. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34-48.
Caitlin M. Fausey and Lera Boroditsky. 2010. Subtle linguistic cues influence perceived blame and financial liability. Psychonomic Bulletin & Review, 17(5):644-650.
Caitlin M. Fausey and Teenie Matlock. 2011. Can grammar win elections? Political Psychology, 32(4):563-574.
Jerome A. Feldman. 2006. From molecule to metaphor: a neural theory of language. MIT Press, Cambridge, MA.