Superlatives in Context: Modeling the Implicit Semantics of Superlatives (2025)

Abstract: Superlatives are used to single out elements with a maximal/minimal property. Semantically, superlatives perform a set comparison: something (or some things) has the min/max property out of a set. As such, superlatives provide an ideal phenomenon for studying implicit phenomena and discourse restrictions. While this comparison set is often not explicitly defined, its (implicit) restrictions can be inferred from the discourse context the expression appears in. In this work we provide an extensive computational study on the semantics of superlatives. We propose a unified account of superlative semantics which allows us to derive a broad-coverage annotation schema. Using this unified schema we annotated a multi-domain dataset of superlatives and their semantic interpretations. We specifically focus on interpreting implicit or ambiguous superlative expressions, by analyzing how the discourse context restricts the set of interpretations. In a set of experiments we then analyze how well models perform at variations of predicting superlative semantics, with and without context. We show that the fine-grained semantics of superlatives in context can be challenging for contemporary models, including GPT-4.

## introduction superlatives are used to express a certain type of comparison in language. they work as domainbased comparisons: an expression like "the smallest fish" means that there is a fish which is smaller than all other fish in a specific set. an interpretation of the superlative comparison requires the human or machine to identify the target, e.g., the entity or event being the max or min of a set, and the comparison set (cs), e.g., the set of entities or events against which you are comparing the target. appropriately defining the comparison set requires understanding the general domain, and it is often essential to read beyond the sentence level, or to draw inferences from world knowledge. take, for example, the following statement: in ex. (1) the sentence with the superlative 'largest' does not provide enough information to properly define the cs, except that one is comparing a fish to other fish. with the help of the previous sentence, one can restrict the comparison set to the fish that were caught by tom and his friends at the lake. in this paper we propose that recognizing the cs hinges on identifying the relevant entities or events from context, which can appear both before or after the expression. specifically here, the catching event is crucial in restricting the cs. being able to automatically interpret the semantics of superlatives can be useful for many downstream applications, such as dialogue state tracking or mining product reviews @xcite . they appear in semantic parsing datasets, like text-to-sql ('book the earliest flight to boston' @xcite ) and their accurate semantic representation might improve question answering or information extraction. to the best of our knowledge, superlatives are understudied in nlp, and so far there has been no systematic work on automatically identifying the cs restrictions from the larger discourse. while @xcite annotated the comparison set of one semantic superlative subtype, they did so only when it is explicitly expressed in the syntactic construction of the sentence. similarly, @xcite mention the problem of appropriately defining the cs, but their annotation is restricted to the sentence-level only. there are many different types of superlatives, either through the way they are expressed in syntax (e.g. adverbial vs. adjectival) or through the way they express a semantic comparison. but most works only focus on single subtypes and do not cover all forms of superlatives. we propose a unified annotation schema for providing a complete semantic reading of superlatives. the schema defines the superlative frames, encapsulating all the elements needed for an interpretation. the frames remain identical whether or not the semantic elements are explicit or implicit, and allow us to specify restrictions from context, which are made explicit in the form of neo-davidsonian semantics, allowing one to show when interpretations are restricted by events or arguments. based on the proposed schema we annotate a dataset of superlatives, called supersem foot_0 , over different domains, ranging from encyclopedic text to dialogue. we show that supersem contains a large variety of superlative types and interesting instances of implicit domain restrictions. given supersem, we investigate models' ability to generate superlative interpretations. this allows us to analyze the effect the discourse context has on restricting possible comparison interpretations and to assess the efficacy of filling in implicit elements from larger contexts. we also show that it is challenging for sota llms, like gpt-4, to appropriately incorporate discourse restrictions for the interpretation of superlatives. ## related work while superlatives have been widely studied in formal semantics, they have been largely neglected by nlp research, except for the following works. @xcite presented an automatic approach for predicting semantic interpretations of superlatives. for this purpose they annotated a corpus foot_1 of attributive superlatives and their cs spans inside of a sentence. @xcite proposed an annotation scheme for identifying syntactic classes of superlatives and a semantic analysis of superlatives in terms of targets and cs. they further also automatically identified superlative surface forms and extracted targets and cs for a specific superlative sub-type @xcite . @xcite also worked on identifying the targets and cs, using silver data from structured knowledge bases. @xcite introduced a framework for comparative constructions, including superlatives, which is also able to model ellipsis, but it is limited to the sentence level. similarly, pesahov et al. (2023) we extended upon previous superlative research by looking at a wider array of phenomena. firstly, we are targeting all syntactic types of superlatives, while @xcite only analyzed the comparison sets of attributive superlatives and @xcite only analyzed predicative superlatives. most importantly, adverbial superlatives have not been studied in nlp. lastly, we are extending the analysis of superlatives beyond the sentence boundary. while previous works did perform a (limited) analysis of implicit superlative phenomena inside of a sentence, we go beyond the sentence-based analysis and show how comparison sets are restricted by the broader domain of discourse. ## the challenge: syntax, semantics and pragmatics of superlatives the particular challenge of superlatives interpretation is somewhat ignored in the study of natural language understanding, despite demonstrating many interesting syntactic and semantic phenomena. while all superlative expressions seem to do the same thing, i.e. pick a maximal entity/event, they appear in different forms, which, for nlp, hinders their uniform interpretation. furthermore, superlatives may appear in various syntactic realizations which do not explicitly express some of the semantic aspects of the comparison, in particular how the comparison set is being restricted by context. in what follows we describe the possible syntactic realizations and semantics of superlatives, and how some of the frames can be implicit or ambiguous. ## superlative frames in what follows we define the set of superlative frames. we propose a formal, event-based, account of superlatives. we first define all the frames and then show how they are able to cover all superlative types from sec. 3.2. these superlative frames provide an intuitive way to achieve (i) annotations and (ii) a straightforward use for computational modeling. the frames are built with a focus on annotating the semantics of the comparison and making discourse restrictions explicit. additionally, we center the frames around events/predicates, when available, using neo-davidsonian semantics. this is motivated by the fact that events can also function as set restrictions for superlatives, such as for relative set comparisons. when no event is restricting the superlative, we annotate the restricting noun phrases. an example annotation using our scheme can be seen in fig. 1 . comparison set in the comparison set we define the set of entities or events that take part in the comparison: @xmath0, ..., e n }. comparisons involving an event are formulated using a neo-davidsonian expression. the argument slots are labeled using verbnet roles @xcite and filled with tokens from context. the cs in fig. 1 consists of a pay event, with four semantic arguments: agent, asset, location and time. property each entity or event in the comparison set has a property along which it is being compared. we use nouns to define these properties. the property in the example is popularity. target the target stands in an is-a relation with the comparison set, i.e. the target is one of the entities or events in the comparison set: @xmath1. specifically, it is the entity or event whose property has the max/min value: max/min(p). anchor the anchor of the cs designates the focus of the comparison. we index its position in the cs, e.g. #2=asset. the cs, expressed in words, would be something like 'visa cards people pay with in romania'. the anchor signals that we are comparing 'visa cards' and not another entity. orientation +/-this field designates if the min or max operation was applied on the property. rank sometimes superlative targets do not denote the entity at the min/max position, but instead they denote an entity at the n-th position. for example: "the second biggest bulgarian port". in these cases we note the given rank (default is 1). implicit +/-this field specifies whether the superlative is restricted by content outside the sentence boundary or alternatively by content that is not mentioned but implied. ## annotating superlatives one of our contributions is the supersem dataset, consisting of more than 4000 annotations of superlatives and their semantic interpretation in terms of the set of frames described in sec. 4. in what follows, we describe the annotation process and provide an analysis of the final dataset itself. ## computational modeling of superlative semantics in what follows we want to examine the computational modeling of superlative semantics. we are interested in multiple aspects. first, we want to establish sequence-to-sequence baselines for predicting our superlative frames, when trained on supersem. additionally, we want to better understand the role of context when predicting the superlative frames, and when superlatives are ambiguous. to answer those questions, we carry out multiple experiments (and more in the appendix). experimental setup we use t5-3b @xcite for all experiments, if not noted otherwise. we use a batch size of 2, a maximum output length of 300 and we train for 3 epochs on 2 a100 gpus. for training, development and testing we create 80-10-10 splits of supersem, by randomly sampling from each domain equally. we report exact match accuracy, the jaccard's index, where we divide the count of overlapping tokens by the count of all tokens, and rouge-n (@xmath2. we additionally finetune llama-3 8b @xcite to predict the comparison set, for 3 epochs, with max sequence length of 4096 and a batch size of 128. ## conclusion superlative comparisons are interesting because their interpretation is closely tied to the context they appear in and because many of their components are often implicit. this paper provides a comprehensive study of superlatives, by proposing a new, unified annotation scheme and an annotated superlative dataset, supersem. we further perform a set of experiments which analyze how models interpret superlative comparisons, how they are able to incorporate context restrictions, and how ambiguity and context interact for superlative interpretations.