Is It Semantics? A Case Study of Discourse Particle Understanding in LLM s (2025)

Abstract: Discourse particles are crucial elements that subtly shape the meaning of text. These words, often polyfunctional, give rise to nuanced and often quite disparate semantic/discourse effects,as exemplified by the diverse uses of the particle *just* (e.g., exclusive, temporal, emphatic). This work investigates the capacity of LLMs to distinguish the fine-grained senses of English *just*, a well-studied example in formal semantics, using data meticulously created and labeled by expert linguists. Our findings reveal that while LLMs exhibit some ability to differentiate between broader categories, they struggle to fully capture more subtle nuances, highlighting a gap in their understanding of discourse particles.

## introduction discourse particles are words that comment on aspects of the discourse context or interlocutor attitudes, giving rise to discourse effects that are often difficult to pin down. in some of their uses, their contribution is straightforward. for example, the just in "betsy just eats chicken nuggets" tells us that chicken nuggets are the only thing betsy eats. without the just, we learn nothing about the other things betsy will (not) eat. but not all uses of a polyfunctional discourse particle are easily unifiable: consider the occurrences of just in "my brother just flew in to town" (just ≈ recently) and "i just won't stand for this injustice" (just ≈ simply), or the latter two in "a just judge just wouldn't stand for the laws just passed". from the view of formal semantics, these particles are difficult to analyze, partly because of their rich diversity of senses bundled into one word and partly because of the difficulty of characterizing each individual use @xcite @xcite . at the same ⋆ work partly done at ut-austin before joining ttic. time, they are extremely frequent in conversational language use and are crucial for comprehending discourse. there has been a great deal of work investigating lms' general proficiency at function words @xcite and overall sensitivity to discourse connectives @xcite @xcite . recent work has also shown that llms struggle to grasp senses of discourse relations @xcite at a broad level. at the same time, it is unclear how well do llms' grasp the meaning (or senses) of discourse particles like just-which, as we've discussed-have peculiarly interesting versatility in their semantics. focusing on this line of work, this work investigates the polyfunctional discourse particle just, which has been particularly well-studied in formal semantics @xcite @xcite @xcite . using data created and labeled by expert linguists, we investigate the metalinguistic capabilities of llms to distinguish the nuanced senses of just described in the formal semantics literature. we find that while they possess basic sense distinctions, language models, especially smaller ones, struggle to fully discern the subtle differences of just's senses, signaling the lack of a nuanced understanding of the meaning of discourse particles. foot_0 2 just and its semantics discourse particles, like english just, are a class of function words that are sensitive to discourse-level contextual information. examples include exclusive particles, such as english just and only, whose salient discourse function is to exclude alternatives from a contextually determined set of alternatives @xcite . exclusive: used to exclude other possibilities or options. a: what does betsy eat? b: betsy just eats chicken nuggets. unelaboratory: used to deny further elaboration on an event or concept. a: what kind of dog is fido? b: fido's just a dog. unexplanatory: used to deny that there is an explanation or to offer a weak explanation with no stronger one available. the lights in this place just turn on and off. (paraphrase: there is no reason why.) emphatic: used to add emphasis to an already strong word or phrase. this pumpkin bisque is just delicious! temporal: used to indicate that something happened very recently, or close to another event. the train just left (recently). adjective: used to describe a person or idea, especially a law or policy, as fair, appropriate, or lawful. that queen was a fair and just ruler. this law is not just! target senses figure 1 : list of just senses in declarative sentences (with target senses in blue). note that all senses other than the adjective just are discourse particle senses and function as adverbs. all examples save the adjective ones come from @xcite . english just is a good candidate to study as it (1) has been thoroughly analyzed and (2) has many senses. @xcite present a unified account for all senses of the discourse particle just, outlining 12 senses of the word, excluding the adjective sense 2 (e.g. "she was a just and fair sovereign"). we target llms' ability to distinguish four of these senses that seem reasonably distinct from a semantic point of view: the exclusive (coppock and beaver, 2014, i.a.), unelaboratory @xcite , unexplanatory @xcite , and emphatic @xcite @xcite ; we also use the temporal and adjective senses as controls (figure 1 ). these four senses warrant further definition. the following examples come from warstadt (2020). the exclusive sense excludes other salient possibilities: in one reading of "betsy just eats chicken nuggets.", the just excludes other options of what betsy could eat besides chicken nuggets. the unelaboratory sense denies the need for further elaboration on an event or concept; on one reading, in response to "what kind of dog is fido?" the just in "fido is just a dog." means that fido is simply a mutt. the unexplanatory sense hinges on the lack of an explanation for something, and so usually has the force of adding 'i don't know why'. for example, in a haunted house someone might say "the lights in this place just turn on and off.", since they are not sure as to why the lights turn on and off. the emphatic sense is used to strengthen an already extreme predicate: "this pumpkin bisque is just delicious!" is stronger with the just. the adjective sense is the most distinct in meaning and occurs in very different syntactic environments. the temporal sense serves as a middle ground between the four target senses and the ad2 the adjective sense is arguably associated with a distinct homophonous word. ## experimental setup data this paper uses two sources of data to study just: (1) hand-constructed: 90 sentences (15 of each sense) carefully created by an expert to have only one sense available for the just of each sentence without any context; and (2) annotated: 149 sentences "in the wild" that contain just, taken from opensubtitles @xcite and annotated by semanticists with their senses, with associated context. the hand-constructed corpus is necessary, as clarity in the reading of the sentence is crucial for targeted metalinguistic experiments, since ambiguity can be pervasive in justs "in the wild". for example, in the sentence "i just saw nancy.", just can either mean the seeing occurred recently (temporal reading), or only nancy was seen (exclusive reading). 3 this data is created by a graduate linguist who has studied discourse particle semantics and is a native speaker of american english. for the annotated corpus, we chose movie subtitles over other texts as they are more conversational, and therefore more likely to contain instances of just as a discourse particle. 4 our volunteer annotation team consists of two senior semanticists whose expertise is in discourse particles, and eight graduate students who have taken a graduate semantics class that extensively discussed particles. we collected annotations for 149 sentences, which were annotated by a variable subset of 8 annotators. when there was disagreement, we fell back on two additional senior annotators, whose labels were both considered regardless of agreement. ## conclusion we find that reasonably sized language models (over 1b parameters) show some basic separation for the complex english discourse particle just's senses, but fail to fully discern the deep subtlety of its senses with two very different prompting strategies. first, in an overt, few-shot sense-labeling setting with definitions and examples for each senses; and second in an open-ended, pairwise comparison setting allowing full freedom from just's senses as described in formal semantics research. this lack of nuanced sensitivity points to a gap in language model performance key for the study of discourse particles and discourse comprehension, echoing the findings of recent works @xcite @xcite .