Post-training for Efficient Communication via Convention Formation
Authors: Yilun Hua
Evan Wang
Yoav Artzi
Venue: COLM 2025 (2025)
Keywords: Interaction
Communication Efficiency
Linguistic Convention
Post-training
Alignment
LLM
In-context learning
TL;DR: We introduce a post-training method and new tasks to test and improve LLMs' abilities in forming conventions for efficient communication.

Abstract: Humans communicate with increasing efficiency in multi-turn interactions, by adapting their language and forming ad-hoc conventions. In contrast, prior work shows that LLMs do not naturally show this behavior. We develop a post-training process to develop this ability through targeted fine-tuning on heuristically identified demonstrations of convention formation. We evaluate with two new benchmarks focused on this capability. First, we design a focused, cognitively-motivated interaction benchmark that consistently elicits strong convention formation trends in humans. Second, we create a new document-grounded reference completion task that reflects in-the-wild convention formation behavior. Our studies show significantly improved convention formation abilities in post-trained LLMs across the two evaluation methods.

Published as a conference paper at COLM 2025
Post-training for Efficient Communication via Convention Formation
Yilun Hua, Evan Wang, and Yoav Artzi
Department of Computer Science and Cornell Tech Cornell University {yilunhua, yoav wow.eel...oreel.eeee. ew447@cornell.ed.
Abstract
Humans communicate with increasing efficiency in multi-turn interactions, by adapting their language and forming ad-hoc conventions. In contrast, prior work shows that LLMs do not naturally show this behavior. We develop post-training process to develop this ability through targeted fine- tuning on heuristically identified demonstrations of convention formation. We evaluate with two new benchmarks focused this capability. First, we design a focused, cognitively-motivate interaction benchmark that consistently elicits strong convention formation trends in humans. Second, we create new document-grounde reference completion task that reflects in-the-wild convention formation behavior. Our studies show significantly improved convention formation abilities in post-trained LLMs across the two evaluation methods.
1 Introduction
Humans naturally display rapid adaptation during linguistic interactions by developing increasingly efficient ways to refer to concepts. This formation of ad-hoc linguistic con- ventions has been repeatedly observed in studies (Krauss & Weinheimer, 1964; Brennan & Clark, 1996; Hawkins et al., 2020a), and is a cornerstone of naturalistic human language interaction. not only improves the accuracy of relaying information, but also reduces its costs. Contemporary large language models (LLMs), on the other hand, do not naturally show this behavior (Hua & Artzi, 2024).
We propose a targeted post-training process to develop this general ability in LLMs, such that models spontaneously form conventions as an in-context behavior. We heuristically extract examples of convention formation from human corpora to construct minimal pairs, where the minimal difference between two paired examples in the demonstration of this behavior. In addition, we enhance the reasoning process of model by introducing reference planning tokens, which mark when a referent is a re-mention. We treat this data as preference pairs for DPO-style policy optimization (Rafailov et al., 2024), and carefully design the optimization process to acquire generalizable convention formation ability.
We design our evaluation process to differ from the training in both data and scenario, with the aim of quantifying the generalizable change in model behavior. We create two evaluation tasks. In the first, follow the reference game design from cognitive science studies (Krauss & Weinheimer, 1964; 1966; Clark & Wilkes-Gibbs, 1986; Hawkins et al., 2020a), where a speaker repeatedly directs a listener select a target from shared set of items. Reference games were designed to show strong effects in humans with relatively short interaction and limited data. While reference games often rely on visual stimuli, we design new text-only variant to avoid any confounding issues due to the visual reasoning abilities of models. This broadly applicable new variant shows similar trends to visual reference games in baseline human studies we perform. The second evaluation scenario we design is a seecumnnt-groundde utterance completion task, rooted in scenarios where assistant uses reference document to answer user's 's questions and explain concepts. This task complements the reference games with a format inspired by retrieval augmented generation, where assume the model is generating response based on given references.

---

Published as a conference paper at COLM 2025
Post-training
Reference Game Eval
Doc-grounded Eval
Coreference based Extraction
Applesauce
Lemonade
Marmalade
Soup
Reference Document
Refference Span
Soft food that is usuatly ground from fruie
Planning SFT
[Conversation History)
Applesance
co hosting the Seattle Christmas
Last question, what does our official registered dealer to size-wise? Agent:
User:
made tratt commaniy Ravared
See you at [rementionn] the parade.
Marmaladee
The slan must meesure at least 3 feet wide by feet tall
Preference Optimization
Purced
Your sign needs to be et leest feet wide by 2 feet tall [+]
Narmalade
Met
[remention] the perade.
Your officiel registered dealer sign needs be loast 3 feet [X]
X: See you [remention] the Seattle Christmas parade.
Applesauce
Figure 1: Overview of our framework. We propose a post-training method to improve LLMs' convention formation (left) and two evaluation tasks (middle and right) covering different interaction scenarios and domains.
We study several state-of-the-art proprietary and open-source LLMs and find that, even in the text-only domain, LLMs still lack the ability to develop ad-hoc concise referring expres- sions and form conventions. We apply our post-training method to the open-source models and observe significant improvements. The post-trained LLMs shorten their messages by up to 26% throughout the reference games, maintain message consistency to form conventions, t and allow the listener to identify the target with increasing accuracy. In the document- grounded task, the post-trained models also outperform their off-the-shelf counterparts by a substantial margin. All these improvements appear even though the model only trained on data from natural conversations and never on the evaluation scenarios.
Overall, our contributions are two-fold. First, our tasks form new avenues to assess the communicative efficiency and adaptability of LLMs, without the potential confounding factor of visual stimuli. Second, we show that contemporary LLMs lack this ability for efficient communication, and design post-training method to mitigate this issue. Our methods lay a foundation for research to further improve LLMs' convention formation and communication efficiency. Our code is available at https://ge thub. com cow/lilolal/////////iii.. communication. git.
post-train-foo--ffiiiiee wwt.rorrerrfiiiieeetooooonniniii
2 Background and Related Work
Repeated Reference Games Repeated reference games with visual stimuli are widely used to study human convention formation (Krauss & Weinheimer, 1964; 1966; Clark & Wilkes-Gibbs, 1986; Hawkins et al., 2020a). A standard repeated reference game involves speaker and a listener interacting over many trials and a recurring set of referents (images). In each trial, target referent is revealed only to the speaker, who needs to produce description to guide the listener to select the target. The two participants see the referents in different orders and cannot use the position information in their referring expression (e.g., cannot say pick the first item). After each trial concludes, the speaker sees the item selected by the listener and the listener knows if their selection is correct. As the participants complete more trials together, the same referent will repeatedly appear as the target. This has been observed to lead the speaker to use increasingly efficient language, the listener to become increasingly accurate in identifying the target. Hua & Artzi (2024) adapts the repeated image reference games for multimodal L.M evaluation. They show that multimodal LIMss struggle with developing more concise referring expressions as the speaker, but achieve near-hunm hear-human performance as listeners. Therefore, they use GPT4 as the listener to automate the evaluation of LLM speakers, a design choice we also adopt.
2

---

Published as a conference paper at COLM 2025
Convention Formation Humans are inclined to minimize the effort used in communi- cation, both the effort one spends to articulate a piece of information and the effort their partner needs to understand it (e.g., Zipf, 1949; Gibson et al., 2019; Yin et al., 2024). This principle is widely observed in conversations, in the form of convention formation (Krauss & Weinheimer, 1964; 1966; Clark& & Wilkes Wilkes-Gibbs, 1986; Hawkins et al., 2020a; Haber et al., 2019), when the referring expression for the same item shortens and the two interlocutors arrive at a mutually understood, concise expression that they will stick to for the rest of the interaction. Importantly, the expression for an item quickly becomes consistent a speaker usually keeps dropping words from their message but rarely introduces new words to refer to the same item (Hawkins et al., 2020a). Hua & Artzi (2024) observed that LLMs do not exhibit this trend. Bringing about this property of convention formation is a critical part of our objective.
Learning for Language Adaptation in Agents Prior works on systems that adapt their language for efficient communication have primarily focused training models for fixed, specific interaction task and using data from the task. Hawkins et al. (2020b) applies continual learning method to elicit language shortening within repeated reference games. Takmaz et al. (2020) trains a model on data from an image reference task, PhotoBook (Haber et al., 2019), and shows that models trained to ground on the image and conversation history can generate more efficient referring expressions for this task. Similar paradigms have also been adopted to study models' adaptation interlocutors with different background information in reference games (Greco et al., 2022). In contrast, our goal is to train general-purpont LLM that can show general language efficiency adaptation, whether in highly controlled lab-like game scenario or scenarios close to how these models are used in production. Importantly, using only the data extracted from natural human conversations, our post-training method significantly improves LLMs' convention formation ability in the repeated reference game, even though we do not train them on this specific task format.
3 Post-training for Ad-hoc Convention Formation
We propose post-training method that includes several stages: (a) constructing preference data from heuristically identified examples of convention formation in human conversations; (b) adding a special planning token to enhance explicit reasoning about references; and (c) policy optimization using the preference pairs. Throughout, we design learning to preserve the model's general capabilities. Appendix B includes additional implementation details.
Preference Data Construction We use a coreference resolution (Bagga, 1998) model on scripts of TVV series, domain of text that is rich with conversational interaction, to identify repeating instances of convention formation, and modify the data to create preference pairs. Coreference resolution generates reference chains, where mentions that refer to the same entity appear in their order in the text. Such chains often display convention formation, with later references being shorter and more consistent.
We heuristically identify examples where a concept is initially referred to (i.e., mentioned) with noun phrase in an utterance and is re-mentioned in later utterance j with more concise referring expression. Each such example may also include intermediate re-mentions, meaning that the re-mention in utterance j is not the first re-mention. Therefore, each reference chain can provide multiple demonstrations. This is important because it shows to the model that the desired behavior is not only in the first re-mention, but persists over the entire reference chain.
We use this data to construct triples of (x, Yw, where is the conversational context (i.c., the history of the interaction), yn is the preferred continuation, and y; is the dis-preferred continuation. We set x to be the conversation history until a reference, and y/v and yt to represent desired and undesired utterance to contain the reference. We create two types of preference pairs. The first type is a demonstration of the observed convention behavior. We set V/w to be the observed re-mention text, which reflects convention formation, and y/ to use the more verbose first-mention text as the re-mention. This aims to suppress the behavior of simply repeating mentions verbatim, and encourage adjusting them expected when
3

---

Published as a conference paper at COLM 2025
conventions can be formed. The second type aims to preserve how the first mentions are generated, because the model must avoid a faux conventionalization behavior in the first mention, where it has no shared common ground. We set y/ to the original first mention, and y1 to the conventionalized re-mention that was observed in utterance j. We extract the first type of pairs from reference chains of at least two re-mentions, to better demonstrate the convention formation process. For the second type, we extract from reference chains of at least one re-mention. We process 2,000 TV scripts from Chen al. (2022), and the total numbers of extracted first and second types of pairs are 11,106 and 10,135 respectively.
Adding Mention Planning Tokens We further modify the preference pairs to explicitly reflect the distinction between mentions and re-mentions using a special token: re-mentions are preceded by a [remention] token. We expect this explicit marking to allow the model to better separate its treatment of initial and later mentions. This separation could allow the model to develop convention formation skills, without hurting how initial mentions are generated. Beyond adding the new token to existing preference pairs where appropriate, we also create new preference pairs focused on training the model to use this token. We add triplets (x, Yw, yt), where if yu captures re-mention, then ye contains the corresponding planning token and 1/j does not. Conversely, if yw captures a first-mention, then yıu does not contain the planning token and y/1 contains a misplaced planning token preceding the first-mention. Empirically, we observe the planning token to help (Section 6). Figure 9 in Appendix E.1 shows an example training instance.
Regularized SFT for Planning Token Learning We precede preference optimization with a supervised finetuning (SFT) phase to help the model use the [remention] token properly. Given a training preference pair for a re-mention (x, J,,,, yi), we construct new SFT input-output pair (w), where yo is the target utterance with the special planning token [remention] inserted before the re-mention. To account for the new [remention], we extend the input and output embedding tensors of the LLM as needed. We design the training process carefully to learn the planning tokens while avoiding overfitting. Instead of the standard cross-entropy language modeling loss, we calculate the cross-entropy loss on the planning tokens only and calculate regularization loss on the rest of the tokens. Let N be the set of all tokens in y, M be the set of planning tokens in y, our loss for training instance (x,y) is!
1
LSFT = M
Σ FEM
log Tep TV((I1y<1,x)) +
Σ (M I4M
(ee (Vl(x))))))) (1)
[N|
M
s.t. (TsD(TTl [Te)t = (Kt((TT))
1 2 =
Tef
2 Pxt (Treil
2
2
where w is the regularization coefficient. Tee is the original weights of the model. Disn the Jensen-Shannon divergence (JSD), which is a symmetric and bounded alternative to KL-divergence. We empirically observed that JSD provides more stable learning.
Preference Optimization We use all preference data Yw, Vi) for this training stage. we use the APO-zero loss (D'Oosterlinck et al., 2024), which directly encourages increased winning example likelihood and a decreased losing example likelihood. We opt for APO- zero over the standard DPO loss (Rafailov et al., 2024), because DPO has been shown to sometimes cause the winning example likelihood to decrease during training and thereby hurts the performance, an issue known as likelihood displacement (Pal et al., 2024; Razin et al., 2025). The minimal difference between our preference pairs (i.e, by single noun phrase or re-mention token) is a known cause likelihood displacement. Our loss for this training stage is:
AAO -o (re(x,yw) + (ro(x,y1)
(2)
where re (xxx =Blog
Male x) and ß is hyperparameter similar to DPO's B. Re (y)x)
We use a somewhat non-conventional set notation here because it simplifies the overall notation, and does not require testing for [orrree [remention] tokens in the objective.
4

---

Published as a conference paper at COLM 2025
4 Evaluation Tasks
We design evaluation tasks to be separate and distinct from our training regime, with the aim of assessing general abilities beyond the training data. Figure illustrates the two evaluation tasks. The two tasks approach the evaluation in different ways, and are complementary to each other.
4.1 Text-only Reference Game
Reference games with visual stimuli are commonly used to study convention formation in human interaction (Krauss & Weinheimer, 1964; 1966; Clark & Wilkes-Gibbs, 1986; Hawkins et al., 2020a) and more recently in multimodal LLMs (Hua Artzi, 2024). We create new text-only variant, loosely inspired by the Taboo game. The dynamics are identical to conventional reference games. There are two participants: speaker and a listener. They both observe set of potential referents (i.e., as context). At each turn, the speaker is asked to describe target for the listener to select. The turn ends successfully if the listener selects the right target. Otherwise, it ends with failure. Instead of images, we design the set of potential referents to be textual names of everyday items. To avoid the degenerate behavior of simply copying the word or part of it (e-g., using pan to refer to duslpan), we restrict the use of the word and its lexemes. For example, valid message for the target dustpan can be the flat, rectangular thing with the handle that you sweep into, but not the pan.
Similar to image reference games, we use repeated version of the game to study convention formation, where each interaction includes multiple turns with repeated targets. Figure 10 and Figure 16 in Appendix E.1 show an example model query and messages evolving as the interaction progresses in our reference game. This evaluation task uses relatively synthetic interaction scenario, but aims to emphasize the behavior we aim to study, so measurements of trends are clear and significant.
We follow prior work to evaluate convention formation, including tracking utterance length (e.g., Hawkins et al., 2020a;b) and tracking message consistency with word nov- elty distance (WND; Hua & Artzi, 2024), which tests the lexical stability of consecutive references to the same item (i.e., if new words are introduced). In our experiments, we con- duct human studies and confirm that the scenario elicits the expected convention formation trends in human interaction (Section 6, gray curves in Figure 2). Indeed, human text-only games show trends similar to those observed in image-based games: turn success increases, utterance length decreases, and WND decreases.
We extract candidate referents from dataset of Taboo games." 3 We create challenging refer- ence sets by sampling related items by clustering similar words using their embeddings. Each set includes four referents. A similar method for constructing challenging reference sets was used for maage-ased reference games (Hawkins al., 2020b). We extract 47 referential contexts for evaluation and adopt the same game length as prior studies: 24 turns (trials) organized into six repetitions. Each repetition includes a turn for each item in the set.
4.2 Document-grounded Utterance Completion
The second task is based on the Doc2Dial dataset (Feng et al., 2020), where user is querying an agent in a -uuuunn---oudeed interaction. At each step, the user asks a question, and the agent responds based on the document. Doc2Dial interactions also include the span of text in the document used to produce each agent utterance. As with other human interactions, Doc2Dial dialogues show convention formation when referents repeated.
Our task requires the model to complete the response of a single turn in a Doc2Dial in- teraction. The model is given both snapshot of human conversation transcript at specific turn following user question and the text span from the document that contains
Pnttps://en. wwtp::/wwiiiiiadodeee.eeo.. wwt.//////wwoooboooooooo//oo/oo (game)
9https://githu com/Kovahh/aabo--at..
4We use the OpenAl text-embedding---nme embedding model.
5

---

Published as a conference paper at COLM 2025
the information to the answer. The model is expected to generate the answer, based on the document, the reference span, and the history of the interaction. Figure 11 in Appendix E.1 provides an example input. If the target re-mention phrase is not in the beginning of the original human agent response, we pre-fill the model's generation with the first part of the human response such that the model would generate from where the re-mention phrase would start. Appendix B provides more implementation detail.
We create the test instance by processing the Doc2Dial dialogues similar to how we prepare the training data in Section 3. We extract examples where the assistant refers to a previously mentioned concept with a more concise expression, least two words shorter. This process generates a total of 481 evaluation instances. This scenario is closer to our training setup, but the interaction data differs significantly. While we use TV scripts to create our training data, Doc2Dial includes information-seeking interactions in four specific domains (social security, veterans affairs, motor vehicles, and public health).
We evaluate model performance as win-rate against another model or human ground truth. The win-rate is computed using a GPT-4o judge that is instructed step-by-step to parse the two utterances and judge if the utterance produced by the evaluated LLM uses more, less, or equally concise referring expressions compared to the counterpart model or human utterance. Figure 12 in Appendix E.1 shows the GPTT-4o judge prompt. Appendix B presents more implementation details and the GPT judge's agreement with human annotation.
The Doccument-groundee utterance completion evaluation differs significantly from the text-based reference game task. It does not include repeated interactions, and therefore does not evaluate interaction trends. Instead, it examines the behavior in more realistic scenarios, where the context (i.e., the document and the interaction history) is more complex.
5 Experimental Setup
Text-only Reference Games We automate our evaluation by using GPT4o-mini as the listener, which is efficient to run, has a low cost, and shows similar performance to human listeners based on our pilot study (Appendix A) and prior work (Hua & Artzi, 2024).
As part of our experiments, we collect human interaction data to quantify how our new setup elicits convention formation in human speakers. We use the crowd-sourcing platform Prolific, and collect 200 interactions with human speakers and GPT4o listeners. We design the instruction and format of the study such that the human participants do not know they are interacting with a model, thus eliciting more natural human behaviors. The only goal for the participants is to help the listener identify the target correctly, and participant receives a monetary bonus every time their partner successfully identifies the target. We pay 2.6 USD per interaction, with bonus of 0.03 USD for each successful turn.
To minimize potential priming effects, the instructions for the participants do not mention " "adaptation" " or " "communication efficiency", and the participants can only complete the in- teraction once. A side-effect of this light instruction is that some participants misinterpreted the instruction, and concluded that they are required to change their referring expressions between turns. Naturally, this led to an increase in WND and sabotaged the creation of conventions. We identify this group by fitting Gaussian mixture models (GMM) on the WND scores. We remove the low-consistency component (35.5% of the participants) and only use the remaining users for evaluation. This latter group displays clear trends of convention formation, the behavior we aim for our models to emulate. We further discuss our human studies in Appendix A.
Models We evaluate Claude-3.5-sonnet, GPT4o, Gemma-2-98 2-b-instruut and Llama-3.1-8b- instruct. We apply our post-training to the open-weight models, Gemma and Llama.
Training We use the coreference resolution model from Zhang et al. (2023) to prepare the data. We use LoRA for both the SFT and APO stages. In the SFT stage, we addition- ally update the embeddings of the added token while keeping the original vocabulary's embeddings frozen. Appendix B provides hyperparameters and implementation details.

---

Published as a conference paper at COLM 2025
Gemma (original)
Gemmna (ours)
Llama (original)
Llama (ours)
GITNoo
Claude-3.5
Human
70
100
60
95
Msg Len (chars)
50
Acc (00)
90
40
85
30
80
20
75
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
Repetition
Figure Reference game results. Shaded areas are bootstrapped 95% confidence intervals.
Ablations We experiment with several ablations, using the same training schedule and hyperparameter search as our post-training method whenever possible: (a) APO only: APO-zero training on the preference data directly, no planning tokens or SFT updates are used; (b) SFT: SFT on the inputs and target completions from the SFT stage (Section 3), but with planning tokens removed and using standard language modeling loss; (c) Ours w/o planning tokens: standard SFT and language modeling loss to initialize the model with small number of updates and then adopt APO training; and (d) Ours w/o JSD: standard SFT on the input and target utterances that include the planning tokens, without adding the JSD loss, and with standard language modeling loss calculated on all the tokens of the target output, followed by the same APO training stage as our main method.
Results
We observe significant deficiencies in convention formation in all the off-the-shelf models evaluate across both evaluation testbeds. Our post-training treatment significantly improves convention formation behavior. Figure shows convention formation trends on the reference game scenario, Table 1 provides the grounded-document scenario results, and Appendix C provides ablation analysis.
6.1 Text-only Reference Games
Figure 2 shows the results for the reference game evaluation, where report the average message length in characters, Word Novelty Distance (WND), and average accuracy across the six repetitions of the interaction. All off-the-shelf LLMs fail to form conventions improve their communication efficiency. They often increase message lengths over time and introduce new words to refer to repeated items, degrading consistency (higher WND).
Our post-trained models show statistically significant improvement in language efficiency, Gemma (ours) shortens its messages by 26% (42.1 characters + 31.0 characters) from Repetition 1 to Repetition 6 and Llama (ours) by 14% (48.4 characters - to 41.6 characters). The post-trained models are also substantially more consistent. Their WNDs consistently decrease as the game progresses, reaching below 1 at the end of the interaction. This means that the post-trained models' messages gradually stabilize and conventionalize.
We observe that small accuracy gap exists between post-trained and original models in Repetition Interestingly, this gap makes the post-trained models' accuracies closer to that of humans at Repetition 1. More importantly, the post-trained models' accuracies improve over time while their messages shorten, eventually using substantially more concise language to achieve accuracies close to the original models.
The human convention formation trends we observe validate our new text-based reference game setup, and closely follow trends observed in past work using visual stimuli (Hawkins
7

---

Published as a conference paper at COLM 2025
Post-trained Original
Wins
Competence Compet. Rate (%)
Ours
Original 45
Tie
Cannot Decide
Gemma
121
299
16
Llama
135
46
291
9
LLM vs. Human
Wins
Competence Compet. Rate (%)
LLM
Human
Tie
Cannot Decide
Gemma (ours)
109
97
259
16
79.1
Gemma (original)
130
19
71.9
Llama (ours)
90
114
257
20
75.3
Llama (original)
83
160
227
11
66.0
Claude GPT
76
168
220
17
63.8
192
203
13
59.0
Table 1: Document-groundeed utterance completion main results. We report win-rate com- pared to humans and the original off-the-shelf model. In rare cases, model mentions different concept than the other does or fails to re-mention any concepts. In this case, the GPT4o judge would output "cannot decide."
et al., 2020a;b). This demonstrates that our formulation is a fitting interaction scenario to study convention formation, not only in models, but also in humans.
Overall, the results show a significant increase the alignment between model and human behavior. Humans still show better communication efficiency improvement than any LLM, eventually reaching 17.3 characters and 0.38 WND. The remaining gap indicates the problem not completely solved, despite the significant improvement by our method. Appendix E.2 presents examples of successful convention formation as well as negative examples.
6.2 Document-grounded Utterance Completion
Table reports the utterance completion results. In addition to the GPT4o-judged win rate, we also report the competence rate, single number for quick comparison:
model wins # ties
competence rate (%) # model wins + # ties + #human wins
(3)
model and its off- offtthe-shelf counterpart, we conduct pairwise evaluation them and directly look at how often one model wins ties with the other model. This allows for a more finegrained comparison because post-trained model may show greater shortening than the original model, but the human utterance shows even more shortening. These instances will not be reflected by the models' competence rates, which are based on win and tie rates with humans.
To compare a specific
Compared with the original models, our post-trained models more often use more concise referring expression the re-mention than the other way around. Llama shows the greatest improvement; the post-trained model wins in 135 instances and the original model only wins 46 instances. The gaps between the post-trained and original models are statisti- cally significant with p < 0.01 under sign test. Our post-trained models' wins are also substantially closer to humans' than the original ones, and achieve higher competence rates.
The weak performance of off-the-shelf models suggests that contemporary LLMs are subop- timal in adapting their language for efficient communication. This result further supports our conclusions from the lab-like setting of reference games and manifests these models' issues in a scenario closer to how LLMs are used production.
6.3 Additional Experiments and Analyses
Ablations and Prompting-based Attempts We also experiment with ablating the [remention] tokens, ments show that all these components are necessary for the model to learn generalizable con- vention formation abilities and achieve balanced improvements across tasks. Appendix C discusses our ablation studies in detail.
Divergence (JSD) Loss, and the SFT stage. The experi-
8

---

Published as a conference paper at COLM 2025
Model
A MixEval (%)
Gemma (ours)
+0.2
Llama (ours)
-0.8
Nvidia Nemotron
-6.3
Mazzaccara et al. (2024)
-1.5
Table 2: Impact of post-training on general LLM capabilities, as measured MixEval-hard score change.
of Occurrence
Drop Phrase
Rephrase
New Feature
Human Gemma (ours)
66
19
15
62
11
26
Llama (ours)
43
35
21
Table 3: Language strategies of efficiency adaptation in the reference games.
Additionally, we experiment with eliciting convention formation with prompting. Hua & Arizi (2024) has shown that prompting is not a satisfactory solution to the convention formation problem, with their results from image-reference games. We conduct similar prompting experiments with our text-only tasks. Overall, our conclusions are similar. Prompting and fhot examples do not work well in eliciting convention formation. Appendix D presents the results of these experiments.
Preserving the LLM Capabilities We evaluate our models on MixEval-hard (Ni et al., 2024), benchmark for general LLM capabilities, which consists of diverse tasks and has state-of-the-arr correlation with human judgment. We calculate the change in MixEval score (an accuracy-based score out of 100) after post-training. We find that our approach led to no degradation on Gemma, and only small decrease of 0.8 on Llama. To better interpret the magnitude of this decrease, we additionally evaluate two other post-trained models and compute their MixEval score changes. Importantly, because we just focus on enhancing model's convention formation ability, our method is not comparable with common post-training works that intend to enhance model's general abilities and cover diverse datasets/ /tasks. Instead, we compare with post-training interventions that are intended to add or enhance specific skills. We compare against Nvidia Nemotron Nano, post-trained version of Llama-3.1-8b-instruu. where the post-training aims to enhance the math and reasoning capabilities. We also compare with the model from Mazzaccara et al. (2024), who post-train Llama-2-7b-chat to enhance its ability ask informative questions. Compared to post-training interventions for these models, our method does not cause substantial performance decrease (Table 2).
Analysis of Language Characteristics of Adaptation We analyze humans' and post- trained LLMs' language changes from one repetition to another in 30 reference games, and identify three categories of adaptation behaviors. Table 3 presents the results. The most common strategy for both humans and the post-trained LLMs to remove phrase from the previous message while keeping the rest of the message the same ("Drop Phrase"). The removed phrase is oftentimes a prepositional phrase or a clause. Another strategy is rephrasing, using concise expression to replace part all of message while conveying similar meaning. For example, the referring expression for a waterbed can change from the mattress that is filled with liquid to the liquid-filled mattress. Finally, both humans and LLMs sometimes replace the previous description by describing à new feature that is more concise. For example, a hammer can be initially described as the tool that is used to fix things around the house and later as the tool that is used to build things. These analyses show that humans and LLMs share similarities in the strategies they use, though LLMs' more frequent use of new features and /or rephrasing likely contributes to their slightly lower language consistency compared with humans (as their WNDs show).
9https://hugginge wttp://.w........a/o//o//////o co/ htt///iidaa/laaaa 3. 1 Nemotron -awronnnnnn------ 8B-v1

---

Published as a conference paper at COLM 2025
% of Words Reduction
NOUN
VERB
ADJ
ADP
PRON
Human
15
13
7
14
18
Gemma (ours)
18
14
33
8
0
Llama (ours)
34
8
24
16
-3
Table 4: Reduction of POS counts from Repetition 1 to Repetition in the reference games (percentage of the total word reduction). Negative value means the count increased.
Additionally, we examine how the numbers of common POS tags change for models and humans over repetitions in the reference games (Table 4). For all systems, we see substantial reduction in the number of nouns, verbs, and adpositions. Notably, the reduction in adjective counts accounts for larger proportion of the overall length reduction in Gemma. This is because the original Gemma model would start the game with longer messages that contain more adjectives than humans. Because our post-training not intended to alter the model behavior for initial mentions, the post-trained model's messages still often contain adjective relatively concise language with only an average of 0.5 adjectives in Repetition 1. Another difference is that human messages more often contain pronouns in early repetitions, which are then dropped later (pronoun count in Repetition 1: 0.7 (human), 0.5 (Gemma)). For Llama, the shortenings in nouns accounted for a greater proportion (34%) of reduction, also because the post-trained Llama (just like the original Llama) started the interaction with more nouns than humans did (average noun counts: 2.7 (Llama), 1.5 (Human)).
adjectives in Repetition (average adjectives count: 1.5). Humans, however, start with
7 Conclusion, Future Work, and Limitations
We introduce post-training method for LLMs to acquire convention formation abilities and thereby improve their naturalness and communication efficiency in multi-turn interactions. We also design two text-only tasks to evaluate convention formation, without the poten- tial confounding factor of visual stimuli present in prior work. We show that convention formation for efficient communication is general challenge to ttate-of-the-art LLMss. Our post-training method effectively exploits the naturally occurring examples of convention- alization from human data. We observe significant improvements reflected in evaluation scenarios that are distinct from the training data. We hope our work will encourage further research improve LLMs' adaptation for communication efficiency.
model behavior, as
Our work leaves various open problems, and points to potential directions for future work. We experiment with our approach isolation. An important future direction to integrate this intervention into the complete L.MM post-training process, so that convention formation acquired alongside the general skills acquired in post-training. Future work can also study the impact of this change the broader set of post-training goals.
Critically, important to consider the incentives of human speakers when they form conventions: reduction of costs and increasing of utility. Models have awareness of these considerations, and instead they learn the data distribution better through our process. Reasoning about cost and utility explicitly is an interesting direction for future work.
While our tasks avoid the potential issues of visual stimuli, they are intended to complement, not to replace, image reference games. Image referents in prior work cover diverse domains and levels of details (e.g., Hua & Artzi, 2024; Eliav et al., 2023), likely influencing language styles and adaptation. Future work can study the behavior differences under these settings.
Despite the significant improvements we demonstrate, our post-trained LLMs still do not show the same level of convention formation humans. Closing the gap further is important future work direction. Our evaluation benchmarks will enable this effort.
Finally, focus special type of adaptation (i.e., convention formation). This is not the only way interlocutors adapt ad-hoc to make interactions more efficient and successful. For example, speaker may sometimes want to switch (back) to more elaborate language, upon seeing the listener's 's confusion. We hope future work expands upon this set of behaviors.
10

---

Published as a conference paper at COLM 2025
Acknowledgments
This research was supported by NSF under grants No. 1750499, gift from Open Philan- thropy, a gift from Apple, an Nvidia Academic Grant, the National Artificial Intelligence Research Resource (NAIRR) Pilot, the Frontera supercomputer supported by the National Science Foundation (award NSF-OAC 1818253) at the Texas Advanced Computing Cen- ter (TACC) at The University of Texas at Austin, and the Delta advanced computing and data resource, which is supported by the National Science Foundation (award NSF-OAC 2005572). We gratefully acknowledge AT Consortium, Inc, with support from the State New York, the Simons Foundation, and the Secunda Family Foundation (Bloom et al., 2025). We thank Robert Hawkins and Marten van Schijndel for insightful discussions. We thank the anonymous reviewers and the area chair for their valuable feedback.
of the research computing resources of the Em Empire
References
Ron Artstein and Massimo Poesio. Inter-coder agreement for computational linguistics. Computational Linguistics, 12 2008. URL https: //tts:://do... org/10.g///// /0t../..... 1112..111cco... 0tt2//col.00003-222 07--344RR2.
Amit Bagga. Evaluation of coreferences and coreference resolution systems. In LREC, pp. 563-572. Citeseer, 1998.
Stacie Bloom, Joshua C. Brumberg, Ian Fisk, Robert J. Harrison, Robert Hull, Melur Ra- masubramanian, Krystyn Van Vliet, and Jeannette Wing. Empire Al: A new model for provisioning AI and HPC for academic research in the public good. In Practice and Experi- ence Adoanced Research Computing (PEARC 225,, pp. 4, Columbus, OH, USA, July 2025. URL sttpps///oo http:////oooooo......oo 1145/3708035 37300000003333770000
ACM. doi: 10.1140 00.1145/37000355333663
Susan E. Brennan and Herbert H. Clark. Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1996.
Mingda Chen, Zewei Chu, Sam Wiseman, and Kevin Gimpel. SummScreen: A dataset for abstractive screenplay summarization. 2022. URL https://acllantholoyy org.///2...2222222/
acl-long. 559/
Herbert H. Clark and Deanna Wilkes-Gibbs. Referring as a collaborative process.
Cognition, 22, 1986. URL https:/www..scieneddiree wwwp:/ncedrreett..ooc/ceeeee/e/ comm//scciennn//rriiccpppp/
0010027786900107
Karel D'Oosterlinck, Winnie Xu, Chris Develder, Thomas Demeester, Amanpreet Singh, Christopher Potts, Douwe Kiela, and Shikib Mehri. Anchored preference optimization and contrastive revisions: Addressing underspecification in alignment. arviv, 2024. URL
06266.
Ron Eliav, Anya Ji, Yoav Artzi, and Robert Hawkins. Semantic uncertainty guides the extension of conventions to new referents. Xiv, 2023, URL http:////xixo http/aaiiiii..ggb///////////// 06539.
Song Feng, Hui Wan, Chulaka Gunasekara, Siva Patel, Sachindra Joshi, and Luis Lastras. doc2dial: goal-oriented gocument-grounded dialogue dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, 2020.
URL https://aclaatt aacatt/ooo/l org/2020 2020. emnlp-main 652/.
Edward Gibson, Richard Futrell, Steven P. Piantadosii, Isabelle Dautriche, Kyle Mahowald, Leon Bergen, and Roger Levy. How efficiency shapes human language. Trends in Cog- nitive Sciences, 23, 2019. URL https: http://wwwwwscceeeeeereee... cww ctt..////cceeeeeeeerrreee/piiiii
S1364661319300588
Claudio Greco, Diksha Bagade, Thu Le, and Raffaella Bernardi. She adapts to her student: An expert pragmatic speaker tailoring her referring expressions to the layman listener. Frontiers in Artificial Intelligence, 2022. URL https: /www. frontiersin. wtt.eeeesiinnoooorraaaiaaaeaeaae jorjjsurals// artificial-interd htttffiiaa-mmeelliignnn//aaraaaa 3389/fraii 2023. 101 7204/fu2 fuull.
11

---

Published as a conference paper at COLM 2025
Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, and Raquel Fernández. The PhotoBook dataset: Building common ground through visually-grounded dialogue. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2019. URL https: httpss///aaaathoogy.oooo org/P13 9- 1..111....p..
Robert Hawkins, Michael C. Frank, and Noah D. Goodman. Characterizing the dynamics of learning in repeated reference games. Cognitive Science, 44, 2020a. URL https:// onlinelibrary. wi wil.yy.o com /tt///////////////// 1111 /111//ogs. 12845.
Robert Hawkins, Minae Kwon, Dorsa Sadigh, and Noah Goodman. Continual adaptation for efficient machine communication. In Proceedings of the Conference on Computational Natural Language Learning, November 2020b.
Yilun Hua and Yoav Artzi Talk less, interact better: Evaluating in-conexxt conversational adaptation in multimodal lims. 2024. URL https: /taps://arxxv..oot http///aii..orrab///////// https////2...0111111.1 01417.
Robert M. Krauss and Sidney Weinheimer. Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study, Psychonomic Science, 1, 1964. URL https: //ink springer com/ ar wttpa/r/cccc//eeeee (0t.033388///3388111
Robert M. Krauss and Sidney Weinheimer. Concurrent feedback, confirmation, and the
encoding of referents in verbal communication. Journal of Personality and Social Psychology, 4, 1966.
Klaus Krippendorff. Reliability in content analysis: Some common misconceptions and recommendations, Human Communication Research, 2006. URL https:::///i..rrg//eee 1111/ j. 1.1468-2953 2004. tb00738. x.
Davide Mazzaccara, Alberto Testoni, and Raffaella Bernardi. Learning to ask informative questions: Enhancing LLMs with preference optimization and expected information gain. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024. URL attps://aclanthoooog org/ findings--nnnnn eenegs-emnl...299 291/.
Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, and Yang You. Mixeval: Deriving wisdom of the crowd from 11m benchmark mixtures. ar arv, 2024.
URL https: wwwpii..oooaaaaeee wttpi//.rrr//a//2.......0 06565.
Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. Smaug: Fixing failure modes of preference optimisation with dpo-positive arXiv, 2024. URL https: //axivv.
http:///aaxxv...//a///////// org/ 13228.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. arXiv, 2024. URL https: httppp/:///aavi.oo/a/bbb/////
18290.
Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, and Boris Hanin. Unintentional unalignment: Likelihood displacement in direct preference opti- 088477.
mization. 2025. /RRLhttss///arriv.or/////
Ece Takmaz, Mario Giulianelli, Sandro Pezzelle, Arabella Sinclair, and Raquel Fernández. Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020. URL https://aclanttho///
org emnlp-main. 353/.
Kayo Yin, Terry Regier, and Dan Klein. American sign language language handshapes argg/bs//206 04024.
for communicative efficiency. arviv, 2024. URL hanggaans reflect pressures
Wenzheng Zhang, Sam Wiseman, and Karl Stratos. Seq2seq is all you need for coreference resolution. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https: http::///caaaatooo/oo/ooo wwwphhhlll....ooogg/eeee org/2023. mmnlp-main 704/.
George Kingsley Zipf. Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Addison- Wesley Press, 1949.
12

---

Published as a conference paper at COLM 2025
A Human Study Details
To identify the segetitoon
of people who likely misinterpreted the task, calculate the average WNDs from Repetitions 4-6, where we expect conventions to have likely formed, and apply Gaussian Mixture Models (GMM) on the distribution of the WNDs for the 200 interactions collected. We tested component numbers from to 8 and find that a 3-component GMM best fits the data according to the Bayesian Information Criterion. Based on this GMM (Figure 3), we identify the low-consistency group (Group in Figure 3), which corresponds to 35.5% of the interactions. As shown by Figure 4, this group (purple curve) is overly verbose and achieves lower accuracy than the remaining interactions (gray curve) even though they start with about the same accuracy. Therefore, we only use the performance of the remaining interactions (Groups 1 and 2) as our goal and baseline for post-training.
In addition to the human-GPT interactions, we conducted a pilot study with interactions where both the speaker and the listener were humans. We use the same GMM obtained from our 200 human-GPT interactions to divide the pilot study data into two groups. The low-consistency group corresponds to 30.6% of the pilot study data. We observe that the high-consistency groups from our human-GPT4o study and human-human study show high agreement in message lengths, WND, and accuracies (Figure 4). Therefore, conclude that GPT4o-mini is valid substitute for human listeners to automate the reference game evaluation.
Distribution of Human Speaker WND Modeled by Gaussian Mixture Model
Group 1: 0.04
Group 2: 0.78
Group 3: 3.45
Overall Distribution
Probability Density
1.5
1
0.5
0
2
3
4
5
6
WND
Figure 3: Distribution of Average Human Speaker WND in the second half of the interaction (Repetitions 4-6). Group 1 and Group correspond to the convention-forming majority group which we use as the goal for our post-training.
13

---

Published as a conference paper at COLM 2025
Human-GPTio (high consist.)
Human- (low consist.)
Human-Human (high consist.)
Human-Hume Human (low consist.)
70
100
60
95
Msg Len (chars)
50
WND
Acc (%)
90
40
85
30
2
80
20
75
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
Repetition #
Figure 4: Comparison of our human-GPT4 interactions and our human-human pilot study. Shaded areas are bootstrapped 95% confidence intervals.
B Additional Implementation Details
Training For both the SFT and APO stages, we search over three learning rates (5e-5, 1e-5, 5e-6) and use linear decay learning rate scheduler. For SFT, we train for up to 300 updates with batch size of 16. For APO, we use batch size of 32 and experiment with two max number of updates (4000 steps and 10000 steps). For the SFT training to learn planning tokens, set the w in Equation 1 to 20 to rescale the JSD term to about the same order of magnitude as the cross-entropy loss. We use LoRA for all model training, using a LoRA rank of 16 for SFT and 32 for APO. We set LoRA alpha to 32 for both stages. For APO, we use the default beta of 0.1.
Reference Games Context Construction To make the task sufficiently challenging, we construct referential contexts with semantically similar items. We use the OpenAl text- embedding-33site model generate item embeddings and follow the practice of Hawkins et al. (2020b) to use k-means clustering and k-nearest neighbors to extract contexts.
Constraining the Generation Trajectory for Document-groundee Evaluation When given the references and past utterances, model may choose to generate any appropriate response and sometimes does not re-mention any concepts from the context. We mitigate this with prefix constrained decoding. This does not guarantee reference, but increases its likelihood by keeping the model roughly on the intended trajectory. Let be the original human utterance containing the re-mention and yp be the first N tokens before the re-mention identified, we append yp to the model input such that the model is completing sentence that already has the first N tokens filled. This constrains the potential trajectory of the model's text generation, so the model is more likely to refer to the same concept as the original human utterance. We use this input style for any test instances where the re-mention is not in the beginning of the original human utterance.
GPT4o Judge for Document-groundee Evaluation To avoid assigning winning and losing labels due to trivial differences, we post-process the GPT4o judge's extracted re-mentions from the two candidate completions and compare the re-mentions' lengths. If the length difference is within characters but GPT considers one candidate is winning, we override GPT's judgment and assign a label of tie.
To determine the reliability of the GPT4o judge, manually annotate 200 instances for Gemma (original) vs. human completions, and calculate Krippendorff alpha and Cohen's kappa to reflect the agreement between our annotation and GPT4o's output. For Krippen- dorff alpha, we consider 'tie' and 'cannot decide' as the same category, so we calculate alpha for three labels ('win', 'lose', 'tie or cannot decide'). We use Cohen's kappa for our
14

---

Published as a conference paper at COLM 2025
competence rating (same definition from Section 6), where competent model completion either wins over or ties with the human completion. We achieve a Krippendorfi alpha of 0.705, greater than the 0.667 threshold to be acceptable for drawing tentative conclu- sions (Krippendorff, 2006). For the competence rating, our Cohen's kappa is 0.749, greater than the 0.6 threshold and indicates substantial inter-annotation agreement (Artstein & Poesio, 2008).
C Ablation Results
We present the reference game ablation results in Figure 5 (Gemma) and Figure 6 (Llama). We present the secument-grounded evaluation results in Table
We observe that using SFT only does not introduce the desired trends, illustrating the importance preference pairs, reaches near zero WNDs, because the model simply copies its first expression again and again, showing no efficiency improvement. The APO only baseline does bring significant improvement on the utterance completion task but not on reference games. Compared with main method, APO only either increases the message lengths (for Gemma) or worsens message consistency (Repetition 6 WND=1.5 for Llama), thus not showing convention formation in reference games. also causes degradation in reference game accuracies by up to 8% (for Gemma). These results show that the convention formation ability learned through APO only not robust enough generalize to an unseen scenario. Finally, the ablations on our method (w /o [remention, and w JSD) show less improvement on the utterance completion task and worse performance on reference games. We note that Llama w/o o JSD is the only case where another post-training method produces overall shorter messages than our method. However, Llama w w/o JSD does not show any length change and almost repeats the messages in exact words throughout the interaction (almost zero WND starting from Repetition 2). Additionally, this method causes lower reference game accuracies and leads to smaller competence rate than method.
These results indicate that both the SFT initialization with JSD regularization and the use of planning tokens necessary components of our post-training pipeline, to bring generalizable convention formation abilities and balanced improvements across the tasks.
Ours
w/o [remention]
w/o JSD
APO only
SFT only
Human
70
100
60
95
Mag Len (churs)
50
Aoc (%)
90
4
40
85
3
30
2
80
20
1
75
1
2
3
4
5
6
1
2
3
4
5
6
2
3
4
5
6
Repetition #
Figure 5: Reference game performance for ablations with Gemma, Shaded areas are boot- strapped 95% confidence intervals.
15

---

observe various extents of shortening on the original Gemma, Published as a conference paper at COLM 2025
Ours
w/o Cremention]
w/oJSD
APO unly
SFT only
Human
70
100
60
95
Mug Len (chans)
50
Aoc (%)
90
40
85
3
30
2
80
20
1
75
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
Repetition #
Figure 6: Reference game performance for ablations with Llama. Shaded areas are boot- strapped 95% confidence intervals.
Method
Wins Tie
Competence
On Gemma
LLM
Human
Cannot Decide
Compet. Rate (%)
Ours
109
97
259
16
79.1
w/o [remention]
104
115
239
23
74.9
JSD
100
115
253
75.4
APO only
126
89
242
24
80.5
SFT only
80
150
238
13
67.9
Llama
Ours
90
114
257
75.3
w/o [remention]
85
130
244
22
71.7
/ JSD
100
130
234
17
72.0
APO only
99
117
246
19
74.7
SFT only
75
166
224
64.3
Table 5: Document-grounded utterance completion results for ablation studies
D Prompt Engineering For Convention Formation
Figure 7 presents the results for various prompting-based attempts. In addition to the metrics we have introduced, we also report Word Novelty Rate (WNR) (Hua & Artzi, 2024), a length-normalized version of WND, which can more easily distinguish models' differences when some models' messages are extremely short (e-g., just 1 word).
We first experiment with prompt based on the Gricean quantity maxim, where we specify that the message should contain enough information for identifying the target but every token generated has cost the model should use a shorter message if it's enough. As Figure 7 ("Gricean") shows, under the Gricean prompt, the average shortening from the first to the last repetitions of the reference game negligible for off-the-shelf models (dotted lines). We then experimented with adding an additional hint to the Gricean instruction: think about how the amount of information needed may change more rounds are completed and based on the listener's performance in previous rounds. With this additional hint specific to reference games,
'In the extreme case where the messages just have word and the model uses a new word every time, the WND will be at most but there is no consistency. Although a model with good consistency will still have significantly lower WND (close to 0), its difference with the concise, inconsistent model may be hard to visualize (e-g., when the plot's Y-axis range needs be much greater than 1 to accommodate for larger WND values from earlier repetitions and/ or other models).
16

---

Published as a conference paper at COLM 2025
Llama, and Claude, but they all show significantly worse consistency than humans, as shown by their higher WND and WNR (Figure 7 "Gricean+HHint",, This lack of consistency undermines communication efficiency, increasing the cognitive load for the partner to process the message and prevents convention formation (Hua & Artzi, 2024).
We also experiment with more explicit instruction, where the prompt states, more rounds are completed and the listener understands you better, make your referring expressions shorter and shorter every round. This brings significant shortening, but three of the four models tend to produce inconsistent messages (Gemma, GPT, and Claude), as shown by their significantly higher WND and WNRs than human's ' in Figure 7, "Explicit". This undesirable behavior of substantial shortening without consistency is also observed in image reference games, as reported by Hua Artzi (2024). We find that only very explicit instructions on how to shorten the messages can bring both consistency and shortening (Figure 7, "Explicit + Consistency").. We have to explicitly instruct the model to extract salient tokens from the previous messages for this item rather than introducing new words... For each item, when you reach a message you think can not be further shortened.... ..., you should keep using that message for the rest of the game. Such heavy-handed prompts will not generalize beyond reference games, so prompting is not a feasible solution to elicit convention formation.
We additionally experiment with in-context examples to explain convention formation. Our prompt first states that convention formation happens when people's referring expressions for an item evolve and become more efficient over time. It then provides examples showing how the expressions for the same item become more efficient over time (Figure 8). This combination of a general explanation of convention formation and in-context examples is more likely to generalize than the explicit, we apply this prompt the document--rounded evaluation well.
instructions tested above. Therefore,
Under this few-shot prompt, we still observe that all models (except for Claude) produce significantly less consistent messages than humans in reference games (as shown by their higher WND and WNRs in Figure 7, "Few-Shot"), thus not showing convention formation. Besides the inconsistency issue, we find that the shortening brought by the few-shot prompt does not generalize well to the brings small improvements in competence rates: Gemma (71.9 + 73.9), Llama (66.0 + 66.9), Claude (63.8 68.2), and GPT (59.0 - 62.7). All of these improved scores are still substantially worse than what our post-trained models achieve under just the standard prompt (Gemma (ours): 79.1, Llama (ours): 75.3). Therefore, conclude that prompting, even with in-context examples, is not a shortcut to elicit convention formation in LLMs.
suuumnt--grouded evaluation. The few-shot prompt only
17

---

Published as a conference paper at COLM 2025
Gemma (original)
Gemma (ours)
Llama (original)
Llama (ours)
GPT4e
Claude-3.5
Human
Gricean
Hricean+Hint
Explicit
Explicit+oomsisenny
Few-shot
70
Mog Len (chars)
40
10
5
WND
2
WAR
0.4
0.2
100
Acc
6
3
1 2
5
3
Figure 7: Attempts to elicit convention formation with prompting. We only use the standard prompt with our post-trained models. Shaded areas are bootstrapped 95% confidence intervals.
18

---

Published as a conference paper at COLM 2025
[System] You will speak naturally just like human and, when applicable, you should show behavior called convention formation. Convention formation happens when people's referring expressions for an item evolve and become more efficient over time. Each of the examples below shows how the referring expressions for an item may evolve when it's mentioned for the first time, second time, third time, and so on. Example 1:
the ceramic with floral patterns
floral ceramic vase
3) the ceramic one
4) the ceramic
Example
1) a bowl full of mixed fruit with black background
2) the mixed fruits, black background
3) the mixed fruits, black background
the black background
Example 3:
1) a building with small fire hydrant in front of big pole the building with the fire hydrant the fire hydrant
4) the fire hydrant
[... Reference Game Instructions... ...]
Figure 8: Few-shot prompt to elicit convention formation.
19

---

Published as a conference paper at COLM 2025
E Examples
E.1
Example Inputs
Write the next line of this excerpt TV show transcript from where it's left off. You will play the role of Frasier.
[Scene Three Radio Station. Frasier is rounding up his show as Roz reads Dr. Mary's book in her booth.]
Frasier: Well, Seattle, thank you for your calls.
[Frasier knocks on the window to Roz who is immersed in [remention] the book. She carries on reading.]
Frasier:
Seattle, thank you for your calls!
Roz: [monotone and bored, still reading] Hey, Frasier, what are you doing over the Christmas weekend?
Frasier: Well, Roz, if you insist on interrogating me, I'll be co-hosting the Seattle Christmas parade tomorrow night...
[Roz begins nonchalantly ringing some Christmas bells whilst still reading. She is obviously not too fussed about Frasier's plans for finishing the show.]
Assistant (Frasier):
[Preferred completion yıu J: hope it will be the beginning of a new holiday tradition. Good mental health, see you at [remention] the parade.
[Dispreferred completion y/l: ... hope it will be the beginning of a new holiday tradition. Good mental health, see you at [remention] the Seattle Christmas parade.
Figure 9: Example training data
20

---

Published as a conference paper at COLM 2025
[System] Complete a repeated reference task with a listener. You will act as the speaker and will the facilitator. This task consists of multiple rounds in which you interact with the listener on the same referential context (represented by 4 word cards). In each round, I give you one of the 4 words the target. You should refer to the target with referring expression describing the target, without mentioning the target word itself or any words the referential contexts. any of the context words is a compound word, you cannot mention their components or any part of the word either. The listener will try to choose the target correctly based on your referring expression. The listener will the 4 words in different order every round you cannot refer to the target by saying where it is the context. You can use whatever description you think will be helpful your expression can distinguish the target from other items. The same item will appear as the target multiple times throughout the task, so you will repeatedly refer to what has appeared before. You can up to 15 words for your referring expression. You can only see the word cards representing the items, not the actual objects. Reply by telling the listener: Please pick [how you refer to the target].
the listener. Make sure
Referential Context: "cleaning bucket", "kitchen broom", "laundry basket", "dustpan"
[Trial 1] Target:
"laundry basket"
[Speaker] Please pick the big, open container with the handle. [System] The listener mistakenly answered "cleaning bucket".
[Trial 2] Target:
"cleaning bucket"
[Speaker] Please pick the yellow container with the handle and the blue sponge inside. [System] The listener answered correctly.
[Trial 3/ Target:
"dustpan"
[Speaker] Please pick the flat, rectangular thing with the handle that you sweep into. [System] The listener answered correctly.
[Trial 24] Target: "laundry basket"
[Speaker]
Figure 10: Example model query for the text-only reference game.
[Reference Document]
"Beginning Summer 2018, there will be new rules regarding dealer record keeping. [1] To register as retail dealer licensed by the Department of Motor Vehicles in New York State consult the DMV Commissioners Regulations for Motor Vehicle Dealers and Transporters ...
DMV does not provide official dealer signs. You may purchase these signs from various sign shops and automotive supply stores. Your sign needs red background and white lettering to measure at least 3 feet wide by 2 feet tall display REGISTERED Facility Number STATE OF NEW YORK MOTOR VEHICLE DEALER in two inch block lettering,
[Reference Span]: "Your sign needs red background and white lettering to measure at least 3 feet wide by 2 feet tall"
Generate the next utterance based the reference document and the specific excerpt of the conversation (the "reference span"). The reference span likely has information about the solution to the user's 's query query/requee reeuust.
[Conversation]
User: We want to open a snowmobile dealership Assistant: Would you like to know how to apply for a snowmobile dealer business certificate?
User: Yeah.
Last question, what does official registered dealer sign have be size-wise? Assistant:
User:
Example model query for the document-groundee utterance completion task. A good model should refer to the "official registered dealer sign" with a more concise phrase.
Figure 11:
21

---

Published as a conference paper at COLM 2025
[Context]
We want to open a snowmobile dealership Assistant: Would you like know how to apply for snowmobile dealer business certificate? User: Yeah.
User:
Last question, what does official registered dealer sign have be size-wise? Assistant:
User:
[Completion A] Your sign needs to be at least 3 feet wide by 2 feet tall
[Completion Your official registered dealer sign needs be least feet...
Above are two candidate completions for the agent's turn. Which completion shows stronger convention formation? Convention formation manifests when the referring expression for an item group of items becomes more concise after the item initially mentioned. For example, item initially mentioned as 'the slim ceramic vase that has floral patterns' may later be re-mentioned as 'the floral ceramic vase' or simply 'the ceramic vase."
Consider the items in the [context] that re-mentioned in both completions. The completion that uses more concise phrases (shorter phrases) overall when re-mentioning those items is considered showing stronger convention formation. Remember that a re-mention is the exact noun phrase referring to the item, which includes all the words, phrases, and clauses modifying the noun. These modifiers will also affect the verbosity (conciseness) of the re-mention. Also, the same item be re-mentioned multiple times in completion and the two completions may eventually reach the same level of conciseness for that item. In this case, one completion may still show stronger convention formation by using the more concise referring expression more often.
Your answer should follow one of the following formats: 1) If one completion shows stronger convention formation overall, output its label (A B) and example (evidence) where item is re-mentioned more concisely that completion than in the other.
2) If the completions show the same level of convention formation, output and an example (evidence) where the re- re-mentions in the two completions have similar conciseness.
you cannot find any example (evidence) make judgment, output D.
Return a json of the following format: { ("initial mention": "", "re-mention in Completion A"; "* "re-mention in Completion B": "", "stronger convention formation in": "<A, B, C, or D>").
Figure 12: Example query for GPT40 judge in document- docuuent--rouunded grounded utterance completion.
22

---

Published as a conference paper at COLM 2025
E.2 Interaction Examples
We present positive examples and various negative examples in the following figures. Each example represents an interaction and shows how the messages for each item evolve across 6 repetitions. We highlight in red the messages that led to the listener's 's mistake. If a message is not highlighted in red, it means the listener correctly identified the target when seeing the message. Unless otherwise specified, all the negative examples represent issues found across the two model families (Gemma and Llama) and in both post-trained models and their off-the-shelf versions.
Figure 13 and Figure 14 present positive examples for convention formation. From Rep- etition to 6, the messages show substantial shortening overall, and the model rarely introduces new words refer the same item. The listener consistently selects the target correctly.
Among the negative examples, the most conspicuous type is where the model does not show any shortening and just repeats the initial messages (Figure 15). Another important type of failure found in all models using a message that is relevant but not specific enough (Figure 16). We also found issues only observed in some models. The first one, found in Llama (both post-trained and original) and Gemma (post-trained), is that the model occasionally produces message that refers to another item in the context rather than the target (Figure 18). The second one is specific to the post-trained Gemma, where the model occasionally hallucinates the location of the target items (Figure 19).
Bowl
Butter Dish
1. The round, white thing that is bigger than the plates smaller the pot
1. The small, square, white thing that holds the yellow stuff
2. The square, white thing that holds the yellow food
2. round thing that bigger than the plates but smaller than the pot
3. The square, white thing holds the yellow food
4. square, white thing
3. The round thing that is bigger than the plates but smaller than the pot
5. square thing
4. The round thing
6. The square thing that holds the yellow food
5. The round thing
6. The round thing
Dishes
Kettle
1. The set of plates and cups
1. The metal thing with the handle that you put water in
2. The plates and cups
2. The metal pot with the handle
3. The plates and cups
3. The pot that you put water
The plates cups
The pot
5. The plates and cups
5. pot with the handle
6. The plates and cups
6. The put
Figure 13: Example showing improved efficiency via convention formation. Model: Gemma (Ours).
23

---

Published as a conference paper at COLM 2025
Coffee Filter
Coffee Machine
1. The thing you separate liquid from solid particles
The device that makes hot drink your kitchen
2. The thin material that holds back small particles
The device that makes drink from bean
3. The thin thing that catches small particles
The device that makes hot drink
4. The thin thing that holds back particles
4. The device that makes hot drink
5. The thin thing that catches particles
The device that makes hot drink
6. The thin thing that catches particles
The device that makes hot drink
Tea Pot
Washing Machine
object you use make
The device that cleans clothes in your home
object that hot liquid
The appliance that uses water to clean clothes
3. The object that holds hot drink
The device that cleans clothes with water
4. The object that holds liquid
The device that cleans clothes in water
5. The object that holds hot liquid
The device that cleans clothes in water
6. The object that holds liquid
The cleans clothes
Figure 14: Example showing improved efficiency via convention formation. Model: Llama
(Ours).
Bowl
Butter Dish
1. The round, open container, bigger than cup
The small container with used holding spread
2.
The round, open container, bigger than cup
The small container with lid, used for holding spread
3. The round, open container, bigger than cup
The small container with lid, used for holding spread
4. The round, container, bigger than
4. The small container with a lid, used fur hulding spread
5. The round, open container, bigger than cup
The small container with a lid, used for holding spread
6. The round, open container, bigger than cup
The small container a lid, used holding spread
Dishes
Kettle
1. The set of plates and cups
The thing you use boil water
plates and cups
The thing you use boil water
3. The set of plates and cups
The metal container with spout, used for boiling water
4. The set of plates and cups
The metal container with spout, used for boiling water
5. The set of plates and cups
5. The thing you use boil water
6. The set of plates and cups
The metal container with spout, used for boiling water
Figure 15: Example showing repetition and no convention formation. Model: Gemma (Ours).
24

---

Published as a conference paper at COLM 2025
Laundry Basket
Cleaning Bucket
big, open container with handle
The yellow sponge inside
handle and blue
2. The big, open container with the handle
3. The big, open container
2.
The yellow container with the sponge
The yellow container
4. The container with the handle
The yellow container with the sponge
5. The container with the handle
The yellow
6. The container with the handle
The yellow container
Dustpan
Kitchen Broom
1. The flat, rectangular thing with the handle that you sweep into
The long, thin thing with the bristles end
The brown, long thing with the bristles
2. The rectangular thing with the handle
3. The brown, long thing with the bristles
3. The thing sweep into
4. The brown, long thing
4. The rectangular thing
The brown, long thing with the hristles
5. The rectangular thing with the handle
The brown, long thing
6. The rectangular thing
Figure 16: Example interaction that started with an ambiguous referring expression. The messages still evolved the game progressed. Model: Gemma (Ours).
Light Bulb
Satellite
1. The thing you turn on in a room to see in dark
The object that orbits the earth and sends signals back to
2. thing that glows when you turn on switch
object that you can see sky night
3. The thing that is often used lamp
The object that is launched into space from rocket
4. The thing that is often used in ceiling fixture
The object that is used for navigation
5. The thing that is used in socket
The object that is launched from rocket
6. The thing that is used in lamp
6. The object that is in arbit around the earth
Smartphone
Solar Cell
1. device you use to people play games
1. The device that converts energy from the sun intu elec- tricity
The device you watch videos and listen music
The thing that powers your phone and other small de- vicus
3. The device that you take pictures and videos
4. The device that you
access the internet
The thing that used to charge your phone
5. The device that you use to make calls
The thing that used to power a hause
6. The device that you use to watch videos
The thing that used to charge battery
The thing that used power house
Figure 17: Example showing inconsistent messages and no convention formation. Model:
Llama (Ours).
25

---

Published as a conference paper at COLM 2025
Steam Iron
Cutting Board
1. The thing you hang your shirts to dry
The flat surface you chop food on in the kitchen
2. The tool you use to make your shirt smooth
kitchen The surface you chop food in the kitchen
The thing you chup food
3. The tool you use get rid wrinkles
4. The tool you use make your shirt smooth
The thing you chop food
the kitchen
tool you use make your smooth
The thing you chop food on in the kitchen
6. The tool you make your shirt smooth
The thing you chup food on in the kitchen
Troning Board
Clothesline
thing you use to get
your
1.
The long string you hang your wet laundry on outside
2. The surface you put your hot tool
The thing you hang your dirty laundry on to dry
3. The flat surface you put your hot tool
The thing you hang your laundry on outside
4. The flat surface you put your hot tool on
The thing you hang your laundry on outside
5. The flat surface you put your hot tool on
The thing you hang your laundry outside
6. The flat surface you put your hot tool on
The thing you hang your laundry outside
Figure 18: Example showing the model erroneously referring to non-target. The first message for the steam iron is erroneously referring the clothesline. The first message for the ironing board is also more suitable for the steam iron. Model: Llama (ours).
Gas Turbine
Light Bulb
1. The thing that uses spinning wheel to produce power
The thing that produces a bright white glow
2. The thing that uses combustion engine
The thing that screwed socket
3. The thing that is in the power plant x
The thing that the lamp
4. The thing is the second biggest
The thing that in the ceiling
thing is
pipes
5.
The thing that in the middle
6. The thing that is front of the control panel
The thing that in front of the pipes x
Washer Dryer
Generator
1. appliance that clothes
The thing that produces power for the other items
2. The appliance that the laundry
The thing that connected the power grid
3. The appliance that is next the sink
The thing that next transformer
4. The appliance that is on the floor
The thing that biggest
5. The thing that is on the left
The thing that next the control panel
6. The thing that is on the right x
The thing that behind the pipes
Figure 19: Example showing the model hallucinating the target's location poosiiiooo..nn..oo The model hallucinates locations such as next to the pipes, on the left, and in the middle, which confuse the listener. Model: Gemma (Ours).
26


Reviews:
[{"review_id": "Tjpf1Nzl6R", "reviewer": "Reviewer_XAPw", "summary": "This paper studies how to make large language models speak more like humans by learning to simplify their words over time in a conversation. The authors create a special training method that teaches models to reuse shorter phrases when talking about the same thing again. They also design two new tests to check if the model learned this behavior, and show big improvements. The work is original, well-structured, and addresses a problem not many others focus on. It is clear, thoughtful, and could inspire better communication in AI systems. However, the model does not truly understand why it should form conventions, and the training method is an add-on rather than built into the model's core training.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper gives a new and useful way to help language models talk more like humans by forming shorter and clearer phrases.\n- It shows strong results on two tasks and may help future research on better model communication.", "reasons_to_reject": "The proposed method helps the model speak more simply, but the model does not really understand why it should do this. It could just copy the patterns from training data without real reasoning. The authors do not discuss if the model can choose between using short or full expressions based on the situation. For example, in some cases, the user needs a full and clear answer to avoid confusion. The paper also does not check if the model can switch back to using full expressions when needed, such as when the short form is ambiguous. This is important for real-world use, but it is not evaluated in the paper.", "questions_to_authors": "1. In Figure 2 (right), the post-trained models do not reach the best accuracy compared to GPT-4o and Claude, even though they improve over time. Can you explain why your method leads to lower early accuracy? \n\n2. Can your model decide when to use a short reference and when to give a full description, depending on the context? For example, if the short form is unclear or the user needs more detail, can the model switch back to using a full expression? This ability seems important in real-world use, but it is not discussed or evaluated in the paper.", "extra_scores": {}}, {"review_id": "xd0v7ggQCR", "reviewer": "Reviewer_uQr9", "summary": "The paper investigates the ability of off-the-shelf LLMs to participate in human linguistic convention formation, more specifically the gradual shortening of referring expressions over the course of a dialog. A fine-tuning method using human data is proposed, implemented, and shown to lead to significant improvements.", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The topic is very relevant and has not been sufficiently studied. The paper is very clear and the work described is well carried out.", "reasons_to_reject": "None.", "questions_to_authors": "- Line 153, is (x, y1, yw) in the wrong order? (x, yw, y1) is used elsewhere,\n\n- Line 214: \"The second task is based on the Doc2Dial dataset (Feng et al., 2020), where a user is querying an agent in a document-grounded interaction.\" - is this referring to a human agent or an LLM (or some other type of articificial agent)? If the latter, how come it exhibits convention formation capabilities?\n\n- Line 235: The win-rate judging model should be confirmed to agree with human judgements. Suggest to add this as future work. \n\n- Line 244 onwards: In the data collection process, human users interact with an LLM, which (as shown in the paper) does not handle concept formation. Could this affect the data collected in some way? If not, why not?\n\n- Line 252 onwards: How do you know for sure that all deviations from the expected reduction in WND was due to participants misunderstanding the instructions? Also, it is not entirely clear what the misunderstanding consists in. As it is, one may wonder if there may be some variation in human convention formation capabilities that is being disregarded here.\n\nFinally, could it be worth trying zero shot prompting as a method to enable convention formation?", "extra_scores": {}}, {"review_id": "6zV8EHJcRQ", "reviewer": "Reviewer_He5h", "summary": "This paper proposes a post-training method for efficient multi-turn conversation. \nFocusing specifically on efficient reference convention formation, the authors first construct a dataset for coreference mention preference, and then formulate a new training strategy with newly introduced “mention planning tokens” and regularized SFT for the tokens.\nFor the evaluation, the authors used two tasks, text-only reference game, and document-grounded utterance completion, to investigate the effectiveness of the proposed method. \nHuman interaction data was also collected to quantify the communication efficiency of the current LLMs in the convention formation task.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The convention formation task is novel and will help to draw the community's attention to the issues of measuring and enhancing the LLMs’ naturalness in multi-turn conversations.", "reasons_to_reject": "While the reviewer appreciates the challenge and importance of the task,  the preferable approach was not necessarily clear for me.Table 2 shows that shortening the reference convention can make the conversation more efficient but can also reduce accuracy and success rates. Detailed explanation/analysis would be helpful", "questions_to_authors": "The dataset construction in section 3 seems to be based on some intuitive observations. It was not clear whether or not there is any bias in the dataset that makes the dataset different from the real human conversation. The claim of the paper is that the effectiveness of the model can be shown extrinsically by its effectiveness on other out-of-domain tasks. However, if the reference convention patterns can be manually analyzed and categorized, we may be able to gain deeper insights into what actually  happens in the human and model conversations. \n\nThe experiments exclude the cases of partial matches (e.g. dustpan --> pan), but in reality, this could happen frequently in daily conversation. What is the motivation for excluding these cases?", "extra_scores": {}}, {"review_id": "keSHhiZvyF", "reviewer": "Reviewer_Usy9", "summary": "This paper examines LLMs' capability of convention formation (CF) -- whether LLMs can learn to use simplified mentions to facilitate efficient multi-turn interactions. The authors propose a DPO-based pairwise training algorithm to improve CF for LLMs by encouraging LLMs to generate simplified mentions while discouraging the exact copy. They evaluate CF using both synthetic reference games (measuring utterance length and word distance) and a realistic conversational document QA benchmark, judged by GPT-4o. Results show that while off-the-shelf LLMs struggle to adopt simplified mentions and enhance conversational efficiency, the proposed training method effectively addresses this limitation. Ablation studies further validate the contribution of each algorithmic component. Further ablation study confirms the effectiveness of each component in the algorithm.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed convention formation task provides a novel interesting angle to evaluate LLM multi-turn interaction capability. The evaluation setup for both synthetic games and realistic applications is well-considered. \n\n2. The proposed DPO-based training method, while simple, yields significant improvements over baselines.", "reasons_to_reject": "1. **Debatable formulation**: The central assumption—that using simplified re-mentions improves multi-turn interaction efficiency—may not universally hold. In cases where acronyms are introduced abruptly, listeners may lack sufficient context to interpret them correctly. Similarly, in formal or high-stakes scenarios, omitting parts of important entity mentions could increase the risk of misinterpretation, potentially undermining clarity and hurting the interaction efficiency. There should be more discussions related to this problem. \n\n2. **Weak baselines**: To establish the necessity of training to improve models' capability of handling CF, the paper should consider stronger inference-time baselines. For instance, can we mitigate CF problems by providing just a few in-context samples in the curated datasets or by carefully designed system prompts for LLMs? LLMs are not explicitly trained to do well on the proposed task, and it is thus expected direct prompting might lead to performance degradation, but with a few prompting tweaks it might work. \n\n3. **Lack of in-depth analysis**: Although the proposed methods show promise based on automatic metrics, it is not clear what happened behind those performance numbers. For example, what would be common strategies that LLMs learn to reduce message length? Do they favor pronouns, drop adjectives, invent acronyms, or employ other tactics?  More in-depth qualitative or quantitative analysis would further strengthen this paper.", "questions_to_authors": "1. It is not that clear to me what is the \"Accuracy\" of text-only reference games (Figure 2, the third subfigure) -- what does it measure, and why it is relevant?\n\n2. Though perhaps not in a multi-round interaction setting, there should exist many evaluation environments related to reference games and featuring speaker-listener setup in LLM/NLP literature (e.g., theory-of-mind, pragmatics). What would be the reason that we need to build a new reference game setup? And what would be the main difference between the proposed evaluation tasks to existing related textual tasks?", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper offers a post-training approach to make large language models improve their communication efficiency over the course of multi-turn dialogues. Specifically, the approach aims to get models to follow a human-like approach of slowly making the referring phrases for an object shorter over time (e.g., “The round, white thing that is bigger than the plates but smaller than the pot” to “the round thing”). The post-training approach uses realistic training data that shows examples of these patterns in practice. Then the work applies the method to two base models and demonstrate improvements on two tasks inspired by prior literature.

All reviewers highlighted notable strengths of this work including:

    Novelty and importance of the research area (uQr9, He5h, Usy9), and that it may inspire community attention and future work in this area (XAPw, He5h)
    The technical approach was original (Usy9), well-structured (Usy9), well-executed (uQr9), and ultimately yielded strong experimental results (XAPw, Usy9)
    Solid evaluation methodology (XAPw, Usy9)
    Clear writing (XAPw, uQr9)

Reviewers mostly rated the paper as marginally above the acceptance threshold due to some remaining concerns:

    Efficiency may not equal improved task accuracy or success (He5h, Usy9, XAPw). Shortening may in some cases reduce task success as short/simple references may not always be appropriate or clear enough (humans may flexibly switch between short and long based on the context, but it is unclear if the approach can handle this).
    May be difficult to generalize for real world use given the post-training “add on” approach as it may not be sustainable to train new models for new situations (XAPw, Usy9)
    Lack of detailed analysis to inform future work (Usy9, He5h)
    Lack of knowledge about model’s understanding vs. just pattern matching (XAPw)
    Missing stronger inference-time baseline (e.g., simple prompting such as requesting the model be as concise as possible) (Usy9)

While some of these issues (1, 2, 4) are a bit more fundamental to the work and difficult to fix, the authors in the discussion promised to include more detailed analysis in the revised version (3) and addressed the point about the baseline (5). I agree with the reviewers that the proposed changes will make this paper stronger, and I find the paper to be an interesting and well-executed investigation into communication efficiency that is likely to inspire future work. I recommend accepting the paper.
Decision: Accept