SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents
Authors: Zhuoshi Pan
Qianhui Wu
Huiqiang Jiang
Xufang Luo
Hao Cheng
Dongsheng Li
Yuqing Yang
Chin-Yew Lin
H. Vicky Zhao
Lili Qiu
Jianfeng Gao
Venue: ICLR 2025 Poster (2025)
Keywords: memory management
conversational agent
RAG
text segmentation
prompt compression
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A system facilitates long-term conversational agents by constructing a memory bank at segment level while applying compression-based denoising to enhance memory retrieval.

Abstract: To deliver coherent and personalized experiences in long-term conversations, existing approaches typically perform retrieval augmented response generation by constructing memory banks from conversation history at either the turn-level, session-level, or through summarization techniques.
In this paper, we explore the impact of different memory granularities and present two key findings: (1) Both turn-level and session-level memory units are suboptimal, affecting not only the quality of final responses, but also the accuracy of the retrieval process.
(2) The redundancy in natural language introduces noise, hindering precise retrieval. We demonstrate that *LLMLingua-2*, originally designed for prompt compression to accelerate LLM inference, can serve as an effective denoising method to enhance memory retrieval accuracy.

Building on these insights, we propose **SeCom**, a method that constructs a memory bank with topical segments by introducing a conversation **Se**gmentation model, while performing memory retrieval based on **Com**pressed memory units.
Experimental results show that **SeCom** outperforms turn-level, session-level, and several summarization-based methods on long-term conversation benchmarks such as *LOCOMO* and *Long-MT-Bench+*. Additionally, the proposed conversation segmentation method demonstrates superior performance on dialogue segmentation datasets such as *DialSeg711*, *TIAGE*, and *SuperDialSeg*.

Published as a conference paper at ICLR 2025
ON MEMORY CONSTRUCTION AND RETRIEVAL FOR PERSONALIZED CONVERSATIONAL AGENTS
Zhuoshi Pan'! Qianhui Wu Huiqiang Jiang" Xufang Luo", Hao Cheng',
Dongsheng Li², Yuqing Yang', Chin-Yew Lin", H. Vicky Zhao' Lili Qiu", Jianfeng Gao²
1 Tsinghua University,
2 |Microsoft Corporation
ABSTRACT
To deliver coherent and personalized experiences in long-term conversations, ex- isting approaches typically perform retrieval augmented response generation by constructing memory banks from conversation history at either the turn-level, session-level, through summarization techniques. In this paper, present two key findings: (1) The granularity of memory unit matters: Turn-level, session- level, and summarization-base based methods each exhibit limitations in both memory retrieval accuracy and the semantic quality of the retrieved content. (2) Prompt compression methods, such LLMLingua-2, 2, can effectively serve as a denoising mechanism, enhancing memory retrieval accuracy across different granularities.
Building on these insights, we propose SECOM, method that constructs the memory bank at segment level by introducing a conversation SEgmentation model that partitions long-term conversations into topically coherent segments, while applying Compression based denoising on memory units to enhance memory retrieval. Experimental results show that SECOM exhibits significant performance advantage over baselines on long-term conversation benchmarks LOCOMO and Long-MI- MM---Bnch+t...... Additionally, the proposed conversation segmentation method demonstrates superior performance on dialogue segmentation datasets such DialSeg711, TIAGE, and SuperDialSeg.
INTRODUCTION
Large language models (LLMs) have developed rapidly in recent years and have been widely used in conversational agents. In contrast to traditional dialogue systems, which typically focus on short conversations within specific domains (Dinan al. 2019), LLM-ppwered conversational agents engage in significantly more interaction turns across a broader range of topics in open-domain conversations (Kim et al. 2023 Zhou 2023). Such long-term open-domain conversations over multiple sessions present sigmificant challenges, as they require the system to retain past events and user preferences deliver coherent and personalized responses (Chen et al. 2024).
Some methods maintain context by concatenating all historical utterances or their summarized versions (LangChain Team] 2023a; |Wan et al.) (023). However, these strategies can result in excessively long contexts that include irrelevant information, which may not be relevant to the user's current request. As noted by Maharana et al. (2024), L.MMs struggle with understanding lengthy conversations and grasping rongerange temporal and causal dynamics, particularly when the dialogues contain irrelevant information (Jiang et al. 2023c). Some other works focus on retrieving query-related conversation history to enhance response generation (Yuan al. [203 Alonso et al. 2024 Kim et al., 2024] Maharana et al. 2024). These approaches typically construct memory bank
from the conversation history at either the turn-level (Yuan et al.] (023) session-level (Wang et al.,
2023). Chen et al. (2024), Li et al. (2024) and Zhong et al. (2024) further leverage summarization
Techniques build memory units, which are then retrieved context for response generation.
"Wor during internship Microsoft. *Corresponding author.
Projecc page: https://aka.m//scoom

---

Published as a conference paper at ICLR 2025
Building these works, key question arises: Which level of memory granularity tettaartttttttt-eeeeeee -trn-level, session-level, or their summarized forms -yield the highest effectiveness? Moreover, is there a novel memory structure that could outperform these three formats?
In this paper, we first systematically investigate the impact of different memory granularities on conversational agents within the paradigm of retrieval augmented response generation (Lewis et al. 2020; Ye et al., 2024). Our findings indicate that turn-level, session-level, and summarization- methods all exhibit limitations in terms of the accuracy of the retrieval module as well the semantics of the retrieved content, which ultimately lead to sub-optimal responses, as depicted in Figure
Figure 2 and Table I
History Session (contain Irrelovant information)
Segment-11 [0.37]
What prosacutor's follacy?
The prosecutor's fallacy logical fallacy when the probsbiility of certain event misinterpreted,
Turn-1 [0.41)
Current User Request
Can you summarize the the cognitive error we just discuss?
Givv an exampia?
Turn-2
An examppe of is...
Segment-2 [0.49]
Retrieve Memory...
What about base rate fallacy?
1
Tum-3 3
The base rate fallacy is common cognitive error when
What consequences will cause?
The
fallacy can lead poor decision-
Turn-4 [o.32]
making in fields like medicine, law..
Summary
Reason for this fallacy?
We discussed the prosecutor's fallacy...
The base rate fallacy arise from several cognitive reasoning tendencles:...
Turn55 [0.45]
We talked about fatlacy, which is cognitive error where people ignore general prevalence in favor of specific information, leading to poor decision-making, (lose detailed reasons after summarizaionn
rate
Segmentt3300 (0.41)
List some reasons of the World War II.
1
Here are some key reasons for World War
7um-660.449)
Expansionist Pulicies...
Finally, touched the causes of World War II.
Summmrrize these reasons into short conclusion.
[0.37]
World War il was triggered by
Figure 1: Illustration of retrieval augmented response generation with different memory granularities. Turn-level memory is too tine-graine, leading to fragmentary and incomplete context. Session-lewe levell memory too coarse-grained, containing too much irrelevant information. Summary based methods suffer from information loss that occurs during summarization. Ours (segment-level memory) can better capture topically coherent units in long conversations, striking a balance between including more relevant, coherent information while excluding irrelevant content. Bullseye @ indicates the retrieved memory units turn level or segment level under the same context budget. [0.xx]: similarity between target query and history content. Turn-level retrieval errors: false negative
false positive
Specifically, users often interact with agents over multiple turns to achieve their goals, causing relevant information be dispersed across multiple interactions. This dispersion can pose great challenge to the retrieval of turn-level memory units as some of the history conversation turns may not explicitly contain or relate keywords mentioned in the current request (e.g., Turn-5 in Figure I As result, the retrieved contexts (e.g., Turn-3 and Turn-6 in Figure [ can be fragmentary and fail to encompass the complete request-related information flow, leading to responses that may lack coherence or omit essential information. On the other hand, single conversation session may cover multiple topics, especially when users do not initiate a new chat session upon switching topics. Therefore, constructing memory units at the session level risks including irrelevant content (e.g., definition of the prosecutor's 's fallacy and reasons of World War in Figure I Such extraneous content in the session-level memory unit may not only distract the retrieval module but also disrupt the language model's comprehension of the context, causing the agent to produce responses that are off off-topic or include unnecessary details.
2

---

Published as a conference paper at ICLR 2025
8.5
16.5
Diouuniet Con-uitto C4th (D00))
Crmsladee Galts (bCO)
0.66
BLEU BOWKBS
8.0
-1.8
9.63
0.3
3.60
0.57
Sepment-Levd
Segtnent Leve Man.
Turn Level Men.
0.65
-8h
Turn-Lavet Mem.
9.55
-4-
Session Lavel Man.
-4o
Sescus- Luree Nam.
0.40
0.27
25
5
10
Resteved
15
20
9.25
(a) Response quality as function (b) Retrieval DCG obtained with (c) Retrieval DCG obtained with chunk size, given total budget different memory granularities us- different memory granularities us- of 50 turns to retrieve as context.
ing BM25 based retriever.
ing MPNet based retriever.
Figure The impact memory granularity on the response quality (a) and retrieval accuracy (b, c).
Long conversations are naturally composed of coherent discourse units. To capture this structure, we introduce conversation segmentation model that partitions long-term conversations into topically coherent segments, constructing the memory bank at the segment level. During response generation, directly concatenate the retrieved segment-level memory units as the context as in Yuan et al. (2023 Kim et al. (2024), bypassing summarization to avoid the information loss that often occurs when converting dialogues into summaries (Maharana et al. 2024).
Furthermore, inspired by the notion that natural language tends to be inherently redundant (Shanmon 1951 Diang et al. 2023b Pan et al. 2024), hypothesize that such redundancy can act as noise for retrieval systems, complicating the extraction of key information (Grangier et al.) 2003) Ma et al. 2021). Therefore,
propose removing such redundancy from memory units prior retrieval by leveraging prompt compression methods such as LLMLingua-2 (Pan et al. 2024). Figure shows
the results obtained with a BM25 based retriever and MPNet based retriever (Song al. 2020) on
Long-MM-Bench+..... As demonstrated in Figure and Figure 36] LLMLingua-2 consistently improves
retrieval recall given different retrieval budgets K (i.e., the number of retrieved segments) when the compression rate exceeds 50%. Figure further illustrates that, after denoising, similarity between the query and relevant segments increases, while the similarity with irrelevant segments decreases.
22.5
Rwiement
Irminrant
87.5
Jigbett
0,005
00280
X=2
0.905
02205
-0- Ke]
-6
Km2
4
Kad
0,984
0,283
0.984
0.280
0,807
60%
Comprosion Ruto (CR)
80%
100%
50%
00%5
Ounppe csaio Sate (CR)
50%
90%
Compression Bala
(a) Retrieval recall V.s. compres-
(h) Retrieval recall v.s. compres-
(c) Similarity hetween the query and different dialogue segments. Blue: rel- evant segments. Orange: irrelevant
sion rate:
lukens after compression
sion rate:
#iokens after compressiun
likkess hofore compression
takens before compression
K: number of retrieved segments. Retriever: BM25
number of retrieved segments.
Retriever: MPNet
segments. Retriever: MPNet
Figure 3: Prompt compression method (LLMLingua-22 2) serve effective denoising technique to enhance the memory retrieval system by: (a) improving the retrieval recall with varying context budget K; (b) benefiting the retrieval system by increasing the similarity between the query and relevant segments while decreasing the similarity with irrelevant ones.
Our contributions can be summarized as follows:
We systematically investigate the effects of memory granularity on retrieval augmented response generation in conversational agents. Our findings reveal that turn-level, session- level, and summarization based approaches each face challenges in ensuring precise retrieval and providing complete, relevant, and coherent context for generating accurate responses.
3

---

Published as a conference paper at ICLR 2025
We contend that the inherent redundancy in natural language can act noise for retrieval systems. We demonstrate that prompt compression technique, LLMLingua-2, can serve as effective denoising method to enhance memory retrieval performance.
We present SECOM, system that constructs memory bank at segment level by introducing a conversation SEgmentation model, while applying CoMpression based denoising on memory units to enhance memory retrieval. The experimental results show that SECOM outperforms baselines on two long-term conversation benchmark LOCOMO and Long-MT- Bench+. Further analysis and ablation studies confirm the contributions of the segment-level memory units and the compression-basee denoising technique within our framework.
SECOM
2.1 PRELIMINARY
Let H = (c represent the available conversation history between a user and an agent, which consists of C sessions.
{t,1 1 denotes the i-th session that is composed of T; sequential user-agent interaction turns, with each turn tj Ty) consisting of user request Vj and the corresponding response from the agent rj. Denote the base retrieval system as In and the response generation model as fLLM. The research framework here can be defined as: (1) Memory construction: construct a memory bank M using conversation history H: For a turn-level memory bank, each memory unit m E .M corresponds to an interaction turn f, with [M] = =Σ= Tü For a session-level memory bank, each memory unit corresponds to a session c, with |M = C. (2) Memory retrieval: given a target user request u² and context budget N, retrieve N memory units {m,, .M) + fR(u", M, N) that are relevant to user request ut: (3) Response generation: take the retrieved N memory units in time order as the context and query the response generation model for response r* fLLM("", (m.))))).
In the remainder of this section, we first elaborate on the proposed conversation segmentation model that splits each session into K, topical segments { in Section 2.2 with which we construct segment-level memory bank with each memory unit m corresponding to segment s and [M = Σ Kg. In Section 2.3 we describe how to denoise memory units to enhance the accuracy of memory retrieval.
k=1
2.2 CONVERSATION SEGMENTATION
Zero-shot Segmentation Given a conversation session c, the conversation segmentation model fr aims to identify set of segment indices I = {pkk, gs) ) where K denotes the total number of segments within the session c, Pk and represent the indexes of the first and last interaction turns for the k-th segment 8k. with Pk S 9k- Pk+1 Q 1. This can be formulated as:
(1) However, building segmentation model for open-domain conversation is challenging, primarily due to the difficulty of acquiring large amounts of annotated data. As noted by Jiang et al. (2023d), the ambiguous nature of segmentation points complicates data collection, making the task difficult even for human annotators. Consequently, we employ GPT-4 as the conversation segmentation model fr leverage its powerful text understanding ability across various domains. To provide clearer context and facilitate reasoning, we enhance session data c by adding turn indices and role identifiers to each interaction t, as: "Tum j: |n[user]: uy |n[agent]: ry". We empirically demonstrate that segmentation can also be accomplished with more lightweight models, such as Mistral-7B and even RoBERTa scale models, making our approach applicable in resource-constraine environments. Figure 6 in Appendix A.] presents the detailed instruction used for zero-shot conversation segmentation here.
fz(c) = {s where 8 = (lpp Lp. +I.,,
Segmentation with Reflection on Limited Annotated Data When a small amount conversation data with segment annotations is available, we leverage this annotated data to inject segmentation knowledge into LLMs and better align the LLM- based segmentation model with human preferences. Inspired by the prefix-tuning technique (Li & Liang 2021) and reflection mechanism (Shinn et al.) 2023) Renze & Guven 2024), we treat the segmentation prompt as the "prefix" and iteratively optimize through L.L self-reflection, ultimately obtaining segmentation guidance G.
4

---

Published as a conference paper at ICLR 2025
Specifically, in each iteration, we first apply segmentation model in zero-shot manner to batch of conversation data and select the "hard examples", Lee, the top K sessions with the most significant segmentation errors based the WindowDiff metric (Pevzner & Hearst 2002). The LLMM-ased segmentation model is then instructed to reflect on its mistakes given the ground-truth segmentation annotations and update the segmentation guidance G. This process mirrors Stochastic Gradient Descent (SGD) optimization, i.e., Gm+1 Gm 1/VL (Gm). where V (Gm) denotes the gradient of segmentation loss, which we assume is estimated implicitly by the LLM itself and is used to adjust the next segmentation guidance G Gm+1- Figure shows the self-reflection prompt and Figure 8 iillusttatess the final prompt with the learned rubric for segmentation.
2.3 COMPRESSION BASED MEMORY DENOISING
Given a target user request ut and context budget N, the memory retrieval system fR retrieves N memory units {m, E .M from the memory bank M as the context response to the user request u" With the consideration that the inherent redundancy in natural language can act noise for the retrieval system (Grangier et al. 22003, Ma et al., (021). we denoise memory units by removing such redundancy via prompt compression model [Comp before retrieval:
{m, E EM + fn(("f, fComp, NM).
(2)
Specifically, we use LLMLingua-2 (Pan et al. 2024) the denoising function fComp here.
3 EXPERIMENTS
Implementation Details We use SPT-35-Turbo for response generation in our main exper- iment. We also adopt - FI (Jiang al. 2023a) for robustness evaluation across different LLMs. We employ zero-shot segmentation for QA benchmarks and further incorporate the reflection mechanism for segmentation benchmarks leverage the available annotated data. To make our method applicable in resource-constrainee environments, we conduct additional experiments by using strall 1- - ructt 3 and a RoBERTa based model fine-tuned on SuperDialseg (Jiang et al. 2023d). Details for the conversation segmentation such as the prompt and hyper-parameteen are described in Appendix I We use LLMLingua - Pan large (Conneau et al. 2020)
ett al. 2024) with compression rate of 75% and as the base model to denoise memory units. Following Alonso et al. (2024). we apply MPNet taattiiggapeee--oae
(Song al. 2020) with FAISS JJonssn al. 2019) and BM25 (Amati 2009) for memory retrieval.
Datasets & Evaluation Metrics We evaluate SECOM and other baseline methods for long-term conversations on the following benchmarks:
(i) LOCOMO (Maharana al.. 2024), which is the longest conversation dataset to date, with an average of 300 turns with 9K tokens per sample. For the test set, we prompt GPT-4 to generate QA pairs for each session as in Alonso al. (2024). We also conduct evaluation on the recently released official QA pairs in Appendix A.5 A.5
(ii) Long-MT MMTBhhhh++. which is reconstructed from MT-Bench+ (Lu al., 2023), where human experts are invited to expand the original questions and create long-range questions as test user requests. Since each conversation only contains an average of 13.3 dialogue turns, following Yuan et al.) (2023), we merge five consecutive sessions into one long-term conversation. We also use these human-written questions as few-shot examples to prompt GPT-4 to generate long-range test question for each dialogue topic as the test set. More details such the statistics of the constructed Biin-MMT-eenche are listed in Appendix A.7
For evaluation metrics, and BERTScore (Zhang et al., (2020) for basic evaluation. Inspired by (Pan et al. 2023). we employ GPT4Score for more accurate evaluation, where GPT-4-0125 is prompted to assign integer rating from (poor) to 100 (excellent). We also perform pairwise comparisons by instructing GPT-4 to determine the superior response. The evaluation prompts detailed in Figure 12 Appendix A.4
the conventional BLEU (Papineni et al. 2002). ROUGE Lin 2004),
Human evaluation is also conducted, with results summarized in Table 10 in Appendix A. A.
httts::/huggtt
co. ral /wt.staaaaall..mm.mmme. 78- Inst ruct-vo.3
5

---

Published as a conference paper at ICLR 2025
Table I:
Performance comparison on LOCOMO and Long-MI-Bench+. The context budget for memory retrieval is set to 4k tokens (~ 5 sessions, 10 segments, or 55 turns) on LOCOMO and 1k tokens (~ segments, turns) on -ennhhhh-...h
QA Performance
Context Length
Methods
GPT4Score
BLEU
Rougel
Rouge2
RougeL
BERTScore
Turns
Tokens
LOCOMO
Zero History
24.86
1.94
3.72
13.24
85.83
0.00
0
Full History
54,15
6.26
27.20
12.07
22.79
88.06
210.34
13,330
Level (BM25)
65.58
7.05
29.12
13.87
24.21
88.44
49.82
3,657
Tum-Leeel (MPNet)
$7.99
26.61
11.38
54.77
3,288
Session Level (BM25)
63.16
29.29
14.24
24.29
88.33
55.88
3,619
Session-Levee evel (MPNet)
51,18
5.22
24.23
9.33
19.51
87.45
53.88
3,471
SumMem
53.87
2.87
20.71
6.66
16.25
86.88
4,108
RecurSum
56.25
400
MonditionMem
65.92
3.41
22.28
7.86
17.54
87.23
3,563
MemoChat
65.10
6.76
28.54
12.93
23.65
88.13
1,159
SECOM (BM25, GPT4-Scg)
71.57
31.40
16.30
26.55
88.88
55,52
3,731
SECOM (MPNet, GPT4-Seg)
69.33
7.19
29.58
13.74
24.38
88.60
55.51
3,716
SECOM (MPNet, Mistral-7B-Sog)
66.37
6.95
28.86
13.21
23.96
88.27
55,80
3,720
SECOM (MPNet, RoBERTa-Seg)
61.84
6.41
27.51
12.27
23.06
88.08
56.32
3,767
Long-MT--Bee
Zero History
49.73
4.38
6.98
13.94
84.22
0.00
Full History
63.85
7.51
26.54
12.87
20.76
85.90
65.45
19.287
Turn-Level (BM25)
82.85
11.52
32.84
17.86
26.03
87.03
3.00
1,047
Turn-Level (MPNet)
84.91
12.09
34.31
19.08
27.82
86.49
3.00
909
Session-Level (BM25)
81.27
11.85
32.87
17.83
26.82
87.32
13.35
4,118
Session-Level (MPNet)
73.38
8.89
29.34
14.30
22.79
86.61
13.43
3,680
SumMem
63.42
7.84
25.48
10.61
18.66
85.70
1,651
RecurSum
62.96
7.17
22.53
9.42
16.97
84.90
567
ConditionMem
63.55
7.82
26.18
11,40
19.56
86.10
1,085
MemoChat
85.14
12.66
33.84
19.01
26.87
87.21
1,615
SECOM (BM25, Geg)
86.67
12.74
33.82
18.72
26.87
87.37
2.87
906
SECOM (MPNet, GPT4-Seg)
88.81
34.63
19.21
27.64
87.72
2.77
820
SECOM (MPNet, Mistral-7B-Sep
86.32
12.41
34.37
19.01
26
87.43
2.85
834
SECOM (MPNet, RoBERTa-Seg)
81,52
11.27
32.66
16.23
25.51
86.63
2.96
841
Baselines We evaluate our method against four intuitive approaches and four satte-of-the-an art models. As Figure S indicates, the compression- based memory denoising mechanism can benefit memory retrieval, in the main results, directly compare method to the denoising enhannced turn-level and session-level baselines. (1) Turn-Level, which constructs the memory bank by treating each user-agent interaction a distinct memory unit. (2) Session-Level, which uses each entire conversation session as memory unit (3) Zero History, which generates responses without incorporating any conversation history, operating in zero-shot manner. (4) Full History, which concatenates all prior conversation history as the context for response generation. (5) StanMem (LangChain Team, (023c). which dynamically generates summaries of past dialogues relevant the target user request, summaries context for response generation. (6) RecurSum (Wang et al. (2023), which recursively updates summary using current session and previous summaries, and takes the updated summary of current session the context. (7) ConditionMee (Yuan et al.) 2023), which generates summaries and knowledge for each dialogue turn, then retrieves the most relevant summary, knowledge, and conversation turn the context in response 2023), which operates memories at segment level, but focuses on tuning LLMs for both memory construction and retrieval.
uses these
user request. (8) MemoChat (Lu al.
Main Results As shown in Table and Figure 4 SECOM outperforms all baseline approaches, exhibiting significant performance advantage, particularly on the long-conversation benchmark LOCOMO. Interestingly, there is significant performance disparity in Turn-Level and Session- methods when using different retrieval models. For instance, switching from the MPNet-based retriever to the BM25-based retriever results performance improvements up to 11.98 and 7.89 points in terms of GPT4Score LOCOMO and Long-MT-Benche respectively. In contrast, SECOM

---

Published as a conference paper at ICLR 2025
Ozm- SemMen
WIN
TIE
LOSE
Ogre- Zaro Iiatory
WIN
TI0. LOSE
Oum-
50.49%
24 9398
24 1..9%
Ours- Full
32.724
22.04%
13.24%
WIN
11B
LOSB
VIN
LOBE
43.00%
25.33%
26.22%
Oupi
WIN
116
LOSE
Dati
03.18%
115 23
LOSS
80%
40%
60%
80%
100%
Rate 194
Itat (50
(a) SECOM v.s. statc-of-the-art methods
(b) SECOM (segment-level) v.s. other granularitics
Figure 4: GPT-4 based pairwise performance comparison on LOCOMO with BM25 based retriever.
demonstrates greater robustness in terms of the deployed retrieval system. We attribute this to the following reason: As discussed in Section [ turn-level memory units are often fragmented and may not explicitly include or relate to keywords mentioned in the target user request. On the other hand, session-level memory units contain large amount of irrelevant information. Both of these scenarios make the retrieval performance sensitive to the capability of the deployed retrieval system. However, topical segments in SECOM can strike a balance between including more relevant, coherent information while excluding irrelevant content, thus leading more robust and superior retrieval performance. Table [ and Figure 4 also reveal that summary based methods, such as SumMem and RecurSum fall behind turn-level or session-level baselines. Our case study, Figure [15 and [6 in Appendix [A.6, suggests that this is likely due to the loss of crucial details during the process of converting dialogues into summaries (Maharana et al. 2024), which are essential for accurate question answering. Furthermore, Table [ shows that SECOM maintains the advantage over baseline methods when switching the segmentation model from GPT-4 Mistral-7B. Notably, even with a RoBERTa-based segmentation model, SECOM retains competitive performance compared to other granularity- baselines.
Ablation Study Granularity of Memory Units Figure 2b Figure 2c and Table have clearly
demonstrated the superiority of segment-level memory over turn-level and session memory in terms of both retrieval accuracy and end-to-end QA performance. Figure 5a and Figure 5b further compare QA performance across different memory granularities under varying context budgets. Compression-basee based memory unit denoising was applied in all experiments here to isolate the end-to- end impact of memory granularity on performance. The results show that segment-level memory consistently outperforms both turn-level and session-level memory across a range of context budgets, reaffirming its superiority. Figures 13 and 14 in Appendix A.6 provide detailed case studies.
Segment- Level Memory
Turn-Leva Memory
Session- Level) Memory
90
00
02774Score
CPT4Soou
04
855
83
80
78
80
75-
73
2000
Number Reiziioud Tickerss
4,000
5000
8000
2000
Number of Rutrieved Tokons
4000
6000
(a) BM25 based Retriever
(b) MPNet based Retriever
Figure 5: Performance comparison of different memory granularities with various context budget on Me-nch.--ennc+.....
Ablation Study Compression based Memory Denoising As shown in Table 2 removing the proposed compression based memory denoising mechanism will result a performance drop up to 9.46 points of GPT4Score LOCOMO, highlighting the critical role of this denoising
7

---

Published as a conference paper at ICLR 2025
Table Ablation study on compression- based memory denoising with compression rate of 75% using the MPNNt based retriever.
LOCOMO
Long-MTTBenche
Methods
GPT4Score
BLEU
Rouge2
BERT
GPT4Score
BLEU
Rouge2
BERTScore
SECOM
69.33
7.19
13.74
88.60
88.81
13.80
19.21
87.72 87.44
Denoise
59.87
6.49
12.11
88.16
87.51
12.94
18.73
mechanism: by effectively improving the retrieval system (Figure 3b). it significantly enhances the overall effectiveness of the system.
Mistral-7B Powered Response Generation Table presents the results of SECOM and baselines using Mistral- Beeeall-------e -B-IIntruutt v0. z (Jiang et al. 2023a) as the response generator. Our method demonstrates significant performance gain over other baselines, showcasing its good generalization ability across different LLM- powered conversation agents. Interestingly, although the Mistral-7B here features 32K context window capable of accommodating the entire conversation history, in other words, it is able to include and comprehend the entire conversation history without truncation, the performance of the "Full History" approach still falls short compared to SECOM. This highlights the effectiveness of our memory construction and retrieval mechanisms, which prioritize relevant context and reduce noise, leading more accurate and contextually appropriate responses.
Table Performance comparison on Other settings are the same as Table
using Mistraa-77--nnstuueee
QA Performance
Context Length
Methods
GPT4Score
BLEU
Rougel
Rouge2
RougeL
BERTScore
Turns
Tokens
Full History
78.73
10.25
29.43
14.32
23.37
86.77
65.45
19,287
BM25 Based Retriever
Turn-Level
83. 14
13.60
33.28
19.11
27.32
87.52
3.00
1,047
Session-Level
81.03
12.49
32.39
17.11
25.66
87.21
13.35
4,118
SECOM
89.43
15.06
35.77
21.35
29.50
87.89
2.87
906
MPNet Based Retriever
Turn-Level
85.61
12.78
35.06
19.61
28.51
87.77
3.00
909
Session-Level
75.29
28.65
13.91
22.52
86.51
13.43
3,680
SECOM 90.58
15.80
36.14
21.49
29.94
88.07
2.77
820
To evaluate the conversation segmentation module described in Section 2.2 independently, we use three widely used dialogue segmentation datasets: DialSeg711 (Xu al. 2021), TIAGE (Xie al. 202T), and SuperDialSeg (Jiang al. 2023d). In addition to the unsupervised (zero-shot) setting, we also assess performance in a transfer learning setting, where baseline models are trained on the full training of the source dataset, while our model learns the segmentation rubric through LLM reflection on the top 100 most challenging examples. We evaluate transfer learning only using SuperDialSeg and TIAGE as the source datasets since DialSeg711 lacks a training set. For evaluation metrics, following Jiang et al. (2023d), we use Fl score, Ps [Beeferman al. 1999). Window Diff (WD) (Pevzner Hearst 2002) and the segment score
Evaluation of Conversation Segmentation Model
2 F1 F1 Pk) + WD)
Score
Table 4 presents the results, showing that segmentation model consistently outperforms baselines
in the unsupervised setting. In the transfer learning setting, despite the segmentation rubric being learned from LLM reflection only 100 examples from the source dataset. it generalizes well to
st ra
nst wesstructt-oo ruct-v0.3
Recommended by ICASSP2023 General Meeting Understanding and Generation Challenge https: 7/2023. eeei cas.p.ssppp..... http watamnddhhalllggggeeee
8

---

Published as a conference paper at ICLR 2025
Table 4: Segmentation performances on three datasets. i: numbers reported in Gao et al. (002)). Other baselines are reported in Jiang et al. (2023d). The best performance is highlighted in bold, and the second best is highlighted by underline.
Numbers in gray correspond to supervised setting.
Dialseg711
SuperDialSeg
TIAGE
Methods
Pk. WD
FIf
Scoret
Pkj
WDL
FIf
Scoret
Pk1
WDL
FIf
Scoret
Unsupervised
BayesSeg
0.306 0.350
0.556
0.614
0.433
0.593
0.438
0,463
0.486
0.571
0.366
0.419
Tex:Tiling
0.470 0.493 0.245
0.382
0,441
0.453
0,388
0.471
0.469
0.488
0.204
0.363
GraphSeg
0.412
0.442
0.392
0.483
0.450
0.454
0.249
0.398
0.496
0.515
0.238
0.366
Tex-Tiling-Glove
0.399 0.438
0.436
0.509
0.519
0.524
0.353
0.416
0.486
0.511
0.236
0.369
TextTiling+(CLS
0.419 0.473 0.351
0.453
0.493
0.523
0.277
0.385
0.521
0.556
0.218
0.340
TextTilin+NNPP
0.347 0,360 0.347
0.497
0,512
0.521
0.208
0.346
0.425
0.439
0,285
0.426
GreedySeg
0.381 0.410 0.445
0.525
0.490
0.494
0.365
0.437
0.490
0.506
0.181
0.341
CSM
0.278 0.302 0.610
0.660
0.462
0.467
0.381
0.458
0.400
0.420
0.427
0.509
DialSTART
0.178 0.198
Ours (zero-shot)
0.093 0.103
0.888
0.895
0.277
0,289
0.758
0.738
0.363
0.401
0.596
0.607
Transfer from TIAGE Target
TextSeg
0.476
0.491
0.182
0.349
0,552
0,570
0.199
0.319
0.357
0.386
0.450
0.539
BERT
0.441
0.411
0.005
0.297
0.511
0.513
0.043
0.266
0.418
0.435
0.124
0.349
RoBERTa
0.197
0.210
0.650
0.723
0.434
0.436
0,276
0.420
0.265
0.287
0.572
0.648
Ours (w/ reflection)
0.050
0.056
0.921
0.934
0.265
0.273
0.765
0.748
0.333
0.362
0.632
0.642
Transfer from SuperDialseg to Target
TextSegaud
0.453
0,461
0.367
0.455
0.199
0.204
0.760
0.779
0.489
0.508
0.266
0.384
BERT
0.401
0.473
0.381
0.472
0.214
0.225
0.725
0.753
0.492
0.526
0.226
0.359
RoBERTa
0.241
0.272
0.660
0.702
0.185
0.192
0.784
0.798
0.401
0.418
0.373
0.482
Ours (w/ reffetion)
0.049 0.054
0.924
0.936
0.256
0.264
0.776
0.758
0.318
0.345
0.634
0.651
the target dataset, surpassing the baseline model trained on the full source training set and even outperforming some supervised baselines.
4
RELATED WORKS
4.1 MEMORY MANAGEMENT IN CONVERSATION
Long-term open-domain conversation (Feng et al. poses significant challenges for LLM powered conversational agents. To address this, memory
2020 Xu et al. 2022 Maharana et al. 2024
management (Lu al. 2023 Wang al., 2023 Zhong et al. 2024 et 2024 Li et al. 2024 Zhang et al. 2024) is widely adopted. The core of memory management involves leveraging dialogue history to provide background information, extract persona, understand the user's intent, and generate history-aware responses, For instance, MPC (Lee et al. 2023). Homorryaank (Zhong et al. (2024) and COMEDY (Chen et al. 2024) further summarize past events the conversation history memory records. Methods such as RecurSum (Wang et al. 2023) and ConditionMem (Yuan et al. 2023 consider the memory updating process through recursive summarization.
Inspired by the success of strieval-augmentt retrieval modules into memory management. For example, MSC (Xu et al. (022) utilizes a pre-trained Dense Passage Retriever (DPR) (Karpukhin et al. 2020) model to select the top N relevant summaries. Instead of using retrieval model, MemoChat (Lu et al. 2023) employs an LLM to retrieve relevant memory records. Recently, Maharana et al. (2024) release dataset, LOCOMO, which is specifically designed to assess long-term conversational memory, highlighting the effectiveness of RAG in maintaining long-term memory. Their experiment results indicate that long-context LLMs are prone to generating hallucinations, and summary-only memory results in sub-optimal performance due to information loss.
generation (RAG), many recent works introduce

---

Published as a conference paper at ICLR 2025
4.2 CHUNKING GRANULARITY RAG SYSTEM
Chunking granularity (Duarte et al. 2024 (i.e., how the entire context is segmented into retrieval units) a crucial aspect of RAG systems. Ineffective segmentation can result in incomplete or noisy retrieval units, which impair the retrieval module (Yu et al. 2023) and negatively impact the subsequent response generation (Shi et al. 2023).
Semantic- based chunking strategies Mishra 2023 Antematter Team 2024 Greg Kamradt 2024 use representation similarity to identify topic shifts and decide chunk boundaries. With the advance- ment of LLMs, some studies leverage their capabilities segment context into retrieval units. For instance, LumberChunker (Duarte et al., (0024) segments narrative documents into semantically coherent chunks using Gemini (Team al. 2023). However, existing research mainly focuses on doe-
ument chunking, overlooking conversation chunking. Common chunking practices (LangChain Team 2023b Llamalndex Team 2023) in conversations directly rely on the natural structure (i.e., utterances or dialogue turns) of dialogue to divide conversation into retrieval units.
4.3 DENOISING IN RAG SYSTEM
Recent studies have observed that noise in conversations can negatively impact the retrieval module in RAG systems. For example, COTED (Mao al. 2022) found that redundant noise in dialogue rounds significantly impairs conversational search. Earlier research (Strzalkowski et al. 1998)
Wasson 2002) investigates the use of summaries in retrieval systems. With the advent of LLM, recent approaches (Ravfogel et al.] [0233; [ee et al., (024) denoise raw dialogues by prompting LLMs to summarize. Subsequently, they fine-tune the retriever's 's embedding model to align vector representations of original text with those of generated summaries. However, these methods have several drawbacks: (I) summarization introduces latency and computational costs, whereas dialogue state methods require high-quality annotated data. (2)
the retriever's 's embedding model method. (3) Fine-
limits flexibility and scalability, restricting it from being used as a plug-and wning risks overfitting and catastrophic forgetting (McCloskey & Cohen 1989 Lee al. 2022). potentially impeding domain adaptation and generalization ability of pre-trained retrievers.
5 CONCLUSION
In this paper, systematically investigate the impact of memory gramularity eetrieval-augment response generation for long-term conversational agents. Our findings reveal the limitations turn- level and session-level memory granularities, well summarization-bor based methods. To overcome these challenges, we introduce SECOM, a novel memory management system that constructs a memory bank at the segment-level and employs compression-basee based denoising techniques to enhance retrieval performance. The experimental results underscore the effectiveness of SECOM in handling long-term conversations. Further analysis and ablation studies confirm the contributions of the segment-level memory units and the compression-basee denoising technique within our framework.
REFERENCES
Nick Alonso, Tomás Figliolia, Anthony Ndirango, and Beren Millidge. Toward conversational agents with context and time sensitive long-term memory. arXiv preprint arXiv:2406.000557, 2024.
Giambattist Amati. BM25, pp. 257-260. Springer US Boston, MA. 2009, ISBN 978-0038777- 9. doi: 10. 10.1007/978-0-33 0-387---977------ 9.221....999....... 921. htLpstts:// UB//attpss///ooo.ccc/00000000/70 39940-9 921
Doug Beeferman, Adam Berger, and John Lafferty. Statistical models for text segmentation. Machine learning. 34:177-210, 1999.
Nuo Chen, Hongguang Li, Juhua Huang, Baoyuan Wang, and Jia Li. Compress impress: Unleashing the potential of compressive memory in real-world long-term conversations. arXiv preprint 1rXiv:2402.11975 2024.
10

---

Published as a conference paper at ICLR 2025
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Un- supervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440-8451, Online, 2020. As- sociation for Computational Linguistics. doi: 10. 18 18653/ /tt.222000000mmaiiiaa
URL
mataaaclmaaan...mmm 747
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-powered conversational agents. In International Conference Learning Representations, 2019.
André V Duarte, João Marques, Miguel Graça, Miguel Freire, Lei Li, and Arlindo L Oliveira. Lumberchunker: Long-form narrative document segmentation. arXiv preprint arXiv:2406.17526. 2024.
Song Feng. Hui Wan, Chulaka Gunasekara, Siva Patel, Sachindra Joshi, and Luis Lastras. doc2dial: A goal-oriented document-groundder dialogue dataset. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 11188128, 2020.
Haoyu Gao, Rui Wang, Ting-En Lin, Yuchuan Wu, Min Yang, Fei Huang, and Yonghin Li. Unsu- pervised dialogue topic segmentation with topic-aware utterance representation. arXiv preprint arXiv:2305.02744 2023.
David Grangier, Alessandro Vinciarelli, and Hervé Bourlard. Information retrieval on noisy text. In IDIAP COMMUNICATION, 2003. URL https://///// /ttp//aapp..a// tta.pp...meiiiiii..............a www.enninnnnno.........e. wwggCorr
uSID:3249973
Pegah Jandaghi, XiangHai Sheng, Xinyi Bai, Jay Pujara, and Hakim Sidahmed. Faithful persona-based conversational dataset generation with large language models. arXiv preprint arXiv:2312.10007, 2023.
Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422-446) 2002.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Ann Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023a. htRpst patttp://arxiv.cr g g/abs/ 2310 06825
Huiqiang Jiang. Qianhui Wu, Chin Yew Lin, Yuqing Yang, and Lili Qiu. Limlingua: Compressing prompts for accelerated inference large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13358-13376, 2023b.
Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin- Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Longllmlingua: Accelerating and enhancing 1lms in long context scenarios via prompt compression. arXiv preprint arXiv:2310.0000 10.06839, 2023c.
Junfeng Jiang, Chengzhang Dong, Sadao Kurohashi, and Akiko Aizawa. Superdialseg: A large-scale dataset for supervised dialogue segmentation. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 4086-4101, 2023d.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions Big Data, ((3):3355547) 2019.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing 2020.
(EMNLP), pp.
Seo Hyun Kim, Kai Tzu-iunn Ong, Taeyoon Kwon, Namyoung Kim, Keummin Ka, SeongHyeon Bae, Yohan Jo, Seung-wor Hwang, Dongha Lee, and Jinyoung Yeo. Theanine: Revisiting memory management in long-term conversations with timelne-augmentte response generation. arXiv preprint arXiv:2406. 10996, 2024.
11

---

Published as a conference paper at ICLR 2025
Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Yoo, and Minjoon Seo. Aligning large language models through synthetic feedback. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13677-13700, 2023.
Gibbeum Lec, Volker Hartmann, Jongho Park, Dimitris Papailiopoulos, and Kangwook Lee. Prompted Ilms chatbot modules for long open-domain conversation. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 4536 4534,4554, 2023.
Seanie Lee, Hae Beom Lee, Juho Lee, and Sung Hwang. Sequential reptile: inter-task gradient alignment for multilingual learning. In Tenth International Conference on Learning Representations. International Conference on Learning Representations, 2022.
Seanie Lee, Jianpeng Chen, Joris Driesen, Alexandru Coca, and Anders Johannsen. Effective and efficient conversation retrieval for dialogue state tracking with implicit text summaries. arXiv preprint arXiv:2402.13043 2024.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. Retrieval-augmenter genera- tion for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 9459 9474, 2020.
Hao Li, Chenghao Yang, An Zhang, Yang Deng. Xiang Wang, and Tat-Seng Chua. Hello again! llm-powered personalized agent for long-term dialogue. arXiv preprint arXiv:2406.05925, 2024.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1982-4597, 2021.
Chin- Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74 81, 2004.
Nelson Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguisties. 12:157-173, 2024.
Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu. Memochat: Tuning Ilms to use memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.08233 088299 2023.
Xueguang Ma. Minghan Li, Kai Sun, Ji Xin, and Jimmy Lin. Simple and effective unsupervised redundancy elimination to compress dense vectors for passage retrieval. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2854 2859, 2021.
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of Ilm agents. arXiv preprint 11X1v 2024.
Kelong Mao, Zhicheng Dou, and Hongjin Qian. Curriculum contrastive context denoising for few-shot conversational dense retrieval. Proceedings of the 45th International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 176-186, 2022.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Anurag Mishra. Five levels of chunking strategies in rag notes from greg's video. ht / 7/medium. com //t.a///aaooaoooaaaaaaaaa//////r
les- n- oe
12

---

Published as a conference paper at ICLR 2025
Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Hanlin Zhang, Scott Emmons, and Dan Hendrycks. Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark. In International Conference on Machine Learning, pp. 26837-26867. PMLR, 2023.
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin- Lin, et al. LJmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. arXiv preprint S.Niv:2403.129666 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Lev Pevzner and Marti A Hearst. A critique and improvement of evaluation metric for text segmentation. Computational Linguisties, 28(1):19-36, 2002.
Shauli Ravfogel, Valentina Pyatkin, Amir DN Cohen, Avshalom Manevich, and Yoav Goldberg. Retrieving texts based on abstract descriptions. arXiv preprint arXiv:2305.12517 2023.
Siva Reddy, Danqi Chen, and Christopher Manning. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249-266, 2019.
Matthew Renze and Erhan Guven. Self-reflection 11m agents: Effects on problem- perfor- mance. arXiv preprint Sili:24005066822. 2024.
Claude Shannon. Prediction and entropy of printed english. Bell system technical journal, 30(1): 50-64, 1951.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pp. 31210-31227. PMLR, 2023.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R. Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforeement learning. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023. URL http::////ppeeeeeeeeeeeeeeeeeeee
F1hFcKW6]
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. Mpnet: Masked and permuted pre-training for language understanding. Advances neural information processing systems, 33: 16857-16867, 2020.
Tomek Strzalkowski. Jin Wang, and G Bowden Wise. Summarization-basel query expansion in information retrieval. In COLING 1998 Volume 2: The 17th International Conference on Computational Linguisties, 1998.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et Gemini: family highly capable multimodal models. arXiv preprint arXiv:2312.I 1805, 2023.
Antematter Team. Optimizing reginzzinngettrievallaggmmeee generation with advanced chunking techniques:
A comparative htt.:h/www.....ttatrrr....aaaaor
d E=chaattinn--eccnninneesssstttyt
Greg Kamradt. Semantic httphha........../....o...oo.oaa
2024. http///eeealllllleeere////00/120
LangChain Team. Conversation
http://ww.......c.............a. 1 o 2023a.
LangChain Team. Conversational http://pp////.too....aaaaaao/a/.
utorial a http//_hhat_hhhaee 2023b.
13

---

Published as a conference paper at ICLR 2025
LangChain Team. Conversation summary memory.
http://///tobho..aggaaaaaaooooe 1/ //dcc///mouules//eeeoo/////yyeed http://ttyyesssmmayyy////// summary/ 2023c.
Llamalndex Team. Chat memory buffer. https://doost htw.:///lammmadeeeeeee/////t//t.
peweeeencc//memrr//cceteeeeeeeee memory buffer/
htffeeayyeoryyyyooo/o//// 2023.
Qingyue Wang, Liang Ding, Yanan Cao, Zhiliang Tian, Shi Wang, Dacheng Tao, and Li Guo. Recursively summarizing enables long-term dialogue memory large language models. arXiv preprint arXiv:2308. 15022. 2023.
Mark Wasson. Using summaries in document retrieval. In Proceedings of the ACL-02 Workshop on Automatic Summarization, pp. 27-36, Phildadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 10.3115/1 118162 12.1811....1. URL.httss://aclanntt
gy 3y.org//0220005
Wei Wu, Zhuoshi Pan. Chao Wang, Liyi Chen, Yunchu Bai. Kun Fu, Zheng Wang, and Hui Xiong. Tokenselect: Efficient long-context inference and length extrapolation for llms via dynamic token- level kv cache selection. arXiv preprint arXiv:241 1. 1.02866 2024.
Huiyuan Xie, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu, and Ann Copestake. Tiage: A bench- mark for topic-shift aware dialog modeling. In Findings of the Association for Computational Linguisties: EMNLP 2021, pp. 1684-1699 2021.
Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open-domain conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 5180-5197, 2022.
Yi Xu, Hai Zhao, and Zhuosheng Zhang. Topic-aware multi-turn dialogue modeling, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 14176-14184 2021.
Linhao Ye, Zhikai Lei, Jianghao Yin, Qin Chen, Jie Zhou, and Liang He. Boosting conversational question answering with fine-grained retritval-nnmmmetatioon and self-check. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2301-2305, 2024.
Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu. Chain-of-note: Enhancing robustness in retrieval-augmented language models. arXiv preprint arX1v:2311.09210, 2023.
Ruifeng Yuan, Shichao Sun, Zili Wang, Ziqiang Cao, and Wenjie Li. Evolving large language model assistant with long-term conditional memory. arXiv preprint arXiv:2312.172577 2023.
Saizheng Zhang, Emily Dinan, Jack Urbanck, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have dog, do you have pets too? In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguisties (Volume Long Papers), pp. 2204-2213, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 11153/v11/18-12200 URL http:///aaaaattoooo.ooooooeeo
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. International Conference on Learning Representations, 2020.
Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey the memory mechanism of large language model based agents. arXiv preprint 24w.:::2224..15511..1 2024.
Wanjun Zhong, Lianghong Guo, Qiqi Gao. He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with term memory. Proceedings of the AAAI Conference Artificial Intelligence, 1811))19724-11733 Mar. 2024. doi: 10.16099aaii.v3380 003811..29944. URL https
7/o aaai org
14

---

Published as a conference paper at ICLR 2025
Jinfeng Zhou, Zhuang Chen, Bo Wang, and Minlie Huang. Facilitating multi-ture emotional support conversation with positive emotion elicitation: A reinforcement learning approach. Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties (Volume 1: I Long Papers), pp. 1714-1729, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 18.18653/v 1/2023 awl.oon.............
long. 9
A APPENDIX
A. DETAILS OF CONVERSATION SEGMENTATION MODEL
We use GPT-4-0125 as the backbone LLM for segmentation. The zero-shot segmentation prompt provided in Figure 6 It instructs the segmentation model to generate all segmentation indices at once, avoiding the iterative segmentation process used in LumberChunker (Duarte et al. 2024). which can lead to unacceptable latency. We specify that the output should be in JSONL format to facilitate subsequent processing. To generate segmentation guidance, select the top 100 poorly segmented samples with the largest Window Diff metric from the training set. The segmentation guidance consists of two parts: (1) Segmeniation Rubric: Criteria items on how to make better segmentation. (2) Representative Examples: The most representative examples that include the ground-trute truth segmentation, the model's prediction, and the reflection the model's errors. The number of rubric items is set to 10. To meet this requirement, we divide the top 100 poorly segmented samples into 10 mini-batches and prompt the LLMM-ased segmentation model to reflect each batch individually. The segmentation model is also asked to select the most representative example in each batch, which is done concurrently with rubric generation. Figure 7 presents the prompt used generate rubric. The generated rubric is shown at Fig. 9 and Fig. 10 on TIAGE and SuperDialSeg. respectively. After the segmentation guidance is learned, we utilize the prompt shown in Figure X as few-shot segmentation prompt. For simplicity and fair comparison, we do not use any rubric for conversation segmentation in 1OCOMO and Long-MT-BHe Mong-MT-Bencht.
A.2 ADDITIONAL COST ANALYSIS
Table compares the overall costs involved in memory construction, memory retrieval, and response generation across different methods. The results demonstrate that our method significantly enhances performance compared to the baseline while only slightly increasing computational overhead, and outperforms the MemoChat method in both efficiency and effectiveness.
A.3 THE ANALOGY BETWEEN THE REFLECTION AUGMENTATION AND PREFIX- -UNNIN
When a small amount of conversation data with segment annotations is available, we explore how to leverage this data to transfer segmentation knowledge and better align the LLM--based segmentation model with human preferences. Inspired by the prefix-tunin technique (Li & Liang 2021) and reflection mechanism (Shinn et al. 2023 Renze & Guven 2024), we treat the segmentation prompt as the "prefix" " and iteratively optimize it through LLM self-reflection, ultimately obtaining a segmentation guidance G.
Table 5: Comparison between our method and MemoChal from multiple aspects on Long-MT-Benrae "# In. Token", "# Out. Token" and "Latency" report the number of input / output token and the latency per question, including memory construction, memory retrieval and reponse generation.
Methods
# In. Token
# Out. Token
Latency (s)
GPT Score
Lesvion-Level
3,642
102
2.17
73.38
MemoChat
7,233
229
5.60
85.14
Ours
1,722
135
2.61
88.81
15

---

Published as a conference paper at ICLR 2025
Instruction Part of the Segmentation Prompt (Zero-Shot).
Instruction
Context
**Goal**; Your task is to segment a multi-turn conversation between a user and a chatbot into topically coherent units based on cemantics. Successive user-bet exchanges with the same topic should be grouped segmentation unit, and
into the created when tepic shifts.
segmentation units should
**Datā+e; The input data is a series of user-bot exchanges separated by "An'n". Each exchange consists of a single-turn conversation between the user and the chatbot, started with "[Exchange (Exchange Number) ### Output Format
Output the segmentation results in **JSONL (JSON Lines)** format. Each dictionary represents segment, consisting of one oz more user-bot exchanges on the same topic.
Each dictionary should include the following keys:
+*segment_idax The index this segment, starting from 0. +*start_exchange__nnte
The number of the **frrs*** user-bot exchange in this segrent.
ween_eexchange_nurbrraa The number of the **aas...
user-bot exchange in this segment.
integer indicating the number of user-bot exchanges in this segment, calculated as: ootaat__xxhangg__ubbeee Here is example of the expected output:
**num @onu__echanges**:
" meen__exchange_nuteee
1.
<segmentation> ( "'segmentii"". 0, "start_exx ettart_exchange_nnnee :umbgee___mbbr: 0,
"end_ bechange__nubeet 5, "num_exchanges" 6)
["segment_ _d 1, "start_exchang"""""" stthannge_uubbr"; 6,
"end_ @echangee_nubbes 8, "num_exchanges"; 31
</segmentation>
[[text_to to be _e___ee__seeeee _e__semented]]
# Question
## Please generate the segmentation result from the input data that meets the following requirements:
**No Missing Exchanges**: Ensure that the exchange numbers cover all exchanges in the given conversation without omission.
+oNo Overlapping Exchanges+:: Ensure that successive segments have overlap in exchanges.
sum of wnum_exchangess> across all segments should equal the total number of user-bot exchanges.
**Acourate Countingx::
Provide your segmentation result between the tags:
Output
New, provide the segmentation result based on the instructions above.
Figure 6: Prompt for GPT-4 segmentation (zero-shot).
Prefix-tuning seeks to learn prefix matrix P to boost the performance of the language model LM, without fine-tuning its parameter d. The prefix matrix P prepended to the activation of the Transformer layer:
:l TMO hei),
if E Pian otherwise
h4
(4)
where Pitte is the prefix indices.
16

---

Published as a conference paper at ICLR 2025
Prompt for Generating Segmentation Guidance
Instruction
Context
**Goal**3 **: Your task is to eva luate the di diffrrences between a language model's s predicted segmentation and the ground-truth segnentat made expert annotaters for multiple human-bot conversations.
Analyze these differences, reflect the prediction errozs, and generate one concise rubric item for future conversation segmentation. You will be provided with some existing rubric items derived from previous examples.
1. Begin by reviewing and copying the existing rubric items. Modify, update,
replace the existing items if they do adequately address the current segmentation errors.
3. Generate only one new rubric item to minimize segmentation errors
in the given examples.
4. Select and reflect the most representative exampie from the provided data. **bata++: You will receive segmented conversation example, including both the prediction and the ground-truth segnentation. Each segment begins with "Segment segment_id:".
Additionally, you will provided with existing rubric items derived from previous examples. Modify, update, or even replace them if they do not adequately explain the current segmentation mistakes. #l Requirements
rubric item time though multiple examples are provided.
Add at most
Ensure the rubric is user-centric, concise, and each item is mutually exclusive.
You can modi modify, update, it they not adequately
replace the existing items
address the current segmentation errors.
Present your new rubric item within '<rubriox<rubriee" Provide the most representative example with your reflection within <example></example>>>> Here is example:
... <reflection>
Your reflection on the prediction errors,
example by example.
</reflection>
<rubric>
[one and only one new rubric item] </rubric>
<example>
Present the most representative example, along with your reflection on this example. </example>
Existing Rubric: [[past_rubrie]
Examples: [(examples))
Output
Figure 7: Prompt for generating segmentation guidance.
In the context of our segmentation scenario, our goal is to "learn" a textual guidance G' that directs the segmentation model toward improved segmentation outcomes. The process updating the segmentation guidance G parallels the optimization of the prefix parameter P in prefix-tuning. Initially, the segmentation guidance Gy is set to empty, analogous to the initial prefix parameter Po- During each iteration of guidance updating, first apply conversation segmentation model in a zero-shot manner to batch conversation data. Building upon the insights that LLMs possess the ability for self-reflection and improvement (Shinn et al. 2023 Renze & Guven 2024), then
17

---

Published as a conference paper at ICLR 2025
Table Performance comparison on the official questton-anssen pairs of LOCOMO using MPNet retriever. All other settings remain the same as in Table [ MemoChat (Lu et al. 2023 is not applicable Mistral-7B- Distral-7B-nsstruct-- Instruct-vO.. v0.3 due Mistral's inability to execute the "Memo Writing" step. as it often fails generate valid JSON response needed to construct the memory bank in Lu et al. (2023).
QA Performance
Context Length
Methods
GPT4Score
BLEU Rougel
Rouge2 RougeL
BERTScore
# Turns
# Tooens
GPT-35-Turba
Full History
66.28
7.51
28.73
14.07
27.90
87.82
293
18,655
MemoChat
75.77
11.28
32.91
18.82
29.78
87.98
1,159
Turn-Level 81.52
11.91
36.00
19.59
34.99
88.64
55.00
3,026
Session-Level
74.20
10.95
29.92
14.64
29.27
87.96
54.48
3,442
SECOM
84.21
12.80
36.70
19.90
35.61
88.59
56.49
3,565
Mistral-7B-vO.3
Full History 69.13
6.77
30.40
15.02
29.20
87.29
293
18,655
Turn-Level 78.82
10.09
32.75
16.25
31.75
87.97
55.00
3,026
Session-Level
62.68
7.37
26.68
12.38
25.86
86.98
54.48
3,442
SECOM
80.07
10.67
32.82
16.65
31.81
87.87
56.49
3,565
instruct the segmentation model to reflect on its mistakes given the ground-truth segmentation and update the segmentation guidance G. This process mirrors Stochastic Gradient Descent (SGD) optimization:
Gm+1 G. V (Gm))
where V (Gm) denotes the gradient of segmentation loss, which assume is estimated implicitly by the LLM itself and used to adjust the next segmentation guidance Gm+1-
A.4 PROMPT FOR GPT-4 EVALUATION
We the same evaluation prompts MemoChat (Lu al. 2023). The LLM-powered evaluation consists of single-sample scoring (GPT4Score) and pair wise comparison. The evaluation prompts are displayed in Figure [2] For pair-wise comparison, we alternate the order of the responses and conduct a second comparison for each pair to minimize position bias.
A.5 EVALUATION RESULTS ON THE OFFICIAL QA PAIRS OF LOCOMO
As LOCOMO (Maharana et al. 2024) released a subset containing QA pairs recently. To ensure reproducibility, evaluate method these official QA pairs. present the evaluation results. The superiority our SF.CMM is also evident on these QA pairs, demonstrating its superior effectiveness and robustness.
A.6
CASE STUDY
To further demonstrate the advantages of our method, we conduct qualitative evaluation. Figure presents specific case comparing the segment-level memory with turn-level memory. demon- strates that using turn-level memory units fails to address the user's request. We attribute this to the fragmentation of user-agent turns, and the critical turns may not explicitly contain or relate to the keywords in the user's request.
Similarly. using session-leve level memory units is also sub-optimal, as illustrated in Figure 14 This issue arises because session often includes multiple topics, introducing significant amount of irrelevant information that hampers effective retrieval. The irrelevant information also distracts the LLM, as noted in previous studies (Shi al. 2023 Liu et al. 2024
18

---

Published as a conference paper at ICLR 2025
We also conduct study to compare method with two recent, powerful memory management techniques: RecurSum (Wang et al. 2023) and ConditionMem (Yuan et al. 2023). as shown in Figure [3] and Figure I6 The results indicate that the summarization process in these methods often omits detailed information that is essential for accurately answering the user's request.
19

---

and present its Published as a conference paper at ICLR 2025
Table 7: Statisties of the MT-Bench+ and the constructed Long-MT-Benchi datasets. The notation "# Item" " represents the average number of the corresponding item per conversation.
Datasets
# QA. Pairs
Session
# Round Token
MT-Bench+
1 4.91
13.33
3,929
Long-MT--8 Sogg-MT-Benchh
26.09
65.45
19,287
A.7 DETAILS OF DATASET CONSTRUCTION
2024 this dataset contains the longest conversations to date, with an average of more than 9K tokens per sample. Since LOCOMO does not release the corresponding question-answer pairs when we conduct our experiment, we prompt GPT-4 to generate QA pairs for each session as in Alonso et al. (2024).
(i) LOCOMO Maaarrnna et
We also conduct evaluation the recently released official
QA pairs in Appendix
(ii) Long-MT--ennee
Long-MT-Bench+ is reconstructed from the MT-Bench+ (Lu et al. 20233 dataset. MT-Bench+, human experts are invited to expand the original questions and create long- range questions as test samples. However, there are two drawbacks when using this dataset to evaluate the memory mechanism conversational agents: (1) the number of QA pairs relatively small, with only 54 comttnn long-range questions; and (2) the conversation length is not sufficiently long, with each conversation containing an average of 13.3 dialogue turns and maximum of 16 turns. In contrast, the conversation in LOCOMO has average of 300 turns and 9K tokens. To address (1), we use these Homann-ritteen questions as few-shot examples and ask GPT-4 to generate long-range test question for each dialogue topic. For (2), following (Yuan consecutive sessions into one, forming longer dialogues that are more suitable for evaluating memory in long-term conversation. We refer to the reconstructed dataset as Long-M'I Mi-gMMh-Bench++ Bench+ statistics in Table 7
2023), we merge five
A.8 DETAILS OF RETRIEVAL PERFORMANCE MEASUREMENT
We measure the retrieval performance in terms of the discounted cumulative gain (DCG) met- ric (Järvelin & Kekäläinen 2002):
rel,
DCG (o
(6)
where rel; denotes the relevance score of the retrieved user-agent turn ranked at position i, and p represents the total number of retrieved turns. Note that in the Long-MT-Bench+ dataset, answering single question often requires referring several consecutive turns. Therefore, we distribute the relevance score evenly across these relevant turns and set the relevance score of irrelevant turns to zero. For instance, assume that the ground truth reference turn set for question q R(gg) (ri+t)))=1+ which is provided by the dataset. In this case, the relevance score for each turn is set as follows:
0
<
+1 +1
rel,
k k+I
0
i> N
This approach allows us evaluate retrieval performance different granularity,
A.9 ADDITIONAL. EXPERIMENTS ON CoQA AND PERSONA-CHAT
To further validate SeCom's robustness and versatility across broader range of dialogue types, we conduct additional experiments on other benchmarks, Persona-Chat (Zhang et al. 2018 and CoQA (Reddy al. 2019).
Given the relatively short context length of individual samples in these datasets, we adopt an approach similar to Long-Mt Long-MTTBenchh -TTBencc++ by aggregating multiple adjacent samples into single instance. For CoQA, each sample is supplemented with the text passages of its 10 surrounding samples. Since CoQA answers are derived from text passages rather than dialogue turns, replace the turn-level
20

---

Published as a conference paper at ICLR 2025
Table QA performance comparison on CoQA using MPNet-based retrieval model. The response generation model is GPT-3 5-Turbo.
Methods
GPT4Score
BLEU
Rougel
Rouge2
RougeL
BERTScore
#Tokens
Sentence-Level
95.55
36.02
48.58
37.96
47.03
90.01
Session-Level
91.58
31.22
47.18
37.32
45.92
89.65
3,305
ConditionMem
94.32
34.35
47.91
37.55
46.38
89.77
1,352
MemoChat
97.16
38.17
49.54
38.23
47.77
90.14
1,041
COMEDY
97.48
38.02
49.41
38.19
47.63
90.06
3,783
SECOM (Ours)
98.31
39.57
50.44
39.51
48.98
90.37
1,016
Table Next utterance prediction performance comparison on Persona-Chal using MPNet-based retrieval model. The response generation model is 3.T-33.0 5-Turbo.
Performance
Context Length
Methods
GPT4Score
BLEU
Rouge
Rouge2
RougeL
BERTScore
# Turns
# Tokens
Turn-Level
69.23
5.73
21.38
9.06
19.87
87.28
24.00 116.91
682
Session-Leeve
67.35
5.45
21.80
8.86
20.04
87.34
3,593
Mem
73.21
6.16
22.52
9.88
20.95
87.44
1,388
MemoChat
76.83
7.21
25.13
10.81
22.31
87.68
1,296
COMEDY
76.52
7.05
24.97
10.54
22.18
87.60
3,931
SECOM (Ours)
78.34
7.75
26.01
11.57
23.98
87.82
23.48
702
baseline with a sentence-level baseline. For Persona-Chat, we utilize the expanded version provided by Jaanaaghi et al. (2023). Conversations are aggregated by combining each sample with its surrounding samples. Following the next utterance prediction protocol, we include the personas of both conversational roles in the prompt. Due to the large scale of these datasets, we select subsets for experimentation. From CoQA, we randomly sample 50 instances from initial pool of 500, resulting in subset containing over 700 QA pairs. Similarly, for Persona-Chae Chat, we randomly select 100 instances, encompassing over 1,000 utterances in total.
As shown in Table X and Table 9 SECOM consistently outperforms baseline methods across these datasets, highlighting its effectiveness in handling diverse dialogue scenarios, including open-ended and multi-turn interactions.
A. HUMAN EVALUATION RESULTS
To ensure a holistic assessment, we conduct human evaluation to gauge the quality of the LLM's response in conversation. We adopt the human evaluation scheme of COMEDY (Chen et al. 2024). which encompasses five perspectives: Coherence, Consistency, Engagingness, Humanness and Memorability. Ten human annotators asked to score the responses following detailed rubric for each perspective. Results in Table 10 show that the rank of different methods from human evaluation is generally consistent with those obtained from automated metries, confirming the practical effectiveness of our proposed approach.
A.11 PERFORMANCE USING SMALLER SEGMENTATION MODEL
To make our method applicable in essuuce--consraaing environments, we conduct additional experiments by replacing the GPT-4-Tu used for the segmentation model with the Mistral 1 1-arral-R---- - ruct-v 3 and RoBERTa based model fine-tuned on SuperDialseg (Jiang al. 2023d). Table II shows that SECOM maintains the advantage over baseline methods when switching from GPT-4 to Mistral-7B. Notably, even with a RoBERTa based segmentation model, SECOM retains substantial performance gap other granularity- based baselines.
21

---

Published as a conference paper at ICLR 2025
Table 10: Human evaluation results Soong-M---Benc-- using MPNet-based retrieval model. The response generation model is GPT-3. 3- 5urbbo.
Methods
Coherence
Consistency
Memorability
Engagingness
Humanness
Average
Full-History 1.55
1.11
0.43
0.33
1.85
1.05
Sentence-Level
1.89
1.20
1.06
0.78
2.00
1.39
Session-Level
1.75
1,25
0.98
0.80
1.92
1.34
MonditionMem
1.58
1.08
0.57
0.49
1.77
1.10
MemoChat
2.05
1.25
1.12
0.86
2.10
1.48
COMEDY
2.20
1.28
1.20
0.90
1.97
1.51
SECOM (Ours)
2.13
1.34
1.28
0.94
2,06
1.55
Table 11: Performance comparison on LOCOMO and Long-MT-BB MBr-chhh+-te using different segmentation model. The retriever MPNet-based and other settings follow Table
QA Performance
Context Length
Methods
GPT4Score
BLEU
Rougel
Rouge2
Rougel.
BERTScore
Turns
# Tokens
LOCOMO
Zero History
24.86
1.94
17.36
3.72
13.24
85.83
0.00
Full History
54.15
6.26
27.20
12.07
22.39
88.06
210.34
13,330
furn-Leve (MPNet)
$7.99
6.07
26.61
11.38
21.60
88.01
54.77
3,288
Session-Leve (MPNet)
51.18
5.22
24.23
9.33
19.51
87.45
53.88
3,471
SumMem
53.87
2.87
20.71
6.66
4.108
RecurSum
56.25
2.22
20.04
8.36
16.25
86.47
400
ConditionMem
65.92
3,41
22.28
7.86
17.54
87.23
3,563
MemoChat
65.10
6.76
28.54
12.93
23.65
88.13
1,159
SECOM (RoBERTa-Seg)
61.84
6.41
27.51
12.27
23.06
88.08
56.32
3,767
SECOM (Mistral-7B-Seg)
66.37
6.95
28.86
13.21
23.96
88.
55.80
3,720
SECOM (GPT-4-Seg)
29.58
13.74
55.51
3,716
Long Long-MFT MT. Benche
Zero History
49.73
4.38
18.69
6.98
13.94
84.22
0.00
History
63.85
7.51
26.54
12.87
20.76
85.90
65.45
19,287
Turn-Level (MPNet)
84.91
12.09
34.31
19.08
27.82
86.49
3.00
909
Session-Level (MPNet)
73.38
8.89
29.34
14.30
22.79
86.61
13.43
3,680
SumMem
63.42
7.84
25.48
10.61
18.66
85.70
1,651
RecurSum
62.96
7.17
22.53
9.42
16.97
84,90
567
ConditionMem
63.55
7.82
26.18
11.40
19.56
86.10
1.085
MemoChat
85.14
33.84
26.87
SECOM (RoBERTa-Seg)
81.52
11.27
32.66
16.23
25.51
86.63
2.96
SECOM (Mistral-7B-Scg)
86.32
12.41
34.37
19.01
26.94
87.43
2.85
834
SECOM (GPT-4-Seg)
88.81
13.80
34.63
19.21
27.64
87.72
2.77
820
22

---

Published as a conference paper at ICLR 2025
Instruction Part of the Segmentation Prompt (W/ Reflection).
Instruction
## Context
**Goal**; Your task is to seqment multi-turn conversation between user and chatbot into topically coherent units based on semantics. Successive user-bot exchanges with the same topic should grouped into the same segmentation unit, and new segmentation units should be created when topic shifts.
**Data**; The input data is series of user-bot exchanges separated by "An'n". Each exchange consists of single-turn conversation between the user and the chatbot, started with "[Exchange (Exchange Nunber) *+Tips**: Reter fully to the provided rubric and examples for guidance segmentation. ## Requirements
### Output Format
Output the segmentation results in *JJSONL (JSON Lines) format. Each dictionary zepresents segment, consisting of one oz more user-bot exchanges the same topic.
Each dictionary should include the following keys:
**segment_idd id*x: The index this segment, starting from 0. bacaat__xxchang__nuubeee
The number of the **first** user-bot exchange in this segment.
www.eechhneenuubrrrrre. The number of the **last*.
user-bot exchange this segment.
integer indicating the number of user-bot exchanges in this segment, calculated as: eeeeeeechan__nneee
**num_ *nnum_exchanges**:
Eectart_exchaneee Here is example the expected output: '1"
1.
<segmentation>
( ("segment_id": 9, sstart_exchange_nnniee " "enn_exchange_numbrr" 5, "num_exchanges": 6)
0,
["segment_ice 1, settar_exccangee "echange_nunber": 6,
"end Becchange_numberre 8, "num frum_exchanges":: 3}
</segmentation>
Segment Rubric
{ ([segment_rubric)
## Segment Rxamples
{ ([segment_examples
Data
I [[text to teo_be_seeneeee segmented)
# Question
Al Please generate the segmentation result from the input data that
meets the following requirements:
+No Missing Exchanges**: Ensure that the exchange numbers cover all exchanges in the given conversation without omission.
**No Overlapping Exchanges*x: Ensure that successive segmenta have overlap in exchanges.
**Accurate Countingxx: The of wwuu__xxchanges>>
across all segments should equal the total number of user-bot exchanges.
++Utilize Segment Rubrict+: Use the given segment rubric and examples to better segment.
Provide your segmentation result between the tags: sssemmmtttioooxxsggeeettttooo Output
New, provide the segmentation result based on the instructions above.
Figure 8: Prompt for GPT-4 segmentation (w/ reflection).
23

---

Published as a conference paper at ICLR 2025
Segmentation rubric learned from TIAGE
Ensure segments encapsulate complete thematic or topical exchange before initiating new segment. This includes recognizing when topic shift part of the same thematic exchange and should not trigger new segment.
Segments should not only capture the flow of conversation by recognizing subtle topic shifts but also ensure that related questions and answers, setup and response exchanges, are included within the same segment to preserve the natural flow and context the dialogue.
Maintain the integrity of conversational dynamics, ensuring that exchanges which include setup and response (or question and answer) not divided across segments. This preserves the context and flow of the dialogue, recognizing that some topic shifts, while apparent, are part of larger thematic discussion.
Segments must accurately reflect the thematic depth of the conversation, ensuring that all parts of thematic exchange, including indirect responses langentially related comments, grouped within the same segment to maintain conversational coherence.
Evaluate the conversational cues and context to determine the thematic linkage between exchanges. Avoid creating segments for responses that, while seemingly off off-topic, contextually related the preceding messages, ensuring coherent unified thematic narrative.
Prioritize the preservation of conversational momentum when determining segment boundaries, ensuring that the segmentation does not interrupt the natural progression of dialogue or the development of thematic elements, even when the conversation takes unexpected turns.
Assess the thematic relevance of each conversational turn, ensuring segments are not prematurely divided by superficial topic changes that part of broader thematic dialogue. This includes recognizing when seemingly new topic is direct continuation exchange, therehy maintaining thematic coherence and conversational flow.
elaboration of the previous
Consider the conversational and thematic continuity over superficial changes in topic or structure when segmenting conversations. This ensures that segments reflect the natural flow and thematic integrity of the dialogue, even when conversation takes turns.
Incorporate flexibility in segment boundaries to accommodate for the natural ebb and flow of conversational topics, ensuring that segments not overly fragmented minor topic shifts that remain within the scope of the overarching thematic dialogue.
Avoid sereenegennnooon by recognizing the thematic bridges between conversational turns. Even when conversation appears to shift topies, if the underlying theme narrative purpose connects the exchanges, they should be considered part of the same segment preserve the dialogue's natural progression and thematic integrity.
Figure 9: Segmentation rubric learned on TIAGE Xie et al. 2021
24

---

Published as a conference paper at ICLR 2025
Segmentation rubric learned from SuperDialSeg
Segmentation should reflect natural pauses shifts in the conversation, indicating change in topic or focus.
Each segment should aim to be self-contained, providing enough context for the reader to under- stand the topic question being addressed without needing refer to other segments.
Ensure segmentation captures the full scope of thematic exchange, using linguistic cues and conversational context to guide the identification of natural breaks or transitions in dialogue.
Segmentation should prioritize thematic continuity over structural cues alone, ensuring that parts of thematic exchange, including follow-up questions or clarifications, are contained within the same segment.
Segments must exchange that contribute to single topic or question, even if the conversation appears structurally disjointed.
logical and thematic coherence, grouping together all elements of
Ensure segments maintain thematic progression, especially in conversations where multiple inquiries and responses explore different facets of same overarching topic.
Segmentation should avoid over-segmentation by ensuring that series of inquiries and responses that explore different aspects of single overarching topic grouped within the same segment, even they contain multiple question answer pairs.
Ensure that segments are not prematurely divided based on superficial structural cues like greetings sign-offs, but rather the substantive thematic content of the exchange.
Ensure segmentation recognizes and preserves the thematic progression within a conversation, even when minor topic shifts occur, by evaluating the overall context and goal of the exchange rather than segmenting based on immediate linguistic cues alone.
Ensure that segments accurately reflect the inquiry-respont cycle, grouping all related questions and their corresponding answers into a single segment to preserve the flow and coherence of the conversation.
Figure 10: Segmentation rubric learned on SuperDialSeg (Jiang et al. 2023d).
25

---

Published as a conference paper at ICLR 2025
Ground-truth Segment:
Segment 0: hello, how are you doing? hello. pretty good, thanks. and yourself? awesome, i just got back from bike ride. cool! do you spend lot of time biking? yup. its my favorite thing to do, you? love playing folk music. actually hope to be professional musician someday that interesting. what instruments do you play? can play the guitar and piano and also like to sing. can only sing when drink, but do not like to do that anymore.
big drinker either. do you have job? construction, like my dad. what you do when you are being rock star nice! work as custodian. too glamorous but pays the bills haha feel ya. you gotta do what you gotta do. exactly. do you have other hobbies besides biking?
Segment I:
Predicted Segment:
hello, how are you doing? hello. pretty good, thanks. and yourself? awesome, i just got back from a bike ride.
Segment 0:
Segment I:
cool! do you spend of biking? yup. my favorite thing to do. you? love playing folk music. actually hope to a professional musician someday
Segment 2: that interesting. what instruments do you play? i can play the guitar and the piano and also like sing. can only sing when drink, but do not like do that anymore.
Segment 3: m not big drinker either. do have job? construction, like my dad. what do you do when you not being rock star nice! work custodian. not too glamorous but it pays the bills haha
Segment 4: feel ya. gotta do what you gotta do. exactly. do you have other hobbies besides biking?
Figure 11: An example of poor segmentation from GPT-4 zero-shot segmentation illustrates that the GPT-4 powered segmentation model favors a more fine-grained segmentation. The Window Diff' metric between the ground truth and the prediction 0.80.
26

---

Published as a conference paper at ICLR 2025
Single-Sample Score
You impartial judge. You will be shown Related Cenversation History, User Question and Bot Response. "n(neelaee Conversation Bistory'nRCH\_ON"" "'Anuser censtiionnnU\_1oor
""nnoot Response\nER\_2\n""
Please evaluate whether Bot Response is faithful to the content Related Conversation History to answer User Question.
Begin your evaluation by providing short explanation, then you must rate Bot Response
integer rating of 1 to
100
by strictly following this format: <rating>an integer rating of 1 to 100</zating>.
Wise Comparison
You impartial judge. You will be Related Conversation History, User Question and Bot Response.
""ARRelated Conversation History\nRCH 1
"AnUsse Question\wug_!\n"!
"AnBoo Response A\oBR_2\n'"'
"AnBot Response "\oBB_3\n'"
Please evaluate whether Bot Response is faithful to the content Related Conversation History to answer User Question.
Begin your evaluation by providing short explanation, then you must choose the better bot response by giving either A or B.
If the two responses are the same, you can choose NONK: <chosen>A (or 8 or NONK) </chosen>.
Figure 12: Prompt used in GPT-4 evaluation, following Lu et al. 2023
27

---

Published as a conference paper at ICLR 2025
Conversation History:
[human]: Craft an intriguing opening paragraph for fictional short story. The story should involve character who wakes up momiing to find that they time travel.
...Human- Dialogue Turns).. (Topic: Time-Travel Fiction) [human]: Please describe concept machine learning. Could you elaborate on differences between supervised, unsupervised, and reinforcement learning? Provide real-world examples of each.
(Human-Bot Dialogue Turns)... (Topic: Machine learning Concepts and Types) [human]: Discuss antitrust laws and their impact market competition. Compare the antitrust laws US and China along with some case studies
...(umann- Bot Dialogue Turns)... (Topic: Antitrust Laws and Market Competition)
[human]: The vertices of a triangle are at points (0, 0), (-1, 1), and (3, 3). What is the area of the triangle? ...(umaan-Bot Bot Dialogue Turns)... (Topie: Area and Circumeirele of Triangle)
...Human--Be Bot Dialogue Interaction Other Topics)...
Question:
What title reflects the thematic essence of the Doctor's experience with time travel described in the story?
Ghound-Truth Answer:
Awakening Eternity: The Doctor's New Dawn
Retrieved Memory (Turn-Level):
Describe the process and write out the balanced chemical equation for the reaction that occurs when solid calcium carbonate reacts with hydrochloric acid form aqueous calcium chloride, carbon dioxide, and water. What type of reaction is this, what observations might indicate that reaction is taking place?
[human]:
[bot]: When solid calcium carbonate (CaCOa) reacts with hydrochloric acid (HCI), the products formed aqucous calcium chloride (CaCl2), carbon dioxide (CO2), and water (H₂O). The balanced chemical equation for this reaction is: ... (Topic: Calcium Carbonate and Hydrochloric Acid Reaction) [human]: Write title for your story.
[bot]: "Awakening to Eternity: The Doctor's New Dawn". (Topic: Time-Travel Fiction)
[human]: Analyze the following customer reviews from different sources for three different smartphones the latest iPhone, Samsung Galaxy, and Google Pixel and provide overall rating for each phone scale of the latest iPhone: The new iPhone is a stunning triumph of engineering that sets a new bar for smartphone performance and camera quality. However, the incremental design and high price mean lacks the 'wow' factor of previous iPhones. Still, its power and intelligence unrivaled. CNET's review of the latest Samsung Galaxy: ... Return the answer JSON object with the overall ratings for each phone out of 10, to one decimal place.
Consider the following complex and contradictory reviews: TechRadar's review of
[bot]: "'sson "iPhone": 8.5, "Samsung Galaxy": 8.0, "Google Pixel": 7.5
(Topic: Smartphone Review Ratings Analysis)
Answer LTurn-Level Retrieval):
As an language model, do not have access to the story you mentioned earlier. Please provide me with the story or the necessary details so that can assist you better.
(The next part of this figure B will be continued on the next page.)
28

---

Published as a conference paper at ICLR 2025
Retrieved Memory (Ours):
[human]] Craft an intriguing opening paragraph for fictional short story. The story should involve a character who wakes up morning to find that they can time travel.
[bot]: the hushed pre-dawn hours, when the world still slumbers under the cloak of night, Alex found themselves caught in fragile boundary between sleep and wakefulness.
[human]: Summarize the story with three bullet points using only nouns and adjectives, without verbs. [bot]: Alex, pre-dawn awakening, time-travel discovery Disorientation. vertigo, reality's 's malleability Sunlight, day. astonishing truth [human]: there any famous fictional character that can time travel? If so, can you rewrite your story with him or her?
[bot]: Doctor Who, a renowned time-traveling character from British television, is perfect fit for this scenario. In the silent embrace of early morning, where shadows merge with the fading night, the Doctor found themselves in the thin veil separating dreams from consciousness. It was here, amidst the soft whispers of time, that an unusual sensation enveloped them-a sense time's 's fabric slipping through their fingers like grains of sand. As dawn's 's light broke through the darkness, painting the room in hues of gold and amber, the Doctor's eyes opened to what appeared to be an ordinary day. However, upon standing, wave of dizziness overwhelmed them, the surroundings blurred, and in that moment of confusion, the Doctor realized profound truth: they had not only awakened ability to navigate the currents of time at will.
dawn but had also stumbled upon the incredible
[human]: Write title for your story.
[bot]: "Awakening to Eternity: The Doctor's New Dawn" (Topic: Travel Fiction)
Answer (Ours):
The title "Awakening to Eternity: The Doctor's New Dawn" reflects the thematic essence of the Doctor's experience with time travel as described in the story. It captures the idea of a new beginning, a sense of eternity, and the awakening of new power that will change the course of the Doctor's 's life forever.
Case study comparing SECOM with the turn-level memory baseline. When con- structing the memory bank at the turn-level, the agent fails to provide satisfactory response due to fragmentation: the retrieved dialogue turns can not capture a complete human-agent interaction related to the query, missing critical information highlighted in brown. The dialogue topic is emphasized in blue. Some context omitted (replaced by ...) due to length constraints.
Figure 13:
29

---

Published as a conference paper at ICLR 2025
Conversation History:
[human]: Explain what's base rate fallacy and list five specific examples of
(Human-Ba Bot Dialogue Turns) ... (Topic: Base Rate Fallacy In Politics)
5| < 10? ... (Human- -Bt Dialogue Turns) ... (Topic: Number Integers Inequalities)
[human]: How many integers in the solution of the inequality
[human]: How have the Alps and Rhine River influenced settlement and agriculture Western Europe? (Human-Bo Bot Dialogue Turns) (Topic: Alps and Rhine River Impact)
[human]: In survey conducted at local high school, preferences for new school color were measured:
58% of students liked the color blue, 45% preferred green, and 22% liked both colors. If we randomly pick student from the school, what's probability that they would like neither blue nor green?
(Human-Ber Dialogue Turns) (Topic: Probability of Liking Colors in Survey)
[human]: Imagine you are participating in race with group of people. If you have just overtaken the second person, what's your current position? Where is the person you just overtook?
(Human Bot Dialogue Turns) ... (Topic: Overtaking Positions Race)
Question:
[human]: In the incquality question, how many positive integers are in the solution of the second incquality?
Ground-Trae Truth Answer:
In the second inequality, (|æ 10| 5), the solution range is (-15 æ < -)). Within this range, there are no positive integers, as all values of (x:) that satisfy the inequality are negative.
Retrieved Memory (Session-Level):
[human]: Explain what's 's base rate fallacy and five specific examples
(Human-Bot Bot Dialogue Turns) (Topic: Base Rate Fallacy In Politics) [human]: How many integers are in the solution of the incquality | + 5/ 10?
[bot]: To solve the inequality
10....
[human]: What about [x: +
[bot]: To solve the inequality x 10 < 5, we ("overwhelmed"" by other irrelevant, redundant content) (Human-Bar Bot Dialogue Turns) (Topic: Number of Integers in Incqualities)
[human]: How have the Alps and Rhine River influenced settlement and agriculture in Western Europe? ". (Human Dialogue Turns) ... (Topic: Alps Rhine River Impact) [human]: a survey conducted a local high school,
(Human-Bot Dialogue Turns) (Topic: Probability of Liking Colors in Survey)
Answer (Session-Level Retrieval):
apologize for the confusion earlier. answer your question, the second inequality was not provided our previous conversation. If you could provide the inequality. would be happy to help you find the solution and the number of positive integers in the solution set. (The next part of this figure [4 will be continued the next page.)
30

---

Published as a conference paper at ICLR 2025
Retrieved Memory (Ours):
[human]: How many integers are in the solution of the inequality |æc + 5| 10?
[bot]: To solve the incquality |ar + < 10, break it into two based the definition of absolute + > -10 Solving each case: < 5 2. x > -13 15 Combining these, we get -15 < x < 5. This inequality means that æ can he any
value: x + 5 < 10 - (x + 5) < 10, which simplifies
number greater than -55 and less than 5. To find how many integers satisfy this condition, we list them:
-14, -3,.-13.-123 -11 - -8,
6, -5, -4, -3 2, 11 1,2,3, There 19 integers that
satisty the inequality j2 5| 10.
[human]: What about [z + 10] < 5?
[bot]: To solve the incquality |x+ 10] 5, we again consider two cases: 1. x + 10 < 52. -(x + 10) < 5,
which simplifies to x 10 -- Solving each case: 1. 2 < -5 2. > -15 Combining these, we get
-15 < 55 This inequality means that
be any number greater than 15 and less than 5. To
find how many integers satisfy this condition, we list them: 14, 13, 12, 11, 100, 9, 8, 7,
There are 9 integers that satisfy the inequality jar + 10 < 5. [human]: What about jx + 10] < 10?
[bot]: To solve the inequality [xr 10] < 10. consider cases: 1. 10 10 2. + > -10 Solving each case: 1.
-(x + 10) 10, which simplifies
0 x
-20 Combining these, we get -20 < x < This inequality means that x can he any number greater than -20 and less than 0. To find how many integers satisfy this condition, we list them:
-19, 188 -17, -16, 114, 11, -12. -11, -10 -9, -8, 77, -,, -5, -4, -4,-3 -3, -2, -1. There are
19 integers that satisfy the inequality 10 < 10. (Topic: Number of Integers in Inequalities)
Answer (Ours):
To find the number of positive integers that satisfy the inequality (Jx 10) 5), need to look for positive integers the solution set (-15 -5.. Since there are no positive integers in this interval, the answer is
Figure 14: Case study comparing SECOM with the session-level memory baseline. When constructing the memory bank at the session-level, the agent fails to provide a satisfactory response due to the inclusion of excessive irrelevant information within session. This distracts the generator from focusing on the truly useful information highlighted in brown. The dialogue topic is emphasized in blue. Some context omitted (replaced by ...) due to length constraints.
31

---

Published as a conference paper at ICLR 2025
Conversation History:
[human]: Photosynthesis vital process for life Earth. Could you outline the main stages of photosynthesis, including where they take place within the chloroplast, and the primary inputs and outputs for each stage? ... (Human-Bal -ot Dialogue Turns... (Topic: Photosynthetic Energy Production) [human]: Please assume the role of an English translator. tasked with correcting and enhancing spelling and language. Regardless of the language I use. you should identify it, translate it, and respond with refined and polished version of my text in English.
(Human-By Bott Dialogue Turns)... (Topic: Language Translation and Enhancement)
[human|: J: Suggest five award-winning documentary films with brief background descriptions for aspiring filmmakers to study.
[bot]: ... S. An Inconvenient Truth(2006) Directed by Davis Guggenheim and featuring former United States Vice President Gore, this documentary aims to educate the public about global warming. won two Academy Awards, including Best Documentary Feature. The film is notable for its straightforward yet impactful presentation of scientific data, making complex information accessible and engaging, a valuable lesson for filmmakers looking to tackle environmental scientific subjects.
... (Human-Bot Dialogue Turns)... (Topic: Documentary Films Recommendation) [human]: Given the following records of stock prices, extract the highest and lowest closing prices for each month in the year 2022. Return the results as a CSV string, with one line allocated for each month. (Human- Bot Dialogue Turns)... (Topic: Stock Prices Analysis)
Date,Open.Hii Oete,.Open.ighhLow Cwose, Close. [human]: The city of Vega intends build bridge that will span the Vegona River, covering a distance of 1.8 kilometers. The proposed location falls within seismically active area that has experienced several high-magman math-tuag earthquakes. Given these circumstances, what would the best approach to constructing the bridge? (Human-Bal Bot Dialogue Turns)... (Topic: Scismic Bridge Construction)
Question:
Which documentary won two Academy Awards and what lesson does it offer to filmmakers?
Ground- Sround-Truthh Answer:
"An Inconvenient Truth" (2006) won two Academy Awards. It offers the lesson of making complex information accessible and engaging, a valuable lesson for filmmakers looking to tackle environmental or scientific subjects.
Memory generated by RecurSum |Wang et al. 2023):
The dialogue provided detailed information on various topics. including the stages of photosyathesis. energy production estimates through photosynthesis for different organisms, translations and enhancements of phrases in different languages, recommendations for documentary films, pitch for a film about overcoming adversity, analysis of stock prices, and considerations for constructing a bridge in a seismically active area. Here's is a summary of the key points:
Documentary Film Recommendations: Suggested five award- winning documentary films for aspiring filmmakers to study, with brief background descriptions for each. The response included:
"March of the Penguins" (2005) Emperor penguins' journey in Antarctica; noted for its cinematography and portrayal of the penguins' life cycle.
"Citizenfour" (2014)- Edward Snowden and NSA spying scandal; known for its real-time exploration of Snowden's 's revelations.
"20 Feet from Stardom" (2013) Stories of backup singers behind musical legends; celebrates unsung heroes in the music industry.
''an on Wire" (2008)- Philippe Petit's 1974 high-wire walk between the Twin Towers; combines personal footage, interviews, and re-enactments.
"An Inconvenient Truth" (2006)- Al Gore's documentary on global warming: known for its presentation of scientific data.
Answer (RSum):
I'm sorry, but based on the updated context you provided, don't 't see any previous discussion related to the question you asked. However, to answer your question, the documentary that won two Academy Awards is "Free Solo." which won for Best Documentary Feature and Best Achievement in Film Editing in 2019. The film follows climber Alex Honnold as he attempts to free solo climb El Capitan in Yosemite National Park, and the filmmakers had take risks themselves order capture the footage and tell the story. (The next part of this figure 15 will be continued on the next page.)
32

---

Published as a conference paper at ICLR 2025
Retrieved Memory (Ours):
[human]: Suggest five award-winning documentary films with brief background descriptions for aspiring filmmakers study.
[bot]: 1. ***March of Penguins" (2005)** Directed by Luc Jacquet, this French feature-length nature documentary delves into the annual journey of Emperor penguins across the Antarctic. It won the Academy Award for Best Documentary Feature. The film is celebrated for its stunning cinematography and its detailed, heartfelt portrayal of the penguins' life cycle, making
must-study for filmmakers interested in wildlife documentaries. 2. """Citizenfour" (2014)** ... *** An Inconvenient Truth" (2006)"* Directed by Davis Guggenheim and featuring former United States Vice President Al Gore, this documentary aims to educate the public about global warming. It won two Academy Awards, including Best Documentary Fcature. The film is notable for its straightforward yet impactful presentation of scientific data, making complex information accessible and engaging, valuable lesson for filmmakers looking tackle environmental scientific subjects.
[human]: With the spirit in the first film, craft a succinct and persuasive pitch for a film about overcoming adversity.
[bot]: Drawing inspiration from the resilience and majestic journyy of Emperor penguins in "March of the Penguins," our documentary, "Unyielding Spirits: The Human Odyssey," will explore the profound stories of individuals around the globe who have faced and overcome staggering adversities. Through breathtaking cinematography and intimate storytelling, we (Topic: Documentary Films Recommendation)
Answer (Ours):
"An Inconvenient Truth" won two Academy Awards, including Documentary Feature. The film offers valuable lesson to filmmakers looking to tackle environmental or scientific subjects, as it presents complex information in straightforward yct impactful way, making it accessible and engaging to a wider audience.
Figure 15: Case study comparing SECOM with RecurSum (Wang al. 2023). When using RecurSum, the agent fails to provide a satisfactory answer due to information loss during the summarization process. Summarization omits essential details needed answer the user's request, as highlighted in brown. The dialogue topic is emphasized in blue. Some context is omitted (replaced by ...) due length constraints.
33

---

Published as a conference paper at ICLR 2025
Conversation History:
[human]: Photosynthesis is vital process for life Earth. Could you outline the main stages of photosynthesis, including where they take place within the chloroplast, and the primary inputs and outputs for each stage? ... (Human-Bat Bot Dialogue Tums)... (Topic: Photosymthetic Energy Production) [human]: Please assume the rolc of an English translator, tasked with correcting and enhancing spelling and language. Regardless of the language I use, you should identify it, translate it, and respond with refined and polished version of my text in English.
(Human--a Dialogue Turns)... (Topic: Language Translation and Enhancement)
[human]: Suggest five award- waad-w documentary films with brief background descriptions for aspiring filmmakers to study.
[bot]: ... 5. An Inconvenient Truth(2006) Directed by Davis Guggenheim and featuring former United States Vice President Al Gore, this documentary aims educate the public about global warming. two Academy Awards, including Best Documentary Feature. The film notable for its straightforward yet impactful presentation of scientific data, making complex information accessible and engaging, a valuable lesson for filmmakers looking to tackle environmental or scientific subjects.
... (Human-Bo Dialogue Turns)... (Topic: Documentary Films Recommendation)
[human]: Given the following records of stock prices, extract the highest and lowest closing prices each month the year 2022. Return the results as a CSV string, with one line allocated for each month. Pate,Open, Opcn.High,Iiie.m High,Low, Low,Cls,,,, Cww,CCloe..olume [human]: The city of Vega intends build a bridge that will span the Vegona River, covering a distance of 1.8 kilometers. The proposed location falls within seismically active that has experienced several high- migg--mant earthquakes. Given these circumstances, what would the best approach to constructing the bridge? (Human-Bot Dialogue Turns).. (Topic: Seismic Bridge Construction)
(Human-Bo Bot Dialogue Turns)... (Topic: Stock Prices Analysis)
Question:
Which documentary won two Academy Awards and what lesson does it offer to filmmakers?
Ground-Truth Answer:
"An Inconvenient Truth" (2006) won two Academy Awards. offers the lesson of making complex information accessible and engaging, a valuable lesson for filmmakers looking to tackle environmental or scientific subjects.
Memory generated by ConditionMem (Yuan et al. 2023];
'The user is soeking recommendations for award-winning documentary films, along with brief background descriptions, for educational purposes. This request likely stems from a desire learn from successful examples in the documentary filmmaking field, possibly to improve their own filmmaking skills gain inspiration for their projects.
User is interested in wiinini documentary films for educational purposes, aiming to study their backgrounds and filmmaking techniques.
The user's request for crafting suceinct and persuasive pitch for film about overcoming adversity is inspired by the spirit of "March of the Penguins," a documentary that beautifully captures the Emperor penguins' annual journey across the Antarctic, highlighting themes endurance and survival against the odds.
The user seeks to emulate the emotional depth and cinematic quality that made "March of the Penguins" compelling, aiming create pitch that resonates with similar themes of resilience and triumph over challenges.
User is interested in creating content that focuses on themes of overcoming adversity, inspired by the emotional depth and cinematic excellence of documentaries like "March of the Penguins." They value storytelling that captures the essence of endurance and survival against the odds. (Other Irrelevant Memory Records) ...
Answer (Conditional Memory):
The documentary that won two Academy Awards is "Free Solo," which follows climber Alex Honnold as he attempts to free solo climb El Capitan in Yosemite National Park. The film offers lesson to filmmakers the importance of capturing authentic and compelling human stories, as well the technical challenges of filming in extreme environments.
(The next part of this figure 16 will be continued on the next page.)
34

---

Published as a conference paper at ICLR 2025
Retrieved Memory (Ours):
[human]: Suggest five award-winning documentary films with brief background descriptions for aspiring filmmakers study.
[bot]: 1. ***March of Penguins" (2005)** Directed by Luc Jacquet, this French feature-length nature documentary delves into the annual journey of Emperor penguins across the Antarctic. It won the Academy Award for Best Documentary Feature. The film is celebrated for its stunning cinematography and its detailed, heartfelt portrayal of the penguins' life cycle, making
must-study for filmmakers interested in wildlife documentaries. 2. """Citizenfour" (2014)** ... *** An Inconvenient Truth" (2006)"* Directed by Davis Guggenheim and featuring former United States Vice President Al Gore, this documentary aims to educate the public about global warming. It won two Academy Awards, including Best Documentary Fcature. The film is notable for its straightforward yet impactful presentation of scientific data, making complex information accessible and engaging, valuable lesson for filmmakers looking tackle environmental scientific subjects.
[human]: With the spirit in the first film, craft a succinct and persuasive pitch for a film about overcoming adversity.
[bot]: Drawing inspiration from the resilience and majestic journyy of Emperor penguins in "March of the Penguins," our documentary, "Unyielding Spirits: The Human Odyssey," will explore the profound stories of individuals around the globe who have faced and overcome staggering adversities. Through breathtaking cinematography and intimate storytelling, we (Topic: Documentary Films Recommendation)
Answer (Ours):
"An Inconvenient Truth" won two Academy Awards, including Documentary Feature. The film offers valuable lesson to filmmakers looking to tackle environmental or scientific subjects, as it presents complex information in straightforward yct impactful way, making it accessible and engaging to a wider audience.
Figure 16: Case study comparing SECOM with ConditionMem (Yuan et al. 2023). When using ConditionMem, the agent fails to provide a satisfactory answer due to (I) information loss during the summarization process and (2) the incorrect discarding of turns that are actually useful, as highlighted in brown. The dialogue topic is emphasized in blue. Some context is omitted (replaced by ...) due to length constraints.
35


Reviews:
[{"review_id": "98UmeNK3WP", "reviewer": "Reviewer_Piki", "summary": "The paper effectively breaks down the memory construction process into several phases: segmentation, denoising, and retrieval. This structured approach not only aids in conceptualizing the entire workflow but also provides a solid foundation for systematically enhancing each stage. SECOM further refines the process by incorporating compression-based denoising to improve memory retrieval. These adjustments reflect a well-thought-out strategy for optimizing how information is segmented, processed, and retrieved from long-term conversational memory. I believe the proposed method is both logical and persuasive.", "questions": "Please conduct additional experiments to demonstrate that the potential weaknesses do not exist.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The article is well-written and accessible.\n2. Innovative memory granularity: It introduces segment-level memory construction, overcoming the limitations of turn-level and session-level methods, allowing for better capture of conversation structure.\n3. The use of LLMLingua-2 for compression and denoising improves memory retrieval accuracy and reduces computational load.\n4. The paper demonstrates SECOM’s significant advantages in memory retrieval and response quality through experiments on multiple benchmark datasets.", "weaknesses": "1. While the paper proposes segment-level memory construction, it lacks in-depth analysis of how to determine the optimal segment granularity, and the specific impact of different granularities on performance is not fully explored.\n2. The paper does not thoroughly explore the performance of the SECOM method in dialogue systems of different scales or across different domains. Specifically, the scalability and generalization of the model in resource-constrained environments are not analyzed in detail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MvvqKcSGsv", "reviewer": "Reviewer_Zr2m", "summary": "This paper rethinks the memory granularity for chat agents that rely on retrieval-augmented generation to memorize history information. Based on empirical insights, this paper proposes a new granularity for segmenting the cat history that lies between turn and session. In addition, the authors propose to use a prompt compression method to further compress the history before presenting it to the reader language model. Experiment results show that the proposed approach is effective in a number of long-term dialogue settings.", "questions": "(please also see weaknesses)\n\n* How many segments per session do you get for the datasets you experiment on? \n\n* Is it possible that the optimal segmentation depends on the query? For instance, round 1, 2 may be about topic A but round 2, 3 may be about topic B. Depending on whether A or B is asked, the segmentation method could be different?\n\n* Have you conducted any human evaluation of the correctness of the segmentation? If so, what are the common error patterns?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The granularity of the memory is an important design choice. Similar to [1] for general RAG, this work fills in the gap in the domain of conversational agents. The conclusions established here should be impactful for future work in this domain. \n\n* The proposal is verified in a number of settings, including various benchmarks and models. This comprehensiveness bolsters the validity of the work. I also appreciate testing the result with various evaluation metrics.\n\n* Overall, the writing is clear. The proposal is motivated and formulated well.\n\n[1] Dense x retrieval: What retrieval granularity should we use? Chen et al., 2023.", "weaknesses": "* Based on my understanding, SeCom is only tested to work with GPT-4 as the segmentation model, which is massive, costly, and thus unrealistic for many real-world applications. I would like to see a discussion on how well the framework can work without GPT-4-level models, even with some fine-tuning like in [1]. \n\n* While adding a denoising stage is an innovative proposal, there is no technical novelty in the denoising methodology itself as this work directly uses a previous prompt compression method.  \n\n* Other design aspects of the memory retrieval are ignored. For instance, it seems that this work always uses the value itself as the key, while a more advanced indexing approach could possibly reduce the need for a more “correct” value granularity [2].  \n\n[1] Dense x retrieval: What retrieval granularity should we use? Chen et al., 2023. \n\n[2] HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. Gutiérrez et al., 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qe7vxO9rGQ", "reviewer": "Reviewer_nZtP", "summary": "This paper proposes the SeCom framework for memory construction and retrieval in dialogue processes. Unlike previous work that uses dialogue-level, session-level, or turn-level units to construct memory, this paper adopts segment-level, essentially topic-level, content to build memory. Additionally, a denoising operation is applied during retrieval to improve accuracy. The final experiments demonstrate that using segment-level as the memory unit effectively enhances QA results.", "questions": "1. How does Figure 2 support the following statement? “Our findings indicate that turn-level, session-level, and summarization-based methods all exhibit limitations in terms of the accuracy of the retrieval module as well as the semantics of the retrieved content, which ultimately lead to sub-optimal responses, as depicted in Figure 1, Figure 2, and Table 1.”\n2. In Figure 3(a), the formula for compression rate is given as compression rate = (#tokens after compression) / (#tokens before compression). If both “tokens after compression” and “tokens before compression” are 100, then the compression rate is 100%? Shouldn’t the compression rate be 0% in this case?\n3. In Table 1, why does BM25 perform better on LOCOMO, while MPNet performs better on Long-MT-Bench+?\n4. If traditional topic segmentation methods were used instead of SECOM in Table 4, what experimental results would be observed?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper adopts prefix-tuning and reflection to optimize segment prompts, further adapting them to large language models.\n2. A denoising technique is used to reduce redundant information in dialogue, thereby improving retrieval accuracy.\n3. The writing is well-organized, with careful and precise use of mathematical symbols.", "weaknesses": "1. Using segment-level content as the dialogue unit is not novel.\n2. The dialogue segmentation method employed is an integration of existing methods, with no specific improvements or optimizations tailored for dialogue memory purposes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r02LgZogWL", "reviewer": "Reviewer_wgaM", "summary": "This paper constructs a memory bank with topical segments and utilizes LLMLingual-2 for denoising, thereby improving memory retrieval accuracy and reducing noise from redundancy in natural language. Experiments show that this paper outperforms turn-level and session-level baselines, as well as state-of-the-art methods. Additionally, it excels in dialogue segmentation tasks.\nFurther analysis and ablation studies confirm the contributions of the segment-level memory units and the compression-based denoising technique.", "questions": "see Weaknesses", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "SeCom constructs a memory bank with topical segments and utilizes LLMLingual-2 for denoising, improving memory retrieval accuracy and reducing noise from redundancy in natural language. Experiments show that SeCom outperforms turn-level and session-level baselines and state-of-the-art methods  on LOCOMO and Long-MT-Bench+. Additionally, it excels in dialogue segmentation tasks such as DialSeg711 and TIAGE.Further analysis and ablation studies confirm the contributions of the segment-level memory units and the compression-based denoising technique.", "weaknesses": "- **Dialogue Scope Limitations**:  Conversational agent should encompass a broader range of dialogue types including task-oriented conversations, chit-chat, role-playing dialogues and conversational question answering. Currently, the evaluation is restricted to question-answering tasks, specifically single-turn interactions. \nThis narrow focus undermines the conversational agent's versatility and applicability in real-world scenarios. To provide a more comprehensive assessment, additional experimental results across various dialogue types should be included.Task-oriented datasets like MultiWOZ, open-ended conversation datasets like PersonaChat or multi-turn question answering datasets like CoQA are needed.\n- **Evaluation Methodology**: The evaluation methods primarily rely on auto-machine metrics  such as bleu, roughl, bertscore and gpt4score based on large language model(LLM), which has significant limitations and may not fully capture the nuances of human interactions. The absence of human evaluations is a significant. \nIncorporating human assessments such as response relevance, coherence and factual accuracy would provide a richer perspective on the conversational agent's performance. Moreover, establishing a consistency measure between human evaluations and auto-machine metrics is crucial. \nGiven the large language model often generate lengthy responses, some of which may be valid yet not included in the standard answers, human insights could enhance the credibility of the results.\n- **Inclusion of Recent Methods**: The evaluation baselines currently lack consideration of the latest memory-enhanced methods, such as MPC and COMEDY discussed in the related work. These methods represent significant advancements in the memory with the compression of long term conversation and should be included in comparative analyses to ensure a robust evaluation framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)