DialRet: Enhancing Dialogue Retention for Multi-Session Conversations
Authors: Yohan Na, Dahye Kim, Dong-Kyu Chae


Reviewer paper summaries: This paper introduces DialRet, an LLM trained on eight multi-session tasks. Additionally, the authors present a new benchmark, MSC-Bench, designed to evaluate multi-session dialogue performance. Experimental results indicate that DialRet surpasses existing methods in multi-session dialogue tasks on MSC-Bench.
Summary of strengths: 1. This paper utilizes Conversation-Chronicle data to instruction finetune an LLM on 8 multi-session dialogue tasks and introduces MSC-Bench, a new benchmark for evaluating multi-session dialogue performance.

2. Experimental results demonstrate that DialRet significantly outperforms existing multi-session methods on the MSC-Bench.
Summary of weaknesses: 1. The evaluation of the proposed method, DialRet, is limited to the newly introduced MSC-Bench. The authors should conduct additional experiments on other benchmarks to demonstrate the generalizability of DialRet.

2. There is a lack of human evaluation results, such as Fleiss' kappa, to assess the agreement between human raters and DialRet's performance.

3. Including a detailed table presenting data statistics and comparisons with other benchmarks would enhance the readability and persuasiveness of the research.

4. The paper omits several important related works on multi-turn dialogue evaluation, which could provide a more comprehensive context for DialRet's contributions.

    [1] MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues. ACL 2024.

    [2] BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues. Arxiv 2023.
Comments, suggestions and typos: N/A
Overall assessment scores: 3.5