mohammed933/MemRevise-models

MemRevise: A Revisable Memory for Conversational Search

TL;DR. In conversational search, users correct misunderstandings, revise requests and return to earlier topics. MemRevise keeps a memory of the conversation that a 3B writer revises one turn at a time, so the search query always targets the topic the user means now. This repository holds the writer's checkpoints.

Overview

(a) The 3B writer is distilled from teacher-written memories. (b) At every turn it adds one node to the memory graph: the topic, the facts, and the operations the turn performs; topic statuses follow from the operations. (c) At the final turn the node's query is combined with a template query and evidence from the memory, and searched with a frozen retriever.

Highlights

• 0.701 mean MRR@10 on the three ConvRevise test sets (TopiOCQA, CORAL, OR-QuAC), against 0.589 for the best competitor.
• 3B and one model call per turn: each turn is read once and folded into the memory.
• Revisable: a dropped topic keeps its facts and can be restored when the user returns to it.

The operations

Each node records how its turn changes the conversation:

Operation	Meaning	Status of the topic
ADD	introduces a new topic	becomes current; the previous one becomes earlier
REFINE	continues the current topic: the turn asks more about the topic already being discussed	stays current
REVISE	changes an established value, e.g. the War of 1812 → the one in 1863	becomes current
ABANDON	drops a topic, e.g. after "no, not that one"	dropped, its facts kept
REACTIVATE	restores an earlier topic when the user returns to it	current again

The writer's prompt also defines RETRACT, for a premise the user withdraws.

Example

A ConvRevise test conversation: the user asks about the red fox, digresses to polo and ancient Macedonia, then asks "Sorry for the detour — how did the one I asked about first evolve?". The memory the released writer builds:

Turn	User	Topic	Operation
1	how long does a red fox live in the wild	red fox	ADD
2	what is red fox?	red fox	REFINE
3	where did it originate?	red fox	REFINE
4	How did polo likely begin, and where did its current form originate?	polo	ADD
5	What are some modern variants of polo?	polo	REFINE
6	Switching topics a bit, did ancient Macedonia contribute any notable inventions or early technologies?	ancient Macedonia	ADD
7	Were there any early glass discoveries linked to Macedonia?	ancient Macedonia	REFINE
8	Sorry for the detour — how did the one I asked about first evolve?	red fox	REACTIVATE

The node it writes for the last turn resolves "the one I asked about first" to the red fox, links it to turns 1–3 and writes the query; the search then ranks the gold passage (the red fox's Evolution section) second among 434,179 passages:

{
  "turn_no": 8,
  "asked": ["red fox"],
  "facts": [],
  "depends_on": [1, 2, 3],
  "contradicts": [],
  "types": {},
  "ops": [{"op": "REACTIVATE", "entity": "red fox", "detail": ""}],
  "query": "how did the red fox evolve?"
}

Files

Folder	Content	Size
writer/	the released writer: LoRA adapter (seed 1), tokenizer and chat template	55 MB
base/	base model: Ministral-3-3B-Instruct-2512, text weights	6.9 GB
writer_seeds/seed_{1,2,3}/	the three seeds reported in the paper (seed_1 = writer/)	55 MB each

Quick start

git clone https://github.com/DataScienceUIBK/MemRevise && cd MemRevise
bash scripts/download.sh                      # this repository's base/ and writer/, and the ConvRevise collections
# R and W: the retriever's and the writer's Python (two environments, see the GitHub README)
$R -m memrevise.encode_corpus --corpus orquac
$W -m memrevise.write_memory --set dev_orquac_value_revisions   # memory and query of each conversation (84 questions)
$R -m memrevise.search --set dev_orquac_value_revisions
$R -m memrevise.evaluate --sets dev_orquac_value_revisions

To load the model alone:

from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

d = snapshot_download("mohammed933/MemRevise-models", allow_patterns=["base/*", "writer/*"])
tok = AutoTokenizer.from_pretrained(f"{d}/writer")
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(f"{d}/base", torch_dtype="bfloat16"), f"{d}/writer")

Results

MRR@10 on the ConvRevise test sets with the frozen retriever:

	TopiOCQA	CORAL	OR-QuAC	Mean
MemRevise (mean of three seeds, as in the paper)	0.686	0.634	0.783	0.701
MemRevise, released writer (seed 1)	0.689	0.633	0.788	0.703

Training

Data	22,163 examples from the TopiOCQA and CORAL training conversations of ConvRevise, built from teacher memories
Adapter	LoRA: rank 16, alpha 32, dropout 0.05, on the q, k, v and o projections
Schedule	3 epochs, learning rate 5e-5, cosine with 5% warm-up, effective batch 12
Checkpoint	lowest held-out loss on the fields written before the query

Licence

The adapters and the base model follow Ministral-3-3B-Instruct-2512 (Apache 2.0). The writer was trained on ConvRevise (CC BY-NC-SA 4.0), so use it for research only.

Citation

The BibTeX entry will be added when the paper is published.