Post
2385
🧠 Remember-R1: Our fix for MLLMs forgetting the image during long reasoning
We noticed a frustrating problem: when multimodal models reason over long chains, they gradually stop looking at the image—and start hallucinating based on their own text.
So we built Remember‑R1, a simple RL framework that directly supervises visual attention on the original reasoning trajectory—no inference overhead, no proxy tasks.
We use three complementary rewards: coverage, persistence, and focus. They encourage the model to keep attending to relevant visual evidence even in later reasoning steps.
Results across 7 benchmarks and 2 model sizes: better reasoning, and—more importantly—visual attention decays much more slowly during generation.
No extra cost at inference, just cleaner supervision where it counts.
📄 Paper: https://arxiv.org/abs/2608.01314
💻 Code: https://github.com/Ch921-cell/Remember-R1
Happy to answer any questions and receive feedback!
#MultimodalAI #RL #MLLM #CoT #VisualReasoning
We noticed a frustrating problem: when multimodal models reason over long chains, they gradually stop looking at the image—and start hallucinating based on their own text.
So we built Remember‑R1, a simple RL framework that directly supervises visual attention on the original reasoning trajectory—no inference overhead, no proxy tasks.
We use three complementary rewards: coverage, persistence, and focus. They encourage the model to keep attending to relevant visual evidence even in later reasoning steps.
Results across 7 benchmarks and 2 model sizes: better reasoning, and—more importantly—visual attention decays much more slowly during generation.
No extra cost at inference, just cleaner supervision where it counts.
📄 Paper: https://arxiv.org/abs/2608.01314
💻 Code: https://github.com/Ch921-cell/Remember-R1
Happy to answer any questions and receive feedback!
#MultimodalAI #RL #MLLM #CoT #VisualReasoning
