When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations
Authors: Kailin Jiang
Yuntao Du
Yukai Ding
Yuchen Ren
Ning Jiang
Zhi Gao
Zilong Zheng
Lei Liu
Bin Li
Qing Li
Venue: ICLR 2026 Poster (2026)
Keywords: Evolving Knowledge Injection; Large multimodal model; Benchmark and Dataset
Primary area: datasets and benchmarks
TL;DR: This work introduces MMEVOKE benchmark to reveal challenges in knowledge injection and explores potential solutions.

Abstract: Large Multimodal Models (LMMs) store vast amounts of pretrained knowledge but struggle to remain aligned with real-world updates, making it difficult to avoid capability degradation when acquiring evolving knowledge. Furthermore, most current work focuses on exploring static textual knowledge injection, neglecting dynamic multimodal evolving knowledge injection, leaving the potential of LMMs for multimodal knowledge injection as an open question. To address this, we first propose a pipeline to construct MMEVOKE, a benchmark for evaluating LMMs' ability in multimodal evolving knowledge injection. MMEVOKE contains 9,422 samples spanning 159 subtypes. Then, based on extensive experiments with MMEVOKE, we reveal challenges such as poor injection performance and capability degradation in existing knowledge injection methods through knowledge injection tests and general capability tests. Finally, to tackle these challenges, we introduce knowledge augmentation and knowledge retention methods, finding that knowledge-aware augmentation strengthens knowledge injection performance, and that Data Replay and MoE methods effectively mitigate capability degradation.

Published as a conference paper at ICLR 2026
WHEN LARGE MULTIMODAL MODELS CONFRONT EVOLVING KNOWLEDGE: CHALLENGES AND EXPLO- RATIONS
Kailin Jiang!.2; Yuntao Dus. Yukai Ding
Yuchen Ren Bin Li", Qing Li
Ning Jiang",
Zhi G Zilong Zheng', Lei Liu't,
Unnivessity of Science and Technology of China
"State Key Laboratory of General Artificial Intelligence, BIGAT
CC-FAT&&schon of software, Shandong University
"State Key Lab. for Novel Software Technology, Nanjing University, P.R. China
Wuhan University TThe University of Sydney
Northeast Forestry University *Beijing Institute of Technology
ABSTRACT
Large Multimodal Models (LMMs) store vast amounts of pretrained knowledge but struggic to remain aligned with real-world updates, making it difficult to avoid capability degradation when acquiring evolving knowledge. Furthermore, most current work focuses on exploring static textual knowledge injection, neglecting dynamic multimodal evolving knowledge injection, leaving the potential of LMMs for multimodal knowledge injection first propose a pipeline to construct MMEVOKE, benchmark for evaluating LMMs' ability in multimodal evolving knowledge injection. MMEVOKE contains 9,422 samples spanning 159 subtypes. Then, based on extensive experiments with MMEVOKE,
open question. To address this, we
reveal challenges such as poor injection performance and capability degradation in existing knowledge injection methods through knowledge injection tests and general capability tests. Finally, tackle these challenges, we introduce knowledge augmentation and knowledge retention methods, finding that knowledge-awand augmentation strengthens knowledge injection performance, and that Data Replay and MoE methods effectively mitigate capability degradation. Project Page: https: /eeoke 1mm. gi thub.
INTRODUCTION
Recent research shows that Large Language Models (LLMs) and Large Multimodal Models (LMMs) gain substantial world knowledge and reasoning capabilities through large-scale pre-training (Brown et al., 2020; Zhao et al., 2023; Liu et al., 2023a). By capturing linguistic patterns and factual informa- tion, they achieve significant advancements across domains, demonstrating significant potential for research and applications (Cui et al., 2024; Su et al., 2025). However, the rapid evolution of global information poses significant challenges for LLMs and LMMs in maintaining knowledge consistency, as the constant updating of knowledge and the emergence of new events and entities hinder their ability to maintain accuracy, leading to knowledge obsolescence and inaccuracies.
As shown in Figure 1, LMM fails to recognize or answer question regarding the newly emerged entity Xiaomi Su7, responding with irrelevant information (e.g., Question: Which company produces the car in the image? Answer: Porsche). This indicates that the static nature of trained LMM causes rapid knowledge obsolescence, resulting in inaccuracies and illusions, thereby undermining the reliability of knowledge intensive tasks and requiring consistency with the constantly evolving knowledge.
Researchers have proposed several methods to inject knowledge into LLMs, including fine-tuning to adapt parameters to specific domains (Singhal et al., 2023; 2025; Zhang et al., 2023), retrieval- augmented generation to integrate external knowledge through retrieval and reasoning tools (Ram et al., 2023; Si et al., 2023; Yao et al., 2023), and real-time knowledge updates through internet search combined with LLMS (Nakano et al., 2021; Jiang et al., 2023). Previous works (Jang et al., 2022; Ovadia et al., 2024) constructed distinct knowledge corpora using CC- CC-RECENTNEWS articles "Equal contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
and U.S. current events, respectively, to explore knowledge injection in LLMs. Researchers begin focusing on LLMs' ability to handle temporal knowledge. Realtime QA (Kasai et al., 2023) and DyKnow (Mousavi et al., 2024) assess knowledge freshness in real-time content. EvoWiki (Tang et al., 2025) provides a malli-dimeenional framework for evolving knowledge, categorizing it into stable, evolved, and uncharted levels. This framework enables comprehensive analysis by incorporating multi-hop reasoning and automatic updates. However, the majority of existing research is confined to the text domain and fails to explore solutions for discovered challenges. This manifests in two ways: first, a lack of real-works multimodal evolving knowledge, such as the iterative updating of entities like Xiaomi Su7 and Xiaomi Yu7 in Figure 1. Second, existing work often overlooks analyzing and exploring solutions for the temporal and evolving knowledge challenges identified during evaluation.
Evolving Knowledge
2023
Nees Rainis
Oiنi
Extity Prodect
de
Eniryy Mari
Techerogy
Exity Sparts Car
ROSÉ 4 BRUND
APT.
A Sovid
NRT
Vinee
Evalving Knowledge for Injection
Which compony produces Irage?
Question
The Raen SIT by Company:
Knowledge Injec tion
User
Respanac Klanni Dempary
REsporin:: PNtrecie
2000CE
LMM
LMM
Figure I: Motivation and Overview of MMEVOKE. A fundamental limitation of trained LMMs is their static nature, which causes their inherent knowledge to become outdated and inaccurate over time. Addressing this requires methods for the efficient acquisition of evolving knowledge. To facilitate research this direction, we propose MMEVOKE to specifically evaluate the knowledge injection performance of LMMs when confronted evolving knowledge.
To address these challenges, we introduce MMEVOKE, a multimodal evolving knowledge benchmark designed systematically assess knowledge injection methods in LMMs. We then conduct types of evaluations: knowledge injection tests (to evaluate knowledge adaptation, ie., the ability to acquire new knowledge) and general capability tests (to evaluate knowledge retention, i.e., the ability to preserve previous knowledge). Based on observations from tests, we identify two challenges inherent current knowledge injection paradigms and explore corresponding attempts to address them. Our efforts are summarized as follows:
We propose a comprehensive benchmark MMEVOKE for multimodal evolving knowledge, which, to our best knowledge, serves as the first evaluation dataset to measure LMMs' evolving knowledge injection capabilitiess
Figure 2, we propose a simple and repro- ducible data construction pipeline capable of continuously collecting evolving knowledge.
MMEVOKE is divided into two primary areas: News and Entity. In Figure 1, News and Entity area encompass the latest and popular news and entities since 2024, respectively. Collectively, these two areas span 159 subfields and contain 9,422 carefully collected multimodal evolving knowledge.
ii Knowledge injection and general capability tests unveil challenges existing knowl- edge injection paradigms. We conduct knowledge injection tests with Supervised Fine- Tuning, Retrieval Augmented Generation, Web Search Engine, and Sufficient Context on MMEVOKE. Obtained the following observations: 1 Existing methods exhibit poor knowl- edge adaptation performance; Contrary to intuition, the performance of LMMs remains imperfect even with sufficient context. Second, conduct knowledge retention ability on LMMs after SFT across 7 capability dimensions, @ revealing significant capability degrada- tion and O consistent ranking of degradation severity. @ Notably, severe degradation in instruction wollowiinnnnoooiiggggg causes cascading failures in other capabilities.
2

---

Published as a conference paper at ICLR 2026
ii.. Knowledge augmentation and retention methods are effective explorations to mitigate knowledge injection's challenges. We articulate the distinction between data augmentation and knowledge augmentation, demonstrating that knowledge-awand approach not only strengthens knowledge adaptation but also mitigates capability degradation. For mitigating capability degradation, find direct knowledge rehearsal (Replay) and structured knowl- edge separation (MoELoRA) to be effective, in contrast to indirect knowledge constraint methods (EWC, LwF), which yield unstable results and even worsen degradation.
2 RELATED WORK
Large Multimodal Models. Demonstrating strong vision-language understanding through extensive knowledge and cross-modal alignment, these models typically combine vision encoder with pretrained large language model via an alignment module. BLIP-2 (Li et al., 2023a) employs lightweight Transformer (Q-Former), while MiniGPT-4 (Zhu et al., 2024) and InstructBLIP (Dai et al., 2023) enhance performance via multimodal instruction tuning. LLaVA (Liu et al., 2024a) uses an MLP alignment layer and self-instruct (Wang et al., 2023b) data generation, and Qwen- VI. (Bai et al., 2023) introduces visual receptor with three-stage training.
Knowledge Injection. Improving factual accuracy remains a critical research focus, often addressed by incorporating external knowledge into models. Central approaches include encoding new infor- mation (Chen et al., 2022) and utilizing retrieval or augmentation from knowledge sources (Fan et al., 2020). These sources extend beyond vector databases in modern RAG systems (Lewis et al., 2020a) to structured resources like knowledge graphs (Martino et al., 2023). Alternatively. model adapters (Lauscher et al., 2020) enable domain adaptation through training only adapter parameters.
Continual Learning. The injection of evolving knowledge fundamentally a continual learning (CL) problem, specifically
centered the acquisition of new factual knowledge (Huo & Tang, 2025; Liu et al., 2025c). A central challenge in CL is mitigating catastrophic forgetting, which is the tendency of models to lose prior knowledge and capabilities while learning new information. Existing CL methods designed to address this can be broadly categorized. Regularization-bur bbaed techniques focus on maintaining the stability of critical parameters (Kirkpatrick al., 2017: Li & Hoiem, 2017; Liu et al., 2024b; Qiao et al., 2024; Wang et al., 2023a; Liu et al., 2025a;b). Architecture-centir approaches introduce parameter isolation (Mallya Lazebnik, 2018; Serra al., 2018; Cao et al., 2024), adaptive structures (Yoon et al., 2018; Jiang et al., 2025b), or modular designs (Shen et al., 2019). Rehearsal-baser methods leverage memory buffers for experience replay (Bonicelli et al., 2022). Finally, prompt-based solutions employ learnable prompts to enhance efficiency without explicit data storage (Wang et al., 2022b; Smith et al., 2023). Unlike previous work which often focuses task-level retention in computer vision, this work investigates the specific challenges of knowledge retention and adaptation when LMMs are confronted with evolving factual knowledge.
MULTI-MODAL EVOLVING KNOWLEDGE BENCHMARK
Knowledge Collection and Filter Papalar
Content Summarization
Data F Farmat
CNN
Knowledge Cuntenl
CNN
twos aoold
The
GPT-
oD D
Rule
Injection Date
ruithey Helr to tw bennei den sensa cat tu emeil comepaaoo ofea eage be oyner ucx bndiger con in detemmadd
Google Scarch
smarldie (att Hoddal Gosth lLan aerala Degga cecoes libass teu aivoseceen
usee"" [Mter bos 1all-wod
Knewiedge
Kiswninige
IMC
Use Wikdpedia's 's Hot Data
Content Information
Summary
Viuldyye prortting do doss
QA Generation Query Image Collection
4
Manual Selection
flonal bedoo
GPT-40+
Rule
Search+ IMG Dowalead
+ K-Menns
Knuwledgy IMG
Leataation Datt
Qean:: Who dbanst Notal Faok Plrncc
kil. Doc.ptoeniiia Lx auge?
Selectioe
Noo fand to Sohc
Search Words
Query
Prts Tlades with
Repensine Pares
Pair Object Hypernym
Outlier Data
Query
(Dons track benteetal
Figure 2: Overview of construction pipeline for MОKV For heuristic query, we manually write multiple templates and randomly select template for each data.
3

---

Published as a conference paper at ICLR 2026
To evaluate evolving knowledge injection in LMMs, propose automated pipeline construct MMEVOKE Multi-Modaa EVOlving KnowledgE), benchmark focused on evolving knowledge. Figure 2 illustrates MMEVOKE benchmark's 's data format via a Nobel Prize in Physics case. Each evolving knowledge consists of two components: injection data D (ik, Tg 3k Bk)k= comprises N triples of knowledge-associatee image ig. heuristic query Tk, and knowledge summary y&: and N
evaluation data De (iq: 2q. yo g=) contains query image ig. query 2g., and ground truth V/-
3.1 BENCHMARK CONSTRUCTION
The simple and reproducible pipeline process for evolving knowledge collection is shown in Figure and continuously provides evolving knowledge for the field of knowledge injection.
Step Knowledge Collection. To ensure data timeliness and authenticity, we collect News and Entity evolving knowledge (starting from 2024) from authoritative CNN and Wikipedia sources. For News evolving knowledge: we extract URLs from CNN's robots,tx, via Gentleman's 's Agreement, collecting Type, Title, Content and Image. For Entity evolving knowledge: We compare offline versions Wikipedia different time points to identify new entries, collecting Type, Entity Name, Descriptinn Image and Pageviews. To select representative data, we filter for popularity. News evolving knowledge cannot be filtered directly due to lacking popularity metrics; we search Google using the Title and select popular data based high-similanee result count. Entity evolving knowledge uses Pageviews directly to obtain popular data.
Step 2: Content Summarization. Whether the Content and Description of evolving knowledge, the textual content is often lengthy, posing challenges for LMMS to utilize effectively, Consequently, we employ GPT-4o to summarize the content by establishing stringent rules and providing rich contextual examples, thereby obtaining Summary for each data.
Step 3: QA Generation and Query Image Collection. We establish stringent rules for GPT- 4o to extract corresponding VQA pair for each News Entity evolving knowledge Summary. Simultaneously, GPT-4o must identify the core Object described in the VQA pair and its Hypernym. For VQA pairs generated in the preceding step, the Query Image is absent. Consequently, we combine Object with Hypernym as search key words (c.g.. Geoffrey Hinton Person), search images via Google, and download images. Given potential inclusion of anomalous images in Google- sourced data, we follow prior work (Li et al., 2024b) by utilizing CLIP to extract image features for clustering, thereby detecting and removing aberrant image data.
Step 4: Manual Selection. Since the data collected from CNN websites and Wikipedia often contain multiple associated images, we meticulously conduct manual review of the images corresponding to each data to ensure the acquisition of high-quality image data.
Table I: Key Statistics of MMEVOKE.
Album
Politics
Film 9.34%
11.58%
Statistic
Number
200%
Total evolving knowledge
9,422
Sports 8.78%
Entity evolving knowledge
4,494 (47.7%)
News evolving knowledge
4,928 (52.3%)
Others 11.91%
Total Areas/Subficlds
2/159
Entity 47,70%
News 52.30%
Regions 8.35%
Number of unique images
18,834
IIjection images
9,422
Evaluation images
9,412
Gntartainment 6.86%
Celebrity
Evolving knowledge length
Maximum length
386
20 89%
Hoetth 2672
Others 5,43
$ 5.70%
Businesi
Average length
95.7
Evaluation question length
Maximum length
38
Average length
23.2
Evaluation answer length
Figure 3: Area and Subfield Distribution of MMEVOKE. "Others" for Entity and
Maximum length
12
Average length
2.3
News includes undisplayed subfields.
Benchmark's 's Quality. For MMEVOKE, data quality paramount. We source popular data from authoritative providers (CNN, Wikipedia) ensure high-quality original data. For data generation,
4

---

Published as a conference paper at ICLR 2026
strict rules and contextual examples are implemented using GPT-4o. Query images collected from Google and refined using K-Means clustering to remove noise. This benchmark construction pipeline ensures MMEVOKE'S 's high quality, further validated by human study in Appendix B.4. More details regarding MMEVOKE are presented in Appendix B and G.
3.2 BENCHMARK ANALYSIS
Key Statistics, Area and Subfield Distribution. MMEVORE comprises 9422 evolving knowledge instances, covering 159 fine- fine-grained subfields shown in Figure 3 (29 News and 130 Entity subfields), highlighting its diversity, with key statistics presented Table
Self-Evolving Properties. Evolving knowledge will continue emerge, self-evolving properties are crucial for MMEVOKE. Our data construction pipeline minimizes manual involvement, only manual selection is not automated. Thus, we develop front-end webpages for each data, accelerating manual selection and reducing average inspection time to 10 seconds. To sum up, we easily reproduce data construction pipeline and update MMEVOKE quarterly.
EXPERIMENT
Under the knowledge injection paradigm, assume a large multi-modal model M can be optimized through access the injection data Dx. The optimization seeks mapping function f e F derive the enhanced model M f(M, Dx) must satisfy dual objectives:
(1) Knowledge Adaptation: Maximize accuracy evaluation data De demonstrate robust generalization capabilities on new knowledge:
max E ( DD [T (M* (1g: Ug) - I y4)1
(1)
(2) Knowledge Retention: Minimize the performance gap between M and M on general capability
tests Dp to maintain the model's previous knowledge and capabilities:
min E (I (M(i,, = I
Up)]
(2)
In this section, we conduct experiments to explore the following research questions:
How do existing knowledge injection methods perform in evolving knowledge injection task? Is the injection performance of knowledge from different fine-grained subfields consistent? RQ2: How do knowledge-injeetee LMMs perform on previous general capability tests? Do the post-injected LMMs successfully retain their inherent capabilities?
RQ1:
4.1 EXPERIMENTAL SETUP
To ensure that the evolving knowledge in MMEVOKE is as unknown as possible LMMs, select two representative models for our experiments: LLa (Liu et al., 2024a) and Qwen-VL-Chat (Bai et al., 2023) (all released in 2023). To verify this, we conduct zero-shot testing on MMЕVOKE (Vanilla in Table 2), where the extremely low performance indicates that the vast majority of the evolving knowledge is indeed unknown to the LMMs. The knowledge injection methods will be detailed below.
Large Multimodal Models.
Supervised Fine- Tuning necessitates datasets comprising labeled input-output pairs. Among the commonly used SFT approaches, instruction tuning (Wang et al., 2022a; Mishra al., 2022; Ouyang et al., 2022; Taori et al., 2023) has been identified highly effective technique to improve model performance. We employ two training strategies: Full-FT and LoRA (Hu et al., 2022).
Retrieval Augmented Generation enhances LMMs in knowledge-intensive tasks by integrating external knowledge sources (Lewis et al., 2020b). While early implementations required task- specific training, recent studies (Neelakantan al., 2022) demonstrate that pre- pre-trained embedding models can achieve significant performance gains without additional training. Here, we focus on the knowledge injection performance of Multimodal Retrieval Augmented Generation (MM-RAG)
5

---

Published as a conference paper at ICLR 2026
on LMMs. Three retrieval strategies are employed: Text-Only (retrieving candidate documents only based on textual features), Image-Only (retrieving candidate documents only based on visual features), UnilR (Wei et al., 2024) (retrieving candidate documents based on multimodal features). Commercial AT Web Search Engine integrates internet search and retrieves the evolving knowl- edge in the reasoning process. To validate the capabilities of commercial Al web search engines, we employ Gemini (Team et al., 2023), Perplexity AI, and GPT-4.1. Sufficient Context can be regarded information required to answer the question and directly serves as context for LMMs' inference. Meanwhile, MMEVOKE's high-quality (Section 3.1) ensures reliable Sufficient Context results.
special case of RAG, where it contains all the necessary
Evaluation Protocol:
Cover Exact Match (CEM) and F1-Score (FI) are used as evaluation metrics in open-domain question answering tasks. The former requires to match model prediction with ground
truth (Xu et al., 2023) and equation is CEM 1, y/ C
0, otherwise" Where Y is model prediction. The
latter evaluates overlap between the model's prediction and ground truth at the word level, balancing Precision and Recall (Chan et al., 2024). Let W(y+) = {(1..... ym be ground truth and W( =
(ūn...., Vn J be model's prediction. The overlap is LY U4) = Enew(r)) 1 E W(Y)), where 1[-] the indicator function. Precision is P(Ý,))
4(Y vz) and Recall is R(Ý Y) u(Y.g.) w(y)
TW(()
4.2 PERFORMANCE OF KNOWLEDGE ADAPTATION (RQ1)
Table 2: Performance of knowledge injection methods on MMEVOKE.
Koostelge: Durig compage
Penrer
aus gummas sharptee sumemite fomter
Presaam Borall frams tiing
growe Thury's
Service apents aritty reemibee easate; Thomae
Nattow Crudkis the lncdeot reabed atteader'
Method
ALL
News,Asg
Katity.Avg
aniernad six and cortacent torpp stepatteeete
CEM
FL
CEM1
FL
CEM1
Question Who treed to ussassitane the person Penmoykants?
campaign raily
LLaVA-27.5
Vanilla
9.34
11:36
2.18
Thomas Matthew Crooks
User
Full-FT
18.02
15:17
21.35
16:34
14.37
13.88
&. LLaVA-vI.5 (SFT, MMRAG)
LoRA
18:31
17.32
19:42
12.51
Fall-FT:
Arrar afae
LokA:
Antaney Gualten
MM RACR
24.05
34.30
37.32
49.39
9.50
17.80
CON:
GEM:
Fi-Snore:
RAG
37.11
RAG-Imagp-Dny: Tranp
BAG--aak: Ganman
BAG-SHit: Benii mundors
MM- RAG
40.68
57.51
40.12
53.21
41.30
62:23
(BN:
FL 0
CBN:
F1
CEM
Owee-VI. VL-Chot
by Qwen- Chh- (SFT, MMRAG)
Vanilla
5.84
10:99
7.75
12.72
3.74
9.10
Pull
Tes--eat- Thonse Wathene
Full-FT
16.61
16:22
CaN:
FL
LoRA
6.95
12.64
9.27
14.55
4.41
10.54
Iteage Only: bua
RAG Unire: hupp
21.79
31.28
EDM:
FE-Seare: CEN:
Fi-Scare:
Fi-Soane
MM RAG
22,31
33.09
17.82
25.15
27.24
41.79
Commercial AI Web Search Engine
46.18
49.28
Geninii 2.0 Flash: No 2 2a Thereas Kalthew
Commorciaf AZ Web Search Enginer
CEM:
Fi-Scar::
Gemini 2.0 Flash
26.52
Perplectly Al: Theras
GPT-11: Tasn mathhr
Gemini-2,5 2,5-Pr
44.19
$2.58
48.86
52.54
39
46.27
Perplexity Al GPT-4.1
62.44
keg Sufficient Context
39.61
42.60 69
41.81
43.08
37.19
42.26
Thomss nottow maske
Suffiieut
Fl
v1.5
56.13
75.77
56.78
72.37
55.43
79.50
Gemtte-25-Pm:
Thanscrottorr mwsic
-L-C-V
48.96
66.00
GaM: F1-3care:
Gemini 2,5 Pro
72.15
80.46
72.61
78.77
71.65
82.33
GPT-4.1
83.74
71.21
Figure 4: Case Study.
We conduct extensive experiments to evaluate knowledge injection methods on MMEVOKE. Addi- tional results are in Appendix C. Key observations from Table 2 include:
Obs 1: Current methods perform poorly on MMEYOKE. cerameter-modifyiinn methods (Full-FT, LoRA) is extremely low. For example, on Qwen- VL-Chat, LoRA achieves only 6.95% CEM, showing minimal performance gain compared to Vanilla. @ MM- RAG outperforms SFT, yet its highest performance is only 40. 50.68% CEM and 57.51% FI. Among these, MM-RAG Text-Onll and MM-RAG AMM-RAGlmg--Oni variants perform comparably, while the version using MM-RAG UniTR yields the best results. 2 Commercial AI Search Engines can also perform multimodal knowledge injection, but their performance varies significantly. Gemini-2, 2.0-hlash achieves only 18.21 1% CEM and 26.52% F1, far below the results of Gemini-2.5- AI, and GPT4.1. 1. Although Perplexity AI and Gemini-2.5-Pa 5-rro achieve 48. 48.27% and 44.19% CEM respectively, this performance still falls short of ideal application standards. Current methods
Specifically, the performance of
Perplexity
6

---

Published as a conference paper at ICLR 2026
exhibit shortcomings such overfitting, instruction refusal, and irrelevant responses in Figure 4. These issues underscore the need for improved knowledge injection methods.
when the context is Obs 2: Contrary to intuition, LMMs still provide incorrect answers sufficient. We often assume that accurate and sufficient information can guarantee correct answers. However, shown in Sufficient Context experiments in Table 2, models still give wrong answers under sufficient context conditions; for instance, Qwen-VL-Chat achieves only 48. 18.96% CEM, and even powerful commercial models like GPT-4. I and Gemini-2.5-prr achieve only 75.02% and 72.15% CEM, respectively, failing to achieve the expected perfect accuracy. This aligns with phenomenon observed by (Joren et al., 2025; Tang et al., 2025). This indicates that providing context is not enough; LMM's s reasoning and utilization skills for evolving knowledge are essential.
Challenges
Challenge 1: Current knowledge injection methods perform poorly on MMEVOKE, failing to achieve optimal performance even with sufficient context.
4.3 PERFORMANCE OF KNOWLEDGE RETENTION (RQ2)
Due to the fact that only fine-tuning methods will modify model parameters in the knowledge injection methods we evaluated, general capability testing focuses on Full-FT and LoRA. To further test general capability of LMMs after knowledge injection, we conduct general capability test on 12 benchmarks across 7 different dimensions (Fu et al., 2024). Specifically, the evaluation tasks include: 1 Comprehensive Evaluation: MME (Fu et al., 2023) and MMBench (Liu et al., 2024c); e OCR: SEEDBench2 Plus (Li et al., 2024a) and OCRBench (Liu et al., 2023b); O Multidisciplinary: ScienceQA (Lu et al., 2022) and MMMU (Yue et al., 2024); 2 Instruction Following: MIA- Bench (Qian al., 2024); O Multi-Round QA: MMDU (Liu al., 2025d); @ Mathematical
Reasoning: Math Vista (Lu et al., 2024) and Math Vision (Wang et al., 2025a); ? Hallucination:
POPE (Li et al., 2023b) and HallusionBench (Guan et al., 2024). More details in Appendix D. Table illustrates quantitative results of Vanilla, Full-FT, and LoRA and we have following observations:
Table 3: The results of VA-v1- on general capability tests. First line shows results of current methods in general capability tests; Second line shows the percentage of capability degradation or improvement of current methods compared to Vanilla. Red value indicates capability degradation, the darker the color, the more severe the degradation, and Green value indicates capability improvement. Ranking includes average degree of degradation and degradation ranking.
Methed
Cangushende
DCR
Indmertinn
Mali Rured
Mathonaticl
Bankdeg
Yalls
1 26
64.90
29.78
30.90
09.33
25.00
66.33
26.37
25.00
15.16
31.76
NE-FT
64:309
505.30
118.0%
52.92
018998
31.44
187778
25.10
15379
05.13
112.000
24.33
06:2399
25.25
005259
1553
15.995
26.70
19277
1126
114.99
3422
037499
927
E30999
125134
55.87
3032
25.20
96.5
21.40
20.6
13.00
25:0
13.55
13.01
8-1
00099
12500
$16399
Kaowlades Www.sctatteesaaoo
Be Teat
Apaznd:
111.3%
16699
033545
11.80
123300
012359
16389
L27/090
Ancen (4
0203955
72219
072979
11.395
19.9%
11409
020305
115099
028589
120729
$7.50
2864
20000
60.0
33.00
21,22
10.09
3000
974
$1.53
6:53
11
023
118.0%
155975
133595
14.15
110000
035395
ENKARS
Stankidge
1248.44
$4.21
26.19
25.00
56922
27.00
15411
10.32
20,9
5:13
77.1
15:72
0331071
11..0%
18.0%
03533%
14179
13.9%%
012399
0352220
1101775
036.99
1260105
Know Kolge Redecttion Misloeds
600%
16.395
06.338
17.10
11.968
11.259
14.11
13.28%
13:15
Ray LIBA
111 90%
16.395
11.33
17.14
11,889
1600
11389
10225
US00R
DWC
022
122.0%
096399
111.0%
18009
1304%
112299
1220090
023.65
15185
$25.050
14315
16510
123310
MaEZ
17.359
11.9%
10.995
11.75%
12.0%
125245
11.18%
13.005
Obs 3: All general capacities of LMMs after injection will degrade, and degree of degradation of capacities varies in different dimensions. Specifically, the average performance of the model significantly decreased MME 441.30% MIA-Bench 458.61% % MMDU 449.3%% and
HallusionBench 458.53% after undergoing Full-FT and LORA, while the degree of decline is relatively small ScienceQA 44.55% Mathvista J6.08% and Mathvision 15.89%
Obs 4: The degradation degree of different capacities of LMMs after injection shows consis- tent rankings in Full-FT and LoRA. Specifically, the ranking from severe mild according to
7

---

Published as a conference paper at ICLR 2026
degree of capacities degradation (calculating the mean under the test) is follows: Instruc- tion Following - Multi-Rounn QA - Hallucination - Comprehensive Evaluation - OCR - Multidisciplinary + Mathematical Reasoning.
Prompt: Is c++ code shown in the picture?
$2,00
301
NIFT
Answer the question using a single word or phrase.
Ground Truth: Yes.
Vanilla's prediction; Yes.
41245
Full-FT's prediction: The "Hello, World!' program in C++, written by Bjarne Stroustrup in 1984, has been compiled and run on a 1950s UNIVAC computer.....
44,77%
20000
nore
Figure 6: structions between Vanilla and Full-FT.
Comparison of violation in-
Figure 5: Violating instruction on MME..
Obs 5: Capacities degradation in different dimensions interrelated. Specifically, the degra- dation of instruction-following capability negatively impacts other capacities. Benchmarks such as MME, SEEDBench2 Plus, etc, which require Yes/No multiple-choice formats, necessitate robust instruction- followinn capability. However, according to Figures 5 and 6 we find that deterioration cnterrcctionffollowwiii capability exerts cascading negative effects on these capabilities.
Challenges
Challenge 2: Parameter modification methods cause capability degradation in injected LMMs, exhibiting a consistent severity ranking and cascading effect.
EXPLORATIONS OF EVOLVING KNOWLEDGE INJECTION
5.1 KNOWLEDGE AUGMENTATION STRENGTHENS KNOWLEDGE ADAPTATION
Section 4.2 shows that existing methods struggle with knowledge injection. Data augmentation, though common for limited-data scenarios, fails improve semantic knowledge learning. Knowledge augmentation, however, substantially enhances model comprehension and adaptation.
The core distinction between data augmentation and knowledge augmentation lies their augmen- tation goals: the former operates solely surface-level features (e.g., pixel transformations in images or replacement of synonyms text), whereas the latter explicitly augments knowledge-relatee related semantic information. In Figure 7, we use Figure I's example of Xiaomi SU7 to illustrate both snowledge-agnoste and knowledge-aware augmentation.
W/o Angronncttion
W/ls Angrectttionn
Knuss Knuwledge Agunotic Augmentation
Kaowledge Aware Augmeriation
The Jiasmn SU7 in fint accnncndann created by Xiaemi Auto Company....
Roowdedge-Aggaee Agsovtiic Suprentation
Niarni Company....
21
23
-20
18
200
17-
-15
Scae
30
Wh Aspanitatio ppelegroggucrs Rowlldee-acc
Angnenltttnn
GEM
Fi Score
GEM
Fi Sconeo
Textual Knowledge Augmentation
(b) Knowledge Augmentation
Figure 7: Examples and performance knowledge-agnostic and knowledge-aware augmen- tation. (a) Performance of textual knowledge augmentation across the entire MMEVOKE. (b) Performance of visual knowledge augmentation solely on Entity subset.
8

---

Published as a conference paper at ICLR 2026
Knowledge-agnostic augmentation is rule-based mechanical augmentation. "created" in description of is replaced by "built" and rotation operation "Xiaomi SU7" image does not require understanding of "iiaomi SU7" related knowledge, only mechanical augmentation.
Knowledge-aware augmentation is a knowledge-driven semantic augmentation. For text, it creatively restated knowledge based on profound understanding of description. Additionally, introducing real-world images greatly enrich model's perception of the concepi of "Xiaomi SU7".
Additional results Appendix E. Based on Figure and Table 3, we have following observations:
Obs 6: Knowledgg-agnostic augmentation leads to negative effects. In Figure 7, text/images knowledge-agnostic augmentation results in decrease of 5.66% and 47.36% in CEM, along with a reduction of 7.78% and 12.76% in F1-Score, respectively. This demonstrates that knowledge- agnostic augmentation fails strength knowledge adaptation.
Obs 7: Knowledge-aware augmentation strengths knowledge adaptation. In Figure 7, using just a single data instance for textual/visual knowledge- aware augmentation yields improvements of 11.32% and 43.28% in CEM, along with 39.82% and 13.19% in FI-Score, respectively. Moreover, performance further improves with increasing data quantity. This demonstrates that knowledge- aware augmentation is crucial for strengthening knowledge adaptation.
Obs 8: Surprisingly, knowledge augmentation can partially mitigate capability degradation. In Table 3, knowledge augmentation outperforms both Full-FT and LoRA on benchmarks such as MMBench, SEEDBench2 Plus, and ScienceQA. Furthermore, text knowledge-awar augmentation surpasses not only Full-FT and LoRA but also conventional knowledge retention techniques (EWC & LwF). This novel discovery points to promising new research direction.
Insights
Insight 1: Knowledge-agnostic augmentation proves detrimental and fails to add semantic knowledge. Conversely, knowledge-awee aware augmentation confirms that knowledge-cenne centric strategies strength knowledge adaptation and concurrently mitigate capability degradation.
5.2 KNOWLEDGE RETENTION MITIGATES CAPABILITY DEGRADATION
To efficiently mitigate capability degradation after knowledge injection, we introduce knowledge retention methods: data replay (e.g. Replay), mixture experts (e.g., MoELoRA (Luo et al., 2024)), parameter regularization (e.g., EWC (Kirkpatrick et al., 2017) and LwF (Li & Hoiem, 2017)). Specifically, we categorize Replay into Rel +10% and ReplayIaRa LoRA, fixed-quantity data (10% of MMEVOKE's data size) from LLaVA-v1.55 pre-training data and MMEVOKE's injection data Dx are mixed and used for fine-tuning employing Full-FT and LoRA strategies. Additional results are in Appendix F. Table shows results and have following observations:
FullFF
+10% randomly sampled
Obs 9: Replay reactivates old knowledge networks by forcing model "review the old". Specifically, ReplayFulli Full-FT (ranked 3rd) and ReplayLoRA (ranked Ist) mitigate model capability degradation across all tests. Notably, Replay.iis MMMU and Math Vision, respectively.
+10%
Full-FT surpasses Vanilla by 1.75% and 2.20% on
Obs 10: MoELoRA carves out dedicated zones for new knowledge to prevent parameter conflicts. Specifically, MoELoRA (ranked 2nd) exhibits minimal degradation of only 2.05% in instruction following and surpasses Vanilla by 1.18% on Vista.
Obs 11: EWC LwF attempt to freeze prior knowledge areas through indirect and rigid constraints. Specifically, EWC (ranked 5th) and LwF (ranked 6th) provide almost no mitigation of degradation on MIA-Bench Bench, MMDU, and HallusionBench Moreover, both EWC on OCRBench, ScienceQA, and Math Vista and LwF MMMU, MMDU, Math Vision, and HallusiinBench underperform standard Full-FT and LoRA, further exacerbating capability degradation.
Insights
Direct Rehearsal (Replay) and Structured Separation (MoELoRA) effectively preserve old knowledge by retraining old data and isolating new knowledge. respectively. Indirect Constraint (EWC, LwF) fails due rigid parameter constraints impairing retention.
Insight 2:

---

Published as a conference paper at ICLR 2026
6 CONCLUSION AND DISCUSSION
In this paper, we systematically investigate multimodal evolving knowledge injection LMMs and propose a diverse benchmark. MMEVOKE. This work reveals two critical challenges, and correspond- ing explorations are conducted. Current research (Allen-Zhu & Li, 2024; Omar et al., 2023; Singhal et al., 2023) indicates that mere "data memorization" and genuine "knowledge internalization" distinct concepts. The former only enables models accurately fit training data, while the latter empowers models to effectively extract and manipulate factual knowledge. Similarly, in our work, knowledge-agnostic and knowledge-aware augmentation exhibit this distinction, with only knowledge-aware augmentation significantly enhancing a model's ability to internalize knowledge.
Although less effective than knowledge retention methods like Replay, MoËLoRA, knowledge-awaas augmentation can partially mitigate capability degradation. Therefore, exploring the synergy between these classes of methods promising research direction. Potential strategies include multi- stage training to decouple knowledge adaptation from retention, hybrid framework that integrates knowledge-aware augmentation into loss function. We posit that the synergy between the "proactive learning" of knowledge augmentation and the "capability preserving" nature of retention methods can more effectively tackle the challenges continuous injection of evolving knowledge.
ACKNOWLEDGEMENT
We would like to express gratitude to the anonymous reviewers for their kind comments. This research is supported by Anhui Provincial Natural Science Foundation (Grant No. No.2448085QF214) the National Key R&D Program of China (Grant No. Y322002233333)340000 the Fundamental Research Funds for the Central Universities (Grant No. WK2080000266), the Opening Project of the State Key Laboratory of General Artificial Intelligence (Project N0.SKLAAG20250006 No.SKLAG120240P100 No.SKLAAG10240P11 1) and project 2R2025QC1570 supported by Shandong Provincial Natural Science Foundation.
ETHICS STATEMENT
The primary motivation for MMEVOKE is to investigate the effectiveness existing knowledge injection methods in learning knowledge. Due the scarcity of evolving knowledge in authentic multimodal contexts, we design pipeline to collect such data from the internet. However, information online is complex and may contain false, harmful, biased content. We therefore urge researchers to collect evolving knowledge responsibly and cautiously, ensuring that the information injected into models accurate and safe, and that the technology is applied ethically.
REPRODUCIBILITY STATEMENT
To guarantee reproducibility. our source code and datasets are available at htttps://evokee github. io/ to facilitate verification and reproduction of our work by the community.
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. In International Conference on Machine Learning, 2024. 10
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arxiv 2023. arXiv preprint arXiv:2308 12966, 1(8), 2023. 3.
Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. Llava steering: Visual instruction tuning with 500x fewer parameters through modality linear prreseentationstteerngg In Annual Meeting of the Association for Computational Linguistics, 2024.
38
10

---

Published as a conference paper at ICLR 2026
Jinhe Bi, Yifan Wang, Danqi Yan, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. Prism: Self-pruning intrinsic selection method for training-free free multimodal data selection. ArXiv, abs/2502.12119, 2025. 35
Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, and Simone Calderara. On the effectiveness of lipschitz-driven rehearsal in continual learning. Advances in Neural Information Processing Systems,
31901, 2022. 3
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877-1901. 2020.
Xusheng Cao, Haori Lu, Linlan Huang, Xialei Liu, and Ming-Ming Cheng. Generative multi-modal models are good class incremental learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo. Wei Xue, Yike Guo, and Jie Fu. Rq-rag: Learning to refine queries for retrieval augmented generation. arXiv preprint arXiv:2404. 24:4.000400 2024. 6
Cheng Chen, Junchen Zhu, Xu Luo, Hengtao Shen, Jingkuan Song, and Lianli Gao. Coin: A benchmark of continual instruction tuning for multimodel large language models. Advances in Neural Information Processing Systems, 37:57817-578400 2025. 22
Qizhou Chen, Chengyu Wang. Dakan Wang. Taolin Zhang, Wangyue Li, and Xiaofeng He. Lifelong knowledge editing for vision language models with low-rank mixture-of-expertt 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9455-9466 2024a. 38
Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, al. Lifelong knowledge editing for Iims with retrieval-augmented continuous prompt learning. In Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing, pp- 13565-13580, 2024b. 38
Qizhou Chen, Taolin Zhang, Dongyang Li, Longtao Huang, Hui Xue, Chengyu Wang, and Xiaofeng He. Lifelong knowledge editing for Iims with cetrieval-augmenne continuous prompt learning, In Conference on Empirical Methods in Natural Language Processing, 2024c.
Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web conference 2022, pp. 2778-2788, 2022. 3
Siyuan Cheng, Bozhong Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, and Ningyu Zhang. Can we edit multimodal large language models? arXiv preprint arXiv:2310.08445, 2023. 38
Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Liao, et al. A survey multimodal large language models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 958-979, 2024.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning. Advances in Neural Information Processing Systems, 2023. 3
Yuntao Du, Kailin Jiang, Zhi Gao, Chenrui Shi, Zilong Zheng, Siyuan Qi, and Qing Li. linke-bencht A multimodal editing benchmark for diverse visual knowledge. ArXiv, abs/2502.19870 2025. 38
Zhihao Fan, Yeyun Gong, Zhongyu Wei, Siyuan Wang, Yameng Huang, Jian Jiao, Xuan-Jing Huang, Nan Duan, and Ruofei Zhang. An enhanced knowledge injection model for commonsense generation. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 2014-2025, 2020. 3
11

---

Published as a conference paper at ICLR 2026
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXt::2306.13399 133944 2023. 7,27
Chaoyou Fu, Yifan Zhang, Shukang Yin, Bo Li, Xinyu Fang, Sirui Zhao, Haodong Duan, Xing Sun, Ziwei Liu, Liang Wang. Caifeng Shan, and Ran He. Mme-survey: A comprehensive survey on evaluation of multimodal llms. CoRR, 0ts/24111.152900 2024. 7
Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entan- gled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14375-14388 2024. 7, 28
Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
Han Huang, Haitian Zhong, Tao Yu, Qiang Liu, Shu Wu, Liang Wang. and Tieniu Tan. Vlkeb: A large vision-language model knowledge editing benchmark. Advances in Neural Information Processing Systems, 37:9257-9280 2024. 38
Yukang Huo and Hao Tang. When continue learning meets multimodal large language model: A survey. arXiv preprini arXiv:2503.0/887, 2025. 3
Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, KIM Gyeonghun, Stan- ley Jungkyu Choi, and Minjoon Seo. Towards continual knowledge learning of language models. In International Conference on Learning Representations, 2022.
Yifan Jia, Kailin Jiang, Yuyang Liang, Qihan Ren, Yi Xin, Rui Yang, Fenze Feng, Mingcai Chen, Hengyang Lu, Haozhe Wang, Xiaoye Qu, Dongrui Liu, Lizhen Cui, and Yuntao Du. Benchmarking multimodal knowledge conflict for large multimodal models. ArXiv, abs/2505. 19509, 2025. 38
Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Guojun Ma, Mingyang Wan, Xiang Wang, Xiangnan He, and Tat-seng Chua. Anyedit: Edit any knowledge encoded in language models. arXiv preprint arXiv:2502.05622. 2025a. 38
Kailin Jiang, Hongbo Jiang, Ning Jiang, Zhi Gao, Jinhe Bi, Yuchen Ren, Bin Li, Yuntao Du, Lei Liu, and Qing Li. Kore: Enhancing knowledge injection for large multimodal models via srowledge-oriented augmentations and constraints. ArXiv, abs/2510.1993 2025b. 3
Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, and Wei Wang. Learning to edit: Aligning llms with knowledge editing. In Annual Meeting of the Association for Computational Linguistics, 2024. 38
Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing, pp. 7969-7992, 2023.
Hailey Joren, Jianyi Zhang, Chun-Sung Ferng, Da-Cheng Juan, Ankur Taly, and Cyrus Rashtchian. Sufficient context: A new lens on retrieval augmented generation systems. International Conference on Learning Representations, 2025. 7
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A Smith, Yejin Choi, Kentaro Inui, et al. Realtime qa: What's the answer right now? Advances in neural information processing systems, 36:49025-49043 2023. 2
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho. Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114 (13):3521-3526, 2017. 3,
12

---

Published as a conference paper at ICLR 2026
Anne Lauscher, Olga Majewska, Leonardo FR Ribeiro, Iryna Gurevych, Nikolai Rozanov, and Goran Glavaš. Common sense or world knowledge? investigating adapter-basee knowledge injection into pretrained transformers. arXiv preprins arXiv:2005.11787 2020. 3
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- genera- tion for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 9459-9474, 2020a.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. Retrieval-augmented genera- tion for knowledge-intensiv nlp tasks. Advances in Neural Information Processing Systems, 33: 9459-9474, 2020b.
Bohao Li, Yuying Ge, Yi Chen, Yixiao Ge, Ruimao Zhang, and Ying Shan. Seed-bench-2-plus: Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint ar Xiv:22400 16799 2024a. 7.27
Chuanhao Li, Zhen Li, Chenchen Jing. Shuo Liu, Wenqi Shao, Yuwei Wu, Ping Luo, Yu Qiao, and Kaipeng Zhang. Searchlvims: A plug-and-play framework for augmenting large vision-language models by searching up-to-date internet knowledge. Advances Neural Information Processing Systems, 38, 2024b. 4
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-in image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, pp. 19740-19742. PMLR, 2023a. 3
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 292-305, 2023b. 7, 28
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-299 2947, 2017. 3,
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems, 36:34892-34999 2023a. 1
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 26296-26306, 2024a. 3
Wenzhuo Liu, Xin-Jian Wu, Fei Zhu, Ming-Ming Yu, Chuang Wang, and Cheng-Lin Liu. Class incremental learning with self-supervised pre-training and prototype learning. Patlern Recognition, 157:110943, 2025a. 3
Wenzhuo Liu, Fei Zhu, Longhui Wei, and Qi Tian. C-clip: Multimodal continual learning for vision-language model. In The Thirteenth International Conference on Learning Representations, 2025b.
Xialei Liu, Jiang-Tian Zhai, Andrew D Bagdanov, Ke Li, and Ming-Ming Cheng. Task-adaptive saliency guidance for exemplar-free free class incremental learning. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2024b.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li. Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European Conference Computer Vision, pp. 216-233 233. Springer, 2024c. 7, 27
Yuliang Liu, Zhang Li, Biao Yang, Chunyuan Li, Xucheng Yin. Cheng-lin Liu, Lianwen Jin, and Xiang Bai. On the hidden mystery of ocr in large multimodal models. arXiv preprint arXiv:2305.07880 075.099.. 2023b. 7,27
Yuyang Liu. Qiuhe Hong, Linlan Huang, Alexandra Gomez- Villa, Dipam Goswami, Xialei Liu, Joost van de Weijer, and Yonghong Tian. Continual learning for vlms: survey and taxonomy beyond forgetting. arXiv preprint a50iv:2508. 04227, 2025c. 3
13

---

Published as a conference paper at ICLR 2026
Ziyu Liu, Tao Chu, Yuhang Zang, Xilin Wei, Xiaoyi Dong, Pan Zhang, Zijian Liang, Yuanjun Xiong, Yu Qiao, Dahua Lin, et al. Mmdu: A multi-turn multi-image dialog understanding benchmark and instruction- 8698-8733, 2025d. 7, 28
dataset for Ivims. Advances in Neural Information Processing Systems, 37:
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-We Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521, 2022. 7
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representations, 2024. 7, 28
Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. Moelora: Contrastive learning guided mixture of experts on parameter-effiient fine-tuning for large language models. arXiv preprint arXiv:2402.12881 2024. 9
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7765-77777 2018.
Ariana Martino, Michael lannelli, and Coleen Truong. Knowledge injection to counter large language model (Ilm) hallucination. In European Semantic Web Conference, pp. 182-185. Springer, 2023.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359- 17372, 2022a. 38
Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in transformer. arXiv preprint arXiv:2210.07229, 2022b. 38
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 3470-3487, 2022. 5
Seyed Mahed Mousavi, Simone Alghisi, and Giuseppe Riccardi. Dyknow: dynamically verifying time-sensitive factual knowledge in llms. arXiv preprint arXiv:2404.08700 0..40.00000, 2024. 2
Reicchiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2 1et:22122.0933 2021.
Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. Text and code embeddings by contrastive pre-training. arXiv preprint urXiv:2201. 10005, 2022. 5
Reham Omar, Omij Mangukiya, Panos Kalnis, and Essam Mansour. Chatgpt versus traditional question answering for knowledge graphs: Current status and future directions towards knowledge graph chatbots. CORR, abs/2302.06466, 2023. 10
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 17730-27744 2022. 5
Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. Fine-tuning retrieval? comparing knowledge injection in 1lms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 237-250, 2024. 1
Siyuan Qi, Bangcheng Yang, Kailin Jiang, Xiaobo Wang, Jiaqi Li, Yifan Zhong, Yaodong Yang, and Zilong Zheng. In-context editing: Learning knowledge from self-induced distributions. ArXiv, abs/2406.1 1194, 2024. 38
14

---

Published as a conference paper at ICLR 2026
Yusu Qian, Hanrong Ye, Jean- Phin-Phil Fauconnier, Peter Grasch, Yinfei Yang, and Zhe Gan. Mia- bench: Towards better instruction following evaluation of multimodal llms. arXiv preprint arXiv:2407.01509, 2024. 28
Jingyang Qiao, Zhizhong Zhang, Xin Tan, Yanyun Qu, Shouhong Ding, and Yuan Xie. Large continual instruction assistant. arXiv preprint 24X.:2440000066 2024. 3
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Meyton-Brown and Yoav Shoham. In-context retrieval- language models. Transactions of the Association for Computational Linguistics, 11:131--1331 2023.
Xuankun Rong, Wenke Huang, Jian Liang, Jinhe Bi. Xun Xiao, Yiming Li, Bo Du, and Mang Ye. Backdoor cleaning without external guidance milm fine-tuning. ArXiv, abs/2505. 16916, 2025. 38
Joan Serra, Didac Suris. Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pp. 4548 4557. PMIR, 2018. 3
Yilin Shen, Xiangyu Zeng, and Hongxia Jin, A progressive model to enable continual learning for semantic slot filling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language P Processing (EMNLP-IJCNLLP), pp. 1279-1284, 2019.
Chenglei Si. Zhe Gan, Zhengyuan Yang. Shuohang Wang, Jianfeng Wang, Jordan Lee Boyd-Graber, and Lijuan Wang. Prompting GPT-3 to be reliable. In The Eleventh International Conference on Learning Representations, 2023.
Karan Singhal, Shekoofch Azizi, Tao Tu, S Sara Mahdavi, Jason Wei. Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620 2023. 1, 10
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulezyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943-950, 2025.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante- Donghyun Kim, Assaf Arbellc, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 119991 1919, 2023. 3
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinkimg: Learning to think with images via visual tool reinforcement learning, arXiv preprint arXiv:2505.088: 08617, 2025.
Wei Tang, Yixin Cao, Yang Deng, Jiahao Ying, Bo Wang, Yizhe Yang, Yuyue Zhao Qi Zhang, Xuan-Jing Huang, Yu-Gang Jiang, et al. Evowiki: Evaluating lims evolving knowledge. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 948-964, 2025. 2.7
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction-follooinng model. Stanford Center for Research on Foundation Models. hilps:/ferfm. stanford. aadu/2022/03/11//apaeu himl, 36))7, 2023.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- cann-Bapistte Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. / /805, 2023. 6
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095-95166 2025a. 7, 28
15

---

Published as a conference paper at ICLR 2026
Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan- Jing Huang. Orthogonal subspace learning for language model continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10658-10671, 2023, 3
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, et al. Generalization via declarative instructions on 1600+ nlp tasks, In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 5085-5109, 2022a.
suppe--ntturalistrrutiooo
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct Aligning language models with self self-generated instructions. In Proceedings of the 6l Annual Meeting of the Association for Computational Linguistics, pp.
13484-1 13508, 2023b.
Yujun Wang, Jinhe Bi, Yunpu Ma, and Soeren Pirk. Ascd: Attention-steerable contrastive decoding for reducing hallucination in milm. ArXiv, 200o/2506.14766, 2025b. 38
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Putlern Recognition, pp. 139-149, 2022b.
Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. Uniir: Training and benchmarking universal multimodal information retrievers. In European Conference on Computer Vision, pp. 387-404. Springer, 2024. 6
Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, and Ping Luo. Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models. arXiv preprint Xr.iv::3306. 09265, 2023.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In International Conference on Learning Representations, 2018.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal under- standing and reasoning benchmark for expert agi. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-95677 2024. 7,27
Gengyuan Zhang, Jinhe Bi, Jindong Gu, and Volker Tresp. Spot! revisiting video-language models for event understanding. ArXiv, abs/231 1. 12919, 2023.
Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286 2024. 38
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. survey of large language models. arXiv preprint arXiv:2303. 18223, 1(2), 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing Dision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Representations, 2024. 3
16

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
A THE USE OF LARGE LANGUAGE MODELS IN MMEVOKE
18
B MORE DETAILS ABOUT MMEVOKE
18
B.1 PRESENTATION OF MMEVOKE BENCHMARK
18
B.2 WORD CLOUD DISTRIBUTION
19
B.3 COMPLETE SUBFIELDS DISTRIBUTION
19
B.4 HUMAN STUDY TOWARDS BENCHMARK QUALITY TEST
19
B.5 DENSITY DISTRIBUTION
20
B.6 FINE-GRAINED DIFFICULTY LEVEL OF MMЕVOKE
20
C. MORE RESULTS ABOUT МMЕVOKE
21
C.I MORE QUANTITATIVE EXPERIMENTAL RESULTS ABOUT RQI
21
C.2 SEQUENTIAL FINE-TUNING,
22
C.3 ABLATION EXPERIMENTS IN MM-RAG
24
C.4 MORE QUALITATIVE RESULTS ABOUT MMEVOKE
25
C.5 ERROR ANALYSIS
25
D MORE DETAILS ON CAPABILITY DEGRADATION
27
D.I CAPABILITY DEGRADATION RANKING
27
D.2 FINE-GRAINED DIMENSIONAL RESULTS ON GENERAL CAPABILITY TESTS
27
D.3 MORE EXAMPLES OF VIOLATING INSTRUCTION
32
E MORE DETAILS ABOUT KNOWLEDGE AUGMENTATION ON MMEVOKE
33
E.1 TRAINING Loss PERSPECTIVES ON EFFECTIVENESS OF KNOWLEDGE AUGMEN-
TATION
33
E.2 PERFORMANCE OF KNOWLEDGE AUGMENTATION IN GENERAL CAPABILITY TESTS
33
F MORE EXPERIMENTAL RESULTS ABOUT KNOWLEDGE RETENTION METH- ODS ON MMEVOKE
35
F.1 THE KNOWLEDGE INJECTION PERFORMANCE OF KNOWLEDGE RETENTION METII ON MMEVOKE
35
F.2 Is BETTER TO HAVE MORE DATA FOR REPLAY?
35
G PROMPT FOR GENERATION
36
H KNOWLEDGE EDITING ON MMEVOKE
38
REASONING CONSISTENCY, HALLUCINATION REDUCTION, AND FAC-
TUAL GROUNDING
39
17

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS IN MMEVOKE
In this section, we elaborate on the precise role of large language models within MMEVOKE, as detailed below.
Usage 1: MMEVOKE's construction. In Section 3.1, we specify that GPT-4o is employed for content summarization and QA generation, which aligns with current research practices.
Usage 2: MMEVOKE's evaluation. Section 4.2, we evaluate MMEVOKE using Gemini-2.0-
Flash, Gemini-2.5-PPP Perplexity AI, and GPT-4.1, 1, following standard benchmarking practices. Usage 3: General capability tests. In Section 4.3, we employ MlAeBench, MMDU, Math Vista, and Math Vision, whose evaluation requires large language models as judges practice consistent with current research standards.
Usage 4: Paper grammar polishing. The paper is initially drafted by humans and subsequently polished for grammar using LMMs, a practice consistent with current research norms.
B MORE DETAILS ABOUT MMEVOKE
In this section, further demonstrate the details of MMEVOKE, including benchmark presentation, complete subfields distribution, word cloud distribution, human study, fine-grained difficulty level results and release plan.
B.1 PRESENTATION OF MMEVOKE BENCHMARK
Figure 8 presents additional examples of MMEVOKE, encompassing four distinct subfields: Politics, Science, Video Game, and Songs. Each subfield showcases relevant Type, Knowledge Summary, Knowledge Image, Query, Query Image. Specifically, four examples are as follows:
tax the
Type:
Vidre Caraee
Kavningg: Deinz a canpein
may in tieter Panaytvanu
Slack Wickong
"Kaznlodge Iasueds
Dauld Traep. fring shoes the gaod Thrrgicorr
monsnat fanar Prouiee
Alou, beries vasio relowal
Waddow Aggant 10, 2004, with
relossed Maystako nnd
Kaankodgg Emnger
MINKING
Who mncostaaee
Query: Hon mony oapies the video
the Lange conpeisa
gance image wend will
ndly in Buta. Paaiylraai?
Imagee
AEW
ROSÉ "UlAI
Typec
SeaEp
Type:
ScRRec
Knowlegge: App!
amorted
South Keeena suger Ridd and
Hoyled add Gootley Harve fox plasering oxtribitses to nackiic
Asseiiei segc-aepvtik ttres
-Kainlelge
Mays was relesed dhroeh The
"Kavedelgg Imaaee-
lerning Swtrreg today's
Oeades Atlant
tncaslogies
sionct the Nadted tirre Plysis with the lnage?
Query: Which county's dreding
fane ingired the sog the tmage?
<Query
Figure 8: Examples News/Entity Evolving Knowledge in MMEVOKE, including Type, Knowl- edge Summary, Knowledge Image, Query, Query Image. Examples are taken from different clusters: Politics for News, Science for News, Video Game for Entity, and Songs for Entity.
Politics: Describes the unsuccessful assassination attempt targeting former U.S. President Donald Trump at campaign rally in Butler, Pennsylvania, on July 13, 2024. The query question asks for the identity of the individual depicted the image.
Science: Details the awarding of the 2024 Nobel Prize in Physics to John Hopfield and Geoffrey Hinton for their contributions. The query question inquires about the person who shared the Nobel
Prize with the individual shown in the image.
Video Game: Lists the video game Black Myth: Wukong, released on August 20, 2024. The query
question focuses on the game's 's sales figures during its first month.
Songs: Introduces the song Apt, performed by Russ and Brumo Mars. The query question concerns the drinking game that served inspiration for the song.
18

---

Published as a conference paper at ICLR 2026
These examples illustrate the diverse subfields of evolving knowledge captured within MMEVOKE, providing more detailed demonstration.
B.2 WORD CLOUD DISTRIBUTION
Donald Trump
neww aill
USS
indued Bileor
ppotentia
Ketrzyn
Ppoadiky
Castle
Ka Higuaa
Malte Delow
Wogse
dony offind
Elijah
Jones CAmericane football)
Jaime
Olias Francesco Mazzel
Cha Woo-min
ts stta Ceniiihiin
First
China
MG-ES5
Big Girls Don't Cry
(TV
Verna
series
Mersereau
Missing
Part
1900 Storm Memorial
remain
knowen
support
season sancem
wwinchallnngeee wwn.chlliooooooonnuue continue Jahangir Nattonn-Naaiin
plan
University
following
New York
Live at the Greek (The. Driver Fra album
(a) News Evolving Knowledge.
(b) Entity Evolving Knowledge.
Figure 9; Word Cloud Distributions of MMEVOKE.
Figure 9a, we show the word cloud distribution of News evolving knowledge. can be found that Trump appears more often, which may be because MMEVOKE contains large number of US political News data. Meanwhile, in Figure 9b, we present the word cloud distribution of entity names in the Entity evolving knowledge.
We have demonstrated the diversity of MMEVOKE benchmark through fine-grained subfields dis- tribution, key statistics, word cloud distribution, and multiple perspectives. At the same time, automated pipeline can continuously collect evolving knowledge and provide injection data for the knowledge injection field.
B.3 COMPLETE SUBFIELDS DISTRIBUTION
Style
Nedia
Investing
50
Scauinn
Asia
Economy
Middieeast
News
Entiyy
SaRKIEW
Science
#eacoins
Americas
Travel
Food
Climate
Ariic
We
Ca
Figure 10: Fine-grained subfields distribution News evolving knowledge.
Figure Fine-grained subfields distribution of Entity evolving knowledge.
In Figures 10 and 11, we comprehensively illustrate the fine-grained subfields distribution of the MMEVOKE benchmark, which includes 29 distinct subfields for News evolving knowledge and 130 subfields for Entity evolving knowledge, underscoring its exceptional diversity. This bench- mark serves as critical resource for the evolving knowledge injection domain, providing robust foundation for advancing research and development in the field.
B.4 HUMAN STUDY TOWARDS BENCHMARK QUALITY TEST
To verify the hallucination level of GPT-4o in data generation, We randomly selected 100 pieces of data from MMEVOKE during manual selection for human study. Specifically, four annotators scored the samples (1-5 scales, higher scores indicate greater purity) from the perspectives of content
19

---

Published as a conference paper at ICLR 2026
summarization, QA generation, and whether the summary contained information necessary answer the question. According to the results in Table 4, MMEVOKE exhibits high quality, demonstrating minimal hallucination during the data construction process.
Table 4: Human Study Towards Benchmark Quality Test.
Dimension
ALL
News
Entity
Q&A
4.86 (=0.01)
4.87 (#0.01)
4.85
MMEVOKE
Summary
4.98
(#0.01)
4.97 (#0.01)
4.98 (#0.02)
B.5
DENSITY DISTRIBUTION
Injuction Krowledga Length
Lengh
Lergth
15-
002
Cemiiy
0.10-
Deality
0.8
0.5-
000
0.00
Figure 12: Density distribution based on evolving knowledge sources.
Injuction Keswladge Length bistribution
Quastion Lungth Distribution
Length Distribution
1:54
003-
o15-
E--
a10-
0.68
20n
>0C
450
20
35
Langth (eord))
Length (vanda)
(wordd))
qum
buileeds
Neahhs
TeinTan works
Figure 13: Density distribution of fine-grained subfields based on evolving knowledge.
B.6 FINE-GRAINED DIFFICULTY LEVEL OF MMEVOKE
To further diversify MMEVOKF, we constructed 568 Counterfactual Reasoning and 3-Hop QA pairs using GPT-4o, and extracted their corresponding SimpleVQA data, yielding experimental results comparing fine-grained difficulty levels. The Simple VQA here refers to the QA data of MMЕVOKE itself. Table 5 shows the difficulty ranking: Counterfactual Reasoning < SimpleVQA < 3-Hop,, and 48.24% 24% (avg) cases have SimpleVQA failing while Counterfactual Reasoning succeeding, and 40.06% (avg) have SimpleVQA. succeeding but H-Hop failing.
Table 5: The performance different difficulty levels on MMEVOKE.
ALL
News
Entity
Task
Method
CEM
F1-Score
CEM
F1-Score
CEM
F1-Score
SimpleVQA
Full-FT
16.55
14.82
17.43
14.12
15,53
15.61
Sufficient Context
55.63
76.00
55.59
72.05
55.68
80.54
Full-FT
12.15
5.65
11.18
5.22
13.26
6.14
3-Hop
Sufficient Context
40.49
52.58
38.16
51.49
43.18
53.82
Full-FT
70.42
70.42
74.01
74.01
66.29
66.29
Counterfactual Reasoning
Sufficient Context
76.58
76.58
65.46
65.46
89.39
89.39
20

---

Published as a conference paper at ICLR 2026
C MORE RESULTS ABOUT MMЕVOKE
C.I MORE QUANTITATIVE EXPERIMENTAL RESULTS ABOUT RQ1
Table 6: Performance of knowledge injection methods on MMEVOKE. ALL. News.Avg, and Entity.Avg respectively show the performance of knowledge injection methods on entire MMEVOKE, News subset, and Entity subset. Orange value marks the best performance of methods on LLaVA- v1.5 and Qwen- Ww-n-VV-CCht,, as well as the best performance of models in Web Search Engine and Sufficient Context (vertical perspective). Red value indicates knowledge subfield with the best performance of the same method and model on different fine-graine greinea subfields, while blue value indicates knowledge subfield with the worst performance (horizontal perspective). PO: Politics; SP: Sports; BU: Business; HE: Health; CE: Celebrity; Fl: Film; AL: Album; WR: Written Work.
CRM
CB3
R1T
CSM
F1
CRM
911
CEMTT
CRMF
C3MI
PIT
CEM
FI
CSN
CRMM
RAT
CSMM
Yoak
4.89
6.34
11.5%
1.52
5.66
9.24
16.50
15:99
11.54
2.26
6.45
1.2
6.4
2.9
5.1
3.7
642
5/79
1500
1:55
16.14
17:00
13.88
13.30
13.0
11
19.3
16.47
$2.5
13.4
1911
1699
13.18
1913
7.66
19.35
16.11
17:23
18.54
17'6
19.11
78.86
18.35
1191
25
15.01
1264
1358
17:9
15:0
15.25
8.77
11
17.44
34.5
36.22
3723
23.18
36.3
24.7
34.87
51.77
3005
1.30
1780
15.0
139
238
13.56
3.58
97.51
41.1
31.50
35.8
58.23
65.94
58.85
35.22
50.00
41.30
6354
46.98
63.02
46.39
1A11
1641
642
11.00
13.11
16.42
23.27
12.42
6.55
14.3
1406
11.50
5.22
13.52
Tull
6:51
1984
0.37
12.55
8.10
0.3
13:07
17.95
17.56
11.00
4.41
1933
535
0.5
013
12:6
5.79
3038
18:17
13:17
MSE
71:94
11.34
11:11
1971
09.01
15:11
13.29
15.30
2017
1516
56.9
1300
500
19:71
11:00
19.90
22.31
33.09
923
18:45
3833
20.36
2734
35.5
33.8
53.00
Chencrma Mah Snerch Anginer
18211
3623
31.33
1591
1981
21.4
22.85
36:33
3.32
1651
25.5
3525
25.4
11.6
23.57
16.35
1981
6768
01:00
1106
74
17.00
15.03
01.11
3735
3276
56.55
07.51
18.30
00.94
18:11
(3.11
36777
5545
3503
Onea-VL-Chrr
40.5
06.00
3535
50.5
12.0
68.0
58.35
57.25
0:35
60.00
07.4
5087
4.28
6011
8228
58.50
15.41
-3.66
05.39
12.15
9145
12A0
16.7
17.0
69.99
3500
71.99
6155
63335
74.05
7165
6337
13.3
01.14
14.00
19.0
18.90
10:0
72.35
GPS44
3502
88.38
65.1
64.44
09.07
7.36
71.1
7068
8324
800
88.18
06.32
24.66
Table 6 presents the quantitative experimental results of RQL, revealing that no method achieves robust
injection performance, with significant performance variance observed across different fine-grained subfields knowledge. Specifically, We have obtained further observations:
Obs 1: In Table 6, across nearly all evaluated methods, News knowledge injection performance consistently outperforms Entity knowledge. We attribute this gap to their fundamental differences in learning difficulty. Entity knowledge introduces entirely novel concepts to model, posing a substantial learning challenge. In contrast, News knowledge primarily establishes new and complex relationships among existing entities, which represents a comparatively lower learning barrier.
Obs 2: The performance of knowledge in the same subfield varies depending on the method used. For example, in Full FT, LoRA, and MM-RAG Text-Olll the performance of film knowledge is poor. In sharp contrast, it performs better when using IM-RAGImage-Oni MM-RAG M Sufficient Context, and Web Search.
Obs 3: A significant performance variance among different strategies within same method. Notably, MM-R AG Text-Only is more effective for injecting News knowledge, while AMM-RA-lmage-Oni is better suited for Entity knowledge. This discrepancy indicates that knowledge injection is optimized when the modality of the feature aligns with the nature of the knowledge source (textual features for News and visual features for Entity).
Obs 4: The performance of the same subfield knowledge differs across models. For instance, Health and Written work perform better on Qwen-VL-Chal VL-Chat; Sport and Business perform better on LLaVA-v1.5 1.5. This is likely due to significant distributional differences in types of knowledge data encountered during pre-training of different models.
Obs 5: Politics knowledge contains wide range of professional terms and complex concepts that are difficult to learn, ranking lowest among almost all methods.
Observations
Observation 1: Current knowledge injection methods have significant domain specificity for different fine- -rained subfield knowledge.
21

---

Published as a conference paper at ICLR 2026
Table 7: The performance of knowledge injection methods on Entity subset of M.EE TEL: Television Series; COM: Company; VID: Video Game; CHU: Church Building; SIN: Single;
OGR: Organization; PAI: Painting; MOT: Motor Car.
TEL
00M
VID
CHU
SIN
ORG
PAI
MOT
CEM
F1
CEM 1
F1
CEM
P1
CRM
F1
CEM
F11
CEM 7
F1
CEM T
FT
CEM 7
PT
LL
1.12
4.55
9:51
2.70
6.31
0.00
11.00
0.00
4.76
FT
13.97
10.29
29.21
14.15
10.34
7.32
36.53
22.07
15.91
8.55
27.03
15.52
17.56
13.53
7.14
6.23
16.33
1.41
MM RAG Tosoe
3.35
6.15
4.49
14.31
5.17
21.81
9.16
18.10
2.27
20.72
2.70
13.59
14:29
21.31
7.14
27.55
RAGi 1h4
54.26
90.75
24.32
88 88
53.57
70.95
43
57.M3
MMS-RAG
63.81
10.34
63.49
12
65.77
14.69
64.90
91.83
61.90
20:35
64:20
83.11
31:43
68. 90
Qbeen- YL-Chut
Nacilla
13.35
1.72
10.20
11.33
0.000
2.58
7.14
13.10
0.00
10.37
Pull-P
8.94
16.49
1.13
11.08
3.45
15.54
1.06
16.91
5,82
15.75
5.41
8.61
10.71
12.95
7.14
15.45
LoRA
7.26
13.55
1.12
8.64
1.72
3.85
2.64
9.50
6.52
13.61
2.70
5.29
10.71
15.95
0.000
8.33
31.20
13.51
23.01
39.19
10..4
$5.00
18.90
56.23
58.76
$3.91
$1.82
56.92
20:35
45.05
39.29
48.45
14.28
46 90
RAGloE
19.67
03.54
59.14
24.57
54.26
34,09
59.51
43.24
63.33
42.56
52.62
14.25
46.9
Commercial
Enginer
Gemini-2
19:55
$1.14
8.99
30.80
10.54
25.01
16.50
21.55 56
9.09
23.56
18.02
25.00
14.29
16.65
10000
76.11
Genini
46.55
20.41
43.15
66.37
43.24
59.98
46.43
38.27
7.14
35.45
Perplesity.. Al
43.90
$4.59
30.00
53.08
13:33
48.41
67.50
73.81
50.00
70.00
31:35
54.00
35.
83.67
15.33
13:39
GPE-4
S0.33
62.06
$2.61
$7.02
55.45
65.22
22.45
20.31
38.54
42.03
45.65
$2.45
17.86
30.53
0.000
15.99
Sufficeent Context
LLaWs-r1.
50.42
81.18
41.57
78.05
34,00
08.72
64.90
72.45
45.45
68.79
45.95
79.70
75.00
90.12
35.71
73:15
Qwen-V Vhhh
$1.56
73.06
19.35
33.60
15.86
65.28
$4.69
60.88
96.16
67.62
43:34
65.69
42.85
55. 60
42.86
73.47
Genini-255
73:70
55.10
68.18
82.73
56.76
75.37
85.02
50.00
75.25
GPP44
73.09
00.32
70.79
36.21
87 34
$3.84
99.18
77.27
29.55
91:44
64.86
83:34
39.79
81. 90
56 66.28
$4.97
Table 8: The performance of knowledge injection methods on News subset of MMEVOKE. ENT:
Entertainment; TEC: Tech; SCT: Science; TRA: Travel; FOO: Food; CLII: Climate; INV: Investing: STY: Style.
ENT
TRC
SCI
TRA
FOO
CLI1
INV
STY
CRM 1
P11
CEM
CEM
P1
CEM
CEM
F11
CEM
F1
CFM
P1
CRM
F1
Sanilla
6.79
9.35
6:79
9.35
6.70
9.35
11.90
18.57
10.26
17.83
8.11
13.87
18:28
23.71
13.90
16.20
FUlLET
11.47
13.02
15.79
12.56
29.57
24.16
35,00
LaRta
16.95
15.20
27.63
25.06
8.75
18:73
29:31
29:91
20.51
18.83
25.13
19.6
19.45
MM RLG Tredey
39.81
48.79
46.05
55.21
36.54
55.71
58.10
$4.50
33.33
50.65
37.84
53.51
37.63
47.06
68.85
78.51
23.03
28.02
21.43
30.09
23.00
MM- AMMRRA
52.16
63.67
62.11
$1.77
33.33
$2.89
47.62
62.33
41.03
57.78
35.14
53.05
38.71
48.23
59.84
67.32
Ovenn VZ-Chat
9.90
14.47
16.10
8.77
14.95
9.52
16.59
16.34
10.81
29.27
13.11
16.19
FT
11.27
14.64
17.31
18.79
8.77
13.78
14.29
23.89
17.95
27.35
19.93
21.42
35.48
35.54
16.35
19.15
LaRA
MM RAG Foat
31.88
38.00
26.71
51.27
42.11
48.99
58.10
50.56
20.51
39.66
35.14
06.65
43.01
52.75
00.66
66.14
2006
34.87
23.37
71.06
33
42.99
21.43
31.67
30.51
37.35
$1.40
50.25
41.45
45.18
47.30
55.69
40.48
90.46
05.21
32.43
51.64
$6.00
Commercial Af Hob Search Eegines
Genini-2 F.Fauh
26,99
20.03
38.52
46.00
15.70
22:97
16.67
30.40
23.08
30.52
Genini-2 5-Pm
59.75
61.43
66.76
31.58
37.64
53.98
63.00
48.72
68.65
89.6
68.11
Peplesity Al
59.65
66.15
47/06
55.20
45:45
49.13
50.00
70.05
33.33
40.74
37.50
64.58
33.38
40.12
71.88
74.36
57.34
59.50
22.81
35.29
50.00
50.29
66.67
56.89
40.54
Suffiiient Context
LLANA-115
65.12
78.31
63.92
73.61
47.30
66.3 30
57.14
72.37
51.28
76.58
51.35
63.07
60.22
72.83
75.41
85.15
Qwes
61.42
68.99
62.50
72,69
43.86
63.14
45.24
58.56
51.28
48.65
75.00
Genini 2.5 Pro
$1.17
83.08
75,00
82.35
61.40
66.34
73:81
82.47
66.67
$1.28
70.27
74.10
75.27
77.29
$2.79
$3.34
85.12
Tables and 8 present richer experimental results of fine-grained subfields, further verifying the significant domain specificity of existing knowledge injection methods and their inability to robustly implement knowledge injection.
C.2
SEQUENTIAL FINE-TUNING
C.2.1 SEQUENTIAL FINE-TUNING BASED ON TASKS
Sequential Fine-Tuuing refers the process of incrementally training models new tasks and data. Specifically, model weights obtained from previous tasks and data are used to initialize model parameters (Chen et al., 2025). In this section, we explore whether Sequential Fine-Tuning is more
22

---

Published as a conference paper at ICLR 2026
effective than One-Time Injection? We employed MMEVOKE for knowledge injection, randomly dividing the data into subsets of 4, 8, and 12 tasks. We consider each subset as a task and use these subsets to Sequential
the model.
Sequential Fine-Tuning impede the effective injection of multimodal evolving knowledge. As illustrated in Figure 14, the performance of LMMs exhibits a declining trend with progressive Sequential Fine-Tuning based on tasks. This degradation primarily stems from the disruption of previously fine-tuning parameters during each subsequent fine-tuning iteration. Consequently, the overall performance of LMMs progressively deteriorates. Furthermore, our investigation into the impact of Sequential Fine-Tuning steps revealed a negative correlation between the number of steps and LMMs performance, as evidenced by the values corresponding to the terminal points in each line graph. These findings underscore the importance of minimizing Sequential Fine-Tuning in practical applications to preserve model efficacy.
14.18
Narbr of Sequrrted PUR-FT
Number Sequrriad PLA-TT
Narser Separiid PUB-F7
Nember of Sepurtted Lara
Number of Sequorticl Lara
Number Sequertial Lard
8
Group -2
Ponil
--- Vanile 3F1
Figure 14: The results of LLaVA-v1.5 on Sequential Fine- Tone-Tuning based on Tasks. The data Dx and Do are evenly divided into g E {4, 8, 12} parts, namely Dx = { di .., d% n=1 and De (cgg de. ".
{ >= Sequential Fine-Tuning based on tasks refer to the situation where if
the current m-th Sequential Fine- Fine-Tuning has ended, it indicates that the model is being trained on
d. dg...... , in sequence; and evaluated on { U de U...UU.. U pos
C.2.2 SEQUENTIAL FINE-TUNING BASED ON SUBSETS
Tart Group
Thrt Group
Group
Sroup
Greup
Accenty
20
Group
Grop?
Greup
18-
10
10
30
Group
Groep
Group
Group
20
20
10
10
Test
Group
10
Grop
20
10
F-l-PT Nueber
Pul-FT Number Sequeriaa
Mumber
Pal-FT
Suqurrtia Pull-77 Namber Sequrriial Pul-i7
Groupin
12
Vonils
Vanillo FullFT
The results of LLaVA-v1.5 on Sequential Full-FT based on Subsets. Sequential Full-FT based on subset refer to the situation where if the current m-th Sequential Full-FT has ended, it indicates that the model being trained on d/. of.....dd di in sequence; and evaluate sequentially on one of d= d2 , di
Figure 15:
The results of Sequential Fine- Fine-Tuning based on subsets are shown in Figure 15 and 16. Each subgraph displays the performance changes of the L.Ms on the same subset as the Sequential Fine-Tuning process progresses. It can be observed that whether using Full-FT or LoRA as training strategies, as the number g of Sequential subset shows a downward trend. This discovery further indicates that Sequential Fine-Tuning is not conducive to injecting up-to-date knowledge into the LMMs.
increases, the performance of the model on the same
Observations
Observation 2: Both sequential task and subset fine-tuning impede the efficacy of knowledge injection, with performance degradation correlating with an increased number of tasks or subsets.
23

---

Published as a conference paper at ICLR 2026
Group
Group
Grou
Text
Test
30
10
3
GrAup
Group
Graph
22
10
Grupp
Group
Group
Test Group
8
30
10-
F10122
9101:12
Grop?
Greus
Grop
Grapp 10
Grop
Grap
Accemay
15
Gagurria LARA
Nasber Sequirial LoRA
Natter Sequetiill LoRA
Namber Geqportid
Nemter Sequertic
Mamber Suqurrtial LoRA
Group 4
Group=
---
Group= 12
Vorilla
Vonils LoRA
Figure 16: The results of 1. on Sequential LoRA based on Subsets. Sequential LoRA based on subset refer to the situation where if the current m-th Sequential LoRA has ended, indicates that the model is being trained on d .i...., de in sequence; and evaluate sequentially on one of ..
dg. de 41
dy
C.3 ABLATION EXPERIMENTS IN MM-RAG
Retrieval strategy, Example Number, and Pool Size are critical factors influencing the performance of MM-RAG, demonstrated by the experimental results presented in Figure 17 and 18.
Overall
40
40
25-
30-
20-
08
0.
100
50
60
50-
10-
25
20
08
066
10
0.6
0a
Reel
Peal
Text-Orly
Irage Only
UnIR
Text:Otly
Image Only
Figure 17: The results of LLaVA-v ablation study on MM-RAG about Retrieval Strategy and Example Number analysis.
Figure 18: The results of LLaVA-v1. ablation study on MM-RAG about Retrieval Strategy and Pool Size analysis.
Effect of Retrieval Strategy in MM-RAG. An interesting observation appears in the "News" subgraph, where the Text-Only approach significantly outperforms the Image-Only strategy. The reason for this difference is that textual information is important for news understanding than visual information, as valuable data cannot be retrieved solely through images. On the contrary, for Entity knowledge, visual information is more valuable than textual information.
Effect of Example Number in MM-RAG. We compared K E (1....,5) 5). and in the first row of Figure 17, the direct correlation between the performance of model and Example Number is shown. Our experiment revealed a convincing trend that the model performs using a monotonically increasing function of Example Number for three retrieval strategies. This observation indicates that an increase in the example number brings more diverse reference information, which has positive effect on the model's understanding and utilization of evolving knowledge.
Effect of Retrieval Pool Size in MM-RAG. Regarding the ablation experiment of pool size, our setup is to randomly select 20% of the corresponding data from D and Dx Do 20% and
20% For instance, when Pool Size 20%, Retrieve Pool D 20% When Pool Size 60%. Retrieve Pool Dx 20% + where D randomly selected 40% data from the Dx Dx 20% The evaluation data is always De 20% The experimental results, presented in the second row of Figure 18, demonstrate inverse correlation between MM-RAG's performance and Pool Size. This suggests that larger pool sizes hinder the retriever's ability to identify relevant information, a critical consideration for practical MM-RAG applications.
24

---

Published as a conference paper at ICLR 2026
Observations
Observation 3: retrieval pool size all contribute to strengthening knowledge injection performance.
Cross-modal retrieval strategies, larger number of examples, and smaller
C.4 MORE QUALITATIVE RESULTS ABOUT ММЕVOKE
Knowelodge:: The J004 Mobed Pricin Plyto tao basn umanted joba ligplicté ana ceetrrr itniee or pizseree coesrbuttee madiee leaning, boser ng oxters e indendingss Hates, the Theeersty Terate, baskd the 'podilld of Al, equouee coner nate gnoeth, propptigg deperture inv Gogge Hotre poiertia becetiu unchecked
User
LLaVA-v1.52
Qwen-VL- La--VLLhaattt
topec
Jann Parties
CBM Etaccree
CER: Scrre
baki:
Uaaal Weckun)
Fi-Soue
hapain
X
BAG-Toak-Oddi: Rkcard foreuus
Rati-testtalf: tees
(DI Fi-for::
CBM): F1:0oe::
RAG-Inage-Olly vayy
CEM:: 0 EI Scare
BAG-Uritn::
kAG-inier: leyeled foefthyy iiletes
CBM:
Fishore: 0.7
Commercial AI Web Search Engine
Sufficient Context
Geotii-20-aa Tetier
MaWkei::: Jcha lapfid))
CEM:: FLSSare:
Perporalty Al: Ichn liepteld
lllptetl
Scare
GPT-41: Head Mopraralieerrrrree
CIM: Fi-icam:
Figure 19; Qualitative example of CNN News science knowledge.
Wikipedia Fatity Automabile Model
Reowlelge: The Ragani Trxelsilee spocnins recnded sil-nngeer lyfral sworedsthe Ciene and ieated to unise ress unedled is celine drean velcie.
Question::
What is the groduction Emi of the automobile model in
griced ralliiee (US561 miltten/o camsel
alte coatiile
User
LLaVA-V1.5-72 V1B
Qwen-VL Boeen-L-Chht Chat
Fall::: 22
Fi-Gore:
X
BuliFT: 100
CEM: Elácare:
X
LaRA: 128
CEM: FLScare:
Latk:
CDI: 0
Fi-Sorn:
X
X
E331: F2.5ore:
RAG-teeag--oaly:
CEM::
Commercial AI Web Search Engine
Sufficient Context
F1.Sare:
Seovil-2.-990: 500
EER: FlSoxe
Periiiitty Al: 500 sots
CEM: Ft St-cer:: as
GBM: Ft Piskeee:
anfa
F1-Gam:::
Figure 20: Qualitative example of Wikipedia Entity automobile model knowledge.
C.5 ERROR ANALYSIS
Observing the qualitative examples Figures 19, 20, and 21, we find that, as demonstrated by the results in Table 6, existing knowledge injection methods perform poorly MMEVOKE, with even sufficient context failing to achieve perfect performance. Here, we conduct a detailed analysis of sufficient context.
25

---

Published as a conference paper at ICLR 2026
Even when provided with sufficient context, the model still generates hallucinations. For instance, in Figure 19, the response given by GPT-4.1 is entirely unrelated to the question and does not appear in the sufficient context, representing severe hallucination phenomenon. A similar hallucination issue persists in Figure 20. These concrete results indicate that merely improving the sufficiency of context far from adequate- the model's inherent reasoning and ability utilize contextual information equally critical. Hallucination remains an urgent problem to be addressed.
Observations
Observation 4: Despite being provided with sufficient context, the model still exhibits severe hallucinations.
Knowniidgg::
Question:
How copres the video gamtes the image were sold in the fins month??
grewn exuesiis
User
gane plameed
LLaVA-v1.5-11 v11.57B
Qwen-VL-Chan
Fati-:: 20 ectiinn
Fistoce:
RakFF:: [Eriiie
CIM: Fldicane 6.4
X
LakA: rillie
IaRK: 15miilinn
CEM: Fi-Soum:
CEM: 0
Fi-Sore: as
Rali-Tal--ett: milkne
X
CEB:: Fl-Sorm:
F2-5nxe
Commercial AI Web Search Engine
Sufficient Context
F1Scned:
Gensini 2:5:::: (12...
CES::
FlSoae: 864
FLScoee:
Perplrslty Al: 20villion
Gemini 2.5ttm: inmittonunts Ft Fi-ccar::
CEM: FE-Suce
capies.
Fiotore:
Figure 21: Qualitative example of Wikipedia Entity video games knowledge.
26

---

Published as a conference paper at ICLR 2026
D MORE DETAILS ON CAPABILITY DEGRADATION
D.1 CAPARILITY DEGRADATION RANKING
Based on Table 3. we calculate the mean degradation levels for each capability dimension. Table reveals that both Full-FT and LoRA exhibit consistent ranking of capability degradation: Instruction Following + Multi-Round QA - Hallucination - Comprehensive Evaluation - OCR - Mul- tidisciplinary - Mathematical Reasoning. The identical ranking is also maintained in knowledge retention. Only Replay-108
LoRA and MoELoRA show significantly alleviated degradation rankings in instruction-followiin rising to 4th and 3rd place respectively.
+10%
Table 9: The degree of general capability degradation results. The displayed values are obtained by calculating the mean based the results in Table 3.
Comprehoniie
OCR
Mutidisciplinnyy
Instruction
Multi-Roud
Mathematical
Hallucination
Method
Less
Bank|
|
Loss
Rank
Loss J
Rank
Loss
Rank
Loss
J
Rank
Loss
Rank
Loss
Rank
Foll-F
033.00
013.5%
3
19.63%
[6119358
7
8505595
18.20%
035 989
LoRA
[25 24%
119.32%
3
415 2016
2818
7
$88.055
45.26%
1
437 2518
Knowledge Angmentation for Text
Knowlodge Agnavic
116.60%
519
[4118875
48554880
85907659
425 16988
215
Knowledge Aware (+3) 616.628
15.26%
2
43.788%
664.36688
7
86000858
417.485
4
$2008995
Knowledgy Angmentation for Images
Knowlolge Aguastic
116.05
119.58
(17:4455
4674115
7
459:465
U22
0756
Keowlodge Aware (+3)
58%
212:75988
14.88%
859
0665
Knowledge Retention Methods
Replay
110002%
13.69%
3
10.091
122
6
E31.4015
11.001
2
L13.099
Replay.nn Fals
18.95%
[4.145%
16.031
126.7755
LOL705E
19.6014
EWC
[24.655
614.0688
18,80%
[4903450
15.83%
38555
LoF
018.9418
U17.1618
3
(16,5818
445
6
16.415
1
(33.4216
MOELORA
14.56%
018:3498
10.978
12.05%
129:2455
11.16%
2
19.3898
D.2 FINE-GRAINED DIMENSIONAL. RESULTS ON GENERAL CAPABILITY TESTS
To effectively evaluate the specific capability degradation caused by knowledge injection in LMMs, we utilized 12 benchmarks across task categories:
1. MME (Fu et al., 2023) is a comprehensive evaluation benchmark designed to assess the perfor- mance of LMMs across 14 distinct tasks, encompassing both perception and cognition abilities. To ensure fair and accurate comparisons, MME provides concise, manually designed instruction- answer pairs, eliminating the need for extensive prompt engineering.
2. MMBench (Liu et al., 2024c) is a bilingual benchmark designed to evaluate the comprehensive capabilities of LMMs across multiple modalities. offers a meticulously curated dataset with over 3,000 multiple-choice questions covering 20 distinct ability dimensions, such as object localization and social reasoning. Additionally, MMBench provides questions in both English and Chinese, enabling comparative evaluations of 1.MM performance across these languages.
3. SEEDBench2.Plus (Li et al., 2024a) comprehensively evaluates LMMs' understanding of text- rich visuals (charts, maps, web pages). Comprising 2.300 multiple-choice questions across these categories, it assesses reasoning capabilities in real-world scenarios where text and visuals inter- twine addressin gap for applications like document analysis and web content understanding.
4. OCRBench (Liu et al., 2023b) is a comprehensive evaluation benchmark designed to assess the CCa)capabilities of LMMs. It encompasses 29 datasets across five key tasks: Text Recognition, Scene Text-Centric VQA, Document-Oriented VQA, Key Information Extraction (KIE), and Handwritten Mathematical Expression Recognition (HMER). The benchmark aims to provide thorough assessment of LMMs' performance in various text-related visual tasks, highlighting their strengths and weaknesses, particularly in handling multilingual text, handwritten text, non- semantic text, and mathematical expressions.
MMMU (Yue et al., 2024) is a comprehensive benchmark designed to evaluate LMMs on tasks that require college-level subject knowledge and deliberate reasoning. It comprises 11,500 meticulously curated multimodal questions sourced from college exams, quizzes, and textbooks, spanning six core disciplines: Art & Design, Business, Science, Health & Medicine. Humanities & Social Science, and Technology & Engineering. These questions cover 30 subjects and 183 subfields, featuring 30 diverse image types such charts, music sheets, and chemical structures.
27

---

Published as a conference paper at ICLR 2026
6. MIA-Bench (Qian al., 2024) is benchmark designed to evaluate the ability of LMMs to adhere strictly to complex instructions. comprises a diverse set of 400 to challenge models' compliance with layered instructions, requiring accurate and contextually. MMDU (Liu et al., 2025d) is a comprehensive evaluation framework designed to assess the capabilities of LMMs in handling multi-tumn, multi-image dialog scenarios. It focuses on under- standing complex interactions involving multiple images and sequential dialog turns, which are critical for real-world applications like visual storytelling, medical diagnosis, and interactive Al systems. The benchmark includes diverse dataset with rich annotations, enabling models to be fine-tuned and evaluated on tasks requiring contextual reasoning, image-text alignment, and temporal coherence.
pairs, each crafted
Math Vista (Lu et al., 2024) evaluates foundation models' mathematical reasoning in visual contexts. It comprises 6, 141 examples from 28 existing multimodal datasets, augmented with three new datasets (IQTest, FunctionQA, PaperQA), requiring fine-grained visual understanding and compositional reasoning.
9. Math Vision ( (Wang et al., 2025a) is a meticulously curated dataset comprising 3,040 high- qualityal mathematical problems, each embedded within visual context and sourced from real mathematics competitions. This benchmark spans 16 distinct mathematical disciplines and is organized across five levels of difficulty, offering a comprehensive platform evaluate the mathematical reasoning abilities of LMMs.
10. Hallusion (Guan et al., 2024) a comprehensive benchmark designed to evaluate LMMs on their ability to accurately interpret and reason about visual data, specifically addressing issues of language hallucination and visual illusion. comprises 346 images paired with 1,129 questions among visual dependent and visual supplement. The benchmark introduces novel structure for visual questions, enabling quantitative analysis of models' response tendencies, logical consistency, and various failure modes.
11. POPE (Li et al., 2023b) is a benchmark designed to systematically assess object hallucination in LMMs. Object hallucination refers to the tendency of these models generate descriptions containing objects not present in the corresponding images. POPE addresses this issue by implementing polling-based query method that evaluates models' accuracy in identifying the existence of specific objects within images. This approach provides a more stable and flexible evaluation of object hallucination, revealing that current LMMs often generate objects inconsistent with the target images.
Toe
PEACE
Figure 22: Fine-grained dimensional results on MME and MMBench.
According to Figures 22, 23, 24, 25, and 26, we conduct result analysis for each benchmark.
1. MME: Results on the MME benchmark indicate that both Full-FT and LoRA significantly degrade LLaVA's perception and cognition capabilities, with perception exhibiting a more pronounced decline. We attribute this primarily to MMEVOKF's focus on cognition tasks and its lack of substantial perception content. While the replay method effectively mitigates forgetting in perception abilities (e.g., outperforming Vanilla in Position tasks), it shows limited efficacy for cognition (e.g., poor performance in Numerical Calculation and Text Translation). This disparity likely stems from VL's original training data heavily emphasizing perception. Overall, EWC and LwF are less effective at mitigating forgetting than MoELoRA, though all three methods perform relatively well the Text Translation task.
28

---

Published as a conference paper at ICLR 2026
2. MMBench: Experimental results show that both Full-FT and LoRA significantly degrade LLaVA's performance in the perceptually demanding Attribute Comparison task, while enabling superior performance in the Physical Relationship task due to MMEVOKE'S relational data. For capability degradation mitigation, Replay and MoELoRA remain most effective. Notably, the EWC method underperforms even Full-FT and LoRA across 16 tasks (including Attribute Comparison. Attribute Recognition, Celebrity Recognition, and Function Reasoning), directly indicating the instability of this parameter-regularizatii approach.
Vanta
NNFT
LetA
2MC
MACLORA
Figure 23: Fine-grained dimensional results on SEEDBench2 Plus, OCRBench and MMMU.
3. SEEDBench2 Plus: Both Full-FT and LoRA reduce LLa'A's performance on SEEDBench2 Plus, with LoRA underperforming compared to Full-FT. Among knowledge retention methods, only Replay outperforms the Vanilla approach in Web tasks.
OCRBench: Experimental result shows Full-FT and LoRA exhibit relatively less degradation in OCR tasks, potentially due to their eextinnformation focus, while outperforming Vanilla in Key Information Extraction. However, LwF and MoELoRA demonstrate unstable degradation miti- underprronnn Full-FT/LoRA. in Text Recognition and Scene Text Centric VQA, yet showing opposite trends to all other methods (Full-FT, LoRA, Replay, EWC) in Key Information Extraction.
5. MMMU: While LoRA demonstrates superior overall performance compared to Full-FT across most tasks, exhibits significantly lower performance on specific MMMU domains (Business, Science, Health & Medicine, Technology & Engineering) We hypothesize this discrepancy stems from the similarity between these tasks' required information and the MMEVOKE data distribution, with Full-FT showing greater efficacy in integrating evolving knowledge from MMEVOKE. Concurrently, LwF consistently underperforms both Full-FT and LoRA across multiple tasks, substantiating its inherent instability for mitigating capability degradation in practical applications.
2200
Condee
Pi MOHDU
Vaxta
MNT
1alA
Hapby
Higpln
2MC
MELLaA
Figure 24: Fine-grained dimensional results on MIA-Bench, MMDU and Math Vista.
6. MIA-Bench: Both Full-FT and LoRA exhibit substantial performance degradation on MIA-Bench particularly in the Perspective task (95.65% and 100% degradation respectively) indicating
29

---

Published as a conference paper at ICLR 2026
significant impairment of tional content in MMEVOKE. degradation mitigation effectiveness varies substantially: EWC shows minimal efficacy (particularly in Perspective with no measurable improvement), while LwF provides only modest mitigation. Conversely, both MoELoRA and Replay +10% demonstrate supe- rior capabilitiie, with Rep1ay110% achieving exceptional Perspective task performance surpassing Vanilla.
capability attributable to the absence of instruc-
LoRA
7. MMDU: Both Full-FT and LoRA exhibit substantial degradation across multiple MMDU tasks, primarily attributed to the absence of multi-round dialogue data in MMEVOKE. Crucially, none of the evaluated continual learning methods effectively mitigate this degradation. substantiating that SFT significantly impairs LLL' multi-round dialogue capability and highlighting a critical area for future improvement.
Math Vista: Full-FT and LoRA exhibit relatively lower degradation rates, outperforming Vanilla in reasoning tasks including Geometry Reasoning, Geometry Problem Solving, Figure Question Answering, and Statistical Reasoning. While knowledge retention methods generally demonstrate satisfactory degradation mitigation, they exhibit notable limitations in Logical Reasoning tasks, likely attributable to the inherent complexity and elevated difficulty of such reasoning.
(1H Madrhho Parl
(T) Heckkkebook
LalA
Raplay
TWC
Figure 25: Fine-grained dimensional results on Math Vision and HallusionBenchh.
9. Math Both Full-FT and LoRA improve performance on Math Vision, outperforming Vanilla in Analytical Geometry, Counting, and Logical Reasoning tasks. However, knowledge retention methods exhibit suboptimal performance in geometry-speciff tasks (Geometry Area, Geometry Length, Solid Geometry, Topology). primarily stemming from the substantial domain- specific knowledge required for these specialized domains.
10. HellusionBench: Both full fine-tuning and LoRA exhibit limited performance on HallusionBench, with complete degradation (100% decrease) in the VS. VS_OCR task and significant reductions in VD figures, VS .charts and VD VDROCR tasks. Notably, EWC and LwF outperform Vanilla in VD math and VS table tasks, while MoELoRA achieves exceptional performance in VS table.
Valla
MIF
LeRA
Rey 1y
Replay
EBC
LxF
MALLORA
Figure 26: Fine-grained dimensional results on POPE
30

---

Published as a conference paper at ICLR 2026
11. POPE:
Both Full-FT and LoRA cause marginal performance degradation POPE, potentially attributable to the benchmark's low complexity. Among all methods, only Replay outperforms Vanilla in mitigating degradation, likely due to partial presence of POPE-related data in LLaVA's original training.
31

---

Published as a conference paper at ICLR 2026
D.3 MORE EXAMPLES OF VIOLATING INSTRUCTION
The example of violating instructions on MME
Prompt: Does this artwork belong to the type of historical?'nAnswerr the question using a single word phrase.
Expected Answer: Yes or No.
The Nuremberg Chronicle (German: No0fcrnbergische Chronik) is the title of two different medieval German chronicles, the one by Henricus (died after 1410) and the other by Hartmann (died after 1420), both Nuremberg. The chronicles continue the work the 14th-century German chronicler Konrad von Megenberg and also influenced by the Saxon chronicle. They were written in Middle High German and cover the period from the creation of the world to the death of Emperor Henry II
Output:
The example of violating instructions on Math Vision
Please answer the question and provide the conrect option letter, e.g., A, B, C. D, at the end. Question: Which of the figures shown bellow cannot be cut out of the figure illustrated nearby? Choices: (A) A (B) B (C) C (D) D (E)
Prompt:
Expected Answer: A B or Co or
The Minesweeper puzzle is based on the game Sudoku. The objective to clear the board of all cells (except for a small number in the upper left comner) by revealing the state of each cell through series clues. Fach clue indicates whether a cell contains a number
Output:
flag. The flags mark cells that are not part of the solution; revealing a flag shows as a number or leaves it as flag. The game progresses through series of levels, with each level having more complex patterns of flags. The game ends when all cells are revealed, and the player clears them all. The game implemented as a web application using HTML, CSS, and JavaScript. The logic for generating the puzzle and checking solutions is implemented in JavaScript. The game is tested using the Bri- enne testing framework. The game has been compared the board game Dangerous Crossing.
The example of violating instructions SEEDBench2 Plus
Prompt: How many sections can be accessed directly from the "Homepage"? (A) (B) (C) 4 (D) 5
Expected Answer: A or B or C or D.
Output: 1. About Me 2. Projects 3. Contact
The example of violating instructions on POPE
Is there snowboard in the image? Answer the question using single word or phrase.
Prompt:
Expected Answer: Yes or No.
Output: A snowboard is a wide, flat board used for riding on snow, typically secured to the rider's feet with bindings.
32

---

Published as a conference paper at ICLR 2026
E
MORE DETAILS ABOUT KNOWLEDGE AUGMENTATION ON MМЕVOKE
E. TRAINING LOSS PERSPECTIVES ON EFFECTIVENESS OF KNOWLEDGE AUGMENTATION
Figure 27 demonstrates that the training loss of LLaVA exhibits a significant decline at the end of each epoch under Full-FT training strategies. This behavior aligns with the LLM'' data memorization patterns during training and overfitting, suggesting that repeated exposure to data is essential for acquiring up-to-date knowledge. This further proves the necessity of knowledge augmentation the training phase, which present evolving knowledge in different variants to the model, facilitate the model store attribute knowledge on entities. and flexibly extract knowledge.
65
Hepoch
Figure 27: Training loss over time for L.L. VA-V1.5 based on the Full-FT training strategy.
E.2 PERFORMANCE OF KNOWLEDGE AUGMENTATION IN GENERAL CAPABILITY TESTS
MME
MMlBerch
SEEDilenc22
1600
40+
54
Eveetegee maare-1)
Pal-F
Late
Kondente opretts
sward.
Lake
Eewelege coaorel
OcRBench
ScienceQA
MMMU
26
20.0
hal-
Lekd
Enveldge te
Leks
halerr
Lake
Eomelddy nareet
Eaweeige
Eamsaage
Eawelade
MIA-Bench
MMDU
NathiVists
14
Sont
13
11
20-
Laka
Laka
Lake
Ktestdge oparns
Fomilage smerself
Lnvaadge
medidge
Kondelge
Lomildge
Fomedge
MathVission
PORe
HollusionBench
75
10+
71
beodldge -eweegge-nenst-- -0
Raturt
Ldke
haur
Lake
Envelage cwand-l)
Kmededge
Kuwdedge Augnetiction for Text
Ercwledge Asgmantittion Images
Figure 28: The performance of knowledge augmentation in general capability tests. According to Figure 28, we have the following observations:
33

---

Published as a conference paper at ICLR 2026
Obs 1: Knowledge augmentation is generally superior to standard Supervision Fine-Tuning. Across all 12 general capability benchmarks evaluated, models enhanced with knowledge augmen- tation, whether through text or images, demonstrated markedly superior performance compared to the model trained with standard Supervised Fine-Tuning. This comprehensive superiority is consistently observed in MME, MMBench, SEEDBench2,Pe Plus, ScienceQA, MMMU, MMDU, POPE, and HallusionBench.
Obs 2: Deficiencies in instruction-folllwingg multi-turn dialogue, and reasoning capabilities remain apparent. On the MIA-Bench, MMDU, Math Vista, and Math Vision benchmarks, the model post-knowledge augmentation underperforms standard Supervised Fine- Tine-Tnnng model. This performance disparity is primarily attributed to the fact that the knowledge augmentation process does not inherently enhance the aforementioned capabilities of reasoning, instruction following, multi-turn dialogue. Consequently, these areas represent critical directions for future improvement and refinement.
Obs 3: Increasing the Volume of Text Augmented Data Correlates Positively with Performance Gains, A clear trend indicates that incrementally increasing the volume of augmentation data, denoted by the progression from "+1" "+3", generally leads to continued performance improvements. This dose-response relationship is evident for text augmentation across most benchmarks. For instance, in MME, MMBench, SEEDBench2_Plur MMMU, MIA-Bench, the "43" versions of the augmented models consistently outperform their "+1" and "+2" counterparts. This finding suggests that the model's capabilities can be further enhanced through the sustained integration of a larger and more diverse set of knowledge-rich data.
34

---

Published as a conference paper at ICLR 2026
F MORE EXPERIMENTAL RESULTS ABOUT KNOWLEDGE RETENTION METHODS ON MMEVOKE
F.1 THE KNOWLEDGE INJECTION PERFORMANCE OF KNOWLEDGE RETENTION METHODS ON МMEVOKE
While focusing on capability degradation mitigation via knowledge retention methods, we also evalu- ate these methods' performance evolving knowledge injection, shown in Table 10. Experimental results show that all knowledge retention methods incur losses in evolving knowledge injection, with MoELoRA experiencing most significant decline, while parameter regularization methods (EWC and LwF) retain relatively better performance. Future work could integrate the strengths of multiple knowledge retention methods to design more comprehensive approaches.
Table 10:
The knowledge injection performance of LLa regarding knowledge retention methods on MMEVOKE. POL: Politics; SPO: Sports; BUS: Business; HEA: Health; CEL: Celebrity; FIL: Film; ALB: Album; WRI: Written Work.
Metkel
Arg
POL
SP0
IEEA
Jue
CTL
F1L
ALB
Wilket Kinwiidgy Netretion
18:33
15:17
1155
13.09
31.9
35.6
1834
33:1
1065
1984
15:97
14:3
1555
1311
1691
13.66
1517
7/6
1-8A
15.25
19.31
1772
1942
1034
12.6
1911
21.3
2855
2480
1751
13.76
12.50
1738
12.00
21.15
12.5
15.52
3672
K72
2034
12.54
16.11
1556
16.96
Presesis dusn tnannnniillll
E9C
21.0
11
23.30
1239
11:11
Observations
Observation 5: Parameter regularization methods achieve superior knowledge injection performance compared data replay and MoF.
F.2 Is IT BETTER TO HAVE MORE DATA FOR REPLAY?
Entity
Percentage Difference
0
10
13.22
13.07
CEM
10%
-20
100%
10%
40%
70%
100%
2
7
9
11 12
Replay
Benchmark
Figure 29; The performance different replay data sizes in multimodal evolving knowledge injection and mitigating capability degradation. The numbers the x-axis of the right subgraph correspond to the order the benchmarks shown in Table 3.
As shown in Figure 29, knowledge injection efficacy and capability degradation mitigation exhibit non-monee non-monotonic correlation with replay data size, accompanied by significant fluctuations. Given the computational cost escalation from data expansion, minimization of replay data size is recom- mended (Bi al., 2025).
Observation 6: More replay data does not significantly strengthen knowledge adaptation and retention.
35

---

Published as a conference paper at ICLR 2026
G PROMPT FOR GENERATION
The prompt templates for summary generation, question-answer generation, and phrase generation are detailed in Figure 31 and Figure 30, respectively. All generation tasks were performed using GPT-4o to ensure consistency and high-quality outputs.
You are pawerful question and answor generator. The usen gives title, a description of the question according the title description of the the generated cuestion, and use hypernyn the entity object to replace the entity, and transfors the text question into a nultimadal question. Output forrat: "Text
news. Extract visual entity ooject
"Text_ Question: question Multimodal, Question: nultinodal question Answer: Entity: entity Hyperny: hypernye".
Ouring the generation process, you nust follos each of the folloxing rules:
1.The genersted question and answer pairs must core fros the content of the title and description,
of words used generated pronles sust be visual entity. The best entities 3. entity selected
choose are: people, teans, organizations, etc.
4. generated answer and selected visual entity cannot be the sane.
converting Text_ Question Nultimndal, Question, hypernyn used replace entity
For exanple:Text Question: which company did Nvidia's narket value surpasst The entity object we extracted from the Text, Question is Wiidia,The entity is Nvidia and hypernyn is company. replace 'Wvidia' with 'the company narket value
inage'. The Multimedsl Question: Which company's company in the isage exceed?
6. 6.Generate answers without punctuation. For exapple, Takyo, Japan is against the rules; Takyo Japan within the rules.
everall workflos as follows:
Step1: Generate a text question and answer according to the title and description of the news. Step2:Extract visual entity object fror the text question, and it connot be the sane entity object the answer.
Step3:Using hypernyn of multinodal question. Heer are two exasples for reference.
entity object,
question transformed
type_list ['politics', 'sport', "entertairment', 'business', 'us', 'health', 'europe', 'style', "tach', 'siddleeast]] type type_dase
exanples, randorly select
the exeap proupt
Here are sone examples:
politics_exxet 1 "Exenple
title: 'Siden will dispatch unofficial delegation Tatxan folloding its :President Joe Biden is - -. while the Us continues to support Taixan's donocratic pracesses, erphasizing tles and the \"One China)" policy."
Description:
Exasple output:
purpose Joe Siden's delegation to Taiwan?' Multimodal Question: 'Whtt is the purpose of the delegation sent by the person in the inage to Taisen?"
Question:
Answer: Muspert"Support desocracy"
Entity: 'Joe
Hypernym: 'person"
sport phort_emaspl__1 3 "Example user
title: Philadeiiil 76ers silence boos from hone crowd to edge past Miani Heat and reach playoffs'
Descriptian: Philadelphia
early struggles and fan 105-184 play-in toumasent, - potentialll out due a knee injury as they prepare for an elisination gare against the chicago Bulls for the last playoff spot." Exarple output:
edge past Miant
playoffs after defeating Miart Multinodal Question: 'ho will the tean in the inage face in the playofts after defeating the Miami Heat?'
Text Questian: 'Mh will the Philadelphia 76ens face
Answer: "Now York Knickks'
Entity: Philadelphia 76ers'
Hyperrym: 'tean'
Figure 30: Prompt for Generation of Questions and Answers.
36

---

Published as a conference paper at ICLR 2026
You are helpful assistant. Please help summarize the news into a new description less then words. when you sumnarize rest of your content, to include the core main objects from the news nuch as possible and important information about time and place. From summary, you need to extract more than 4 entities. This entity corresponding to it in the search engine, which can be people, countries, companies, etc. extracted entitys given the title and content. The output format is Sumnrized: #sumnarized description.
unique existence. You find the unique image
sunnarize content.
Example User:
Input:
Title Israel ramps up multiple fronts, nobody what Netanyahu's
endgane 1s
Content When Israels forces killed Hamas leader Yahya Sinwar Gaza last week, many inside and Benjamin Netanyahu would declare victory and scale back the Gaza operation in hopes securing ceasefire and hostage release deal. AnA week after Sinwar's death,
of Israel hoped could
Prime Minister
increasingly clear they have been wrong. VnNetanyahu, -. o. say to hinself, enough is enough," " he said. An"And then his nission would be to strike sone kind of deal with the prosecution, maybe they'll let his go and he will be able to abroad, have any criminal record, he'll be able to sit in all kinds of advisory boards earn lots money,
lectures the who defeated terror and if he won't
lacking.
Output:
Example Assistant:
Sunnarized; Anid Israel's escalating conflicts Hamas Hezbollah, Minister Benjamin Netanyahu remains determined continue military operations, despite growing internal and international pressure for a ceasefire. The recent killing of
retaliatory missile strike heighten tensions, Netanyahu navigates political complexities, balancing U.S. and donestic pressures while siming to establish lasting legacy. With potential implications for U.S.-Israel relations the American elections, Netanyahu's strategy remains uncertain, potentially aimed at broader regional influence.
Hezbollah
Figure 31: Prompt Summary Generation.
37

---

Published as a conference paper at ICLR 2026
H KNOWLEDGE EDITING ON MMEVOKE
The knowledge base stored in Large Multimodal Models (LMMs) is essentially static, which leads to outdated and inaccurate knowledge. Knowledge Editing (KE) is widely used technique for efficiently updating the knowledge, resolving knowledge conflicts, and alleviating knowledge hallucination of large models (Zhang et al., 2024; Jia et al., 2025; Wang et al., 2025b).
For example, research explores how to locate and edit factual associations in GPT and how to mass- edit memory in a Transformer (Meng et al., 2022a;b). Knowledge Editing encompasses multiple methods and types, such as parameter-modificalle based and meta-learning approaches.
Given that MMEVOKE is a multimodal and evolving benchmark, recent research also extends into the fields of multimodal editing (Cheng et al., 2023; Huang et al., 2024; Du et al., 2025; Bi et al., 2024; Rong et al., 2025) and lifelong knowledge editing (Chen et al., 2024b; Jiang et al., 2025a; Qi al., 2024). Therefore, we also apply knowledge editing methods (i.e. FT-LLM (Huang al., 2024), FT-VIS (Huang et al., 2024), RECIPE (Chen et al., 2024c), LTE (Jiang et al., 2024), LiveEdit (Chen al., 2024a)) on MMEVOKE, and the experimental results shown Table 11.
Table 11: The performance of knowledge editing MMEVOKE.
#Edit
FT-LLM
FT-VIS
RECIPE
LTE
LiveEdit
10
50.01
31.55
32.77
62.48
41.77
1000
42.51
29.63
31.40
52.59
42.15
2000
32.93
21.13
26.65
43.52
37.08
4000
28.14
13.54
16.54
32.88
36.97
38

---

Published as a conference paper at ICLR 2026
I REASONING CONSISTENCY, HALLUCINATION REDUCTION, AND FACTUAL GROUNDING
Reasoning Consistency
Question:
"Did the gunman successfully wound the person in the image during the campaign rally?
Answer with Yes or No."
Expected:
"Yes"
Before Injection:
"No"
After Injection:
"Yes"
Question:
"Where did the gunman fire shots that resulted in minor injury to the person in the im- age? A. The arm B. The chest C. The car D. The leg. Answer with the option's letter directly."
Expected:
"C"
Before Injection:
"A"
After Injection:
"A"
Question:
"What injury did the person in the image sustain during the attempted assassination? An- swer with brief description."
Expected:
"A wound on the ear"
Before Injection:
"a bullet wound in the abdomen"
After Injection:
"A minor injury the car"
39

---

Published as a conference paper at ICLR 2026
Hallucination Reduction 1
Question:
"What position was the human in the image appointed to in October 2024?
Ground truth:
"Deputy minister"
Before Injection:
"President"
After Injection:
"President"
Question:
"Which team did the player in the image help defeat to win the World Series MVP?"
Ground truth:
"New York Yankees"
Before Injection:
"Red Sox"
After Injection:
"New York Yankees"
Question:
"What is the release date of the album in the image?"
Ground truth:
"31 May 2024"
Before Injection:
"2014"
After Injection:
"31 May 2024"
40

---

Published as a conference paper at ICLR 2026
Hallucination Reduction 2
Question:
"Which countries co-produced the film in the image?"
Ground truth:
"Belgium, Netherlands, Germany, Iraq"
Before Injection:
"Spain"
After Injection:
"Argentina, Spain, Qatar"
Question:
"Which summit is the leader in the image preparing for amid health scrutiny?"
Ground truth:
"NATO summit"
Before Injection:
"G20"
After Injection:
"G7 summit"
Question:
"Which division did the human in the image commander of in the IDF?"
Ground truth:
"98th Paratroopers Division"
Before Injection:
"101"
After Injection:
"Galilee Formation"
41

---

Published as a conference paper at ICLR 2026
Factual Grounding 1
Question: "Please help me describe the content of the following news: "Quinta Brunson first Black woman to win best comedic actress Emmy in over 40 years"."
Injection knowledge: "This provides an overview of the news. Quinta Brunson made history by winning the Emmy for best actress in a comedy series for her role in 'Ab- bott Elementary.' becoming the first Black woman to win this category since 1981. Brunson, who also created and writes for the show, delivered tance speech at the Emmy Awards. She previously won in 2022 for outstanding writ- ing in comedy series. 'Abbott Elementary,' sitcom about teachers in a Philadel- phia public school, is also nominated for an outstanding comedy series Emmy."
emotional accep-
Before Injection: "The news is about Quinta Brunson, an American actress, who has made history by becoming the first Black woman to win the best comedic actress Emmy in over 40 years. This achievement highlights her talent and contributions to the enter- tainment industry, as well as the progress made in representation and diversity in the field. Brunson is known for her roles in shows like "A Black Lady Sketch Show" and "The Good Place," " and her success at the Emmys is a testament to her talent and hard work." After Injection: "This provides an overview of the news. Quinta Brunson made his- tory by winning the Emmy for best actress in comedy series for her role in 'Abbott Elementary.' becoming the first Black woman to win in this category in over 40 years. Brunson, who also created and writes for the show, delivered an emotional accep- tance speech at the Emmy Awards. She previously won in 2022 for outstanding writ- ing in a comedy series. 'Abbott Elementary,' sitcom about teachers in a Philadel- phia public school, is also nominated for an outstanding comedy series Emmy."
Question: "Please help me describe the content of the following news: 'How the assassi- nation attempt on Trump unfolded'." "
Injection knowledge: "During campaign rally in Butler, Pennsylvania on July 13, 2024, gunman attempted to assassinate former President Donald Trump, firing shots that grazed Trump's car. Secret Service agents swiftly neutralized the shooter, Thomas Matthew Crooks, but the incident resulted in one attendee's death and two severe injuries. The attack marks the first attempt on a current or former U.S. president since 1981. As investigations continue into the security breach, Trump reassured supporters of his safety while expressing condolences for the victims. President Joe Biden condemned the vio- Ience and contacted Trump afterwards."
Before Injection: "The news article is about an attempted assassination on former Presi- dent Donald Trump during a speech in 2021. The article provides details about the event, including the shooter, the location, and the response of the Secret Service It also dis- cusses the impact of the incident Trump's campaign and the political climate the time. The article emphasizes the seriousness of the situation and the measures taken to ensure the safety of the former president and the public."
After Injection: "This provides an overview of the news. On June 13, 2024, near his Mar-a-Lago club in Florida, a gunman attempted to assassinate Donald Trump. who was the time the presumptive Republican presidential nominee. The shooting, which oc- curred during golf tournament, resulted in injuries to Trump's Secret Service protective detail. The gunman, identified as Yuan Gong, was killed in the exchange. The incident is being investigated by the FBI as an act of domestic terrorism, with Gong having ex- pressed political motivations and targeting Trump due to his perceived influence on global events."
42

---

Published as a conference paper at ICLR 2026
Factual Grounding 2
Question: "Please help me describe the content of the following news: "Schumer leads congressional delegation to Ukraine to mark 2-year anniversary of Russian invasion"."
Injection knowledge: "Senate Majority Leader Chuck Schumer leads a congressional delegation to Ukraine, marking the second anniversary of the Russian invasion, and urging U.S. House Republicans to pass a $95.3 billion foreign aid bill for Ukraine and Is- rael. The delegation, including Democratic Senators Jack Reed, Michael Bennet, Richard Blumenthal, and Maggie Hassan, meets with Ukrainian President Volodymyr Zelensky, who emphasized Ukraine's need for U.S. aid to win against Russia. Schumer presses House Speaker Mike Johnson, highlighting the bill's significance for NATO and European security amid new U.S. sanctions on Russia."
Before Injection: "The news is about a congressional delegation led by Senator Charles Schumer visiting Ukraine to mark the two-year anniversary of the Russian invasion. The delegation is expected to meet with Ukrainian President Petro Poroshenko and other offi- cials to discuss the ongoing conflict and the United States' support for Ukraine. The visit aims to demonstrate solidarity and commitment to Ukraine's ' sovereignty and territorial integrity."
After Injection: "This serves as recap of the news. Senate Majority Leader Chuck Schumer leads a congressional delegation to Ukraine, marking the second anniver- sary of Russia's invasion with visit Kyiv. They intend to reaffirm support for Ukraine's sovereignty and territorial integrity, as well discussing humanitarian aid and military assistance. The delegation includes House Speaker Mike Johnson, House Foreign Affairs Committee Chairman Michael McCaul, and Representative Michael Waltz. This visit coincides with the Feb. 24 anniversary of the invasion and a recent $95.3 billion U.S.-Ukraine security aid deal."
Question: "Please help me describe the content of the following news: 'Wayfair is open- ing its first-ever physical furniture store'."
Injection knowledge: "Wayfair is opening its first physical store on May 23 in Edens Plaza, Wilmette, Illinois, aiming to boost sluggish sales amid high advertising costs. The 150,000-sqqare-foon store will feature a restaurant, 'The Porch,' mirroring Ikea's concept. This move responds to challenges faced in online retail, particularly the decline in demand post-pandemic and persistent inflation affecting big-ticket item purchases. Wayfair's CEO, Niraj Shah, acknowledged over-hiring during the pandemic's demand surge. This store could enhance Wayfair's market presence and support its online business by providing an omnichannel shopping experience."
Before Injection: "Wayfair, an online furniture retailer, is opening its first-ever physical furniture store. The store will be located in Boston and will offer customers the opportu- nity to see and touch the furniture before making purchase. This move is an expansion of Wayfair's business model and aims to provide more personalized shopping experience for customers."
After Injection: "This is summary of the news. Wayfair, the online furniture retailer, venturing into brick-and-morte with the opening of its first physical store in London. This move aims to enhance customer experience by allowing visual product demonstra- tions and foster a sense of community. The 100,000 sq ft store, located the White City area, is set to create 1,000 jobs. This expansion follows Wayfair's previous physical store experiments in Boston and New Jersey, The London store will showcase the com- pany's extensive furniture and home decor range."
43

---

Published as a conference paper at ICLR 2026
Factual Grounding 3
Question:
"Please help me describe the content of the following news: 'Provisional re- sults in Rwanda's presidential election show incumbent Paul Kagame winning in land- slide'."
Injection knowledge: "In Rwanda's presidential election, provisional results show incumbent Paul Kagame securing a landslide victory, likely extending his tenure to a fourth term. The election witnessed high voter turnout of 98%, with Kagame leading with 99.15% of the votes counted. Despite accusations of government crackdowns, Kagame emphasizes stability and regional development. The election also included votes for Rwanda's lower house of parliament. Political tensions persist, with some candidates barred from running, while opposition figures report progress in gaining support."
Before Injection: The news article reports that the provisional results of Rwanda's presidential election show incumbent Paul Kagame winning in landslide. The article does not provide further details about the margin of victory or the percentage of the vote that Kagame received."
After Injection:
"This is summary of the news. Incumbent Paul Kagame is likely to secure a landslide victory in Rwanda's presidential election, with provisional results indicating win with over 99% of votes counted. Kagame, who has led since 2000 and recently assumed the East African Community chair, is expected to start a third term. His victory, amidst accusations of suppressing opposition, maintains stability in Rwanda but faces criticism from the West for his stance on Ukraine and support for Russia. Re- gional leaders congratulated Kagame on his win."
Question: "Please help me describe the content of the following news: 'Elon Musk's Neuralink is seeking a second person to test its brain chip'."
Injection knowledge: "Elon Musk's Neuralink is seeking a second human trial par- ticipant test its brain chip, after the first trial with Noland Arbaugh encountered performance issues. Despite the setback, Arbaugh, quadriplegic, reports positive life changes, controlling a computer cursor with the implant. Neuralink aiming to help people with paralysis and other conditions by connecting human brains to computers, and enrolling more candidates for its PRIME Study to assess the safety and functionality of its device. Regulatory approval is required before broader market release."
Before Injection: "The news article discusses Elon Musk's company, Neuralink, and its efforts to develop brain chip. The company is currently seeking a second person to test the brain chip, which is designed to improve communication between the brain and computers. The article provides some background information on the technology and its potential applications, as well as the challenges that the company faces bringing the product to market."
After Injection:
"This provides an overview of the news. Neuralink, Elon Musk's brain- computer interface company, is recruiting a second test subject for its PRIME Study, following the successful implantation in Noland Arbaugh. The study aims assess the safety and functionality of Neuralink's 's device, which aims to treat paralysis. Potential participants must have a similar condition to Arbaugh and be willing to undergo a 90-day evaluation period. The trial reflects Musk's ambition to help people with limb paralysis and other conditions, as he seeks to expand Neuralink's capabilities beyond its current focus on treating brain injuries."
44

---

Published as a conference paper at ICLR 2026
Factual Grounding 4
Question: "Please help me describe the content of the following news: 'NHL players to return Olympics in 2026 and 2030'."
Injection knowledge: "NHI players will return to the Olympics in 2026 and 2030, participating after missing two Winter Games. The NHI, NHI Players Association, and International Ice Hockey Federation agreed on their participation, with NHL Commissioner Gary Bettman highlighting its importance. The 2026 Olympics will be in Milan and Cortina A'peezo, Italy, while the 2030 host unannounced. This marks the first Olympic appearance since 2014, with prior absences due to COVID- 19 and scheduling issues. Additionally, new "4 Nations Face-off" international tournament will debut in 2025, featuring Canada, the U.S., Sweden, and Finland."
Before Injection: "The news is about the National Hockey League (NHL) announcing that its players will return to the Winter Olympics in 2026 and 2030. The NHL had previ- ously decided not to send its players to the 2018 Winter Olympics in Pyeongchang. South Korea, due to scheduling conflicts with the league's regular season. The NHL's decision to rejoin the Olympic Games is a significant development, as marks the first time since 1998 that the league's top players will compete in the Olympics."
After Injection: "This provides an overview of the news. NHL players will return the Olympics in 2026 in Italy and 2030, after missing two Winter Games. The NHL, NHL Players Association, and International Ice Hockey Federation agreed on their participation, with NHL Commissioner Gary Bettman highlighting its importance. This marks the first Olympic appearance since 2014, with prior absences due to COVID-19 and scheduling issues. Additionally, new '4 Nations Face-off' tourna- ment with Canada, the U.S., Sweden, and Finland is set for 2025."
Question: "Please help me describe the content of the following news: 'Russian figure skater Kamila Valieva banned for four years by Court of Arbitration for Sport'."
Injection knowledge: "Russian figure skater Kamila Valieva, found guilty of an anti- doping rule violation, has been handed a four-year ban by the Court of Arbitration for Sport. The decision, backdated to December 2021, follows her positive test before the 2022 Beijing Winter Olympics. The ban nullifies her competitive results, impacting the team event standings. The ruling is praised by global anti-doping bodies, with future medal reallocations dependent on the International Skating Union's decisions. Despite her claims of accidental ingestion of the banned substance, the ruling stands final and binding."
Before Injection: "The news is about a Russian figure skater named Kamila Valieva who has been banned from competing for four years by the Court of Arbitration for Sport (CAS). The ban is due to positive drug test result, which has resulted in the athlete being disqualified from the 2020 Winter Youth Olympics. The CAS has also imposed a fine on the Russian Skating Federation for not fulfilling its obligations under the World Anti-Doping Code."
After Injection: "The Court of Arbitration for Sport has banned Russian figure skater Kamila Valieva for four years following her positive test for the banned drug trimetazidine during the Beijing 2022 Winter Olympics. The decision, backdated to De- cember 2022, means Valieva cannot compete until December 2026. The Russian Skating Federation plans to appeal. Trimetazidine, intended for angina, can enhance endurance and is prohibited by WADA. The ban overshadows the controversy surrounding Valieva's 's performances, which raised questions about her age. Despite claims of conspiracy, evi- dence doping was confirmed by the Stockholm Laboratory."
45

---

Published as a conference paper at ICLR 2026
Factual Grounding 5
Question: "Please help me describe the content of the following news: 'Iranian rapper Toomaj Salehi sentenced death for protesting". "
Injection knowledge: "Iranian rapper Toomaj Salehi has been sentenced to death for his role in the 2022 protests in Iran, confirmed by his lawyer. The sentence was upheld by court in Isfahan, reversing the Supreme Court's prior decision. Salehi, who has been outspoken against the Iranian regime, was detained, allegedly tortured, and spent 252 days in solitary confinement. His sentencing has garnered international criticism, with calls for his release from the UN and the U.S. Special Envoy for Iran, highlighting the regime's repression following 22-year-old Mahsa Amini's death." "
Before Injection: "The news article is about an Iranian rapper named Toomaj Salehi who has been sentenced death for protesting against the government. The article provides details about his arrest, trial, and the charges against him. It also highlights the concerns about the use of capital punishment in Iran and the impact of the sentence on the rapper's family and supporters."
After Injection:
"This provides overview of the news. Toomaj Salehi, Iranian rapper and activist, has been sentenced to death for his role in the 2022 protests in Iran, as confirmed by his lawyer. The sentence was upheld by a court in Isfahan, reversing the Supreme Court's prior decision. Salehi, who has been in custody for over 15 months, was allegedly tortured into making a confession. His sentencing has garnered international criticism, with calls for his release from the UN and the U.S. Special Envoy for Iran, highlighting the regime's repression following 22-year-old Mahsa Amini's death."
46


Reviews:
[{"review_id": "dCgmfyMZy9", "reviewer": "Reviewer_PjCG", "summary": "This work investigates the knowledge retention and adaptation capabilities of LMMs in evolving knowledge. A benchmark named MMEVOKE is constructed to evaluate the performance. Two types of evaluations are conducted: 1) knowledge injection tests, which assess the ability of models to acquire newly introduced knowledge. 2) general capability tests, which examine the ability to preserve previously learned knowledge. Several approaches, including Supervised Fine-Tuning, Retrieval-Augmented Generation (RAG), Commercial AI Web Search Engines, and Sufficient Context Provision, are evaluated for their effectiveness in knowledge injection. Experimental results show that current methods perform poorly on the MMEVOKE benchmark, and LMMs still tend to produce incorrect answers even when sufficient contextual information is provided. Knowledge-aware augmentation demonstrates a clear improvement in knowledge injection performance. Regarding knowledge retention, the study finds that the general capabilities of LMMs degrade after knowledge injection. Methods based on Replay and MoELoRA are shown to effectively mitigate this degradation and help maintain the overall performance of LMMs.", "questions": "Please refer to the Weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed MMEVOKE benchmark serves as the first evaluation dataset designed to measure the evolving knowledge injection capabilities of Large Multimodal Models.\n2. This work systematically evaluates a wide range of approaches for their effectiveness in knowledge injection, including Supervised Fine-Tuning, Retrieval-Augmented Generation (RAG), Web Search Engines, and Sufficient Context Provision. The results indicate that knowledge augmentation substantially enhances model comprehension and adaptability in dynamic knowledge environments.\n3. This work further examines multiple approaches for knowledge retention, encompassing Replay-based methods, two classical continual learning techniques (EWC and LwF), and MoELoRA. The findings reveal that Direct Rehearsal (Replay) and Structured Separation (MoELoRA) effectively preserve previously acquired knowledge through retraining on historical data and isolating newly injected knowledge, respectively.", "weaknesses": "1. It appears that the proposed concept of evolving knowledge injection is essentially a continual learning problem. The connection between evolving knowledge injection and continual learning remains unclear. A more detailed discussion of this relationship should be included in the Introduction or Related Work sections to better position this study within the broader research context.\n2. In the field of continual learning, several recent studies have explored the potential of large multimodal models, e.g., Large Continual Instruction Assistant (ICML 2025), Generative Multi-modal Models are Good Class-Incremental Learners (CVPR 2024). The CL methods evaluated in this work, such as EWC and LwF, are relatively outdated. It would be valuable to evaluate more recent and advanced approaches on the proposed MMEVOKE benchmark and to analyze whether their results align with or challenge the current findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3DUHoWO8hk", "reviewer": "Reviewer_kvMY", "summary": "In this paper，the authors proposed MMEVOKE for multimodal evolving knowledge, which, serves as a evaluation dataset to measure LMMs’ evolving knowledge injection capabilities.  The authors conduct knowledge injection tests with Supervised FineTuning, Retrieval Augmented Generation, Web Search Engine, and Sufficient Context on MMEVOKE. Based on the experimental results, the authors find that existing methods exhibit poor knowledge adaptation performance and the performance of LMMs remains imperfect even with sufficient context.", "questions": "See in Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. In this paper，the authors proposed MMEVOKE for multimodal evolving knowledge, which, serves as a evaluation dataset to measure LMMs’ evolving knowledge injection capabilities.  \n2. The authors conduct knowledge injection tests with Supervised FineTuning, Retrieval Augmented Generation, Web Search Engine, and Sufficient Context on MMEVOKE. Based on the experimental results, the authors find that existing methods exhibit poor knowledge adaptation performance and the performance of LMMs remains imperfect even with sufficient context.", "weaknesses": "1. In my view, the authors overlook a type of method, knowledge editing, such as ROME ( Locating and Editing Factual Associations in GPT), AnyEdit (AnyEdit: Edit Any Knowledge Encoded in Language Models.) and MEMIT (Mass-Editing Memory in a Transformer). \n2.  In Benchmark construction, the authors compare offline versions of Wikipedia at different time points to identify new entries. But such a way cannot cannot guarantee that these entities will be unfamiliar to LMMs, since LMMs are pre-triained on a much larger dataset than Wikipedia.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KyNnVEAACx", "reviewer": "Reviewer_fBUi", "summary": "The paper introduces MMEvoke, a benchmark for evaluating LMMs’ ability to incorporate evolving multimodal knowledge. MMEvoke is built via an automated pipeline with manual curation, using images and accompanying text scraped from Wikipedia and CNN across 159 subfields. It evaluates adaptation and retention for SFT, RAG, AI web agents, and Sufficient Context. The authors show that MMEvoke is challenging for current knowledge-injection methods, which also degrade general capabilities (e.g., instruction following, multi-turn QA). Additional studies indicate that knowledge-aware augmentation substantially improves adaptation, whereas knowledge-agnostic augmentation harms it; and that Replay/MoE-LoRA mitigates degradation better than EWC/LwF.", "questions": "1. Among the 9,422 knowledge updates, how many correspond to *known entities* versus *new entities* for the evaluated LMMs?\n2. Do you provide a baseline where the LMM is trained **only on text-based knowledge updates** (without images)?\n3. How do you ensure that knowledge updates scraped from CNN or Wikipedia do not already exist in the LMM’s **parametric knowledge**, given the counterexamples mentioned?\n4. How are the **knowledge-aware variants** constructed for both text and images?\n5. What are the causes of the **failure cases** for RAG, AI Web Agent, and Sufficient Context? Can you provide more details for error analysis.\n6. How is **Sufficient Context** constructed? Please provide more methodological details on this setup.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. MMEvoke leverages real-world data to benchmark LMM’s abilities to adapt to evolving knowledge. The benchmark is comprehensive, containing 9,422 knowledge and covering 159 subfields.\n2. The paper conducts a comprehensive evaluation (12 benchmarks) spanning training- and retrieval-based methods, e.g., SFT (Full/LoRA) to RAG, commercial agents, and sufficient context, yielding useful cross-method comparisons.\n3. The analysis of knowledge-aware vs knowledge-agonistic augmentation and Replay/ MoELoRA provides practical insights into improving knowledge adaption while preserving  general capabilities.\n4. The writing quality is good. It is easy to follow the benchmark pipeline and evaluation details.", "weaknesses": "1. It is unclear why injecting images of new knowledge is necessary, since the text often suffices to answer questions once the model recognizes the person and recalls textual knowledge. In both examples of “Geoffrey Hinton + Nobel Prize” and “Donald Trump + Assassination”, the text provides all the information needed to answer the question. Or, in the case of news for “Region” or “Business” categories, the images often don’t provide closely related information to the text.\n2. The work does not differentiate among types of knowledge injection, e.g., (1) **new entities** (Xiaomi SU7), (2) **known entities with new facts** (Geoffrey Hinton + Nobel Prize), and (3) **known entities with conflicting facts** (Lee Jae-myung: party leader vs. president).\n3. The pipeline cannot guarantee that knowledge is truly new to the pre-trained LMM, since the article date is not necessarily the first online appearance (e.g., iPhone 16, Xiaomi SU7 existed online before 2024; links below).\n4. For RAG, AI web agents, and Sufficient Context, performance is surprisingly low without an accompanying error analysis; failures could stem from (1) failure to recognize the image, (2) incorrect images from Google, or (3) inability to leverage the provided context.\n\nLinks:\n- https://carnewschina.com/2023/11/15/xiaomis-first-ev-revealed-in-china-to-be-called-xiaomi-su7/\n- https://www.forbes.com/sites/davidphelan/2023/11/03/apple-iphone-16-pro-will-bring-remarkable-upgrade-report-claims/", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2bgI9WMe5j", "reviewer": "Reviewer_SWqj", "summary": "This paper addresses a crucial but underexplored challenge for LMMs, i.e., their ability to acquire and retain evolving multimodal knowledge. The authors argue that while LMMs possess vast pretrained knowledge, they struggle to remain consistent with the dynamically changing world. To tackle this, they introduce MMEVOKE, a large-scale benchmark comprising 9,422 multimodal samples spanning 159 subfields, collected from evolving entities and news since 2024. The authors further propose knowledge-aware augmentation (versus naive data augmentation) and retention strategies (e.g., replay and MoELoRA) to mitigate these challenges. They demonstrate that knowledge-aware augmentation improves both adaptation and retention, and replay/MoELoRA effectively alleviate degradation.", "questions": "1. Besides CEM/F1, could you provide qualitative or human evaluation on reasoning consistency, hallucination reduction, or factual grounding after injection?\n2. Could you elaborate on how GPT-4o’s summarization and augmentation process ensures semantic fidelity and avoids introducing bias?\n3. The conclusion hints at multi-stage or hybrid strategies. Have you conducted any preliminary experiments combining augmentation and replay/MoELoRA?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The work addresses a critical gap in current LMM research, i.e., the handling of evolving multimodal knowledge, which is becoming increasingly relevant as real-world data changes rapidly.\n2. The MMEVOKE benchmark is well designed, covering diverse modalities (text and images) and dynamic sources (CNN and Wikipedia). The pipeline is largely automated, reproducible, and designed for continuous updates.\n3. The paper evaluates a broad spectrum of existing paradigms (SFT, LoRA, RAG, Web Search, Commercial AI systems) and general capabilities across 12 benchmarks in 7 dimensions, offering a panoramic view of how LMMs behave under evolving knowledge injection.", "weaknesses": "1. The proposed “knowledge-aware augmentation” and “retention” methods largely adapt existing paradigms (e.g., replay, MoE) rather than introducing fundamentally new algorithms. Their novelty lies in application and synthesis, not algorithmic innovation.\n2. Despite the automated data pipeline, Step 4 (manual selection) introduces subjectivity and may constrain full automation or large-scale scalability.\n3. Evolving knowledge has been widely studied in the context of LLMs, but there is few discussion in the related works and literature review, such as RealtimeQA, DyKnow, EvoWiki, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)