Title: Conflict-Aware Knowledge Editing in the Wild: Semantic-Augmented Graph Representation for Unstructured Text
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Models

Model Editing

Unstructured Text

Conflict-Aware
Abstract: Large Language Models (LLMs) have demonstrated broad applications but suffer from issues like hallucinations, erroneous outputs and outdated knowledge. Model editing emerges as an effective solution to refine knowledge in LLMs, yet existing methods typically depend on structured knowledge representations.  
However, real-world knowledge is primarily embedded within complex, unstructured text. Existing structured knowledge editing approaches face significant challenges when handling the entangled and intricate knowledge present in unstructured text, resulting in issues such as representation ambiguity and editing conflicts. 
To address these challenges, we propose a Conflict-Aware Knowledge Editing in the Wild (CAKE) framework, the first framework explicitly designed for editing knowledge extracted from wild unstructured text. 
CAKE comprises two core components: a Semantic-augmented Graph Representation module and a Conflict-aware Knowledge Editing strategy. The Semantic-augmented Graph Representation module enhances knowledge encoding through structural disambiguation, relational enrichment, and semantic diversification. Meanwhile, the Conflict-aware Knowledge Editing strategy utilizes a graph-theoretic coloring algorithm to disentangle conflicted edits by allocating them to orthogonal parameter subspaces, thereby effectively mitigating editing conflicts. Experimental results on the AKEW benchmark demonstrate that CAKE significantly outperforms existing methods, achieving a 15.43\% improvement in accuracy on llama3 editing tasks. Our framework successfully bridges the gap between unstructured textual knowledge and reliable model editing, enabling more robust and scalable updates for practical LLM applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose conflict-aware knowledge editing, which is the first knowledge editing framework specifically designed for unstructured text, which effectively addresses the challenges of representation ambiguity and editing conflicts in the wild unstructured text editing.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1.\tTo enhance knowledge editing on unstructured text, the authors mitigate knowledge ambiguity via a semantic-enhanced graph representation, and resolve editing conflicts through a conflict-aware knowledge editing strategy based on graph-theoretic coloring.\n2.\tThe experimental results validate the effectiveness of the method proposed by the authors across different datasets and models of varying scales.\n\n\nWeaknesses: \n\n1.\tThe three hyperparameters $\\alpha$, $\\beta$, and $\\gamma$ in Formula (7) are all set to a default value of 0.8. However, the paper does not provide any justification or guidance on how these values are chosen or how to determine their optimal settings.\n2.\tThe proposed method did not outperform the baseline on the Loc. metric in most cases, could you provide a detailed analysis of the underlying reasons for this phenomenon?\n3.\tThere are some typos, for example, in Line 127 “can be denotes as” $\\rightarrow$ “can be denoted as”.\n\nQUESTIONS:\n1.\tThe proposed method performs better on MQuAKE-CF after removing CKE. Could this be attributed to MQuAKE-CF's specific features?\n2.\tIn Section 2.2, how exactly are the prompt templates designed, could you list the prompt templates?\n3.\tCan the conflict definition in Section 2.3 cover all scenarios, such as triples with the same relation and object but subjects are different?\n\nLIMITATIONS:\nThe description of the experimental setups need to be more detailed, such as the specific design of prompt templates and the selection of hyperparameters in Formula (7), to facilitate readers' understanding.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose the CAKE framework, which tackles the knowledge editing in LLM with natural, unstructured, and possibly concept-conflicted knowledge. Specifically, the CAKE first uses a carefully designed pipeline to extract triplets from the unstructured text. Next, the knowledge edit is conducted by first constructing a conflict graph using the approximated conflict score. Then, the graph coloring algorithm is used to partition the conflict graph into different parts. Finally, each subgraph is edited by training different LoRA. Experiments confirm the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n- The writing of the paper and the workflow of the proposed method are clear to me.\n- The proposed pipeline is effective based on the experimental results. The design of conflict graph construction and graph coloring is interesting. \n\n## Weaknesses:\n- The proposed method is based on two motivations: representation ambiguity and editing conflict. If I understand it correctly, the representation ambiguity is solved by the SGR, and the editing conflict is solved by CKE. I have some questions on both parts.\n- For representation ambiguity. There is no analysis on how often the problem exists in the real-world data and how such ambiguity will affect the knowledge editing performance. One possible way is that maybe include an ablation study on each submodule in SGR that affects the performance (I only see the ablation study on the whole SGR part). It would be even better to include results on how much the effect is in the raw data and how the SGR mitigates this effect.  \n- For the editing conflict part, it is also not clear to me how large the effect of that is on the downstream performance.\n- For CKE module, I am wondering what the distribution of the results conflict graph (average node, edges), graph partition, and how many LoRA are actually used.  When comparing with a single expert, have authors consider the effect of parameter size?\n\nQUESTIONS:\nSee the weakness section.\n\nLIMITATIONS:\nThe biggest limitation to me is how the problems tackled by the proposed method actually caused a significant burden to the downstream editing performance. And how is the improvement of introducing the proposed method worth the cost?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the Conflict-Aware Knowledge Editing in the Wild (CAKE) framework, a novel approach for updating the knowledge of Large Language Models (LLMs) directly from unstructured text. The authors identify two key challenges in this process: representation ambiguity, where the complex relationships in text are hard to encode, and editing conflicts, where updating one piece of information negatively impacts related but distinct facts. The authors evaluated CAKE on the AKEW benchmark using several LLMs, including GPT2-XL, Llama3.2-3b, and Llama3-8b. The experimental results demonstrate that CAKE significantly outperforms existing model editing methods in terms of reliability while maintaining performance on unrelated knowledge.\n\nSTRENGTHS AND WEAKNESSES:\n**strength:** \n\nIt tackles the challenge of editing LLMs using real-world, unstructured text, moving beyond the limitation of methods that require pre-structured knowledge triples or question-answer pairs. The CAKE framework is an innovative solution that systematically addresses the core problems of ambiguity and conflict. The two-module design is logical, with each component targeting a specific, well-defined challenge. The paper provides robust empirical evidence of CAKE's effectiveness across multiple models and datasets from the AKEW benchmark. The authors include detailed ablation studies that validate the contribution of each component of the framework.\n\n\n**weaknesses:**\n1. The SGR module requires several interactions with a large language model to construct and enrich the knowledge graph (for extraction, multi-hop reasoning, and paraphrasing).  This process appears computationally intensive, but the paper does not analyze the associated overhead in terms of time or resources, which could be a significant barrier to practical implementation.\n\n2. The entire framework's performance hinges on the ability of a foundational LLM to accurately extract entities and relations to form the initial knowledge graph.  The paper does not sufficiently discuss how errors or biases from this initial step might propagate through the system and affect the final editing performance.\n\n3. The conflict detection formula includes several hyperparameters (α, β, γ) that are crucial for determining whether two pieces of knowledge are in conflict.  The authors provide default values but do not present an analysis of how sensitive the model's performance is to variations in these values, which is important for understanding the framework's robustness.\n\n4. The mechanism for choosing between the original model and the edited parameters during inference is described as a \"memory routing mechanism.\"  However, the paper lacks specific details about how this router is implemented, how activation memories are stored, or how the similarity threshold is determined, making this critical component difficult to reproduce or evaluate.\n\n\nminor mistake: L208 missing reference of LoRA\n\nQUESTIONS:\nThe SGR module seems to introduce significant computational overhead due to its reliance on an LLM for graph construction, enrichment, and diversification. Could you provide an analysis of the computational cost (e.g., time, GPU hours) of CAKE compared to the baseline methods?\n\nThe quality of the initial knowledge graph is fundamental to CAKE's success. How does the framework handle inaccuracies, such as missed entities or incorrect relations, from the LLM-based graph extraction step? Have you studied how the quality of this initial graph correlates with final editing performance?\n\nThe conflict detection mechanism relies on hyperparameters α, β, and γ. How did you arrive at the default value of 0.8 for each? How sensitive is the framework's performance, particularly the trade-off between reliability and locality, to changes in these values?\n\nCould you please elaborate on the implementation of the \"memory routing mechanism\" used at inference time? Specifically, how are the \"activation memories\" for edited knowledge created and stored, and what method is used to set the similarity threshold for engaging the LoRA experts?\n\nWhat is the typical number of \"colors\" (i.e., LoRA experts) required for the datasets in your experiments? How does the system's performance and parameter count scale as the textual input becomes more complex and the number of knowledge conflicts increases?\n\nYour ablation study shows that using a single expert for all edits is suboptimal.  To further isolate the benefit of your conflict-aware strategy, have you considered comparing the graph-coloring allocation to a simpler baseline, such as randomly assigning conflicting edits to a set of available experts?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the challenge of “wild, unstructured knowledge editing” (WUKE) for large language models—updating facts when supervision comes only from raw text—by proposing CAKE, a two-stage framework that first derives a Semantic-augmented Graph Representation (SGR) of each paragraph through LLM-prompted extraction, multi-hop inference, and paraphrase diversification, then performs Conflict-aware Knowledge Editing (CAKE) by treating each triple as a node in a conflict graph, colouring that graph with the Welsh–Powell algorithm, and mapping each colour to an orthogonal LoRA sub-space so conflicting edits do not interfere during fine-tuning. Tested on CounterFact, WikiUpdate, and MQuAKE-CF with GPT-2-XL and Llama-3 (3 B & 8 B), CAKE improves reliability by 11–21 percentage points over strong baselines while preserving locality, establishing the first conflict-aware framework for unstructured text edits, introducing a richer graph encoding of paragraph knowledge, and offering an elegant, parameter-efficient gradient-isolation mechanism via multi-expert LoRA.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Clear motivation\nEditing directly from unstructured text is both realistic and under-explored; the paper articulates why traditional triple-based editors fail and illustrates the two core difficulties with concrete examples (Fig. 1).\n2. Methodological originality\nSGR cleverly blends LLM-prompted extraction, reasoning, and paraphrasing to densify the knowledge graph, a departure from pipeline OpenIE. CKE uses a classical colouring algorithm to allocate orthogonal LoRA experts—an elegant, lightweight way to resolve conflicting updates.\n3.  Solid empirical gains\nReliability improvements (up to +20.9 pp on Llama-3 3 B) are large, consistent across datasets, and ablations isolate the contribution of each module (Table 2).\n\nWeaknesses\n1. Scalability concerns\nWelsh-Powell is O(|E| log |E|); constructing and colouring a graph for thousands of triples per edit may bottleneck. Suggestion: discuss complexity, memory footprint, and potential batching or heuristic colourings for very large documents.\n2. Clarity & writing\nSome sections (e.g., 2.2 “directed multigraph G′”) use dense prose and could benefit from concrete pseudo-code. Typographical errors (“editing confilicts” l. 86) and inconsistent notation (rij vs. r′ij) should be fixed.\n\nQUESTIONS:\n1. Equation 7 uses fixed thresholds (α, β, γ) and a single hidden layer to compute conflict edges; how robust are results to these choices?\n2. The paper briefly mentions “activation-similarity routing”; what exact algorithm, similarity metric, and thresholds are used, and how does averaging vs. hard selection affect results?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}