Title: K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling
Venue: NeurIPS 2025
Accepted: yes
Keywords: structured knowledge reasoning

continual learning

large language models
Abstract: Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structured queries grounded in structured knowledge. Existing general continual learning approaches face significant challenges when applied to this task, including poor generalization to heterogeneous structured knowledge and inefficient reasoning due to parameter growth as tasks increase. To address these limitations, we propose a novel CSKR framework, \textsc{K-DeCore}, which operates with a fixed number of tunable parameters. 
Unlike prior methods, \textsc{K-DeCore} introduces a knowledge decoupling mechanism that disentangles the reasoning process into task-specific and task-agnostic stages, effectively bridging the gaps across diverse tasks. Building on this foundation, \textsc{K-DeCore} integrates a dual-perspective memory consolidation mechanism for distinct stages and introduces a structure-guided pseudo-data synthesis strategy to further enhance the model's generalization capabilities.
Extensive experiments on four benchmark datasets demonstrate the superiority of \textsc{K-DeCore} over existing continual learning methods across multiple metrics, leveraging various backbone large language models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work studies continual structured knowledge reasoning, where a model must translate natural‑language questions into structured queries while it moves through a sequence of tasks that involve different forms of knowledge, such as relational tables, knowledge graphs, and dialogue states. Current continual learning methods either add new parameters for each task or generalize poorly across heterogeneous schemas. The authors introduce K‑DeCore, which keeps one frozen backbone language model and attaches two small, trainable modules: a schema filter that selects the relevant parts of a given schema and a query builder that forms the final query from the filtered schema. By separating these two stages, the approach transfers knowledge across tasks without increasing the parameter budget. K‑DeCore also maintains dual replay memories, one that stores representative schema examples and another that stores diverse query structures, and enriches them with synthetically generated queries to raise coverage.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper frames continual structured knowledge reasoning as a sequence of heterogeneous tasks and tackles it with a fixed‑parameter backbone plus two small LoRA modules, one for schema filtering and one for query building. By separating these roles, the design allows the same parameters to serve databases, knowledge graphs, and dialogue states without growing with the number of tasks, directly addressing the scalability problem that affects many prior rehearsal and adapter methods.\n\n- The knowledge decoupling idea is paired with a dual‑perspective replay memory: schema‑guided clusters preserve coverage of previously seen schema elements, while structure‑guided clusters store diverse query skeletons. This two‑view memory gives a clear, task‑specific reason for each stored sample and is lightweight, which keeps continual training practical on a single GPU.\n\n- Experiments span four public datasets, formed into three task streams, and the study tests three backbones.\n\n**Weaknesses**\n\n- Both schema‑guided and structure‑guided memories use cosine distance over embeddings from a separate encoder, which may leave important but embedding‑close cases unprotected.\n\n- Mapping every schema into a flat table format removes relation direction, edge types, and slot hierarchy. \n\n- The query-building generator stitches together random structure templates and schema fragments and then discards any sample that fails to execute. This generate-and-filter loop can bias memory toward structures that are easier for the SQL engine to validate rather than those that improve model generalization.\n\nQUESTIONS:\nPlease refer to the Strengths And Weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nK-DECORE addresses challenges in Continual Structured Knowledge Reasoning by decoupling reasoning into task-specific (query building) and task-agnostic (schema filtering) components. It uses a fixed parameter model and integrates memory consolidation and pseudo-data synthesis for better generalization and reduced catastrophic forgetting. Experiments on benchmark datasets with LLM backbones show K-DECORE’s superiority over existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. The paper makes a significant contribution by being the first to systematically explore continual learning across heterogeneous Structured Knowledge Reasoning tasks. This addresses a crucial real-world challenge where models need to continuously adapt to new reasoning tasks with varying structured knowledge types.\n\n2. The core idea of decoupling the reasoning process into schema filtering (task-agnostic) and query building (task-specific) is highly innovative and well-motivated. This design effectively bridges gaps across diverse tasks by identifying schema filtering as a shared, reusable component.\n\n3.  K-DECORE operates with a fixed number of tunable parameters, which is a major advantage over prior methods that suffer from parameter growth as tasks increase. This contributes to more efficient reasoning and scalability.\n\n4. The experimental results are comprehensive and convincingly demonstrate K-DECORE's superior performance across all evaluation metrics (AA, BWT, FWT) compared to strong baselines, including rehearsal-based and PEFT-based methods. The use of three different LLM backbones further strengthens the validity of the findings.\n\nWeakness:\n1. While K-DECORE significantly improves BWT compared to baselines like FINE-TUNING, the BWT scores for LLAMA3 and QWEN2.5 backbones are still negative (e.g., -16.7 for Llama3 Stream1, -8.2 for Qwen2.5 Stream1). This indicates that catastrophic forgetting, while mitigated, remains a persistent challenge. The BWT sometimes remains comparable to or slightly worse than some rehearsal-based methods (e.g., EMAR) in certain streams for Llama3.\n\n2. For the structure-guided query synthesis, the paper mentions that novel structures are synthesized \"guided by carefully curated demonstrations\" . While Appendix B is referenced for examples, more explicit details on demonstration would enhance reproducibility and understanding of this crucial step.\n\nQUESTIONS:\nCould you elaborate on why the \"limited convergence of prompt-tuning\" (for methods like C3 and SAPT) poses a constraint, and how K-DECORE's decoupled design inherently overcomes this issue, leading to its superior results without merely introducing additional parameters?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes K-DECORE, a continual learning framework for Structured Knowledge Reasoning (CSKR) tasks, which involve translating natural language questions into structured queries over heterogeneous structured knowledge sources. The central idea is a knowledge decoupling mechanism splitting the reasoning process into schema filtering and query construction. K-DECORE maintains a fixed-size set of PEFT modules, utilizing dual-perspective memory and a strategy for synthesizing pseudo-queries to improve generalization and reduce forgetting. Experimental evaluation on four benchmarks spanning diverse SKR tasks and three LLM backbones shows that K-DECORE achieves consistent gains over existing continual learning baselines, as validated through multiple metrics and ablation studies.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The experimental evaluation is thorough: methods are compared across three diverse task streams using three different LLM backbones, and baselines include a strong set of rehearsal-based and PEFT-based continual learning algorithms. Ablation studies dissect methodological contributions.\n2. The decoupled architecture, i.e., splitting schema filtering and query construction, offers a principled way to enable transfer and mitigate forgetting between heterogeneous tasks. The method shows a commendable level of innovation and delivers effective results.\n3. The paper is well-organized and clearly written, with illustrative figures and tables that effectively support the presentation.\n\n**Weaknesses:**\n1. It seems that the main experiments do not include a breakdown of the distribution of various SKR problem types across each stream, which could be important for interpreting the results more precisely.\n2. The necessity of the designed submodules in this work is not sufficiently demonstrated. In Figure 3, the performance differences between the curves are minimal. Similarly, in Table 2, K-DeCore achieves consistently the best performance only on the AA metric.\n\nQUESTIONS:\n1. Could the authors provide a more detailed distribution of different problem types within the data stream? In scenarios where there is a significant difference between the training and test distributions, or where there is a large imbalance among different SKR categories, can the method still maintain robust performance?\n\n2. The performance drop after ablating certain components, such as Random Memory, appears to be minimal. Could the authors provide additional experiments or explanations to clarify this observation?\n\n3. As the proposed approach introduces several modules, it may significantly increase training cost. Although Section 4.6 compares training efficiency, the paper lacks sufficient implementation details. Could the authors provide additional experiments or elaborate further on the implementation aspects related to this comparison?\n\nLIMITATIONS:\nSee Weaknesses and Questions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces K-DeCore, a new continual learning framework for Structured Knowledge Reasoning across heterogeneous tasks. By decoupling reasoning into schema filtering and query building stages and leveraging a unified schema representation, the approach aims to facilitate knowledge transfer without increasing parameter count. A dual-perspective memory mechanism—covering schema and query structure—and a structure-guided pseudo-data synthesis strategy are incorporated to boost generalization and retention. Experiments on four SKR benchmarks with multiple LLM backbones show gains across accuracy, forgetting, and forward transfer metrics compared to baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. This paper clearly motivates that schema filtering is a common subtask across different SKR forms and proposes a systematic decoupling between schema filtering and query building. \n\n2. Unified schema representation: By mapping various structured knowledge forms (SQL, KG, dialogue) to a DB-style schema, the model reduces task heterogeneity in schema filtering. \n\n3. The core methodology, especially the knowledge decoupling and memory construction strategies, is clearly explained.\n\n\nWeaknesses\n1. All experiments constrain task train/test samples to fixed small sizes (1,000/300), which, while making experiments tractable, limits generality to much larger-scale or more realistic continual environments where distribution drift is more gradual or data is less segmented.\n\n2. The described DB-like mapping unifies most SKR schemas, but some structured knowledge forms may not map easily (e.g., hierarchical, nested, or graph-centric schemas with types or attributes that do not naturally correspond to columns/tables). \n\n3. The work highlights that most baselines cannot be directly ported to decoder-only LLMs, but the consequence is that comparative results with these (Llama3, Qwen2.5) have fewer baselines and thus weaker empirical competition, risking over-claiming.\n\nQUESTIONS:\nPlease refer to Strengths And Weaknesses\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}