Title: Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Machine Unlearning

Large Language Models

Constrained Optimization

Primal‑Dual Algorithms

Entropy‑Based Objectives
Abstract: Large Language Models (LLMs) deployed in real-world settings increasingly face the need to unlearn sensitive, outdated, or proprietary information. Existing unlearning methods typically formulate forgetting and retention as a regularized trade-off, combining both objectives into a single scalarized loss. This often leads to unstable optimization and degraded performance on retained data, especially under aggressive forgetting. We propose a new formulation of LLM unlearning as a constrained optimization problem: forgetting is enforced via a novel logit-margin flattening loss that explicitly drives the output distribution toward uniformity on a designated forget set, while retention is preserved through a hard constraint on a separate retain set. Compared to entropy-based objectives, our loss is softmax-free, numerically stable, and maintains non-vanishing gradients, enabling more efficient and robust optimization. 
    We solve the constrained problem using a scalable primal-dual algorithm that exposes the trade-off between forgetting and retention through the dynamics of the dual variable, all without any extra computational overhead.
    Evaluations on the TOFU and MUSE benchmarks across diverse LLM architectures demonstrate that our approach consistently matches or exceeds state-of-the-art baselines, effectively removing targeted information while preserving downstream utility.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new formulation for LLM unlearning as a constrained optimization problem. Forgetting is enforced through a novel logit-margin flattening loss, which explicitly drives the model’s output distribution toward uniformity over a designated forget set. Meanwhile, retention is maintained via a hard constraint applied to a separate retain set.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed method formalizes the unlearning problem as a constrained optimization task, which effectively separates the forgetting and retention objectives, allowing for explicit control over each. While the formulation is intuitively appealing, the experimental results are not particularly strong. One possible explanation could be limited computational resources or insufficient hyperparameter tuning. However, this also highlights a broader concern—that the method may lack efficiency and practical scalability.\n\nQUESTIONS:\nTable 1 presents performance on the TOFU dataset under a forget10/retain90 setting. Does this mean the experiments evaluate unlearning 10% of the original training data?\n\nFrom the results, it appears that SimNPO for many times outperforms the proposed method. In fact, in roughly 50% of the experiments, the proposed method does not achieve the best performance. Given this, can the results convincingly demonstrate the efficacy of the method?\nSame question for Table 2. \n\nAdditionally, how does the efficiency of the proposed method compare to the other baselines in terms of computational cost or runtime?\n\nLIMITATIONS:\nIt would be beneficial for the paper to include an analysis of the efficiency and computational complexity of the proposed method, and to compare it directly with traditional unlearning methods.  This would help clarify the practical advantages and trade-offs in terms of scalability and resource requirements.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-04"}

comments: SUMMARY:
The paper proposes a novel framework for unlearning in Large Language Models called Constrained Entropic Unlearning, which formulates the problem as a constrained optimization task.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The constrained optimization approach provides explicit control over forgetting and retention, addressing limitations of scalarized loss methods.

2. The logit-margin flattening loss is numerically stable, softmax-free, and maintains non-vanishing gradients, enabling efficient optimization.

3.Experiments on multiple benchmarks (TOFU, MUSE) and model architectures (Llama 2/3, Gemma) demonstrate consistent improvements over baselines in both traditional metrics and LLM-judged evaluations.

Weaknesses:

1.The method slightly reduces fluency on the forget set, as noted in the results, which may affect generation quality in some applications.

2. The paper does not provide a formal theoretical analysis of PDU's convergence properties or guarantees.

3. The motivations of this paper are not well illustrated. For instance, in the introduction section, the three key obstacles lack of supportive references or some visualizations. Moreover, the motivation of the proposed modules is not clarified.

QUESTIONS:
See weakness.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

comments: SUMMARY:
This paper proposes a new framework for machine unlearning in LLMs by casting unlearning as a constrained optimization problem. The authors solve the constrained problem using a scalable primal-dual algorithm, which dynamically balances forgetting and retention through updates of the dual variable. Experiments on the TOFU and MUSE benchmarks show strong results, across multiple LLaMA and Gemma models.

STRENGTHS AND WEAKNESSES:
Strengths:

- The formulation of unlearning as a constrained optimization problem is well-motivated.
- The primal-dual optimization framework provides a principled way to enforce retention constraints.
- Experiments demonstrate robustness across TOFU and MUSE benchmarks and several base models.

Weaknesses:
- The method's effectiveness is evaluated mainly on QA and text-based benchmarks. It's unclear how well it would work on challenging tasks like code generation or reasoning benchmarks.
- There is no exploration of how the method handles overlapping semantics between the forget and retain sets, which is common in realistic unlearning scenarios.
-  Computational cost (especially the primal-dual updates) on very large models or frequent unlearning requests could pose practical challenges. And this is not discussed or evaluated in the paper.

QUESTIONS:
How sensitive is your method to the choice of constraint threshold and the initial dual variable?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Constrained Entropic Unlearning (CEU), framing unlearning for large language models as a constrained optimization problem. The idea is to explicitly enforce forgetting on a specified dataset using a logit-margin flattening loss, while preserving performance on retained data by imposing a hard constraint. A primal-dual algorithm is used to balance the forgetting–retention trade-off dynamically, yielding stable optimization even under aggressive forgetting settings. Experiments on TOFU and MUSE benchmarks across multiple LLM architectures show that CEU matches or exceeds current best methods in forgetting efficacy and retention utility.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* **Reframe existing problem in a novel way**. The paper provides a principled method for balancing retention and forgetting. This method reframes the unlearning problem as a constrained optimization problem. This reframing is novel to me while being sensible to address the limitations of current unlearning methods.\n* **Stability**. Most unlearning methods are very sensitive to learning hyper-parameters. The proposed method seems to be numerically stable. The proposed logit-margin loss is softmax-free, avoids vanishing gradients, and supports robust optimization for forgetting tasks.\n* **Competitive performance** on TOFU and MUSE benchmarks.\n\n\n**Weaknesses**\n\n* **Lack of interpretability**. The paper does not investigate where forgetting happens in the model (e.g., which layers or features). There's no mechanism-level insight.\n* **Theoretical clarity**. The rationale behind the loss and constraints isn’t deeply motivated, even though properties are proven, practical impact of hyperparameter settings is unexplored. However, this has been a shared issue for most existing methods.\n* **Scalability concerns**. It isn’t clear how well CEU extends to large models or continual unlearning workflows. The computational cost of repeated constraint enforcement may be prohibitive.\n* **Missing references**:\n\n  * There are a couple of recent works in the unlearning area. For example, *UniErase: Unlearning Token as a Universal Erasure (2025)*, a token-level unlearning mechanism, and the *Digital Forgetting Survey (2024)*, which offers comprehensive coverage of unlearning methods and benchmarks. These would help position this work within the most up-to-date unlearning landscape.\n  * Current metrics may miss internal traces of forgotten content. Tools like **Logit Lens** (*\"Interpreting GPT\"*, Nostalgebraist, 2020), **Tuned Lens** (*\"Feature Visualization...\"*, Belrose et al., 2023), and **Jet Lens** (*\"Jet Expansions of Residual Computation\"*, Chen et al., 2024) offer insight into intermediate layer behavior. These could help verify whether forgetting is real or just suppressed at the output layer.\n\nQUESTIONS:\n1. What are the failure modes when constraints are overly tight?\n\n2. Can CEU be adapted to continual or streaming unlearning scenarios?\n\n3. Is it possible to use performance-efficient fine-tuning methods (like LoRA) in conjunction with CEU?\n\n4. Have you considered visualizing internal representations (e.g., attention, residual path flows) or use mechanistic interpretability tools to better explain where forgetting occurs?\n\nLIMITATIONS:\ndiscussed in limitation", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}