Title: DR.PO: Dual Reference and Preference Optimization for Machine Unlearning in Large Language Model
Primary area: generative models
Keywords: dual-positive-preference，dual-reference-model，similarity-scores-retain

Abstract: Most high-performing LLM unlearning methods use preference learning but have a critical flaw: insufficient or singular positive preferences make post-unlearning models generate meaningless, inconsistent, or single-category outputs. These differ in probability distribution from "fully unlearned" models, causing suboptimal unlearning quality and privacy risks of unlearned data. We propose DR.PO (Dual Reference and Preference Optimization). It adopts answer with incorrect fact and answer indicating information deficiency as positive preferences, with distinct reference models for each. Instead of random sampling the full dataset for the retain set (as in existing methods), we match each forget sample to a corresponding retain sample via similarity scores—reducing repeated sampling of the retain set. Experiments show our method not only achieves better unlearning quality and better privacy protection but also effectively preserves the model’s original capabilities.

TL;DR: A dual-positive-preference and dual-reference-model optimization form LLM unlearning