OPTML-Group/SimNPO-TOFU-forget10-Llama-2-7b-chat

SimNPO-Unlearned Model on Task "TOFU - Forget05"

Model Details

• Unlearning:
  • Task: 🤗datasets/locuslab/TOFU - Forget10
  • Method: SimNPO
• Origin Model: 🤗OPTML-Group/TOFU-origin-Llama-2-7b-chat
• Code Base: github.com/OPTML-Group/Unlearn-Simple
• Research Paper: "Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning"

Unlearning Algorithm

This model uses the SimNPO unlearning algorithm with the following optimization objective: $${\mathrm{\ell }}_{SimNPO}(\mathbf{\theta })={\mathbb{E}}_{(x,y)\in {\mathcal{D}}_f}[-\frac{2}{\beta }\log \sigma (-\frac{\beta }{\mathrm{\mid }y\mathrm{\mid }}\log {\pi }_{\mathbf{\theta }}(y\mathrm{\mid }x)-\gamma )]+\lambda {\mathbb{E}}_{(x,y)\in {\mathcal{D}}_r}[-\log {\pi }_{\mathbf{\theta }}(y\mathrm{\mid }x)]\backslash ell\_\{SimNPO\}(\backslash mathbf\{\backslash theta\})\; =\; \backslash mathbb\{E\}\_\{(x,\; y)\; \backslash in\; \backslash mathcal\{D\}\_f\}\backslash left[-\backslash frac\{2\}\{\backslash beta\}\backslash log\backslash sigma\backslash left(-\backslash frac\{\backslash beta\}\{|y|\}\backslash log\backslash pi\_\{\backslash mathbf\{\backslash theta\}\}(y|x)\; -\; \backslash gamma\backslash right)\backslash right]\; +\; \backslash lambda\; \backslash mathbb\{E\}\_\{(x,\; y)\; \backslash in\; \backslash mathcal\{D\}\_r\}[-\backslash log\backslash pi\_\{\backslash mathbf\{\backslash theta\}\}\; (y|x)]$$ Unlearning hyper-parameters:

• Learning Rate: 1e-5
• beta: 4.5
• lambda: 0.125
• gamma: 0.0

Loading the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("OPTML-Group/SimNPO-TOFU-forget10-Llama-2-7b-chat", use_flash_attention_2=True, torch_dtype=torch.bfloat16, trust_remote_code=True)

Evaluation Results

	Forgeting Quality (FQ)	Model Utility (MU)
Origin	0.00	0.62
Retrain	1.00	0.62
NPO	0.29	0.55
SimNPO	0.45	0.62

Citation

If you use this model in your research, please cite:

@article{fan2024simplicity,
  title={Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning},
  author={Fan, Chongyu and Liu, Jiancheng and Lin, Licong and Jia, Jinghan and Zhang, Ruiqi and Mei, Song and Liu, Sijia},
  journal={arXiv preprint arXiv:2410.07163},
  year={2024}
}

Reporting Issues

Reporting issues with the model: github.com/OPTML-Group/Unlearn-Simple