Unlearned Llama-3.1-8B-Instruct checkpoints on TOFU forget10
Eleven full-parameter unlearned checkpoints derived from
open-unlearning/tofu_Llama-3.1-8B-Instruct_full
on the TOFU forget10 split. Released because no unlearned checkpoint at this scale appears to be
publicly available, and training them is the main cost of working on 8B unlearning evaluation.
| directory | algorithm | notes |
|---|---|---|
GradAscent, GradAscent_w |
gradient ascent | ⚠️ both diverged (training loss → −100); outputs are degenerate |
GradDiff, GradDiff_s |
gradient difference | |
NPO, NPO_s |
negative preference optimisation | |
SimNPO, SimNPO_w |
reference-free NPO | |
RMU, RMU_s, RMU_w |
representation misdirection | localised edit at one block |
Suffixes: none = standard setting, _s = stronger, _w = weaker.
Setup. Full-parameter (not LoRA) bf16, gradient checkpointing, paged 8-bit AdamW, batch size 1,
gradient clipping 1.0; peak 32.4–34.5 GB on an A100-40GB. Saved in fp16. Objectives follow
locuslab/open-unlearning's trainer configs.
Please read before use. Several of these are deliberately over- or under-trained to span a range of forgetting strength, and two diverged outright. Per-checkpoint hyperparameters and measured behaviour (pre-attack extraction rate, post-attack recovery) are not included here. Ask if you need them.
Base model is Llama-3.1; the Llama 3.1 Community License applies.