lomahony's picture
Update README.md
5eef764 verified
metadata
language:
  - en
tags:
  - pytorch
  - causal-lm
  - pythia
license: apache-2.0
datasets:
  - Anthropic/hh-rlhf

Pythia-70m finetuned using original DPO code with the helpful subset of Anthropic-hh-rlhf dataset for 1 epoch.

Checkpoints are also uploaded.

Fully reproducible finetuning code is available on GitHub

wandb log

See Pythia-70m for model details (paper).

See further details of these models in the paper Attributing Mode Collapse in the Fine-Tuning of Large Language Models.

You can cite these models if they are helpful as follows:

@inproceedings{o2024attributing,
  title={Attributing Mode Collapse in the Fine-Tuning of Large Language Models},
  author={O’Mahony, Laura and Grinsztajn, Leo and Schoelkopf, Hailey and Biderman, Stella},
  booktitle={ICLR 2024, Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) workshop},
  year={2024}
}

hf (pretrained=lomahony/pythia-70m-helpful-dpo), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 16

Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 0 acc 0.1724 ± 0.0110
none 0 acc_norm 0.2201 ± 0.0121
arc_easy 1 none 0 acc 0.3350 ± 0.0097
none 0 acc_norm 0.3380 ± 0.0097
boolq 2 none 0 acc 0.4315 ± 0.0087
hellaswag 1 none 0 acc 0.2614 ± 0.0044
none 0 acc_norm 0.2665 ± 0.0044
lambada_openai 1 none 0 perplexity 5951.7544 ± 428.5435
none 0 acc 0.0309 ± 0.0024
openbookqa 1 none 0 acc 0.1460 ± 0.0158
none 0 acc_norm 0.2440 ± 0.0192
piqa 1 none 0 acc 0.5550 ± 0.0116
none 0 acc_norm 0.5501 ± 0.0116
sciq 1 none 0 acc 0.4010 ± 0.0155
none 0 acc_norm 0.5070 ± 0.0158
wikitext 2 none 0 word_perplexity 547.6920 ± N/A
none 0 byte_perplexity 3.2518 ± N/A
none 0 bits_per_byte 1.7012 ± N/A
winogrande 1 none 0 acc 0.4822 ± 0.0140

hf (pretrained=lomahony/pythia-70m-helpful-dpo), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 16

Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 5 acc 0.1886 ± 0.0114
none 5 acc_norm 0.2338 ± 0.0124
arc_easy 1 none 5 acc 0.3346 ± 0.0097
none 5 acc_norm 0.3308 ± 0.0097
boolq 2 none 5 acc 0.4028 ± 0.0086
hellaswag 1 none 5 acc 0.2617 ± 0.0044
none 5 acc_norm 0.2648 ± 0.0044
lambada_openai 1 none 5 perplexity 22676.7987 ± 1626.4435
none 5 acc 0.0173 ± 0.0018
openbookqa 1 none 5 acc 0.1640 ± 0.0166
none 5 acc_norm 0.2460 ± 0.0193
piqa 1 none 5 acc 0.5528 ± 0.0116
none 5 acc_norm 0.5462 ± 0.0116
sciq 1 none 5 acc 0.3100 ± 0.0146
none 5 acc_norm 0.4220 ± 0.0156
wikitext 2 none 5 word_perplexity 547.6920 ± N/A
none 5 byte_perplexity 3.2518 ± N/A
none 5 bits_per_byte 1.7012 ± N/A
winogrande 1 none 5 acc 0.5201 ± 0.0140