| license: apache-2.0 | |
| datasets: | |
| - stanfordnlp/SHP | |
| - Anthropic/hh-rlhf | |
| - OpenAssistant/oasst1 | |
| language: | |
| - en | |
| metrics: | |
| - accuracy | |
| tags: | |
| - human feedback | |
| - rlhf | |
| - preferences | |
| - alignment | |
| - HALO | |
| - halos | |
| - dpo | |
| - rl | |
|  | |
| This repo contains the model checkpoints for: | |
| - model family <b>pythia1-4b</b> | |
| - optimized with the loss <b>SFT+KTO</b> | |
| - aligned using the SHP, Anthropic HH and Open Assistant datasets. | |
| Please refer to our [code repository](https://github.com/ContextualAI/HALOs) or [blog](https://contextual.ai/better-cheaper-faster-llm-alignment-with-kto/) which contains intructions for training your own HALOs and links to our model cards. | |
| If you find this repo or the technical paper useful in your research, please feel free to cite [our work](https://github.com/ContextualAI/HALOs/blob/main/assets/report.pdf): | |
| ``` | |
| @techreport{ethayarajh2023halos, | |
| author = {Ethayarajh, Kawin and Xu, Winnie, and Jurafsky, Dan and Kiela, Douwe}, | |
| title = {Human-Centered Loss Functions (HALOs)}, | |
| institution = {Contextual AI}, | |
| note = {https://github.com/ContextualAI/HALOs/blob/main/assets/report.pdf}, | |
| year = {2023}, | |
| } | |
| ``` |