jondurbin
/

bagel-7b-v0.4

 ---
 license: apache-2.0
+datasets:
+- ai2_arc
+- allenai/ultrafeedback_binarized_cleaned
+- argilla/distilabel-intel-orca-dpo-pairs
+- jondurbin/airoboros-3.2
+- codeparrot/apps
+- facebook/belebele
+- bluemoon-fandom-1-1-rp-cleaned
+- boolq
+- camel-ai/biology
+- camel-ai/chemistry
+- camel-ai/math
+- camel-ai/physics
+- jondurbin/contextual-dpo-v0.1
+- jondurbin/gutenberg-dpo-v0.1
+- jondurbin/py-dpo-v0.1
+- jondurbin/truthy-dpo-v0.1
+- LDJnr/Capybara
+- jondurbin/cinematika-v0.1
+- WizardLM/WizardLM_evol_instruct_70k
+- glaiveai/glaive-function-calling-v2
+- jondurbin/gutenberg-dpo-v0.1
+- grimulkan/LimaRP-augmented
+- lmsys/lmsys-chat-1m
+- ParisNeo/lollms_aware_dataset
+- TIGER-Lab/MathInstruct
+- Muennighoff/natural-instructions
+- openbookqa
+- kingbri/PIPPA-shareGPT
+- piqa
+- Vezora/Tested-22k-Python-Alpaca
+- ropes
+- cakiki/rosetta-code
+- Open-Orca/SlimOrca
+- b-mc2/sql-create-context
+- squad_v2
+- mattpscott/airoboros-summarization
+- migtissera/Synthia-v1.3
+- unalignment/toxic-dpo-v0.2
+- WhiteRabbitNeo/WRN-Chapter-1
+- WhiteRabbitNeo/WRN-Chapter-2
+- winogrande
+# A bagel, with everything (except DPO)
+![bagel](bagel.png)
+## Overview
+This is the pre-DPO version of the mistral-7b model fine-tuned with https://github.com/jondurbin/bagel
+The DPO counterpart can be found here: https://huggingface.co/jondurbin/bagel-dpo-7b-v0.4
+This model is likely better for roleplay usage.
+### Data sources
+There are many data sources used in the bagel models.  See https://github.com/jondurbin/bagel for more information.
+__*Only train splits are used, and a decontamination by cosine similarity is performed at the end as a sanity check against common benchmarks.  If you don't know the difference between train and test, please learn.*__
+<details>
+  <summary>SFT data sources</summary>
+  - [ai2_arc](https://huggingface.co/datasets/ai2_arc)
+    - Abstraction and reasoning dataset, useful in measuring "intelligence" to a certain extent.
+  - [airoboros](https://huggingface.co/datasets/unalignment/spicy-3.1)
+    - Variety of categories of synthetic instructions generated by gpt-4.
+  - [apps](https://huggingface.co/datasets/codeparrot/apps)
+    - Python coding dataset with 10k problems.
+  - [belebele](https://huggingface.co/datasets/facebook/belebele)
+    - Multi-lingual reading comprehension dataset.
+  - [bluemoon](https://huggingface.co/datasets/Squish42/bluemoon-fandom-1-1-rp-cleaned)
+    - Roleplay data scraped from Bluemoon, then cleaned and formatted as ShareGPT.
+  - [boolq](https://huggingface.co/datasets/boolq)
+    - Corpus of yes/no questions (which can be surprisingly difficult for AI to answer apparently?)
+  - [camel-ai biology](https://huggingface.co/datasets/camel-ai/biology)
+    - GPT-4 generated biology instructions.
+  - [camel-ai chemistry](https://huggingface.co/datasets/camel-ai/chemistry)
+    - GPT-4 generated chemistryinstructions.
+  - [camel-ai math](https://huggingface.co/datasets/camel-ai/math)
+    - GPT-4 generated math instructions.
+  - [camel-ai physics](https://huggingface.co/datasets/camel-ai/physics)
+    - GPT-4 generated physics instructions.
+  - [capybara](https://huggingface.co/datasets/LDJnr/Capybara)
+    - Multi-turn dataset used to create the capybara models.
+  - [cinematika](https://huggingface.co/datasets/jondurbin/cinematika-v0.1) (instruction and plain text)
+    - RP-style data synthesized from movie scripts so the model isn't quite as boring as it otherwise would be.
+  - [emobank](https://github.com/JULIELab/EmoBank)
+    - Emotion annotations using the Valence-Arousal-Domninance scheme.
+  - [evol-instruct](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k)
+    - WizardLM's evol instruct 70k dataset.
+  - [glaive-function-calling-v2](https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2)
+    - GlaiveAI function calling dataset.
+  - [gutenberg](https://www.gutenberg.org/) (plain text)
+    - Books/plain text, again to make the model less boring, only a handful of examples supported by [chapterize](https://github.com/JonathanReeve/chapterize)
+  - [limarp-augmented](https://huggingface.co/datasets/grimulkan/LimaRP-augmented)
+    - Augmented and further modified version of [LimaRP](https://huggingface.co/datasets/lemonilia/LimaRP)
+  - [lmsys_chat_1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) (only gpt-4 items, also used for DPO)
+    - Chats collected by the lmsys chat arena, containing a wide variety of chats with various models.
+  - [lollms](https://huggingface.co/datasets/ParisNeo/lollms_aware_dataset)
+    - LoLLMs question answering dataset by ParisNeo, with helpful question answer pairs for using LoLLMs.
+  - [mathinstruct](https://huggingface.co/datasets/TIGER-Lab/MathInstruct)
+    - Composite dataset with a variety of math-related tasks and problem/question formats.
+  - [natural_instructions](https://huggingface.co/datasets/Muennighoff/natural-instructions)
+    - Millions of instructions from 1600+ task categories (sampled down substantially, stratified by task type)
+  - [openbookqa](https://huggingface.co/datasets/openbookqa)
+    - Question answering dataset.
+  - [pippa](https://huggingface.co/datasets/kingbri/PIPPA-shareGPT)
+    - Deduped version of [PIPPA](https://huggingface.co/datasets/PygmalionAI/PIPPA) in ShareGPT format.
+  - [piqa](https://huggingface.co/datasets/piqa)
+    - Phyiscal interaction question answering.
+  - [python_alpaca](https://huggingface.co/datasets/Vezora/Tested-22k-Python-Alpaca)
+    - Python instruction response pairs, validated as functional.
+  - [ropes](https://huggingface.co/datasets/ropes)
+    - Reasoning Over PAragraph Effects in Situations - enhances ability to apply knowledge from a passage of text to a new situation.
+  - [rosetta_code](https://huggingface.co/datasets/cakiki/rosetta-code)
+    - Code problems and solutions in a variety of programming languages taken from rosettacode.org.
+  - [slimorca](https://huggingface.co/datasets/Open-Orca/SlimOrca)
+    - Collection of ~500k gpt-4 verified chats from OpenOrca.
+  - [sql-create-context](https://huggingface.co/datasets/b-mc2/sql-create-context)
+    - SQL-targeted dataset, combining WikiSQL and Spider.
+  - [squad_v2](https://huggingface.co/datasets/squad_v2)
+    - Contextual question answering (RAG).
+  - [airoboros-summarization](https://huggingface.co/datasets/mattpscott/airoboros-summarization)
+    - Combination of various summarization datasets, formatted into the airoboros context-obedient format.
+  - [synthia](https://huggingface.co/datasets/migtissera/Synthia-v1.3)
+    - GPT-4 generated data using advanced prompting from Migel Tissera.
+  - whiterabbitneo [chapter 1](https://huggingface.co/datasets/WhiteRabbitNeo/WRN-Chapter-1) and [chapter 2](https://huggingface.co/datasets/WhiteRabbitNeo/WRN-Chapter-2)
+    - Offensive cybersecurity dataset by WhiteRabbitNeo/Migel Tissera
+  - [winogrande](https://huggingface.co/datasets/winogrande)
+    - Fill in the blank style prompts.
+</details>
+<details>
+  <summary>DPO data sources</summary>
+  - [airoboros 3.2](https://huggingface.co/datasets/jondurbin/airoboros-3.2) vs [airoboros m2.0](https://huggingface.co/datasets/jondurbin/airoboros-gpt4-m2.0)
+    - The creative/writing tasks from airoboros-2.2.1 were re-generated using gpt4-0314 and a custom prompt to get longer, more creative, less clichè responses for airoboros 3.1, so we can use the shorter/boring version as the "rejected" value and the rerolled response as "chosen"
+  - [contextual-dpo](https://huggingface.co/datasets/jondurbin/contextual-dpo-v0.1)
+    - Contextual prompt/response dataset using the airoboros context-obedient question answering format.
+  - [helpsteer](https://huggingface.co/datasets/nvidia/HelpSteer)
+    - Really neat dataset provided by the folks at NVidia with human annotation across a variety of metrics.  Only items with the highest "correctness" value were used for DPO here, with the highest scoring output as "chosen" and random lower scoring value as "rejected"
+  - [distilabel_orca_dpo_pairs](https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs)
+    - Another interesting dataset, originally by Intel, enhanced by argilla with [distilabel](https://github.com/argilla-io/distilabel) which provides various DPO pairs generated from prompts included in the SlimOrca dataset.
+  - [gutenberg-dpo](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1)
+    - DPO pairs meant to increase the models novel writing abilities, using public domain books from https://gutenberg.org/
+  - [py-dpo](https://huggingface.co/datasets/jondurbin/py-dpo-v0.1)
+    - Python DPO dataset (based on the SFT python_alpaca dataset above)
+  - [toxic-dpo](https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2)
+    - __*highly toxic and potentially illegal content!*__ De-censorship, for academic and lawful purposes only, of course.  Generated by llama-2-70b via prompt engineering.
+  - [truthy](https://huggingface.co/datasets/jondurbin/truthy-dpo-v0.1)
+    - DPO pairs meant to increase truthfulness of the model, e.g. common misconceptions, differentiate between AI assistants and roleplayed human in terms of corporeal awareness/locality/etc.
+  - [ultrafeedback](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned)
+    - One of the bits of magic behind the Zephyr model.  Only the items with a chosen score of 8 or higher were included.
+</details>
+## Prompt formatting
+In sticking with the theme of the bagel, I didn't want to use a single prompt format, so I used 4 - vicuna, llama-2, alpaca, and chat-ml.
+I also didn't want to randomly select a single prompt format for each item (hoping each instruction would generalize more when used in a variety of prompt formats), so each instruction is converted into every prompt format (with 0.75 probability).
+This means each epoch of our fine-tune is the equivalent of 3 epochs.
+<details>
+  <summary>Alpaca (sort of)</summary>
+  The only caveat here for alpaca format is that most of the datasets didn't have a separate `"input"` value, so there is no `### Input:` block - any additional input should just be in the instruction section.
+  ```
+  Below is an instruction that describes a task.  Write a response that appropriately completes the request.
+  ### Instruction:
+  {system prompt, if provided}
+  {instruction}
+  ### Response:
+  ```
+  The main difference here is that because of the dataset formatting and variety of data sources, it would have been much to tedious to add an `### Input:` block, so the inputs are just in the instruction section.
+  ### Vicuna
+  ```
+  {system prompt, if provided, randomly defaulting to "A chat between a user and an unbiased, uncensored assistant."}
+  USER: {instruction}
+  ASSISTANT:
+  ```
+</details>
+### ChatML (sort of)
+```text
+{bos}<|im_start|>{role}
+{text}
+<|im_end|>{eos}
+```
+### Llama-2 chat (recommended)
+```
+[INST] <<SYS>>
+{system}
+<</SYS>>
+{instruction} [/INST]
+```