ehartford commited on
Commit
10fda0b
·
1 Parent(s): 560c310

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -2,6 +2,8 @@
2
  datasets:
3
  - ehartford/dolphin
4
  - jondurbin/airoboros-2.2.1
 
 
5
  language:
6
  - en
7
  license: llama2
@@ -10,6 +12,8 @@ license: llama2
10
  Dolphin 2.2 🐬
11
  https://erichartford.com/dolphin
12
 
 
 
13
  <img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/KqsVXIvBd3akEjvijzww7.png" width="600" />
14
 
15
  Dolphin-2.2-70b's training was sponsored by [a16z](https://a16z.com/supporting-the-open-source-ai-community/).
@@ -29,7 +33,7 @@ I modified the dataset for uncensoring, deduping, cleaning, and quality.
29
 
30
  I added Jon Durbin's excellent Airoboros dataset to increase creativity.
31
 
32
- I added a curated subset of Samantha and WizardLM data to train it for multi-turn conversation.
33
 
34
  ## Training
35
  It took 5 days to train 3 epochs on 4x A100s using qLoRA and Axolotl
 
2
  datasets:
3
  - ehartford/dolphin
4
  - jondurbin/airoboros-2.2.1
5
+ - ehartford/samantha-data
6
+ - WizardLM/WizardLM_evol_instruct_V2_196k
7
  language:
8
  - en
9
  license: llama2
 
12
  Dolphin 2.2 🐬
13
  https://erichartford.com/dolphin
14
 
15
+ New in this release: The EOS token works now, and I have added multi-turn conversational data so it has learned to integrate the history with its response when appropriate.
16
+
17
  <img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/KqsVXIvBd3akEjvijzww7.png" width="600" />
18
 
19
  Dolphin-2.2-70b's training was sponsored by [a16z](https://a16z.com/supporting-the-open-source-ai-community/).
 
33
 
34
  I added Jon Durbin's excellent Airoboros dataset to increase creativity.
35
 
36
+ I added a curated subset of Samantha (sans identity and relationship stuff) and WizardLM data to train it for multi-turn conversation.
37
 
38
  ## Training
39
  It took 5 days to train 3 epochs on 4x A100s using qLoRA and Axolotl