bleysg commited on
Commit
b58e599
1 Parent(s): 7becd74

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -21
README.md CHANGED
@@ -29,31 +29,18 @@ The only significant improvement was with TruthfulQA.
29
 
30
  ## MT-bench Performance
31
 
32
- Epoch 4.5 result:
33
- ```
34
- Mode: single
35
- Input file: data/mt_bench/model_judgment/gpt-4_single.jsonl
36
-
37
- ########## First turn ##########
38
- score
39
- model turn
40
- oo-phi-1_5 1 6.0375
41
-
42
- ########## Second turn ##########
43
- score
44
- model turn
45
- oo-phi-1_5 2 4.025
46
-
47
- ########## Average ##########
48
- score
49
- model
50
- oo-phi-1_5 5.03125
51
- ```
52
 
53
 
54
  # Training
55
 
56
- Trained with full-paramaters fine-tuning on 8x RTX A6000-48GB (Ampere) for 5 epochs for 62 hours (12.5h/epoch) at a commodity cost of $390 ($80/epoch).
57
  We did not use [MultiPack](https://github.com/imoneoi/multipack_sampler) packing, as training was begun prior to implementing support for it in Axolotl for this new model type.
58
 
59
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 
29
 
30
  ## MT-bench Performance
31
 
32
+ | Epoch | Average | Turn 1 | Turn 2 |
33
+ |:----------|:----------|:----------|:----------|
34
+ | 3 | 4.85 | 5.69 | 4.01 |
35
+ | 3.5 | 5.19 | 5.91 | 4.46 |
36
+ | 4 | 4.89 | 5.74 | 4.05 |
37
+ | 4.5 | 5.03 | 6.04 | 4.03 |
38
+ | 5 | 4.94 | 5.76 | 4.11 |
 
 
 
 
 
 
 
 
 
 
 
 
 
39
 
40
 
41
  # Training
42
 
43
+ Trained with full-parameters fine-tuning on 8x RTX A6000-48GB (Ampere) for 5 epochs for 62 hours (12.5h/epoch) at a commodity cost of $390 ($80/epoch).
44
  We did not use [MultiPack](https://github.com/imoneoi/multipack_sampler) packing, as training was begun prior to implementing support for it in Axolotl for this new model type.
45
 
46
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)