dvilasuero HF staff commited on
Commit
2ebbfb4
1 Parent(s): 49945e6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -9
README.md CHANGED
@@ -13,14 +13,15 @@ tags:
13
  - dpo
14
  - preference
15
  - ultrafeedback
16
- license: apache-2.0
17
  ---
 
 
 
18
 
19
  # Model Card for Notus 7B v1
20
 
21
- <div align="center">
22
- <img src="https://cdn-uploads.huggingface.co/production/uploads/60f0608166e5701b80ed3f02/LU-vKiC0R7UxxITrwE1F_.png" alt="Image was artificially generated by Dalle-3 via ChatGPT Pro"/>
23
- </div>
24
 
25
  Notus is a collection of fine-tuned models using Direct Preference Optimization (DPO) and related RLHF techniques. This model is version 1, fine-tuned with DPO starting with zephyr-7b-beta's SFT model.
26
 
@@ -41,7 +42,7 @@ with the original Zephyr dDPO model and other 7B models.
41
  - **Shared by:** Argilla
42
  - **Model type:** GPT-like 7B model DPO fine-tuned
43
  - **Language(s) (NLP):** Mainly English
44
- - **License:** Apache 2.0 (same as Zephyr 7B SFT and Mistral 7B v0.1)
45
  - **Finetuned from model:** [`alignment-handbook/zephyr-7b-sft-full`](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full)
46
 
47
  ### Model Sources
@@ -53,18 +54,18 @@ with the original Zephyr dDPO model and other 7B models.
53
  ## Performance
54
 
55
  ### Chat benchmarks
56
- This shows the updated table, based on Zephyr-7b-β original table for [MT-Bench](https://huggingface.co/spaces/lmsys/mt-bench) and [AlpacaEval](https://tatsu-lab.github.io/alpaca_eval/) benchmarks:
57
 
58
  | Model | Size | Alignment | MT-Bench (score) | AlpacaEval (win rate %) |
59
  |-------------|-----|----|---------------|--------------|
60
  | MPT-Chat | 7B |dSFT |5.42| -|
61
  | Xwin-LMv0.1 | 7B| dPPO| 6.19| 87.83|
62
  | Mistral-Instructv0.1 | 7B| - | 6.84 |-|
63
- | Zephyr-7b-β 🪁 | 7B | **dDPO** | **7.34** | 90.60 |
64
- | **Notus-7b-v1** | 7B | **dDPO** | 7.30 | **91.42** |
65
  | GPT-3.5-turbo | - |RLHF |7.94 |89.37|
66
  | Claude 2 | - |RLHF |8.06| 91.36|
67
- | Cohere Command 2 | - |RLHF |-| 90.62|
68
  | GPT-4 | -| RLHF |8.99| 95.28|
69
  | Falcon-Instruct | 40B |dSFT |5.17 |45.71|
70
  | Guanaco | 65B | SFT |6.41| 71.80|
@@ -75,6 +76,8 @@ This shows the updated table, based on Zephyr-7b-β original table for [MT-Bench
75
 
76
  ## Academic benchmarks
77
 
 
 
78
  | Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | DROP |
79
  |-----------------------------------------------|---------|-------|-----------|-------|------------|------------|-------|-------|
80
  | Zephyr 7B dDPO (HuggingFaceH4/zephyr-7b-beta) | 52.15 | 62.03 | 84.36 | 61.07 | **57.45** | 77.74 | 12.74 | **9.66** |
 
13
  - dpo
14
  - preference
15
  - ultrafeedback
16
+ license: mit
17
  ---
18
+ <div align="center">
19
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/60420dccc15e823a685f2b03/CuMO3IjJfymC94_5qd15T.png" alt="Image was artificially generated by Dalle-3 via ChatGPT Pro" width="50%"/>
20
+ </div>
21
 
22
  # Model Card for Notus 7B v1
23
 
24
+
 
 
25
 
26
  Notus is a collection of fine-tuned models using Direct Preference Optimization (DPO) and related RLHF techniques. This model is version 1, fine-tuned with DPO starting with zephyr-7b-beta's SFT model.
27
 
 
42
  - **Shared by:** Argilla
43
  - **Model type:** GPT-like 7B model DPO fine-tuned
44
  - **Language(s) (NLP):** Mainly English
45
+ - **License:** MIT (same as Zephyr 7B-beta)
46
  - **Finetuned from model:** [`alignment-handbook/zephyr-7b-sft-full`](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full)
47
 
48
  ### Model Sources
 
54
  ## Performance
55
 
56
  ### Chat benchmarks
57
+ Table adapted from Zephyr-7b-β original table for [MT-Bench](https://huggingface.co/spaces/lmsys/mt-bench) and [AlpacaEval](https://tatsu-lab.github.io/alpaca_eval/) benchmarks. Notus stays on par with Zephyr on MT-Bench, while surpassing Zephyr, Claude 2, and Cohere Command on AlpacaEval. Making Notus the most-competitive 7B commercial model on AlpacaEval.
58
 
59
  | Model | Size | Alignment | MT-Bench (score) | AlpacaEval (win rate %) |
60
  |-------------|-----|----|---------------|--------------|
61
  | MPT-Chat | 7B |dSFT |5.42| -|
62
  | Xwin-LMv0.1 | 7B| dPPO| 6.19| 87.83|
63
  | Mistral-Instructv0.1 | 7B| - | 6.84 |-|
64
+ | Zephyr-7b-β 🪁 | 7B | dDPO | **7.34** | 90.60 |
65
+ | **Notus-7b-v1** | 7B | dDPO | 7.30 | **91.42** |
66
  | GPT-3.5-turbo | - |RLHF |7.94 |89.37|
67
  | Claude 2 | - |RLHF |8.06| 91.36|
68
+ | Cohere Command | - |RLHF |-| 90.62|
69
  | GPT-4 | -| RLHF |8.99| 95.28|
70
  | Falcon-Instruct | 40B |dSFT |5.17 |45.71|
71
  | Guanaco | 65B | SFT |6.41| 71.80|
 
76
 
77
  ## Academic benchmarks
78
 
79
+ Results from [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard):
80
+
81
  | Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | DROP |
82
  |-----------------------------------------------|---------|-------|-----------|-------|------------|------------|-------|-------|
83
  | Zephyr 7B dDPO (HuggingFaceH4/zephyr-7b-beta) | 52.15 | 62.03 | 84.36 | 61.07 | **57.45** | 77.74 | 12.74 | **9.66** |