Yhyu13 commited on
Commit
4909fcf
1 Parent(s): d927329

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -0
README.md CHANGED
@@ -3,3 +3,38 @@ license: other
3
  license_name: microsoft-research-license
4
  license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
5
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  license_name: microsoft-research-license
4
  license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
5
  ---
6
+
7
+ This is the merged model for LoRA https://huggingface.co/Yhyu13/phi-2-sft-alpaca_gpt4_en-ep1-lora
8
+
9
+ ---
10
+
11
+ From this
12
+ https://huggingface.co/microsoft/phi-2/discussions/38
13
+
14
+ Since phi2 requires remote code which HF open llm leaderboard would not accept at this moment,
15
+
16
+ I ran phi2 and my sft to the AlpcaEval benchmark
17
+
18
+ https://tatsu-lab.github.io/alpaca_eval/
19
+
20
+ Here is result evaluated by chatpgpt https://github.com/tatsu-lab/alpaca_eval/pull/183
21
+
22
+ ```
23
+ win_rate standard_error n_total avg_length
24
+ gpt4 73.79 1.54 805 1365
25
+ claude 70.37 1.60 805 1082
26
+ chatgpt 66.09 1.66 805 811
27
+ wizardlm-13b 65.16 1.67 805 985
28
+ vicuna-13b 64.10 1.69 805 1037
29
+ guanaco-65b 62.36 1.71 805 1249
30
+ oasst-rlhf-llama-33b 62.05 1.71 805 1079
31
+ alpaca-farm-ppo-human 60.25 1.72 805 803
32
+ falcon-40b-instruct 56.52 1.74 805 662
33
+ phi-2-alpaca-gpt4(new) 54.23 1.75 804 1138
34
+ text_davinci_003 50.00 0.00 805 307
35
+ alpaca-7b 45.22 1.74 805 396
36
+ phi-2(new) 43.79 1.74 805 924
37
+ text_davinci_001 28.07 1.56 805 296
38
+ ```
39
+
40
+ It could be a milestone for small models, we finally have one open model can run for everyone which surpass GPT3.5!