Text Generation
Transformers
PyTorch
English
llama
medical
Inference Endpoints
text-generation-inference
satyamt commited on
Commit
4a39413
1 Parent(s): f7b7e4b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -22
README.md CHANGED
@@ -9,17 +9,17 @@ language:
9
  tags:
10
  - medical
11
  ---
 
12
  [Technoculture/MD7b-alpha](https://huggingface.co/Technoculture/MD7b-alpha) adapter merged with its Base Model (Meditron 7B)
13
 
14
  # Evaluations
15
-
16
  ## Open LLM Leaderboard
17
 
18
  | Model | ARC |HellaSwag| MMLU |TruthfulQA|Winogrande|GSM8K|
19
  |---------------------------------------------------|----:|--------:|--------------------------|---------:|---------:|----:|
20
  |[MT7Bi](https://huggingface.co/Technoculture/MT7Bi)|50.94| 73.24|Error: File does not exist| 43.04| 72.06|22.52|
21
 
22
- ### ARC
23
  | Task |Version| Metric | Value | |Stderr|
24
  |-------------|-------|--------------------|-------------|---|------|
25
  |arc_challenge|Yaml |acc,none | 0.48| | |
@@ -28,9 +28,7 @@ tags:
28
  | | |acc_norm_stderr,none| 0.01| | |
29
  | | |alias |arc_challenge| | |
30
 
31
- Average: 50.94%
32
-
33
- ### HellaSwag
34
  | Task |Version| Metric | Value | |Stderr|
35
  |---------|-------|--------------------|---------|---|------|
36
  |hellaswag|Yaml |acc,none | 0.54| | |
@@ -39,13 +37,7 @@ Average: 50.94%
39
  | | |acc_norm_stderr,none| 0| | |
40
  | | |alias |hellaswag| | |
41
 
42
- Average: 73.24%
43
-
44
- ### MMLU
45
-
46
- Average: Error: File does not exist%
47
-
48
- ### TruthfulQA
49
  | Task |Version| Metric | Value | |Stderr|
50
  |--------------|-------|-----------------------|-----------------|---|------|
51
  |truthfulqa |N/A |bleu_max,none | 16.17| | |
@@ -107,26 +99,18 @@ Average: Error: File does not exist%
107
  | | |acc_stderr,none | 0.01| | |
108
  | | |alias | - truthfulqa_mc2| | |
109
 
110
- Average: 43.04%
111
-
112
- ### Winogrande
113
  | Task |Version| Metric | Value | |Stderr|
114
  |----------|-------|---------------|----------|---|------|
115
  |winogrande|Yaml |acc,none | 0.72| | |
116
  | | |acc_stderr,none| 0.01| | |
117
  | | |alias |winogrande| | |
118
 
119
- Average: 72.06%
120
-
121
- ### GSM8K
122
  |Task |Version| Metric |Value| |Stderr|
123
  |-----|-------|-----------------------------|-----|---|------|
124
  |gsm8k|Yaml |exact_match,get-answer | 0.23| | |
125
  | | |exact_match_stderr,get-answer| 0.01| | |
126
  | | |alias |gsm8k| | |
127
 
128
- Average: 22.52%
129
-
130
- Average score: Not available due to errors
131
-
132
  Elapsed time: 03:56:55
 
9
  tags:
10
  - medical
11
  ---
12
+
13
  [Technoculture/MD7b-alpha](https://huggingface.co/Technoculture/MD7b-alpha) adapter merged with its Base Model (Meditron 7B)
14
 
15
  # Evaluations
 
16
  ## Open LLM Leaderboard
17
 
18
  | Model | ARC |HellaSwag| MMLU |TruthfulQA|Winogrande|GSM8K|
19
  |---------------------------------------------------|----:|--------:|--------------------------|---------:|---------:|----:|
20
  |[MT7Bi](https://huggingface.co/Technoculture/MT7Bi)|50.94| 73.24|Error: File does not exist| 43.04| 72.06|22.52|
21
 
22
+ ### ARC: 50.94%
23
  | Task |Version| Metric | Value | |Stderr|
24
  |-------------|-------|--------------------|-------------|---|------|
25
  |arc_challenge|Yaml |acc,none | 0.48| | |
 
28
  | | |acc_norm_stderr,none| 0.01| | |
29
  | | |alias |arc_challenge| | |
30
 
31
+ ### HellaSwag: 73.24%
 
 
32
  | Task |Version| Metric | Value | |Stderr|
33
  |---------|-------|--------------------|---------|---|------|
34
  |hellaswag|Yaml |acc,none | 0.54| | |
 
37
  | | |acc_norm_stderr,none| 0| | |
38
  | | |alias |hellaswag| | |
39
 
40
+ ### TruthfulQA: 43.04%
 
 
 
 
 
 
41
  | Task |Version| Metric | Value | |Stderr|
42
  |--------------|-------|-----------------------|-----------------|---|------|
43
  |truthfulqa |N/A |bleu_max,none | 16.17| | |
 
99
  | | |acc_stderr,none | 0.01| | |
100
  | | |alias | - truthfulqa_mc2| | |
101
 
102
+ ### Winogrande: 72.06%
 
 
103
  | Task |Version| Metric | Value | |Stderr|
104
  |----------|-------|---------------|----------|---|------|
105
  |winogrande|Yaml |acc,none | 0.72| | |
106
  | | |acc_stderr,none| 0.01| | |
107
  | | |alias |winogrande| | |
108
 
109
+ ### GSM8K: 22.52%
 
 
110
  |Task |Version| Metric |Value| |Stderr|
111
  |-----|-------|-----------------------------|-----|---|------|
112
  |gsm8k|Yaml |exact_match,get-answer | 0.23| | |
113
  | | |exact_match_stderr,get-answer| 0.01| | |
114
  | | |alias |gsm8k| | |
115
 
 
 
 
 
116
  Elapsed time: 03:56:55