Hugging Face's logo

Technoculture
/
MT7Bi-sft

Text Generation
Transformers
PyTorch
4 datasets
English
llama
medical
text-generation-inference
Inference Endpoints
Model card Files Community
1
satyamt commited on
Commit
33efe92
•
1 Parent(s): e386947

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -43,6 +43,17 @@ tags:
43
  |Avg | | 72.0 | 69.2 | 63.6 | 57.4 |
44
  | | | | | | | |
45
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
  | Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
48
  | ------------------ | -------- | --------- | ---- | ---------- | ---------- | -------- |
 
43
  |Avg | | 72.0 | 69.2 | 63.6 | 57.4 |
44
  | | | | | | | |
45
 
46
+ | | | | | | | |
47
+ | --- | ------ |----- |----- |----- |----- |------ |
48
+ |Dataset | meditron-7b | llama-2-7b | pmc-llama-7b | Zephyr-7B-beta* | Mistral-7B-instruct* | MT7Bi |
49
+ |MMLU-Medical | 54.2 | 53.7 | 56.4 | 63.3 | 60.0 | 46.9 |
50
+ |PubMedQA | 74.4 | 61.8 | 59.2 | 46.0 | 17.8 | 65.2 |
51
+ |MedMCQA | 59.2 | 54.4 | 57.6 | 43.0 | 40.2 | 42.7 |
52
+ |MedQA | 47.9 | 44.0 | 42.4 | 42.8 | 32.4 | |
53
+ |MedQA-4-Option| 52.0 | 49.6 | 49.2 | 48.5 | 41.1 | |
54
+ |Avg | 57.5 | 52.7 | 53.0 | 48.7 | 38.3 | 44.3 |
55
+ | | | | | | | |
56
+
57
 
58
  | Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
59
  | ------------------ | -------- | --------- | ---- | ---------- | ---------- | -------- |