chenhao
commited on
Commit
•
06c7838
1
Parent(s):
c9ec533
Update README.md
Browse files
README.md
CHANGED
|
@@ -56,4 +56,13 @@ hf (pretrained=mistralai/Mistral-7B-v0.1,parallelize=True,load_in_4bit=True,peft
|
|
| 56 |
|
| 57 |
### Original Performance (mistralai/Mistral-7B-v0.1)
|
| 58 |
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
### Original Performance (mistralai/Mistral-7B-v0.1)
|
| 58 |
|
| 59 |
+
hf (pretrained=mistralai/Mistral-7B-v0.1,parallelize=True,load_in_4bit=True), gen_kwargs: (None), limit: 100.0, num_fewshot: None, batch_size: 1
|
| 60 |
+
| Tasks |Version|Filter|n-shot| Metric |Value| |Stderr|
|
| 61 |
+
|---------------------|-------|------|-----:|--------|----:|---|-----:|
|
| 62 |
+
|pubmedqa | 1|none | 0|acc | 0.98|± |0.0141|
|
| 63 |
+
|medmcqa |Yaml |none | 0|acc | 0.45|± |0.0500|
|
| 64 |
+
|professional_medicine| 0|none | 0|acc | 0.64|± |0.0482|
|
| 65 |
+
|college_medicine | 0|none | 0|acc | 0.65|± |0.0479|
|
| 66 |
+
|clinical_knowledge | 0|none | 0|acc | 0.68|± |0.0469|
|
| 67 |
+
|ocn |Yaml |none | 0|acc | 0.62|± |0.0488|
|
| 68 |
+
|aocnp |Yaml |none | 0|acc | 0.47|± |0.0502|
|