Update README.md
#1
by
ChenWeiLi
- opened
README.md
CHANGED
@@ -33,3 +33,10 @@ The training process was geared towards simulating verbal exchanges between doct
|
|
33 |
- The user can take on the role of a doctor, and the model can engage in conversation with you as if it were a patient.
|
34 |
- You can provide the model with a brief patient background in the system prompt, and the model will respond based on that prompt. (see [Examples](http://www.tame.org.tw/webmag/news/newsFile/508/3.110%E5%B9%B4%E7%AC%AC%E4%B8%80%E6%AC%A1OSCE%20SP%E5%8A%87%E6%83%85%E6%91%98%E8%A6%81%E9%A1%8C%E5%9E%8B%E7%AF%84%E4%BE%8B(63%E4%BE%8B).pdf))
|
35 |
- Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
- The user can take on the role of a doctor, and the model can engage in conversation with you as if it were a patient.
|
34 |
- You can provide the model with a brief patient background in the system prompt, and the model will respond based on that prompt. (see [Examples](http://www.tame.org.tw/webmag/news/newsFile/508/3.110%E5%B9%B4%E7%AC%AC%E4%B8%80%E6%AC%A1OSCE%20SP%E5%8A%87%E6%83%85%E6%91%98%E8%A6%81%E9%A1%8C%E5%9E%8B%E7%AF%84%E4%BE%8B(63%E4%BE%8B).pdf))
|
35 |
- Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)**
|
36 |
+
|
37 |
+
### Model Evaluation
|
38 |
+
The model got the same **TMMLU+** (0 shot) performance when evaluated using [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with default settings.
|
39 |
+
| Details on TMMLU+ (0 shot):<br/>Model | STEM | Social Science | Humanities | Other | AVG |
|
40 |
+
|-----------------------------------------------------|--------------|----------------|------------|------------|---------|
|
41 |
+
| Taiwan-inquiry_7B_v2.0 | 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |
|
42 |
+
| Breeze-7B-Instruct-v1_0 | 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |
|