ChenWeiLi commited on
Commit
349ba54
1 Parent(s): ca73e60

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -1
README.md CHANGED
@@ -35,4 +35,10 @@ The training process was geared towards simulating verbal exchanges between doct
35
  - Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)**
36
 
37
  ### Model evaluation
38
- The model achieved the same **TMMLU+** performance when evaluated using [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with default settings.
 
 
 
 
 
 
 
35
  - Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)**
36
 
37
  ### Model evaluation
38
+ The model got the **TMMLU+** (0 shot) performance using [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with default settings.
39
+
40
+ |Details on TMMLU+ (0 shot):<br/>Model | Base Model | STEM | Social Science | Humanities | Other | AVG |
41
+ |-----------------------------------------------------|-----------------------|-----------------|----------------|------------|--------- |---------|
42
+ | Taiwan-inquiry_7B_v2.0 |Breeze-7B-Instruct-v1_0| 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |
43
+ | Taiwan-inquiry_7B_v1.1 |Taiwan-inquiry_7B_v1.0 | 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |
44
+ | Taiwan-inquiry_7B_v1.0 |Taiwan-LLM-7B-v2.1-chat| 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |