cosmoquester commited on
Commit
2ea066e
1 Parent(s): 808ffb8

docs: Add Benchamark table

Browse files
Files changed (1) hide show
  1. README.md +79 -0
README.md CHANGED
@@ -23,6 +23,85 @@ You can also test mask filling performance using `[MASK]` token like this.
23
  [BOS] [MASK] 먹었어? [EOS]
24
  ```
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## Used Datasets
27
 
28
  ### [모두의 말뭉치](https://corpus.korean.go.kr/)
 
23
  [BOS] [MASK] 먹었어? [EOS]
24
  ```
25
 
26
+ ## Benchmark
27
+
28
+ <table>
29
+ <tr>
30
+ <th>Dataset</th>
31
+
32
+ <td>KLUE NLI dev</th>
33
+ <td>NSMC test</td>
34
+ <td>QuestionPair test</td>
35
+ <td colspan="2">KLUE TC dev</td>
36
+ <td colspan="3">KLUE STS dev</td>
37
+ <td colspan="3">KorSTS dev</td>
38
+ <td colspan="2">HateSpeech dev</td>
39
+ </tr>
40
+ <tr>
41
+ <th>Metric</th>
42
+
43
+ <!-- KLUE NLI -->
44
+ <td>Acc</th>
45
+
46
+ <!-- NSMC -->
47
+ <td>Acc</td>
48
+
49
+ <!-- QuestionPair -->
50
+ <td>Acc</td>
51
+
52
+ <!-- KLUE TC -->
53
+ <td>Acc</td>
54
+ <td>F1</td>
55
+
56
+ <!-- KLUE STS -->
57
+ <td>F1</td>
58
+ <td>Pearson</td>
59
+ <td>Spearman</td>
60
+
61
+ <!-- KorSTS -->
62
+ <td>F1</td>
63
+ <td>Pearson</td>
64
+ <td>Spearman</td>
65
+
66
+ <!-- HateSpeech -->
67
+ <td>Bias Acc</td>
68
+ <td>Hate Acc</td>
69
+ </tr>
70
+
71
+ <tr>
72
+ <th>Score</th>
73
+
74
+ <!-- KLUE NLI -->
75
+ <td>0.5253</th>
76
+
77
+ <!-- NSMC -->
78
+ <td>0.8425</td>
79
+
80
+ <!-- QuestionPair -->
81
+ <td>0.8945</td>
82
+
83
+ <!-- KLUE TC -->
84
+ <td>0.8047</td>
85
+ <td>0.7988</td>
86
+
87
+ <!-- KLUE STS -->
88
+ <td>0.7411</td>
89
+ <td>0.7471</td>
90
+ <td>0.7399</td>
91
+
92
+ <!-- KorSTS -->
93
+ <td>0.7725</td>
94
+ <td>0.6503</td>
95
+ <td>0.6191</td>
96
+
97
+ <!-- HateSpeech -->
98
+ <td>0.7537</td>
99
+ <td>0.5605</td>
100
+ </tr>
101
+ </table>
102
+
103
+ - The performance was measured using [the notebooks here](https://github.com/cosmoquester/transformers-bart-finetune) with colab.
104
+
105
  ## Used Datasets
106
 
107
  ### [모두의 말뭉치](https://corpus.korean.go.kr/)