cosmoquester commited on
Commit
ed86142
1 Parent(s): e89f23e

docs: Add Benchmark table

Browse files
Files changed (1) hide show
  1. README.md +92 -0
README.md CHANGED
@@ -23,6 +23,98 @@ You can also test mask filling performance using `[MASK]` token like this.
23
  [BOS] [MASK] 먹었어? [EOS]
24
  ```
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## Used Datasets
27
 
28
  ### [모두의 말뭉치](https://corpus.korean.go.kr/)
23
  [BOS] [MASK] 먹었어? [EOS]
24
  ```
25
 
26
+ ## Benchmark
27
+
28
+ <style>
29
+ table {
30
+ border-collapse: collapse;
31
+ border-style: hidden;
32
+ width: 100%;
33
+ }
34
+
35
+ td, th {
36
+ border: 1px solid #4d5562;
37
+ padding: 8px;
38
+ }
39
+ </style>
40
+
41
+ <table>
42
+ <tr>
43
+ <th>Dataset</th>
44
+
45
+ <td>KLUE NLI dev</th>
46
+ <td>NSMC test</td>
47
+ <td>QuestionPair test</td>
48
+ <td colspan="2">KLUE TC dev</td>
49
+ <td colspan="3">KLUE STS dev</td>
50
+ <td colspan="3">KorSTS dev</td>
51
+ <td colspan="2">HateSpeech dev</td>
52
+ </tr>
53
+ <tr>
54
+ <th>Metric</th>
55
+
56
+ <!-- KLUE NLI -->
57
+ <td>Acc</th>
58
+
59
+ <!-- NSMC -->
60
+ <td>Acc</td>
61
+
62
+ <!-- QuestionPair -->
63
+ <td>Acc</td>
64
+
65
+ <!-- KLUE TC -->
66
+ <td>Acc</td>
67
+ <td>F1</td>
68
+
69
+ <!-- KLUE STS -->
70
+ <td>F1</td>
71
+ <td>Pearson</td>
72
+ <td>Spearman</td>
73
+
74
+ <!-- KorSTS -->
75
+ <td>F1</td>
76
+ <td>Pearson</td>
77
+ <td>Spearman</td>
78
+
79
+ <!-- HateSpeech -->
80
+ <td>Bias Acc</td>
81
+ <td>Hate Acc</td>
82
+ </tr>
83
+
84
+ <tr>
85
+ <th>Score</th>
86
+
87
+ <!-- KLUE NLI -->
88
+ <td>0.7390</th>
89
+
90
+ <!-- NSMC -->
91
+ <td>0.8877</td>
92
+
93
+ <!-- QuestionPair -->
94
+ <td>0.9208</td>
95
+
96
+ <!-- KLUE TC -->
97
+ <td>0.8667</td>
98
+ <td>0.8637</td>
99
+
100
+ <!-- KLUE STS -->
101
+ <td>0.7654</td>
102
+ <td>0.8090</td>
103
+ <td>0.8040</td>
104
+
105
+ <!-- KorSTS -->
106
+ <td>0.8067</td>
107
+ <td>0.7909</td>
108
+ <td>0.7784</td>
109
+
110
+ <!-- HateSpeech -->
111
+ <td>0.8280</td>
112
+ <td>0.5669</td>
113
+ </tr>
114
+ </table>
115
+
116
+ - The performance was measured using [the notebooks here](https://github.com/cosmoquester/transformers-bart-finetune) with colab.
117
+
118
  ## Used Datasets
119
 
120
  ### [모두의 말뭉치](https://corpus.korean.go.kr/)