Update README.md
Browse files
README.md
CHANGED
@@ -22,16 +22,15 @@ https://github.com/JunnYu/RoFormer_pytorch
|
|
22 |
| | iflytek | tnews | afqmc | cmnli | ocnli | wsc | csl |
|
23 |
| :-----: | :-----: | :---: | :---: | :---: | :---: | :---: | :---: |
|
24 |
| BERT | 60.06 | 56.80 | 72.41 | 79.56 | 73.93 | 78.62 | 83.93 |
|
25 |
-
| RoBERTa | 60.64 | 58.06 | 74.05 |
|
26 |
| RoFormer | 60.91 | 57.54 | 73.52 | 80.92 | 76.07 | 86.84 | 84.63 |
|
27 |
| RoFormer-pytorch(本仓库代码) | 60.60 | 57.51 | 74.44 | 80.79 | 75.67 | 86.84 | 84.77 |
|
28 |
| RoFormerV2<sup>*</sup> | 60.87 | 56.54 | 72.75 | 80.34 | 75.36 | 80.92 | 84.67 |
|
29 |
-
| RoFormerV2<sup>*</sup>-pytorch(本仓库代码) | **62.87** |
|
30 |
| GAU-α | 61.41 | 57.76 | 74.17 | 81.82 | 75.86 | 79.93 | 85.67 |
|
31 |
| GAU-α-pytorch(Adafactor) | 61.18 | 57.52 | 73.42 | 80.91 | 75.69 | 80.59 | 85.5 |
|
32 |
| GAU-α-pytorch(AdamW wd0.01 warm_decay0.1) | 60.68 | 57.95 | 73.08 | 81.02 | 75.36 | 81.25 | 83.93 |
|
33 |
-
|
34 |
-
|
35 |
|
36 |
|
37 |
### CLUE-1.0-test榜单分类任务结果,base版本。
|
@@ -39,9 +38,10 @@ https://github.com/JunnYu/RoFormer_pytorch
|
|
39 |
| | iflytek | tnews | afqmc | cmnli | ocnli | wsc | csl |
|
40 |
| :-----: | :-----: | :---: | :---: | :---: | :---: | :---: | :---: |
|
41 |
| RoFormer-pytorch(本仓库代码) | 59.54 | 57.34 | 74.46 | 80.23 | 73.67 | 80.69 | 84.57 |
|
42 |
-
| RoFormerV2<sup>*</sup>-pytorch(本仓库代码) | 63.15 | 58.24 | 75.42 | 80.59 | 74.17 | 83.79 | 83.73 |
|
43 |
-
| GAU-α-pytorch(Adafactor) | 61.38 | 57.08 | 74.05 | 80.37 | 73.53 | 74.83 | 85.6 |
|
44 |
| GAU-α-pytorch(AdamW wd0.01 warm_decay0.1) | 60.54 | 57.67 | 72.44 | 80.32 | 72.97 | 76.55 | 84.13 |
|
|
|
45 |
|
46 |
## pytorch & tf2.0使用
|
47 |
```python
|
22 |
| | iflytek | tnews | afqmc | cmnli | ocnli | wsc | csl |
|
23 |
| :-----: | :-----: | :---: | :---: | :---: | :---: | :---: | :---: |
|
24 |
| BERT | 60.06 | 56.80 | 72.41 | 79.56 | 73.93 | 78.62 | 83.93 |
|
25 |
+
| RoBERTa | 60.64 | 58.06 | 74.05 | 81.24 | 76.00 | 87.50 | 84.50 |
|
26 |
| RoFormer | 60.91 | 57.54 | 73.52 | 80.92 | 76.07 | 86.84 | 84.63 |
|
27 |
| RoFormer-pytorch(本仓库代码) | 60.60 | 57.51 | 74.44 | 80.79 | 75.67 | 86.84 | 84.77 |
|
28 |
| RoFormerV2<sup>*</sup> | 60.87 | 56.54 | 72.75 | 80.34 | 75.36 | 80.92 | 84.67 |
|
29 |
+
| RoFormerV2<sup>*</sup>-pytorch(本仓库代码) | **62.87** | 59.03 | **76.20** | 80.85 | 79.73 | 87.82 | **91.87** |
|
30 |
| GAU-α | 61.41 | 57.76 | 74.17 | 81.82 | 75.86 | 79.93 | 85.67 |
|
31 |
| GAU-α-pytorch(Adafactor) | 61.18 | 57.52 | 73.42 | 80.91 | 75.69 | 80.59 | 85.5 |
|
32 |
| GAU-α-pytorch(AdamW wd0.01 warm_decay0.1) | 60.68 | 57.95 | 73.08 | 81.02 | 75.36 | 81.25 | 83.93 |
|
33 |
+
| RoFormerV2<sup>*</sup>-large-pytorch(本仓库代码) | 61.75 | **59.21** | 76.14 | **82.35** | **81.73** | **91.45** | 91.5 |
|
|
|
34 |
|
35 |
|
36 |
### CLUE-1.0-test榜单分类任务结果,base版本。
|
38 |
| | iflytek | tnews | afqmc | cmnli | ocnli | wsc | csl |
|
39 |
| :-----: | :-----: | :---: | :---: | :---: | :---: | :---: | :---: |
|
40 |
| RoFormer-pytorch(本仓库代码) | 59.54 | 57.34 | 74.46 | 80.23 | 73.67 | 80.69 | 84.57 |
|
41 |
+
| RoFormerV2<sup>*</sup>-pytorch(本仓库代码) | **63.15** | 58.24 | 75.42 | 80.59 | 74.17 | 83.79 | 83.73 |
|
42 |
+
| GAU-α-pytorch(Adafactor) | 61.38 | 57.08 | 74.05 | 80.37 | 73.53 | 74.83 | **85.6** |
|
43 |
| GAU-α-pytorch(AdamW wd0.01 warm_decay0.1) | 60.54 | 57.67 | 72.44 | 80.32 | 72.97 | 76.55 | 84.13 |
|
44 |
+
| RoFormerV2-large<sup>*</sup>-pytorch(本仓库代码) | 61.85 | **59.13** | **76.38** | **80.97** | **76.23** | **85.86** | 84.33 |
|
45 |
|
46 |
## pytorch & tf2.0使用
|
47 |
```python
|