Update README.md
Browse files
README.md
CHANGED
|
@@ -8,3 +8,8 @@ widget:
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# roberta-base-ko
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# roberta-base-ko
|
| 11 |
+
|
| 12 |
+
A roberta model trained from the scratch. It has almost the same structure as [roberta-base](roberta-base), but with a few differences.
|
| 13 |
+
|
| 14 |
+
- hidden_act: `mish`
|
| 15 |
+
- position_embedding_type: "relative_key_query"
|