Commit
β’
85281fd
1
Parent(s):
1fe96d0
Update README.md
Browse files
README.md
CHANGED
@@ -18,14 +18,15 @@ ByT5-Korean was pre-trained on [mC4](https://www.tensorflow.org/datasets/catalog
|
|
18 |
```text
|
19 |
id: token
|
20 |
0: <pad>
|
21 |
-
1: <
|
22 |
-
2: <
|
23 |
3~258: utf-8 encoding
|
24 |
-
259~277: beginning consonants(μ΄μ±),
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
```
|
|
|
29 |
## Example Inference
|
30 |
|
31 |
```python
|
|
|
18 |
```text
|
19 |
id: token
|
20 |
0: <pad>
|
21 |
+
1: <eos>
|
22 |
+
2: <unk>
|
23 |
3~258: utf-8 encoding
|
24 |
+
259~277: beginning consonants(μ΄μ±), 19κ°(γ±γ²γ΄γ·γΈγΉγ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
)
|
25 |
+
278~298: middle vowel(μ€μ±), 21κ°(γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
‘γ
’γ
£)
|
26 |
+
299~326: final consonant(μ’
μ±), 무μ’
μ±+27κ°(γ±γ²γ³γ΄γ΅γΆγ·γΉγΊγ»γΌγ½γΎγΏγ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
)
|
27 |
+
327~384: from <extra_id_0> to <extra_id_57>
|
28 |
```
|
29 |
+
|
30 |
## Example Inference
|
31 |
|
32 |
```python
|