Commit
•
326dbc8
1
Parent(s):
bd46f0e
Update README.md
Browse files
README.md
CHANGED
|
@@ -119,6 +119,8 @@ SMaLL-100 is a compact and fast massively multilingual machine translation model
|
|
| 119 |
|
| 120 |
The model architecture and config are the same as [M2M-100](https://huggingface.co/facebook/m2m100_418M/tree/main) implementation, but the tokenizer is modified to adjust language codes. So, you should load the tokenizer locally from [tokenization_small100.py](https://huggingface.co/alirezamsh/small100/blob/main/tokenization_small100.py) file for the moment.
|
| 121 |
|
|
|
|
|
|
|
| 122 |
```
|
| 123 |
from transformers import M2M100ForConditionalGeneration
|
| 124 |
from tokenization_small100 import SMALL100Tokenizer
|
|
|
|
| 119 |
|
| 120 |
The model architecture and config are the same as [M2M-100](https://huggingface.co/facebook/m2m100_418M/tree/main) implementation, but the tokenizer is modified to adjust language codes. So, you should load the tokenizer locally from [tokenization_small100.py](https://huggingface.co/alirezamsh/small100/blob/main/tokenization_small100.py) file for the moment.
|
| 121 |
|
| 122 |
+
**Note**: SMALL100Tokenizer requires sentencepiece, so make sure to install it by ```pip install sentencepiece```
|
| 123 |
+
|
| 124 |
```
|
| 125 |
from transformers import M2M100ForConditionalGeneration
|
| 126 |
from tokenization_small100 import SMALL100Tokenizer
|