Bauwens commited on
Commit
c38ef14
1 Parent(s): d96215e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -1,5 +1,8 @@
1
  ---
2
  license: apache-2.0
3
  ---
4
-
5
- Requires using the Dutch BPE-knockout tokeniser loadable with the [bpe_knockout](https://github.com/bauwenst/BPE-knockout) Python package.
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ To load the tokeniser for this model, do *not* use `AutoTokenizer`. Instead, install [TkTkT](https://github.com/bauwenst/TkTkT) and run:
5
+ ```python
6
+ from tktkt.models.bpe.knockout import BPEKnockout
7
+ tokenizer = BPEKnockout.from_pretrained("Bauwens/RoBERTa-nl_BPE-knockout_30k")
8
+ ```