Hugging Face's logo

lsmpp
/
qwenillustrious

Model card Files
xet
Community
qwenillustrious / transformers /utils /test_module /custom_tokenization.py
lsmpp's picture
lsmpp
Add files using upload-large-folder tool
d477207 verified
raw
history blame contribute delete
88 Bytes
 from transformers import BertTokenizer
class CustomTokenizer(BertTokenizer):
pass