manta-lm-base / tokenization_manta.py
nthngdy's picture
Create tokenization_manta.py
d607139
raw
history blame
38 Bytes
from transformers import ByT5Tokenizer