manta-lm-base / tokenization_manta.py
nthngdy's picture
Create tokenization_manta.py
d607139
from transformers import ByT5Tokenizer