xu-song's picture
add aya
44c3329
raw
history blame
106 Bytes
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("CohereForAI/aya-101")