eson's picture
add compression leaderboard
1b7fc74
raw history blame
No virus
113 Bytes
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")