Spaces:

eson
/

tokenizer-arena

Running

tokenizer-arena / vocab /amber /__init__.py

add amber and crystal_coder

5db13e0 4 months ago

No virus

120 Bytes

	from transformers import LlamaTokenizer

	tokenizer = LlamaTokenizer.from_pretrained("LLM360/Amber", revision="ckpt_356")