CMA 1M Byte Experiment
1,018,344-parameter byte-level CMA model with a 256-token vocabulary and 4096-byte context. This repository tracks the latest evaluated language-modeling checkpoint, currently step 16,000.
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "User01110/cma-1M-exp"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)
- Downloads last month
- 1,167