tokyotech-llm

/

Swallow-MX-8x7b-NVE-v0.1

like 26

readmeのsample codeですが､一般的には､

・original
tokenizer = AutoTokenizer.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
model = AutoModelForCausalLM.from_pretrained(model_name)

・提案
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name,torch_dtype=torch.bfloat16, device_map="auto")

ではないかと思います!