Isotonic
/

smol_llama-4x220M-MoE

Text Generation

Mixture of Experts

BEE-spoke-data/smol_llama-220M-openhermes

BEE-spoke-data/beecoder-220M-python

BEE-spoke-data/zephyr-220m-sft-full

BEE-spoke-data/zephyr-220m-dpo-full

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Isotonic commited on Feb 5

Commit

62257b4

•

1 Parent(s): f272e9c

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -44,7 +44,7 @@ tokenizer = AutoTokenizer.from_pretrained(model)
 pipeline = transformers.pipeline(
     "text-generation",
     model=model,
-    model_kwargs={"torch_dtype": torch.float16},
 )
 messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]

 pipeline = transformers.pipeline(
     "text-generation",
     model=model,
+    model_kwargs={"torch_dtype": torch.bfloat16},
 )
 messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]