Severian
/

Jamba-Hercules

Text Generation

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Severian commited on Apr 2

Commit

4206bf1

•

1 Parent(s): 7269076

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -56,14 +56,14 @@ double_quant_config = BitsAndBytesConfig(
 )
 model = AutoModelForCausalLM.from_pretrained(
-    "Severian/Jamba-Open-Hermes",
     device_map="auto",
     trust_remote_code=True,
     torch_dtype=torch.bfloat16,
     attn_implementation="flash_attention_2",
     quantization_config=double_quant_config,
 )
-tokenizer = AutoTokenizer.from_pretrained("Severian/Jamba-Open-Hermes")
 input_text = """<|im_start|>system
 You are a helpful AI assistant.

 )
 model = AutoModelForCausalLM.from_pretrained(
+    "Severian/Jamba-Hercules",
     device_map="auto",
     trust_remote_code=True,
     torch_dtype=torch.bfloat16,
     attn_implementation="flash_attention_2",
     quantization_config=double_quant_config,
 )
+tokenizer = AutoTokenizer.from_pretrained("Severian/Jamba-Hercules")
 input_text = """<|im_start|>system
 You are a helpful AI assistant.