BAAI
/

AquilaChat2-34B-16K

Text Generation

Model card Files Files and versions Community

Anhforth commited on Oct 20, 2023

Commit

f70ac68

•

1 Parent(s): a5dbc58

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -35,7 +35,7 @@ quantization_config=BitsAndBytesConfig(
                         bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16,
                     )
-model = AutoModelForCausalLM.from_pretrained(model_info, trust_remote_code=True, torch_dtype=torch.bfloat16
                                                 # quantization_config=quantization_config, # Uncomment this line for 4bit quantization
                                                 )
 model.eval()

                         bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16,
                     )
+model = AutoModelForCausalLM.from_pretrained(model_info, trust_remote_code=True, torch_dtype=torch.bfloat16,
                                                 # quantization_config=quantization_config, # Uncomment this line for 4bit quantization
                                                 )
 model.eval()