neuralmagic
/

granite-3.1-2b-instruct-quantized.w8a8

Text Generation

Inference Endpoints

8-bit precision

compressed-tensors

Model card Files Files and versions Community

nm-research commited on 19 days ago

Commit

f781824

·

verified ·

1 Parent(s): debee57

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -148,8 +148,8 @@ oneshot(
 )
 # Save to disk compressed.
-model.save_pretrained(SAVE_DIR, save_compressed=True)
-tokenizer.save_pretrained(SAVE_DIR)
 ```
 ## Evaluation

 )
 # Save to disk compressed.
+model.save_pretrained(quant_path, save_compressed=True)
+tokenizer.save_pretrained(quant_path)
 ```
 ## Evaluation