jbochi commited on
Commit
67bb427
1 Parent(s): 624e9bd

Add instructions to quantize model (#1)

Browse files

- Add instructions to quantize model (e40158bf0962d8ab7acd6ef5388b1f1f4e550ea8)

Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -11,3 +11,11 @@ tags:
11
 
12
  Quantized weights of [coedit](https://github.com/vipulraheja/coedit) for inference with [candle](https://github.com/huggingface/candle/tree/main/candle-examples/examples/quantized-t5).
13
 
 
 
 
 
 
 
 
 
 
11
 
12
  Quantized weights of [coedit](https://github.com/vipulraheja/coedit) for inference with [candle](https://github.com/huggingface/candle/tree/main/candle-examples/examples/quantized-t5).
13
 
14
+ Conversion command, using candle:
15
+
16
+ ```shell
17
+ cargo run --example tensor-tools --release -- quantize \
18
+ --quantization q6k \
19
+ /path/to/coedit-<version>/model.safetensors \
20
+ --out-file model<version>.gguf
21
+ ```