GPTQ - examples
#4
by
davide221
- opened
If you used AutoGPTQ, can you explain what example did you use for the quantization? Thanks
I quantized with AutoGPTQ, using the wikitext2 dataset. Here's the script I used: https://gist.github.com/TheBloke/b47c50a70dd4fe653f64a12928286682
Shouldn't be better to provide a coding dataset to quantise a coding model?
Yes, potentially.
I've not yet experimented with per-model datasets but it could potentially help in certain cases, like coding models and non-English models.