Intel
/

Mixtral-8x7B-Instruct-v0.1-int4-inc

Model card Files Files and versions Community

yintongl commited on May 11

Commit

8caf5c9

•

1 Parent(s): 540157f

Update README.md

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -11,7 +11,9 @@ datasets:
 ## Model Details: Mixtral-8x7B-Instruct-v0.1-int4-inc
-This model is an int4 model with group_size 128 of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)  generated by [intel/auto-round](https://github.com/intel/auto-round).  Layers "block_sparse_moe.gate" have not been quantized due to the exporting issue of  AutoGPTQ format.
 ## How To Use

 ## Model Details: Mixtral-8x7B-Instruct-v0.1-int4-inc
+This model is an int4 model with group_size 128 of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)  generated by [intel/auto-round](https://github.com/intel/auto-round).  Layers "block_sparse_moe.gate" have not been quantized due to the exporting issue of  AutoGPTQ format.
+Inference of this model is compatible with AutoGPTQ's Kernel.
 ## How To Use