Quantized version?
#24
by
MichielBontenbal
- opened
Will quantized versions of Idefics be published? Look forward to it!
hi
@MichielBontenbal
did you check the section https://huggingface.co/HuggingFaceM4/idefics2-8b#model-optimizations?
i am about to add some more information on memory requirements for 4 bit quantized versions. as a sneak peak, there are lots of possibilities to run inference on a <16GB chip!
VictorSanh
changed discussion status to
closed