Molmo-7B-O BnB 4bit quant

30GB -> 7GB

approx. 12GB VRAM required

base model for more information:

https://huggingface.co/allenai/Molmo-7B-O-0924

example code:

https://github.com/cyan2k/molmo-7b-bnb-4bit

performance metrics & benchmarks to compare with base will follow over the next week

Downloads last month
1,213
Safetensors
Model size
4.35B params
Tensor type
F32
ยท
U8
ยท
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The HF Inference API does not support model that require custom code execution.

Model tree for cyan2k/molmo-7B-O-bnb-4bit

Quantized
(2)
this model

Spaces using cyan2k/molmo-7B-O-bnb-4bit 2