nintwentydo/dolphin-vision-72b-4.0bpw-h6-exl2

DolphinVision 72b - 4.0bpw EXL2 🐬

Base model: cognitivecomputations/dolphin-vision-72b

Language model quantized to 4.0bpw with FP16 vision layers merged back in.

Text working in exllamav2/tabbyapi. Vision input not working yet.

n.b. architecture in config.json has been changed from "BunnyQwenForCausalLM" to "Qwen2ForCausalLM" to prevent model from being loaded as llama in tabbyapi.