Where is Image to text projector
#3
by
rv2307
- opened
As you are following Llava's approach you need a mm projector to conver the image space to text space. But how are you doing this without projector, Or is projector available in the model bin?
They are in the model bin / safe tensor files:
"multi_modal_projector.linear_1.bias": "model-00001-of-00003.safetensors",
"multi_modal_projector.linear_1.weight": "model-00001-of-00003.safetensors",
"multi_modal_projector.linear_2.bias": "model-00001-of-00003.safetensors",
"multi_modal_projector.linear_2.weight": "model-00001-of-00003.safetensors",
matthewlyleolson
changed discussion status to
closed