How to get Image embedding using Fuyu
#37
by
oaishi
- opened
Hi, Thanks so much for sharing the pretrained model. The results look amazing. I wanted to use Fuyu to get the embedding of chart images and compare them to each other. Anyone has any tips how to achieve it?
Hi, I do not think that is good idea. You have no image embeddings or encoder to begin with, I guess your best bet is just put in the image without the text prompt and try to compare the features in some layer below the head. As far as I know this release is also not the model finetuned for inference on the chart images.
Thanks, @ludeksvoboda ! I appreciate your feedback!