Spaces:
Running
on
Zero
Please add VQA capability!
One of the most used tasks (at least in my opinion) is visual question answering aka VQA. Unfortunately, the model does not include this task. Would it be possible to extend the demo to support it as well? The HF-blog article [1] mentions how to finetune Florence-2. However, I don't have the resources for finetuning it :-)
I think you can find a dataset and try it free on google colab. Its very easy to fine tune and it takes ~1.5 hours with T4 on colab for 3500 train data samples.
Huggingface team finetuned it, here is the model card https://huggingface.co/HuggingFaceM4/Florence-2-DocVQA and related space https://huggingface.co/spaces/andito/Florence-2-DocVQA
How to interact with this space using python??
How to interact with this space using python??
https://huggingface.co/microsoft/Florence-2-large/blob/main/sample_inference.ipynb
Florence-2 with VQA support