What should be the input text?
Hi 👋 thanks for your great work!
I was trying to play around with google/matcha-chart2text-statista and google/matcha-chart2text-pew by following the instructions presented in the google/matcha-chartqa model card.
My question is about the text input that should be associated with the chart image, for the chart summarization task. I left it blank, and got acceptable results:
inputs = processor(images=image, text="", return_tensors="pt")
predictions = model.generate(**inputs, max_new_tokens=512)
but I was wondering whether you fine-tuned it with a text input stating the instruction, something like "summarize this chart".
Thanks!
Hey thanks for your question. I tried both with and without a prefix like "summarize the chart" and the difference was little to none since this ckpt is only fine-tuned to summarize the chart anyways. Let me double-check which ckpt this was and get back to you :)
update: it should be without the prefix.