SBS Figures: Pre-training Figure QA from Stage-by-Stage Synthesized
  Images
Authors: name: Risa Shinoda; name: Kuniaki Saito; name: Shohei Tanaka; name: Tosho Hirasawa; name: Yoshitaka Ushiku
Published: 2024-12-23 14:25:33

Abstract: Building a large-scale figure QA dataset requires a considerable amount of
work, from gathering and selecting figures to extracting attributes like text,
numbers, and colors, and generating QAs. Although recent developments in LLMs
have led to efforts to synthesize figures, most of these focus primarily on QA
generation. Additionally, creating figures directly using LLMs often encounters
issues such as code errors, similar-looking figures, and repetitive content in
figures. To address this issue, we present SBSFigures (Stage-by-Stage Synthetic
Figures), a dataset for pre-training figure QA. Our proposed pipeline enables
the creation of chart figures with complete annotations of the visualized data
and dense QA annotations without any manual annotation process. Our
stage-by-stage pipeline makes it possible to create diverse topic and
appearance figures efficiently while minimizing code errors. Our SBSFigures
demonstrate a strong pre-training effect, making it possible to achieve
efficient training with a limited amount of real-world chart data starting from
our pre-trained weights.