Unchecked and Overlooked: Addressing the Checkbox Blind Spot in Large
  Language Models with CheckboxQA
Authors: name: Michał Turski, name: Mateusz Chiliński, name: Łukasz Borchmann

Checkboxes are critical in real-world document processing where the presence
or absence of ticks directly informs data extraction and decision-making
processes. Yet, despite the strong performance of Large Vision and Language
Models across a wide range of tasks, they struggle with interpreting checkable
content. This challenge becomes particularly pressing in industries where a
single overlooked checkbox may lead to costly regulatory or contractual
oversights. To address this gap, we introduce the CheckboxQA dataset, a
targeted resource designed to evaluate and improve model performance on
checkbox-related tasks. It reveals the limitations of current models and serves
as a valuable tool for advancing document comprehension systems, with
significant implications for applications in sectors such as legal tech and
finance.
  The dataset is publicly available at:
https://github.com/Snowflake-Labs/CheckboxQA