Éclair -- Extracting Content and Layout with Integrated Reading Order
  for Documents
Authors: name: Ilia Karmanov; name: Amala Sanjay Deshmukh; name: Lukas Voegtle; name: Philipp Fischer; name: Kateryna Chumachenko; name: Timo Roman; name: Jarno Seppänen; name: Jupinder Parmar; name: Joseph Jennings; name: Andrew Tao; name: Karan Sapra
Published: 2025-02-06 17:07:22

Abstract: Optical Character Recognition (OCR) technology is widely used to extract text
from images of documents, facilitating efficient digitization and data
retrieval. However, merely extracting text is insufficient when dealing with
complex documents. Fully comprehending such documents requires an understanding
of their structure -- including formatting, formulas, tables, and the reading
order of multiple blocks and columns across multiple pages -- as well as
semantic information for detecting elements like footnotes and image captions.
This comprehensive understanding is crucial for downstream tasks such as
retrieval, document question answering, and data curation for training Large
Language Models (LLMs) and Vision Language Models (VLMs). To address this, we
introduce \'Eclair, a general-purpose text-extraction tool specifically
designed to process a wide range of document types. Given an image, \'Eclair is
able to extract formatted text in reading order, along with bounding boxes and
their corresponding semantic classes. To thoroughly evaluate these novel
capabilities, we introduce our diverse human-annotated benchmark for
document-level OCR and semantic classification. \'Eclair achieves
state-of-the-art accuracy on this benchmark, outperforming other methods across
key metrics. Additionally, we evaluate \'Eclair on established benchmarks,
demonstrating its versatility and strength across several evaluation standards.