# Towards better identification of fact tables in statistical spreadsheets

Paul Kronlund-Drouault, Ioana Manolescu

<figure><img src="image_1.png" /></figure>

## To cite this version:

Paul Kronlund-Drouault, Ioana Manolescu. Towards better identification of fact tables in statistical spreadsheets. Inria Saclay; LIX – Ecole polytechnique; ENS Lyon. 2025. hal-05219883

HAL Id: hal-05219883

https://inria.hal.science/hal-05219883v1

Submitted on 22 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Towards better identification of fact tables in statistical spreadsheets

<figure><img src="image_3.png" /></figure>

ENS DE LYON

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

Paul Kronlund-Drouault

ENS Lyon, Inria & Institut Polytechnique de Paris

Supervisor: Ioana Manolescu

Inria & Institut Polytechnique de Paris
# 1 Overview

To strengthen public trust and counter disinformation, computational fact-checking leveraging digital data sources attracts interest from journalists and the computer science community (Balalau et al., 2022). In this context, statistical data sources represent a valuable class of information resources, that can act as ground truth in the political scene. Statistics are often multidimensional datasets, where multiple dimensions characterize one value, and dimensions may be organized in hierarchies (Balalau et al., 2022). These sources, compiled at significant expense by national and international organizations, present fundamental challenges that have limited the adoption of automated fact-checking systems. The complexity stems from the heterogeneous nature of these data sources: they are typically published in various formats ranging from HTML web tables to Excel spreadsheet files, with no standardized structure or layout conventions (Balalau et al., 2025).

This heterogeneity is particularly pronounced in Excel spreadsheets, where organizations follow diverse formatting practices, often destined to human readers, making automated extraction and analysis particularly difficult. The problem is made even harder by the multidimensional nature of statistical data. Fact tables from statistical datasets often contain complex header hierarchies, mixed data types, partial aggregations, and embedded metadata (Balalau et al., 2022). These characteristics require sophisticated parsing techniques that can understand not just the structural layout, but also the semantic relationships between different components of the table that reflects itself in visual structure features.

## 1.1 Journalistic fact-checking and the information ecosystem

In an era where political discourse is increasingly dominated by numerical claims, often exaggerated, taken out of context, or fabricated, to persuade, terrorize, or discredit, the role of journalism has evolved to center more and more around data mastery. This transformation has given rise to data journalism, where professional journalists with specialized training in statistical analysis serve as gatekeepers of numerical accuracy. However, the scale of contemporary information flows far exceeds human capacity for manual handling and verification. The emergence of "fake news" phenomena and the rise in utilization of statistics in public debate have created an urgent need for automated tools that can assist journalists in real-time fact-checking scenarios, from live interviews and debates to social media monitoring and investigative reporting. Unfortunately, the volume and complexity of statistical data ("big data") require powerful computational tools that can process, understand, and verify numerical claims at scale.

## 1.2 The StatCheck project

As a response to these challenges and as a tool for combating misinformation, the STATCHECK system represents an ambitious attempt to automatically gather, structure, and utilize statistical data from trusted institutional sources Balalau et al. (2022). The system targets a comprehensive pipeline that includes **web scraping** of official statistics, **automated data extraction** from complex spreadsheet formats, and **indexing** for rapid claim verification. STATCHECK's architecture enables the ingestion of high-dimensional statistical data from sources such as INSEE and Eurostat, converting heterogeneous spreadsheet formats into structured, queryable databases (Balalau et al., 2022). This structured data can then be integrated into knowledge graphs and accessed through direct queries or combined with Named Entity Recognition systems for fully automated claim verification workflows. The work presented in this paper addresses a critical bottleneck in this pipeline: the accurate identification and structural understanding of fact tables within statistical spreadsheets.<fnref n="1" /> By developing machine learning approaches specifically designed for this task, we aim to improve the robustness and scalability of automated statistical fact-checking systems, ultimately supporting both real-time verification needs and long-term investigative journalism workflows.

<footnote n="1">¹By analyzing STATCHECK extraction log, we observed up to 40% failed extraction from files, mostly due to detection failures. Accounting for the proportion of files with no tables we get around 30% of tables missed by the current heuristic based pipeline defined in Balalau et al. (2025).</footnote>
<figure><img src="image_6.png" /><figcaption>Figure 1: STATCHECK architecture overview.</figcaption></figure>

# 2 State of the Art

The field of table analysis has evolved significantly in recent years, with research efforts primarily concentrated on two fundamental tasks: table detection (identifying and localizing tables within documents through bounding boxes) and structural recognition (parsing table organization into rows, columns, and cells). However, the existing literature reveals a substantial bias toward PDF document processing, leaving a critical gap in understanding and analyzing statistical fact tables commonly found in Excel spreadsheets and other structured data formats.

Traditional approaches in table analysis have predominantly focused on extracting layout information that is often already inherently available in structured formats, such as positional relationships, cell alignment data, and basic structural elements. This emphasis has resulted in limited advancement in semantic understanding and content-aware analysis of tabular data, particularly for complex statistical tables that require deeper comprehension of data relationships and contextual meaning.

* Section 2.1 describes table recognition methods based on DETection Transformers models (DETR)

* Section 2.2 discusses table recognition methods based on Large Language Models (LLMs).

## 2.1 Transformer-Based Table Structure Recognition

### Table Transformer (TATR)

The Table Transformer models introduced by Smock et al. (2022) represent a significant advancement in table extraction methodology. TATR is an object detection model that recognizes tables from image input, utilizing a hybrid architecture that combines convolutional neural networks (CNNs) with transformer encoders. The framework consists of two specialized components:

* **TATR-detect:** Identifies table regions within documents and outputs precise bounding boxes

* **TATR-structure:** Performs fine-grained structural analysis, identifying rows, columns, headers, and cell relationships

Building upon previous integration work (Soric, 2025), we experimented with TATR on our spreadsheet dataset by incorporating a spreadsheet-to-image conversion preprocessing step. However, the results proved suboptimal, as the model was primarily designed to extract alignment information that was already available in our structured layout data.

### TSRFormer

TSRFormer (Lin et al., 2022) builds on the TATR model but addresses a critical limitation of previous methods that relied on heuristic mask-to-line modules, which often struggled with low-quality segmentation masks, particularly in challenging scenarios involving distorted or curved tables.

The architecture centers around Separator Regression Transformer (SepRETR), a DETR-based framework specifically designed for predicting row/column separation lines directly from table images.

TSRFormer incorporates several key innovations to address the inherent challenges of applying DETR to table structure recognition:
1. **Prior-Enhanced Matching Strategy:** Instead of relying on the unstable Hungarian matching of vanilla DETR, this approach matches each reference point with its closest ground-truth separator based on spatial proximity, significantly accelerating training convergence.

2. **High-Resolution Cross-Attention Module:** Using a cross-attention mechanism that samples features directly from high-resolution convolutional feature maps, the model can more accurately capture fine-grained structural details essential for accurate separation line prediction.

3. **Context-Enhanced Feature Representation:** The method employs Spatial CNN (SCNN) modules to propagate contextual information across the feature map in both horizontal and vertical directions, enabling each pixel to leverage structural information from the complete table layout.

The model's robustness is particularly evident in its ability to handle complex table variations including borderless cells, large blank spaces, spanning cells, and geometrically distorted or curved table structures. Again, this is more relevant in the context of PDF table extraction, rather than fact tables identification.

The strength of the TSFormer approach lies in its elimination of heuristic post-processing components that is usually necessary for segmentation-based approaches.

## 2.2 Large Language Model for Table Understanding

### SpreadsheetLLM Framework

The SPREADSHEETLLM method, introduced by Dong et al. (2025), leverages the power of modern large language models for spreadsheet understanding. The framework introduces SHEETENCODER, an efficient encoding method designed to handle the extensive two-dimensional grids, flexible layouts, and varied formatting options that characterize spreadsheets.

The SPREADSHEETLLM pipeline consists of three modules:

1. **SheetCompressor:** Implements structural-anchor-based compression to reduce the complexity of spreadsheet representations while preserving critical structural information

2. **Inverse Index Translation:** Provides efficient indexation mechanisms for navigating complex spreadsheet structures

3. **Data-Format-Aware Aggregation:** Performs context-aware data aggregation that respects the semantic meaning of different data formats

Experimental results demonstrate that SPREADSHEETLLM significantly outperforms existing approaches, achieving improvements of 25.6% in GPT-4's in-context learning setting for spreadsheet table detection tasks, and 12.3% improvement over previous methods in general spreadsheet table detection.

### Applications

The potential impact of SPREADSHEETLLM extends beyond extraction purposes, with industry observers noting that such technology could enable non-technical users to query and manipulate spreadsheet data using natural language prompts.

## 2.3 Limitations and Research Gaps

Despite these advances, several critical limitations persist in current table analysis research:

1. **Domain Specificity:** Most existing approaches are optimized for document-based tables rather than statistical fact tables with complex hierarchical structures and semantic relationships.

2. **Scalability:** Performance of LLMs degrades significantly with long contexts, limiting their effectiveness on large, complex spreadsheets.
3. Compute & relative complexity: For cloud-scale data extraction, the huge compute cost of modern LLMs that often have more than a hundred billion parameters makes their use impractical and increases cost by several order of magnitude compared to more specialized approaches.

While State of the Art approaches manage to get very good results on PDF extraction and table identi-
fication from pictures or other unstructured documents, none has yet to overcome the challenge efficiently
gathering structured data from fact tables, with the size constraints of a statistical dataset.

3 Proposed Approach

As highlighted in previous sections, the structure and position of fact tables in statistics spreadsheets can be
very heterogeneous, even within publications from the same institution. While a heuristic-based approach
can achieve good performance, some edge cases still prove difficult to solve with traditional algorithms.
Furthermore, even powerful ML based solution can struggle, especially on large tables, as they are not
optimized for the specific task.

Indeed, while tables all share common components such as *column headers (ch)*, *row headers (rh)* and data zones, they can be laid out in different formats, topologies (organizations of rows, columns, a large variety of header structures), and may contain data of different formats. In order to improve fact extraction performance from *difficult to extract* tables, we decided to try a custom ML-based approach that would identify the table positions and structure based on patterns at the cell level.

In Section 3.1, we present the overall structure of our approach, describe the AI models and heuristics we employ, and explain how these components interact within our extraction pipeline.

In Section 3.2, we present in-depth technical analysis of our algorithmic implementations, with precise
charts of our processes and data structures, including some sample from a labeled dataset.

In Section 3.3, we focus on our data pipeline, detailing in particular:

* The data gathering process;

* Our synthetic data generation method;

* Challenges due to the nature of our data; and

* Optimization techniques we applied.

In Section 3.4 we present the training method, with optimization goals, challenges and evaluation metrics.

3.1 Architecture Overview

Our approach employs a two-stage pipeline that progressively refines table detection and structure identifica-
tion. The pipeline consists of two U-Net (Ronneberger et al., 2015) inspired Convolutional Neural Networks
(CNNs) and intermediate processing steps:

* **Grid pre-processing:** A labeling system classifies the raw cell values from each sheet into 9 data categories (`EMPTY`, `MISSING VALUE`, `TEXT`, `INTEGER`, `FLOAT`, `DATE`, `AMBIGUOUS DATE`, `MIXED`, `WEIRD`, listed also in Table 1 below).

* **Stage 1 - Table Detection:** A trained detection network is used to identify **table regions** from the entire cell grid, producing a binary segmentation map with two classes: **background** and **table**. Note that in this binary segmentation, the cells identified as part of a table do not necessarily form rectangular areas.

* **Intermediate post-processing:** An algorithm converts the probabilistic cell-level predictions into bounding boxes, localizing rectangular candidate table regions for further analysis.

* **Stage 2 - Structure Classification:** A refinement network processes cell grids cropped to the detected bounding boxes, and produces a 4-class segmentation into **background**, **column headers**, **row headers**, and **data** regions. A region of one of these four types is not necessarily rectangular.

This two-stage approach allows us to first broadly locate tables within complex spreadsheets, then pre-
cisely identify their internal structure components. The pure CNN architecture also allows us to process
variable-sized data very reliably, as the convolutional layers apply learned kernels across the spatial dimen-
sions of the inputs, regardless of its size, like a sliding window.
<figure><img src="image_7.png" /><figcaption>Figure 2: Architecture overview diagram</figcaption></figure>

**TableGridCNN** The Detection CNN as well as the Structure CNN in Figure 2 consist of a **CNN Backbone** together with **Embedding layers**. We call this TableGridCNN.

The actual CNN backbone has the same structure for both stages, with the only difference being the output classes. It is composed of the following components:

1. **Encoder Blocks**: The input is progressively downsampled through four encoder blocks:

$$e_1 = \text{Encoder}_1(\mathbf{x}_{emb}) \qquad (1)$$

$$e_2 = \text{Encoder}_2(\text{Pool}(e_1)) \qquad (2)$$

$$e_3 = \text{Encoder}_3(\text{Pool}(e_2)) \qquad (3)$$

$$e_4 = \text{Encoder}_4(\text{Pool}(e_3)) \qquad (4)$$

where Pool(·) represents the pooling operation that reduces spatial dimensions.

2. **Bottleneck**: The deepest feature representation is computed as:

$$b = \text{Bottleneck}(\text{Pool}(e_4)) \qquad (5)$$

3. **Decoder Blocks**: The feature maps are progressively upsampled through four decoder blocks:

$$d_4 = \text{Decoder}_4(\text{Upsample}(b)) \qquad (6)$$

$$d_3 = \text{Decoder}_3(\text{Upsample}(d_4)) \qquad (7)$$

$$d_2 = \text{Decoder}_2(\text{Upsample}(d_3)) \qquad (8)$$

$$d_1 = \text{Decoder}_1(\text{Upsample}(d_2)) \qquad (9)$$

where Upsample(·) performs nearest-neighbor interpolation with a scale factor of 2.

4. **Final Classification**: A final convolutional layer produces the output segmentation map:

$$y = \text{FinalConv}(d_1) \qquad (10)$$

The actual size of the Encoder and symmetrical Decoder layers can vary. In our benchmarks and experiments, we used two different versions (sizes) of the same model:

1. **TableGridCNN** is our smallest model, with 16 embedding dimensions, encoder blocks of size 32 to 512 and the symmetrical decoders. It performs well on small grids, as long as there are not too many concurrent structures to process.

2. **TableGridCNN-Large** is our big model, with 24 embedding dimensions, encoder blocks of size 48 to 768 and the symmetrical decoders. It has been tested and performs well in large grids, with large amounts of content.

Note that the models are still relatively small, thus they are able to process tables with a speed comparable to heuristics-based methods (especially considering the amount of overhead python adds to the operations used in the heuristics, compared to the very optimized runtimes powering `torch` for convolutions).
## 3.2 Method details

We now detail the steps of our method, starting with the cell classification. We then describe table detection and structure identification.

### Cell classification

Cells in statistics data sheets are grouped in different categories depending on the nature of their content; this can then be used to understand the structure of the underlying data.

In our work, we decided to use 9 categories for classification, as shown in Table 1.

<table><caption>Table 1: Target categories used for each cell (r, c) of the sheet.</caption><thead><tr><td>Id</td><td>Label</td><td>Interpretation</td></tr></thead><tbody><tr><td>0</td><td>EMPTY</td><td>The cell is truly empty or blank ("")</td></tr><tr><td>1</td><td>TEXT</td><td>Non-numeric, non-date string</td></tr><tr><td>2</td><td>FLOAT</td><td>Non-integer IEEE-754 value (e.g. 3.14, 1.2e-3)</td></tr><tr><td>3</td><td>INTEGER</td><td>Signed or zero integer that is likely not a year/date, e.g., 12, or 120 321</td></tr><tr><td>4</td><td>MIXED</td><td>Alphanumeric value with a high digit ratio (e.g. "A12b")</td></tr><tr><td>5</td><td>DATE</td><td>Cell already typed as DATE by the spreadsheet editor or a string matching strict date regexes (e.g. 2020-12-31)</td></tr><tr><td>6</td><td>AMBIGUOUS DATE</td><td>Cell that contains an integer number that could (with good probability) be a year (thus, a <span style="color:blue;">date</span>) and a short (1-4 characters) <span style="color:red;">string</span> e.g. "2020 Q4"</td></tr><tr><td>7</td><td>WEIRD</td><td>Anything that falls through the above rules</td></tr><tr><td>8</td><td>MISSING VALUE</td><td>A set of predefined values used by the data provider to represent missing data (see Table 2)</td></tr></tbody></table>

<table><caption>Table 2: Symbols used in statistics datasets to represent missing data.</caption><thead><tr><td>Flag</td><td>Description</td></tr></thead><tbody><tr><td>-</td><td>Not applicable</td></tr><tr><td>:</td><td>Not available</td></tr><tr><td>b, —</td><td>Break in time series</td></tr><tr><td>d</td><td>Definition differs</td></tr><tr><td>e</td><td>Estimated</td></tr><tr><td>f</td><td>Forecast</td></tr><tr><td>i</td><td>Imputed by Eurostat or other sources</td></tr><tr><td>m</td><td>Missing value; data cannot exist</td></tr><tr><td>n</td><td>Not significant</td></tr><tr><td>p</td><td>Provisional</td></tr><tr><td>u</td><td>Low reliability</td></tr><tr><td>0n</td><td>Less than half of the unit used, different from real zero</td></tr></tbody></table>
<figure><img src="image_8.png" /><figcaption>Figure 3: Sample labeled grid, with one color for each cell category</figcaption></figure>

Table Detection

In order to identify fact table areas in the labeled grid, the *Table Detection Pipeline* is composed of a **TableGridCNN** that identifies the potential zones of interest via binary classification between a **table** class and a **background** class, and a **BoxProcessor** that converts the output classification to rectangular bounding boxes. The model size can vary, as described above.

The **BoxProcessor** uses a combination of heuristics and graph algorithms to extract good table zones from the model output, and eliminate errors. It first converts the output of the model to probabilities for each cell, stored in array (torch Tensors) with shape (batch size, rows, columns), then builds gradient maps in each direction (left, right, up, down) for each cell. This gradient map has the shape (batch size, rows, columns, 4) with gradient values in the 4 directions for every pixel. It is then converted it to a (sparse) matrix representation of the graph with pixels in the grid cell as nodes, and 4-directional gradients as edge values. Finally an adaptive thresholding is applied to get connected components, which can then be easily converted to bounding boxes after some sanitization (average probability, rectangularity, etc.).

Structure identification

The structure identification step is relatively similar to the first part of the detection, as it uses the same type of model, to process a labeled grid, cropped to only table bounding boxes, and possibly padded.

Here, instead of detecting bounding boxes, we employ a sampling mechanism that can be either deterministic (argmax) or probabilistic (multinomial) to select the final grid layout from the output class distributions. The sampling strategy is passed as a pipeline parameter. Theoretically, probabilistic sampling should get superior results by exploring the full probability space rather than greedily selecting the most confident prediction at each location, but this approach also introduces a risk of sampling poor configurations from lower probability regions of the distribution.

We then obtain a grid of cells classified between **ch**, **rh** or **data**, that could later be directly converted to an *Area* object and used in `STATCHECK` (Balalau et al., 2025).
<figure><img src="image_9.png" /><figcaption>Figure 4: Table Detection Pipeline: from labeled grid to bounding boxes through CNN classification and graph-based postprocessing.</figcaption></figure>

## 3.3 Data generation and processing

Gathering training data was one of the main challenges of this project, as such tasks are novel, and thus no academic references exists. One one hand, we can produce some hand-annotated data for the training, but it is hard to get more than a few hundred samples in this way. On the other hand, we have large amounts of data produced by the basic `STATCHECK` extractor, which can be converted to bounding boxes, and table labels. Both of the above methods presented serious limitations, as the `STATCHECK` data was very limited for *hard* (*difficult to extract*) tables which are our main optimization goal, and often imprecise.

In the end, we built a mix of a hand annotated dataset of selected hard cases from the INSEE corpus, `STATCHECK`-produced curated data, and finally a big set of synthetic tables, with a generation process carefully tweaked to be as close to ground truth as possible. We describe in Section 3.3 the general outline of the process, while Section 3.3 details a useful optimization applied to scale to large spreadsheets (containing many rows).

### Synthetic dataset generation

Our synthetic data generation system creates configurable, realistic tabular data labeled using the classification system defined in Table 1. The generated data includes **fact table zones**, made up of **column headers**, **row headers** and data. Using empirical data gathered over the INSEE dataset, headers are generated with a mix of `TEXT`, `MIXED`, `DATE`, `AMBIGUOUS DATE` and more rarely `INTEGER`. For the data, we mostly use `FLOAT` and `INT`, and some sparse random `AMBIGUOUS DATE`, `MISSING VALUE` and `EMPTY`. In order to better fit the real data, we also add randomly distributed empty lines.

The generation framework consists of three main components: a `GridTablesDataset` container for managing table collections, a `SyntheticGridTableConfig` system controlling generation parameters, and the core table generation engine. The generated tables are grids of numbers, following the label scheme defined in Table 1.
<figure><img src="image_10.png" /></figure>

ch: column header, rh: row header, data: data cell

<table><caption>Figure 5: Structure Classification Pipeline: from labeled grid and table bounding boxes to classified cell grid through CNN multi-class classification and sampling.</caption><thead><tr><th>Category</th><th>Parameter</th><th>Description</th></tr></thead><tbody><tr><td rowspan="5">Global</td><td>min_table_size</td><td>Minimum number of rows/columns for tables</td></tr><tr><td>max_table_size</td><td>Maximum number of rows/columns for tables</td></tr><tr><td>max_table_num</td><td>Maximum number of tables per image</td></tr><tr><td>min_tableSpacing</td><td>Minimum spacing between multiple tables</td></tr><tr><td>table_size_dist</td><td>Distribution used to sample table sizes</td></tr><tr><td rowspan="3">Column Header</td><td>max_height</td><td>Maximum header height (rows)</td></tr><tr><td>multiLayer-prob</td><td>Probability of a multi-layer column header</td></tr><tr><td>empty-prob</td><td>Probability of an empty column header cell</td></tr><tr><td rowspan="3">Row Header</td><td>max_width</td><td>Maximum header width (columns)</td></tr><tr><td>multiLayer-prob</td><td>Probability of a multi-layer row header</td></tr><tr><td>empty-prob</td><td>Probability of an empty row header cell</td></tr><tr><td rowspan="2">Data Region</td><td>emptycells ratio</td><td>Fraction of data cells left intentionally empty</td></tr><tr><td>missing-ratio</td><td>Fraction of data cells replaced with MISSING VALUE</td></tr><tr><td rowspan="6">Noise &amp; Layout</td><td>blank_line-prob</td><td>Chance of inserting a single empty line</td></tr><tr><td>big_empty_line-prob</td><td>Chance of inserting a large empty line block</td></tr><tr><td>big_empty_line_max_size</td><td>Max height of a large empty line block</td></tr><tr><td>big_empty_patch-prob</td><td>Chance of adding a rectangular empty patch</td></tr><tr><td>no_tables-prob</td><td>Probability of generating a tableless grid</td></tr><tr><td>max_random_text</td><td>Max number of stray text blocks on the page</td></tr><tr><td rowspan="4">Comments</td><td>max_comment patches</td><td>Maximum number of comment boxes per image</td></tr><tr><td>max_comment_patch_size</td><td>Max width and height of comment boxes</td></tr><tr><td>default_top padding</td><td>Vertical padding around comment patches</td></tr><tr><td>default_left padding</td><td>Horizontal padding around comment patches</td></tr></tbody></table>

<caption>Table 3: Parameters used in the synthetic grid table generation</caption>
<figure><img src="image_11.png" /><figcaption>Figure 6: Visualization of the synthetic generation system output. At left, the Grid Table Structure, with fact table elements and a comment patch; at right, the result padded to fit a common batch grid size.</figcaption></figure>

Additional features enhance realism: comment patches (2±1 patches with 4±2 × 6±2 cells) containing primarily **TEXT** labels, random isolated text cells throughout the grid, and empty regions including horizontal strips and rectangular patches within data areas. The hierarchical configuration system allows fine-grained control over the generation parameters through specialized sub-configurations for each component (Table 3).

All generated tables are normalized to a consistent grid size through bottom-right padding, adding **EMPTY** cells to match the defined batch grid size, as illustrated in Figure 6.

Actual parameter values defined in Table 3 are adjusted experimentally according to the training metrics, and the evaluation result.

## Grid compression

To handle the computational challenges posed by large spreadsheets encountered in the INSEE corpus, we implemented a grid compression method inspired by **SHEETCOMPRESSOR** Dong et al. (2025). The method identifies consecutive "duplicate" (or similar) rows using SHA1 hash comparisons, and merges them under the assumption that such rows carry equivalent structural information. This optimization achieved up to 70% reduction of the number of rows in very large tables, significantly improving processing efficiency while preserving complete reconstruction metadata.
<figure><img src="image_12.png" /><figcaption>Figure 7: Grid compression algorithm inspired by SpreadsheetLLM. The method detects and removes consecutive duplicate rows (highlighted in red) using SHA1 hash comparison, achieving significant space reduction (64% in this example). The method preserves metadata allowing to completely reconstruct the grid.</figcaption></figure>

## 3.4 Training

As the post-processing of the detection stage was made up of complex non-differentiable steps such as graph thresholding, we split the training into two parts:

* The TableDetectionCNN training with supervised learning on table zone targets (Section 3.4);

* The TableStructureCNN training with supervised learning on cell class targets (Section 3.4).

We also experimented a Reinforcement Learning (RL)-based approach to train the whole pipeline. We describe it in Section 3.4.

### Training the detection model

The detection model was trained using a **composite loss function**, expressed as:

$$ \mathcal{L}_{\text{total}} = \mathcal{L}_{\text{focal}} + \lambda_w \cdot \mathcal{L}_{\text{spatial}} $$

with:

* $\mathcal{L}_{\text{focal}}$ the focal loss component

* $\mathcal{L}_{\text{spatial}}$ the spatial regularization loss

* $\lambda_w$ the spatial regularization weight

The focal loss is a cross-entropy loss, modified to suit the class imbalance of our scenario where most cells are *easy classifications* in the background class (Lin et al., 2018):

$$ \begin{aligned} \mathcal{L}_{\text{focal}} &= -\alpha(1-p)^{\gamma} \log(p) & \text{if } y=1 \\ &= -(1-\alpha)p^{\gamma} \log(1-p) & \text{if } y=0 \end{aligned} $$
In the above, $p \in [0, 1]$ is the predicted probability for the positive class, $\alpha$ balances positive/negative examples, and $\gamma$ reduces loss for confident predictions. After some experiments we adopted the commonly used values of $\gamma = 2.0$ and $\alpha = 0.25$.

The spatial regularization component combines Total Variation (TV) and a compactness term. It is meant to improve the fitting of the post-processed bounding boxes to the detected table region:

$$ \mathcal{L}_{\text{spatial}} = \mathcal{L}_{\text{TV}} + \mathcal{L}_{\text{comp}} $$

with:

$$ \mathcal{L}_{\text{TV}} = \frac{1}{H \times W} \times \left[ \sum_{i,j}^{H,W} (p_{i,j-1} - p_{i,j}) + \sum_{i,j}^{H,W} (p_{i-1,j} - p_{i,j}) \right] $$

and

$$ \mathcal{L}_{\text{comp}} = \frac{\left[ \sum_{i,j}^{H,W} (p_{i,j-1} - p_{i,j}) + \sum_{i,j}^{H,W} (p_{i-1,j} - p_{i,j}) \right]^2}{\sum_{i,j}^{H,W} \mathbf{1}_{p_{i,j} > 0.5}} $$

where:

* $H$ is the height of our grid (number of rows)

* $W$ is the width of our grid (number of columns)

* $p_{i,j}$ is the probability of the table class for the cell at (column $i$, row $j$).

## Structure training

The structure training is done mostly interdependently from the detection model. We skip the first detection step of the extraction process by taking ground-truth bounding and adding some random bounding box offset noise to mimic real conditions.

As described in 5 we use the bounding boxes to crop the grid, and pass them through the model to get raw classifications.

The loss is computed as standard cross-entropy between the target classification and our model output:

$$ \mathcal{L}_{CE} = - \sum_{i=1}^{C} y_i \log(\hat{y}_i) $$

## Reinforcement learning tuning

In order to get rid of some misalignment on our tasks, we also experimented with full pipeline training using Reinforcement Learning (RL), by optimizing our models for our actual target (bounding boxes or classifications). While supervised learning requires fully differentiable steps in the loss function in order to allow gradient flow, RL allows us to use non-differentiable goals (our evaluation metrics) as reward functions.

Using a simple REINFORCE policy gradient method (Wikipedia contributors, 2024), this approach showed some interesting results as a **fine-tuning** step, to re-align the model on annotated data after fully synthetic runs. Using calibrated rewards, it also allowed us to easily modify the "focus" of the model, from a metric to another with **reward weight tuning**.

The **detection reward** is computed using a mix of different components defined in Equation (11), weighted according to the training objectives:

$$ \text{Reward} = \lambda_P \times \text{Precision} + \lambda_R \times \text{Recall} + \lambda_O \times \text{Average IoU} $$

The **Structure reward** is computed using simple normalized Per-Pixel (Per-Cell) Accuracy as defined in Equation (4.3).

The REINFORCE method computes the gradient using the following equation:

$$ \nabla_{\theta} J(\theta) = \mathbb{E} [\nabla_{\theta} \log \pi_{\theta}(a|s) \cdot (\mathcal{R} - \mathcal{B})] \quad (11) $$

where
* $\nabla_{\theta}$ is the gradient of the expected reward;

* $\pi_{\theta}(a|s)$ is our **policy**, defining the probability of taking action $a$ when in state $s$;

* $\mathcal{R}$ is the reward;

* $\mathcal{B}$ is the baseline, computed as an average on previous rewards.

In our case, since we want to use the final post-processed **bounding boxes**, we define our policy as $\pi_{\theta}(\text{boxes}|\text{grid})$, and approximate it with $\pi_{\theta}(\text{table class}|\text{pixels in boxes})$. Theoretically, it's arguably a bit of a shot in the dark, since the direct signal between the actual grid and the detections is very diluted through the non-differentiable complex *Box Processing* steps, but it should at least lead to gradients with the right direction.

## 3.5 Limitations

Compared to modern approaches to object detection and classification like TATR (Smock et al., 2022), our method uses simple and quite low-tech solutions. This was mostly by design, as we tried to keep a performance on par with algorithmic alternatives, but it has severe drawbacks. The constraints imposed by the pure-CNN design forces us to use heavy postprocessing, that sometimes reveals inefficient. Indeed, the training is done for intermediate steps, and makes it hard to align the model on our end goal.

Better results could come from taking the same path as TATR and going will a full-ML pipeline, allowing us to optimize closer to our actual goal.

# 4 Performance evaluation

The approach described above has been implemented in python. It is available in the StatCheck project within the Inria Gitlab, on the branch **table-detection-fun-rework**. The trained models used for these experiments are stored on the CEDAR HuggingFace repository.

## 4.1 Datasets

For our evaluations, we used different manually annotated datasets:

* DS1 made of 121 sheets from INSEE with 164 tables.

* DS2 made of 114 sheets representing STATCHECK heuristics failure cases with manually annotated bounding-boxes. The Sheets' URLs were obtained using STATCHECK structured logging system, and extracting table detection failures from the logs.

We also employed procedurally generated synthetic datasets (denoted as SDS\_size) and mixed datasets as described in Section 3.3 (denoted as MDS\_size). To ensure our evaluation reflects true generalization capability rather than overfitting to specific patterns, we generate datasets randomly for each experimental run instead of using fixed Excel sheets. Sufficient dataset scale provides low-variance performance estimates while ensuring comprehensive coverage of the input distribution, better reflecting real-world deployment scenarios where models must handle previously unseen Excel sheet structures and formatting variations.

The DS2 dataset is particularly significant as it represents cases where existing heuristic-based approaches in StatCheck failed to correctly identify tables, making it a crucial benchmark for demonstrating the practical value of our ML-based approach as a complementary solution to current production systems.

## 4.2 Table detection accuracy

We compute several metrics to evaluate our model. We used the standard TP, TN, FN, FP names to denote, respectively, the true positive, true negative, false negative and false positive rates and IoU the **Intersection over Union**. We match predicted and ground-truth boxes using 1–1 Hungarian matching at IoU ≥ 0.5. Average IoU is computed over matched pairs only. Their formulas are recalled in Table 4, and we report their values in Tables 5 and 6.
<table><caption>Table 4: Detection accuracy metrics.</caption><thead><tr><th>Metric</th><th>Formula</th></tr></thead><tbody><tr><td>Precision</td><td><sup>TP</sup>&frasl;<sub>TP + FP</sub></td></tr><tr><td>Recall</td><td><sup>TP</sup>&frasl;<sub>TP + FN</sub></td></tr><tr><td>Average IoU</td><td><sup>1</sup>&frasl;<sub>M</sub> &sum;<sub>i=1</sub><sup>M</sup> IoU<sub>i</sub> (for matched detections: IoU &ge; 0.5)</td></tr></tbody></table>

<table><caption>Table 5: Quality metrics on DS1 for the different models</caption><thead><tr><th>Model</th><th>Precision</th><th>Recall</th><th>Average IoU</th></tr></thead><tbody><tr><td>TableDetectionCNN</td><td>0.48</td><td>0.55</td><td>0.59</td></tr><tr><td>TableDetectionCNN-large</td><td>0.56</td><td>0.71</td><td>0.74</td></tr><tr><td>TableDetectionCNN-large + RL + Compression</td><td>0.73</td><td>0.51</td><td>0.67</td></tr></tbody></table>

<table><caption>Table 6: Quality metrics on DS2 (STATCHECK failure cases) for the different models</caption><thead><tr><th>Model</th><th>Precision</th><th>Recall</th><th>Average IoU</th></tr></thead><tbody><tr><td>TableDetectionCNN</td><td>0.04</td><td>0.03</td><td>0.10</td></tr><tr><td>TableDetectionCNN-large</td><td>0.62</td><td>0.58</td><td>0.61</td></tr><tr><td>TableDetectionCNN-large + Compression</td><td>0.45</td><td>0.14</td><td>0.47</td></tr></tbody></table>

The results on DS2 are particularly noteworthy as they demonstrate the potential of our approach to handle cases where traditional heuristic methods fail. The TableDetectionCNN-large model achieves a Recall, showing that machine learning approaches can successfully identify tables in challenging scenarios that defeat rule-based systems. However, the performance drop when compression is enabled on this dataset suggests that the failure cases often involve complex table structures that benefit from higher resolution processing.

## Reinforcement Learning

Figure 8 shows the reward weights we used for RL. Qualitatively, the impact was mixed, as follows:

* On DS1 we noticed non-negligible improvement in detection accuracy, where a high F1 Score weight allowed us to increase the Precision score, from 0.40 up to 0.73 on compressed data (5)

* After some experiments on SDS65535 and MDS4096, we noticed that RL had little to no impact on the model performance with this type of data.

* On DS2, reinforcement learning was not applied due to the challenging nature of these failure cases, focusing instead on architectural improvements.

<table><caption>Figure 8: Reward weights used for RL tuning.</caption><thead><tr><th>Metric</th><th>Weight</th></tr></thead><tbody><tr><td>F1 Score</td><td>0.3</td></tr><tr><td>Precision</td><td>0.1</td></tr><tr><td>Recall</td><td>0.3</td></tr><tr><td>Average IoU</td><td>0.3</td></tr></tbody></table>
## 4.3 Table Structure precision

To measure table structure precision as a standalone operation, we use our reference table bounding boxes, with some random noise to mimic actual table detection results, and compute the per-pixel accuracy of the final classified cells:

$$ \text{Per-Pixel Accuracy} = \frac{\sum_{c,r} \mathbf{1}_{p_{c,r}=t_{c,r}}}{|P|} $$

where *T* the **target class** and *P* the **predicted class**.

The per-pixel accuracy varies significantly across the different classes, as Table 7 shows. The results are quite poor. With .5 accuracy on 3 classes, we are barely above what an *all data* classifier would get. This is easily explained by the important class imbalance towards "data".

<table><caption>Table 7: Structure classification per-pixel accuracy on DS1.</caption><thead><tr><th>Model</th><th>Overall</th><th>Col Header</th><th>Row Header</th><th>Data</th></tr></thead><tbody><tr><td>TableStructureCNN</td><td>0.62</td><td>0.43</td><td>0.46</td><td>0.47</td></tr><tr><td>TableStructureCNN-Large</td><td>0.56</td><td>0.06</td><td>0.34</td><td>0.75</td></tr><tr><td>TableStructureCNN-Large + Compression</td><td>0.64</td><td>0.21</td><td>0.50</td><td>0.76</td></tr></tbody></table>

## 4.4 Full pipeline

The full pipeline evaluation is similar to the structure evaluation, with the essential difference that the bounding boxes used for the classification and evaluation are different. Indeed, per-pixel accuracy is only computed in *target* table bounding boxes, which might differ from the predicted ones. This penalizes badly aligned or cropped detections with a low score.

<table><caption>Table 8: Per-pixel Accuracy evaluation for full pipeline on DS1</caption><thead><tr><th>Metric</th><th>Overall Per-pixel accuracy</th></tr></thead><tbody><tr><td>CNN</td><td>51.70%</td></tr><tr><td>CNN-Large</td><td>61.26%</td></tr><tr><td>CNN-Large + Compression & RL</td><td>57.96%</td></tr></tbody></table>

## 4.5 Experiment conclusion

From our evaluation, we can draw the following conclusions:

* Table detection tasks can reach high accuracy scores on difficult datasets even while using very simple and efficient ML models, and could be ready as a drop-in replacement to StatCheck's existing table *Boundary Finder*.

* Our approach demonstrates significant value on StatCheck failure cases (DS2), where the TableDetectionCNN-Large model achieves a 0.58 Recall, proving that ML methods can complement heuristic-based systems by handling their failure modes.

* Mostly synthetic training can get good results on unknown datasets, as evidenced by consistent performance across DS1 and the challenging DS2 scenarios.

* On full table data, our models for structure identification tasks do not seem to converge as well as they could, even underperforming heuristics-based methods, which highlights the limitation of our architecture.
* When combined with table compression, reducing the class imbalance, structure identification models can achieve better results on standard datasets, though this improvement does not extend to the most challenging failure cases.

* The table detection component shows production readiness for deployment as a complementary solution to existing heuristic systems, particularly for handling edge cases where traditional methods fail.

* The combined results for structure recognition are not good enough to integrate into production systems, indicating the need for architectural improvements in this component.

# 5 Conclusion

We introduced a grid-based two-stage approach for identifying and parsing fact tables in heterogeneous statistical spreadsheets. By operating directly on cell-type grids and coupling a lightweight U-Net style backbone with practical components: synthetic data generation, grid compression, and optional RL fine-tuning; our method targets the realities of institutions-produced statistics spreadsheets. In extensive experiments on INSEE sheets and mixed synthetic datasets, the detection stage proved robust and efficient, transferring well from synthetic pretraining and benefiting from RL fine-tuning on real data. Compression delivered substantial speedups on large sheets with negligible loss in quality, when combined with specific training. In contrast, structure classification lagged behind, due to severe class imbalance, noise introduced by detection and architecture limitations. The full pipeline approach still presents significant error rates, and isn't ready for production without error checking and fallback heuristics.

From an application standpoint, the detection stage is ready to be integrated into the STATCHECK pipeline. Deployed as a fallback for detection on tables were the regular heuristics fail, it can reduce a substantial portion of current extraction failures attributable to missed tables, and improve coverage on non-standard layouts. We recommend logging low-confidence cases, using the STATCHECK structured logging system, to drive active learning, and continuously improve performance on the long tail of table formats.

The main limitations are the reliance on non-differentiable post-processing for detection and the underperformance of header labeling. To address these, we propose three concrete directions:

* First, replace the complex post-processing with an end-to-end detection head, similar to approaches like DETR and train with set-based losses to align optimization with final objectives.

* Second, strengthen structure learning with class-balanced and structured losses.

* Third, expand and calibrate synthetic data to better match real distributions (header depths, data types, missing-value flags, empty blocks, comments), and report distribution alignment to demonstrate realism.

Further gains might come from exploring mixed approaches, combining CNN for spatial feature extraction and language models with special encoders for complete data extraction, opening the way to a fully trainable extraction pipeline, delivering fine-grained details, and even improving the identification of *Area* structures, as defined in Balalau et al. (2025).
Bibliography

Balalau, O., S. Ebel, T. Galizzi, I. Manolescu, A. Gauquier, and H. Galhardas (2025). Towards more accessible statistic data. Technical report, Inria. Unpublished report.

Balalau, O., S. Ebel, T. Galizzi, I. Manolescu, Q. Massonnat, A. Deiana, E. Gautreau, A. Krempf, T. Pontillon, G. Roux, et al. (2022, October). Fact-checking multidimensional statistic claims in french. In *TTO 2022 - Truth and Trust Online*, Boston [Hybrid Event], United States.

Dong, H., J. Zhao, Y. Tian, J. Xiong, S. Xia, M. Zhou, Y. Lin, J. Cambronero, Y. He, S. Han, and D. Zhang (2025). SpreadsheetLLM: Encoding spreadsheets for large language models.

Lin, T.-Y., P. Goyal, R. Girshick, K. He, and P. Dollár (2018). Focal loss for dense object detection.

Lin, W., Z. Sun, C. Ma, M. Li, J. Wang, L. Sun, and Q. Huo (2022). Tsrformer: Table structure recognition with transformers.

Ronneberger, O., P. Fischer, and T. Brox (2015). U-net: Convolutional networks for biomedical image segmentation. Cham. Springer International Publishing.

Smock, B., R. Pesala, and R. Abraham (2022, June). Pubtables-1m: Towards comprehensive table extraction from unstructured documents.

Soric, M. (2025). Understanding and extracting table information from brgm documents. https://hal.science/hal-04974179. Computer Science [cs].

Wikipedia contributors (2024). Reinforce. Wikipédia, l'encyclopédie libre. [Online; accessed 22-July-2025].