TABLET: A Large-Scale Dataset for Robust Visual Table Understanding

## Abstract

Abstract While table understanding increasingly relies on pixel-only settings where tables are processed as visual representations, current benchmarks predominantly use synthetic renderings that lack the complexity and visual diversity of real-world tables. Additionally, existing visual table understanding (VTU) datasets offer fixed examples with single visualizations and pre-defined instructions, providing no access to underlying serialized data for reformulation. We introduce Tablet , a large-scale VTU dataset with 4 million examples across 20 tasks, grounded in 2 million unique tables where 88% preserve original visualizations. Each example includes paired image-HTML representations, comprehensive metadata, and provenance information linking back to the source datasets. Fine-tuning vision-language models like Qwen2.5-VL-7B on Tablet improves performance on seen and unseen VTU tasks while increasing robustness on real-world table visualizations. By preserving original visualizations and maintaining example traceability in a unified large-scale collection, Tablet establishes a foundation for robust training and extensible evaluation of future VTU models. 1 1 1 Dataset, code, and other resources are available at https://github.com/alonsoapp/TABLET

## 1 Introduction

The field of table understanding focuses on techniques for representing and interpreting tabular data to support a wide range of practical tasks such as question answering, summarization, and information extraction. Research in this area has traditionally representated tables as structured text, encoding their content and layout through linearized or graph-based representations (see Figure [1](https://arxiv.org/html/2509.21205v2#S1.F1)b; herzig-etal-2020-tapas; zhang2020graph; liu2022tapex). While this unimodal view remains effective in certain domains, many tables found in documents and webpages contain irregular structures, rely on visual formatting (e.g., merged cells, background colors, font variations), or embed multimodal elements such as images (see Figure [1](https://arxiv.org/html/2509.21205v2#S1.F1)a). Advances in Vision-Language Models (VLMs; pmlr-v139-radford21a; liu2023llava) have provided impetus for treating tables as images, eschewing the step of rendering them as text sequences (like Markdown or HTML).
The conceptual simplicity of this approach, coupled with improved performance on several tabular tasks (alonso-etal-2024-pixt3; zhou-etal-2025-texts) has driven significant research interest (zheng-etal-2024-multimodal; su2024tablegpt2largemultimodalmodel; jiang2025multimodaltabularreasoningprivileged) in Visual Table Understanding (also known as Multimodal Table Understanding). Visual representations of tables are not only merely convenient but in many cases necessary, particularly for VLM agents that interact with the world exclusively through pixels (e.g., on a screen) and must interpret tables directly in their visual form (deng2023mind2web; zheng2024gptvision; lu2024omniparserpurevisionbased).

Despite the growing relevance of VTU, there are few resources that support training models *directly* on image-based representations of tables. Existing benchmarks like MMTab (zheng-etal-2024-multimodal) consist of web tables (e.g., from Wikipedia which is a common source for many tabular datasets), that are serialized and subsequently rendered as synthetic images (see Figures [1](https://arxiv.org/html/2509.21205v2#S1.F1)b,c). These images do not preserve the original visual characteristics of the tables and do not reflect the diversity and complexity of real-world layouts. As a result, models trained on such data face a train-test mismatch, since the visual patterns learned from serialized renderings do not generalize well to naturally occurring tables failing to capture critical visual cues like subtle ruling lines, intricate merged cell layouts, background colors, font variations, or embedded images that are inherent to real-world table comprehension (compare Figure [1](https://arxiv.org/html/2509.21205v2#S1.F1)a and [1](https://arxiv.org/html/2509.21205v2#S1.F1)c). An exception is WikiDT shi2024wikidT, which provides access to original table visualizations from web pages but focuses exclusively on table question answering.
Likewise, datasets designed for processing PDF documents or screenshots (zhong2020image; lu2023dynamic) preserve original table visualizations but support only a single task.

In this work, we introduce Tablet(^2^22Our dataset is called Tablet partly as a nod to the Scottish sugary confection which is often cut into uniform squares or rectangles, resembling a table.), a large-scale dataset designed to enable vision-language models to learn generalizable skills for table understanding by leveraging visual features from table images. Unlike existing datasets that focus on a single task or use synthetic renderings, Tablet preserves *lossless representations* of real-world table images and spans a *diverse set of tasks*, including table-to-text generation (parikh-etal-2020-totto), table fact verification (chen2020hybridqa), and table-based question answering (pasupat-liang-2015-compositional). Tablet facilitates training and evaluation on table formats that align with those encountered in downstream applications, particularly in pixel-based agents that process visual input directly. It contains 4 million examples
across 20 tasks, derived from 2 million unique tables, 88% of which retain their original visualization. To promote flexibility and foster future research, we provide both image and HTML representations of tables, together with metadata and links to the source datasets. Pairing table images with HTML enables models to learn from naturally occurring and synthetic table images when the former are not available. Tablet’s large-scale and task aggregation provide a unique, foundational resource that complements research integrating textual and visual modalities for training and reasoning (liu2025hippoenhancingtableunderstanding; jiang2025multimodaltabularreasoningprivileged).
We experimentally demonstrate that fine-tuning on Tablet improves performance on in- and out-of-domain benchmarks. Our work makes the following contributions:

- •
Tablet addresses a critical gap in large-scale, visually faithful datasets for VTU, with 4 million examples and 2 million tables, most of which are preserved in their original form.
- •
We show that supervised fine-tuning on Tablet  enhances VLM performance across seen and unseen table understanding tasks. We release a VLM model trained on Tablet that achieves state-of-the-art results on 10 out of 14 table understanding benchmarks.
- •
Tablet  is designed for extensibility: all tasks are provided in a unified format, paired with HTML table serializations that can be rendered back into the original visualization, along with example metadata and links to the source datasets to facilitate reuse and reformulation.

Figure: Figure 1: Previous datasets render table images from serialized tables, losing original visual details. In contrast, Tablet locates and retrieves the original table visualizations across 14 tabular datasets, resulting in 4M examples grounded in 2M unique tables.
Refer to caption: x1.png

## 2 Related Work

Early work approached the problem of table understanding from a unimodal perspective, treating tables as structured text and developing general-purpose models that generalized across multiple tasks (li2023tablegpttabletunedgptdiverse; zhang-etal-2024-tablellama). Recent progress in natural image understanding afforded by increasingly better VLMs (pmlr-v139-radford21a; liu2023llava) has led to development of models that can successfully process visually rendered text beyond natural imagery and perform
table understanding holistically, within a multimodal framework
(kim2022donut; lee2023pix2structscreenshotparsingpretraining; zhang2023llavar; ye-etal-2023-ureader; hu-etal-2024-mplug; alonso-etal-2024-pixt3; jiang2025multimodaltabularreasoningprivileged; su2024tablegpt2largemultimodalmodel; zhao2024tabpedia; zheng-etal-2024-multimodal; su2024tablegpt2largemultimodalmodel).

To this effect, several benchmarks have been proposed recently to evaluate progress in VTU. For instance, TableVQA-Bench (kim2024tablevqabenchvisualquestionanswering) focuses on visual table question answering but relies on synthetic table images. In contrast, MMTBench (titiya2025mmtbenchunifiedbenchmarkcomplex) includes original table visualizations, along with visually rich images and interleaved charts. While both are valuable for evaluation, they are limited in scope and do not provide large-scale training data, restricting their utility for developing generalizable models.

Larger training datasets (with training splits) have historically focused on image-based Table Structure Recognition (TSR), including PubTabNet (zhong2020image), TableBank (li-etal-2020-tablebank), and TabComp (gautam-etal-2025-tabcomp). Beyond TSR, alonso-etal-2024-pixt3 created image-rendered versions of existing datasets like ToTTo (parikh-etal-2020-totto), relying on lossy renderings that discard visual features. WikiDT (shi2024wikidT) preserves original table visualizations but only targets Visual TableQA. MMTab (zheng-etal-2024-multimodal) is a large VTU dataset, with 150k TSR pre-training examples and 232k instruction examples across 19 tasks. While MMTab is a useful resource that we build upon, it relies on synthetic renderings
of serialized tables, lacks traceability to original data sources, and remains limited in size compared
to the scale needed for general-purpose VTU.

In this work, we do not introduce a task-specific dataset or model.
Instead, we create a large-scale resource aimed at enhancing table understanding in general-purpose vision-language models like Gemma-3 (gemmateam2025gemma3technicalreport) or Qwen2-VL (qwen2vl), under the assumption that tables for many tasks are seen as images, and therefore can be naturally processed by these models. This distinguishes Tablet from many recent efforts that introduce task-specific datasets or specialized architectures as it is designed to be a holist resource for advancing VLM capabilities for tables.

## 3 The Tablet Dataset

### 3.1 Overview

Tablet is a large-scale dataset that aggregates a total of 4,066,545 examples, combining 20 TU tasks sourced from 14 datasets (see Section [3.3](https://arxiv.org/html/2509.21205v2#S3.SS3) for a breakdown of examples per task). Tablet includes 2,031,256 unique table images with visualizations from Wikipedia (61.5%), PubTabNet (25.1%; zhong2020image), TabMWP (1.9%; lu2023dynamic, and synthetically rendered images (11.5%) from serialized tables (see Section [3.2](https://arxiv.org/html/2509.21205v2#S3.SS2) for details).

Examples in Tablet are framed as instructions, leveraging the benefits of instruction-tuning for large language models.Tablet does not introduce new data instances; rather it repurposes examples from existing datasets, referred to as *seed* datasets, rephrasing their tasks into an instruction format. Examples are drawn from 14 seed datasets: TURL (deng2020turl), ToTTo (parikh-etal-2020-totto), TabFact (2019TabFactA), WikiTableQuestions (pasupat-liang-2015-compositional), HybridQA (chen2020hybridqa), HiTab (cheng-etal-2022-hitab), PubTabNet (zhong2020image), TabMWP (lu2023dynamic), TAT-QA (zhu-etal-2021-tat), InfoTabs (gupta-etal-2020-infotabs), WikiBIO (lebret-etal-2016-neural), FeTaQA (nan-etal-2022-fetaqa), MMTab zheng-etal-2024-multimodal, and DocStruct4M (hu-etal-2024-mplug).

Datasets that include only images and fixed prompts restrict researchers to the provided visualizations and instruction formulations, limiting the exploration of new prompting techniques or the use of table data to create additional examples.
To avoid this, each example in Tablet is provided in a unified format, including the instruction used in this work, its corresponding atomic data, the HTML version of the table, the original source ID, and additional metadata (detailed in Appendix[7.4](https://arxiv.org/html/2509.21205v2#S7.SS4)).

**Table 1: Tablet splits, tasks, and seed datasets; M^1: InfoTabs, NSF, StatCan, TabMWP, TAT-QA, TabFact, ToTTo, WikiBIO, WikiTableQuestions; M^2: InfoTabs, NSF, StatCan, TAT-QA, TabFact, ToTTo, WikiBIO; M^3: PubTabNet, TabFact, WikiTableQuestions. Proportion of examples in each set (train/dev/test) is shown in gray within parentheses.**
| Task | Seeds | Train | Dev | Test |
| --- | --- | --- | --- | --- |
| ent_link | TURL | 1,236,128 (35.3%) | 74,282 (35.4%) | 213,494 (60.8%) |
| col_type | TURL | 602,406 (17.2%) | 13,188 (6.3%) | 12,802 (3.6%) |
| struct_aware_parse | M^3 | 513,482 (14.6%) | 9,115 (4.3%) | 1,102 (0.3%) |
| wikibio | WikiBIO | 582,659 (16.6%) | 72,831 (34.7%) | 72,831 (20.7%) |
| hybridqa | HybridQA | 62,670 (1.8%) | 3,466 (1.7%) | 3,463 (1.0%) |
| fetaqa | ToTTo | 3,006 (0.1%) | 577 (0.3%) | 1,079 (0.3%) |
| hitab | NSF, StatCan, ToTTo | 7,417 (0.2%) | 1,670 (0.8%) | 1,584 (0.5%) |
| infotabs | InfoTabs | 16,538 (0.5%) | 1,800 (0.9%) | 5,400 (1.5%) |
| tabfact | TabFact | 87,717 (2.5%) | 12,389 (5.9%) | 12,326 (3.5%) |
| tabmwp | TabMWP | 23,059 (0.7%) | 7,686 (3.7%) | 7,686 (2.2%) |
| tat-qa | TAT-QA | 2,201 (0.1%) | 278 (0.1%) | 277 (0.1%) |
| totto | ToTTo | 110,934 (3.2%) | 7,077 (3.4%) | 7,084 (2.0%) |
| wikitq | WikiTableQuestions | 14,152 (0.4%) | 3,537 (1.7%) | 4,344 (1.2%) |
| rel_extraction | TURL | 60,615 (1.7%) | 2,145 (1.0%) | 2,030 (0.6%) |
| table_instruction | M^1 | 136,944 (3.9%) | 0 (0.0%) | 0 (0.0%) |
| row_column_extraction | M^1 | 7,721 (0.2%) | 0 (0.0%) | 957 (0.3%) |
| table_cell_extraction | M^1 | 7,727 (0.2%) | 0 (0.0%) | 966 (0.3%) |
| table_cell_location | M^1 | 7,708 (0.2%) | 0 (0.0%) | 956 (0.3%) |
| table_recognition | M^1 | 6,927 (0.2%) | 0 (0.0%) | 912 (0.3%) |
| table_size_detection | M^1 | 7,800 (0.2%) | 0 (0.0%) | 950 (0.3%) |
| merged_cell_detection | M^2 | 7,500 (0.2%) | 0 (0.0%) | 950 (0.3%) |
| Total |  | 3,505,311 | 210,041 | 351,193 |

### 3.2 Image sources

Wikipedia Tables
Wikipedia tables generally follow a hierarchical table structure and are often rich in visual information. 61.5% of the tables in Tablet are lossless visualizations sourced from Wikipedia (See Appendix [7.2](https://arxiv.org/html/2509.21205v2#S7.SS2) for a breakdown of image sources per task). These tables are referenced in 67.4% of all examples in the dataset. Each table from the seed datasets was traced back to its original visualization in the corresponding Wikipedia article and captured in a screenshot.(^3^33Table images were rendered using Firefox in headless mode (version 142.0.1 with GeckoDriver version 0.36.0) at an effective rendering density of 96 PPI (which stands for Pixels Per Inch).).
A key challenge in retrieving the original visualizations was that the seed datasets were created at different times, while Wikipedia articles are continuously updated. To address this, we relied on the metadata released with the seed datasets and also reached out to their authors to find out when the tables were harvested (see Appendix [7.3](https://arxiv.org/html/2509.21205v2#S7.SS3)). We used Wikipedia’s archiving API to retrieve each article as it appeared at the time of crawling. All Wikipedia tables in Tablet are linked to their source via both the page identifier and the corresponding revision identifier (oldid).
As Wikipedia pages often contain multiple tables, we needed to identify which one corresponded to the seed example. We did this by computing the levenshtein1966binary edit distance between Wikipedia tables and seed tables, both represented in markdown format, and selecting those with the highest similarity value. We set a minimum similarity threshold of 0.70; in cases without a match, the serialized seed table was rendered as an image.

Synthetic Tables
Wikipedia tables that could not be retrieved, either due to low similarity scores or other issues, were synthetically generated by converting their serialized format to HTML and rendering them using the same browser configuration used for the original Wikipedia tables. These synthetic tables account for 11.5% of the dataset, contributing 747,002 additional examples (18.4% of the total). In Section [5](https://arxiv.org/html/2509.21205v2#S5), we show that combining original table images with synthetic visualizations during training allows for an increase in example count and visualization diversity that leads to better model performance compared to using synthetic or original tables alone.

Document Tables
We also incorporate tables from other domains to improve model generalization. These include tables from scientific articles in PubTabNet (zhong2020image) and rendered tables from TabMWP (lu2023dynamic). PubTabNet contributes 509,892 tables to our dataset (25.1% of the total), accounting for 97.5% of the structure-aware learning task incorporated from DocStruct4M (hu-etal-2024-mplug). TabMWP adds 38,431 tables (1.9% of the total); while smaller in scale, TabMWP offers valuable coverage of numerical reasoning examples.

### 3.3 Table Understanding Tasks

Tablet aggregates a total of 20 TU tasks grouped into 7 broad categories: Table Interpretation, Question Answering, Table-to-Text Generation, Table Numerical Reasoning, Table Natural Language Inference (NLI), and Table Structure Understanding. Subtasks within each category require different skills and demonstrate various degrees of complexity, ranging from basic table structure understanding to downstream tasks that combine tabular reasoning with other skills, such as information retrieval, numerical reasoning, or natural language inference. Table [1](https://arxiv.org/html/2509.21205v2#S3.T1) provides a breakdown of examples per task and their source dataset. We briefly describe each task below and provide more details in Appendix [7.5](https://arxiv.org/html/2509.21205v2#S7.SS5).

Table Interpretation comprises three tasks, namely Column Type Annotation, Entity Linking, and Relation Extraction (ent_link, col_type, and rel_extraction in Table [1](https://arxiv.org/html/2509.21205v2#S3.T1)). Sourced from TURL (deng2020turl), these tasks do not represent downstream applications but target basic table understanding skills essential for tackling more complex tasks. All instructions for these tasks were aggregated from TURL. They constitute the largest group in our training set, with 1,899,149 examples (54.2%).

Table Question Answering is represented by Free-form Table QA (FeTaQA, nan-etal-2022-fetaqa),
Hierarchical Table QA (HiTabQA; cheng-etal-2022-hitab), Hybrid QA (HybridQA; chen2020hybridqa),
and Table QA (WikiTableQuestions; pasupat-liang-2015-compositional).
These tasks range from simple question answering over table content to multi-hop reasoning that combines textual and tabular information. They also vary in terms of table complexity and the expected answer format. Question answering represents 2.5% of Tablet’s training set (87,245 examples).

Table-to-Text Generation is exemplified by two subtasks, namely Highlighted Cell Text Generation (ToTTo; parikh-etal-2020-totto) and Wikipedia Biography Generation (WikiBio; lebret-etal-2016-neural). Both tasks involve generating text based on table content, concise biographies in the case of WikiBio, and short descriptions based on visually highlighted cells, in the case of ToTTo. Together, they account for 693,593 training examples (19.8%).

Table Numerical Reasoning combines Tabular Math Word Problem Solving (TabMWP; lu2023dynamic) and Hybrid-context Financial QA (TAT-QA; zhu-etal-2021-tat). Given a table and a mathematical question, the model must perform reasoning over table values. This type of numerical reasoning represents 0.8% of Tablet’s training set (25,260 examples).

Table NLI includes two entailment tasks: Table Fact Verification (TabFact; 2019TabFactA) and Infobox Natural Language Inference (InfoTabs; gupta-etal-2020-infotabs) where statements as supported or refuted based on table content. They represent 104,255 examples (3%) in Tablet’s training set.

Table Structure Understanding is an umbrella category for tasks designed to facilitate structural understanding of tables: Merged Cell Detection, Row and Column Extraction, Table Cell Extraction, Table Cell Location, Table Recognition, Table Size Detection, and Structure-aware Parsing in Table [1](https://arxiv.org/html/2509.21205v2#S3.T1)). Examples for the first seven tasks are aggregated from MMTab, while Structure-aware Parsing is derived from DocStruct4M (hu-etal-2024-mplug). These tasks are based on tables from multiple seed datasets (see rows with seeds M^2, M^1, and M^3), including InfoTabs (gupta-etal-2020-infotabs), NSF (nsf_science_engineering_indicators_2019), StatCan (statistics_canada_2024), TabMWP (lu2023dynamic), TAT-QA (zhu-etal-2021-tat), TabFact (2019TabFactA), ToTTo (parikh-etal-2020-totto), WikiBio (lebret-etal-2016-neural), WikiTableQuestions (pasupat-liang-2015-compositional), and PubTabNet zhong2020image. While we maintain the original MMTab instructions, we use our own table visualizations. This group contributes 558,865 examples (15.8%) to our training.

Instruction Following To facilitate evaluation, we fine-tune and evaluate all models using instructions that explicitly require the final answer to be encapsulated in a JSON object, regardless of any additional tokens the model may generate. To increase instruction diversity and mitigate catastrophic forgetting, we include a dedicated instruction-following set from MMTab. These instructions rephrase a subset of examples from the above tasks using a different template and do not require the JSON output format. While not a task in itself, this set adds instruction diversity and reduces overfitting to a single output style. We aggregate these examples directly from MMTab while using our visualizations. This set contributes 136,944 training examples (3.9%).

### 3.4 Linking to Sources and Highlighting

Tablet is designed with extensibility in mind. Existing datasets such as TableInstruct (zhang-etal-2024-tablellama), DocStruct4M (hu-etal-2024-mplug), and MMTab (zheng-etal-2024-multimodal) do not provide a clear reference to the original examples from which their instructions were derived. However, the absence of such pointers, makes reuse, modification, and augmentation difficult. Additionally, image-based datasets such as MMTab lack serialized table versions in text format, providing only rendered images as representations. To address these issues, each example in Tablet includes both the identifier of the original example and the identifier of the corresponding table from the source dataset. Identifiers follow a simple, human-readable format, and the released code provides a function to retrieve the original examples given their IDs.

To support future extensions and new tasks, Tablet also provides serialized versions of its tables in HTML. For the tables successfully retrieved from Wikipedia, this HTML corresponds to the raw table object in the original HTML of the article. These tables can be rendered with their full visualization using Wikipedia’s official CSS stylesheets or through the rendering functions provided with our code. The remaining tables are either provided as HTML in the seed datasets or converted into HTML from their serialized representations.

This feature is particularly useful in tasks involving cell pointing or highlighting. Prior work
has shown that models can achieve competitive results by relying uniquely on highlighted values (an2022cont; alonso-etal-2024-pixt3). We also observed in our experiments that as long as highlighted cell values are mentioned in the prompt, models are able to perform the task (e.g., Column Type Annotation, Entity Linking, Relation Extraction, FeTaQA, and ToTTo) regardless of the table provided. In other words, the model learns to ignore the table during training, which can be detrimental for tasks requiring genuine table understanding.
We mitigate this, by exploiting the HTML serialization to directly locate the highlighted cells in the source tables. By cross-referencing the highlighted spans from the seed datasets with the actual table content, we generate versions of the tables with explicit highlights while preserving the original visual features.

### 3.5 Comparison with MMTab

MMTab (zheng-etal-2024-multimodal) is also a dataset for visual table understanding; however, unlike Tablet, it does not include the full set of training examples from the original datasets (e.g., only 12.4% of ToTTo compared to our 91.8%; see Appendix [7.6](https://arxiv.org/html/2509.21205v2#S7.SS6)), and it provides no development splits. MMTab covers 20 tasks while Tablet covers 21 (inculding instruction tuning); four tasks are unique to the former and five to the latter, and overall our dataset is 9.36 times larger (comprising 4,066,545 examples compared to MMTab’s approximate 433,376 examples). Moreover, only a few tasks in MMTab include identifiers that can be linked back to their source examples, limiting developers to the provided instruction text. While MMTab relies on synthetically rendered tables with predefined styles, which omit original formatting and embedded images, Tablet retains the original visualizations, enabling models to exploit a fuller range of visual features.

## 4 Experimental Setting

Our experiments are motivated by two questions:
(1) Does training VTU models with lossless table visualizations improve performance on VTU tasks? and
(2) Is Tablet diverse enough so that supervised fine-tuning (SFT) improves generalization on unseen VTU tasks? We also examine whether training only with original visualizations performs better than mixing original and synthetic ones, whether a balanced task distribution is preferable to the full dataset distribution, and whether including various table interpretation tasks contributes to VTU performance.

Backbone Model
All experiments employ the same backbone model, namely Qwen2.5-VL with 7B parameters (qwen2025qwen25technicalreport). This model provides a good tradeoff between performance and computational requirements.
Together with InternVL3 (zhu2025internvl3exploringadvancedtraining), it was the best-performing open-weight VLM in its class, setting a new benchmark for Single Page Document VQA (mathew2021docvqa) at the time of writing.
Exploring the best backbone model for VTU or advanced SFT strategies is outside the scope of this work. Full SFT configurations are detailed in Appendix [8](https://arxiv.org/html/2509.21205v2#S8); results for other open-weight models on Tablet (test set) are reported in Appendix [8.4](https://arxiv.org/html/2509.21205v2#S8.SS4).

Evaluation
We evaluate fine-tuned models on eight *held-in* tasks: three focus on Table QA (WikiTableQuestions, HiTabQA, FeTaQA), two on text generation (WikiBio, ToTTo), two on Table Numerical Reasoning tasks (TabMWP, TAT-QA), and one on Table NLI (TabFact). To test generalization, we further evaluate on six *held-out *tasks: HybridQA, InfoTabs, TabMCQ (jauhar2016tabmcqdatasetgeneralknowledge), AIT-QA (katsis-etal-2022-ait), Table Recognition, and PubHealthTab (akhtar-etal-2022-pubhealthtab). The last four are directly extracted from MMTab and use instructions generated with their own templates, adding diversity in both table and instruction styles. Although included in Tablet, the 86,135 training examples from HybridQA, InfoTabs, and Table Recognition were excluded from fine-tuning to enable held-out evaluation. Models trained on the full dataset would potentially achieve even stronger results than those reported here.

## 5 Results and Analysis

Lossless vs Synthetic Table Visualizations We first assess whether models benefit from being trained with original table visualizations, in particular when exposed to real-world tabular images rather than synthetic ones at test time. This experiment was conducted on a subset of Tablet corresponding to MMTab
as this benchmark represents the current state of the art in multimodal table understanding. We refer to this subset as Tablet-Base (Tablet-B). We replaced all Wikipedia-based table images in MMTab with their original visualizations and kept tables that did not undergo lossy serialization unchanged (TabMWP, PubHealthTab, TabMCQ, AIT-QA); we also left MMTab’s instructions unaltered. We refer to this partition of the dataset as Tablet-B${}_{\text{org}}$ and contrast it with Tablet-B${}_{\text{synth}}$ which is the same version but with synthetic table images. Tablet-B contains 238,980 examples (in both original and synthetic versions). We also create Tablet-B${}_{\text{mix}}$ which combines original with synthetic tables. Specifically, we extend Tablet-B${}_{\text{org}}$ with all examples whose original visualizations could not be retrieved, using our synthetic renders. This set contains 371,292 examples (43.5% original visualizations and 35.6% synthetic). We perform full fine-tuning (without LoRA) on Qwen2.5-VL-7B for 3 epochs and evaluate in the 8 held-in tasks
and 4 held-out tasks
mentioned above. These tasks contain either original visualizations or synthetic ones to simulate various test cases (e.g., a model has seen only synthetic/original table images but is tested on original/synthetic ones).

**Table 2: Comparison of Qwen2.5-VL-7B-Instruct models fine-tuned on different Tablet-B variants and in zero-shot mode; models are evaluated on original (lossless) table visualizations and synthetic ones across held-in benchmarks. Results for TabMWP and TAT-QA exclude the mixed setting, as all tables already use original visualizations. We report exact match accuracy for WikiTQ, TabMWP, TabFact, and TAT-QA; BLEU for WikiBio, ToTTo, and FeTaQA; and F1 for HiTab. ToTTo results are reported on its development set. Best model per setting (synth/org) shown in bold.**
|  | WikiBio | ToTTo | WikiTQ | TabMWP | HiTAB | TabFact | FeTaQA | TAT-QA |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | synth | org | synth | org | synth | org | org | synth | org | synth | org | synth | org | org |
| 0-Shot | 2.0 | 2.5 | 10.8 | 8.9 | 57.7 | 50.8 | 51.9 | 11.1 | 2.5 | 70.4 | 59.4 | 9.9 | 9.8 | 48.7 |
| Tablet-B${}_{\text{synth}}$ | 11.3 | 4.8 | 12.4 | 14.4 | 58.4 | 56.5 | 84.3 | 33.5 | 16.0 | 63.1 | 65.5 | 28.6 | 27.9 | 33.4 |
| Tablet-B${}_{\text{org}}$ | 9.6 | 11.5 | 13.7 | 15.9 | 58.5 | 56.2 | 84.2 | 35.1 | 24.7 | 62.1 | 63.7 | 28.4 | 28.4 | 33.0 |
| Tablet-B${}_{\text{mix}}$ | 10.3 | 12.0 | 12.5 | 14.1 | 58.9 | 56.5 | 84.4 | 38.5 | 26.9 | 68.2 | 70.0 | 28.6 | 28.5 | 52.5 |

Our results in Table [2](https://arxiv.org/html/2509.21205v2#S5.T2) (held-in tasks) show that lossless models outperform synthetic ones in two tasks (WikiBio,
HiTab), match performance in four (WikiTQ, TabMWP, FeTaQA, TAT-QA), and underperform in one
(TabFact). For reference, we also report the 0-shot performance of Qwen2.5-VL-7B. The fact that there is no pronounced difference
between lossless and synthetic visualizations is perhaps not surprising since most benchmarks were originally designed around serialized, text-only representations, without paying heed to visual cues (TURL, WikiBIO, ToTTo, TAT-QA). Thus, while lossless visualizations contain more information, current benchmarks do not test whether models benefit from them. Developing benchmarks with high visual variability and challenging layouts would better highlight scenarios where lossless visualizations offer a distinct advantage.

When comparing models trained with lossless table visualizations (Tablet-B${}_{\text{org}}$) against a mixture of lossless and synthetic (Tablet-B${}_{\text{mix}}$), we find that mixed training improves results on four tasks (WikiBio, HiTab, TabFact, TAT-QA) and obtains equivalent performance on three (WikiTQ, TabMWP, FeTaQA). However, note that this improvement could be due to Tablet-B${}_{\text{mix}}$ having more samples compared to the other partitions. In general, we observe that model performance degrades when evaluating on original visualizations. This setting appears to be consistently more challenging, but robustness improves when adding original tables into the training (see Tablet-B${}_{\text{mix}}$ row in table [2](https://arxiv.org/html/2509.21205v2#S5.T2)). We measure degradation as the average percentage-point change in performance between original and mixed visualizations. The baseline model degrades the most, with a drop of 28.9 percentage points, particularly in tasks like HiTab and TabFact. The model trained on Tablet-B${}_{\text{synth}}$ degrades by 22.35 points, compared to 6.63 and 7.87 points for models trained on Tablet-B${}_{\text{org}}$ and Tablet-B${}_{\text{mix}}$, respectively. The model trained purely with lossless tables is most robust, maintaining stable performance when exposed to both original and synthetic styles (a detailed breakdown of degradation per task and model is in Appendix [8.2](https://arxiv.org/html/2509.21205v2#S8.SS2)).
On held-out tasks models trained on Tablet-B${}_{\text{mix}}$ have a slight advantage (see Appendix [8.3](https://arxiv.org/html/2509.21205v2#S8.SS3)).

Performance on Unseen VTU Tasks with Tablet
We next evaluate whether Tablet’s task diversity improves performance on unseen datasets. While fine-tuned models are expected to perform better on tasks seen during training, improved performance on held-out datasets would suggest that training on Tablet enhances the model’s VTU capabilities. Tables [3](https://arxiv.org/html/2509.21205v2#S5.T3) and [4](https://arxiv.org/html/2509.21205v2#S5.T4.fig1) report our results on held-in and held-out tasks, respectively. We present comparisons between zero- and one-shot Qwen2.5-VL-7B(^4^44We should note that Qwen2.5-VL-7B is a competitive baseline that has undergone extensive pre-training and instruction-tuning, achieving top scores in many TU and Visual Document Understanding benchmarks.) and its fine-tuned instantiations trained on MMTab and different sizes of Tablet. For MMTab, all Wikipedia-sourced images are synthetic (61.8% of all tables in their dataset), whereas all Tablet versions use a mix of original (88%) and synthetic (12%) (See Appendix [7.2](https://arxiv.org/html/2509.21205v2#S7.SS2) for the distribution of image sources per task). For now, we focus solely on Tablet-L (a shorthand for Tablet-Large), which is our biggest dataset comprising 4M examples (we discuss smaller sizes in the next sections).

**Table 3: Evaluation on eight held-in datasets with Qwen2.5-VL-7B-Instruct in 0/1-shot mode and fine-tuned on MMTab and different Tablet sizes. Results for ToTTo correspond to evaluation on the development set. We report exact match accuracy for WikiTQ, TabMWP, TabFact, and TAT-QA; BLEU for WikiBio, ToTTo, and FeTaQA; and F1 for HiTab. Best performing model per task is shown in bold; we mark with $\ast$ models significantly different ($p<0.05$, using bootstrap resampling) from those trained on Tablet (highlighted in gray). Models that perform significantly better within the Tablet group are underlined.**
| Held-in Datasets |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | WikiBio | ToTTo | WikiTQ | TabMWP | Hitab | TabFact | FeTaQA | TAT-QA |
| 0-Shot | 2.5 | 9.1^∗ | 53.4^∗ | 59.1^∗ | 31.2^∗ | 73.9^∗ | 7.0^∗ | 6.9^∗ |
| 1-Shot | 4.4 | 16.1^∗ | 49.8^∗ | 57.7^∗ | 37.3^∗ | 72.8^∗ | 8.5^∗ | 10.1^∗ |
| MMTab | 6.4^∗ | 12.6^∗ | 48.7^∗ | 80.4^∗ | 41.5^∗ | 57.0^∗ | 1.7^∗ | 9.4^∗ |
| \rowcolorgray!20<br>Tablet-S | 2.9 | 28.9 | 56.8 | 84.0 | 64.8 | 78.9 | 28.7 | 27.8 |
| \rowcolorgray!20<br>Tablet-M | 3.1 | 29.3 | 56.6 | 84.0 | 67.0 | 79.5 | 30.7 | 31.0 |
| \rowcolorgray!20<br>Tablet-L | 3.8 | 30.4 | 55.5 | 84.5 | 67.5 | 79.5 | 31.5 | 32.5 |

**Table 4: Left: Evaluation on five held-out datasets with Qwen2.5-VL-7B-Instruct (0/1-shot) and fine-tuned on MMTab and different Tablet sizes. We report Tree-Edit- Distance-based Similarity for Table Recognition (TabRec) and accuracy for all other benchmarks. HybridQA results are reported on the development set. Best performing model per task is shown in bold; we mark with $\ast$ models significantly different ($p<0.05$, using bootstrap resampling) from those trained on Tablet (highlighted in gray). Models that perform significantly better within the Tablet group are underlined. Right: Distribution of examples per task in Tablet-L (b) and Tablet-M (a).**
| Held-out-Datasets |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Infotabs | TabMCQ | AIT-QA | PubHealthTab | HybridQA | TabRec |
| 0-Shot | 64.5^∗ | 84.4 | 51.7^∗ | 64.7^∗ | 34.2^∗ | 24.5^∗ |
| 1-Shot | 65.5^∗ | 61.0^∗ | 43.1^∗ | 70.0^∗ | 30.5^∗ | 27.9^∗ |
| MMTab | 56.9^∗ | 89.1 | 56.6^∗ | 63.9^∗ | 27.1 | 43.6^∗ |
| \rowcolorgray!20<br>Tablet-S | 57.2 | 88.2 | 58.9 | 64.7 | 22.8 | 43.9 |
| \rowcolorgray!20<br>Tablet-M | 61.5 | 88.3 | 62.4 | 66.3 | 27.7 | 44.1 |
| \rowcolorgray!20<br>Tablet-L | 61.4 | 87.9 | 70.8 | 70.2 | 25.1 | 45.4 |

We observe that the model fine-tuned on Tablet-L  performs overwhelmingly better on held-in *and* held-out datasets. It dominates in all QA benchmarks except HybridQA, where long-context proficiency is likely reduced since its training set was excluded from fine-tuning. Notably, in tasks such as HiTab, FeTaQA, and TAT-QA, where Tablet includes fewer training examples than MMTab, the model still performs considerably better, suggesting benefits from transfer across tasks, original visualizations, or simply due to a higher-quality subset of examples. Finally, tasks that include hierarchical table structures such as ToTTo and HiTab show clear gains. Table cell hierarchy is not lost in synthetic tables, but it is often conveyed more clearly in their original visualizations.
Our results are encouraging, particularly when considering that portions
of the held-out datasets, including test sets, may have been
encountered during petraining. Models trained on Tablet outperform 0/1-shot Qwen2.5-VL-7B on 12 out 14 (held-in and held-out) tasks.

Interestingly, our gains extend beyond unseen tasks. For instance, ToTTo, HiTab, TabFact, FeTaQA, and TAT-QA contain the *same* number of training examples across *all* Tablet variants, yet performance consistently improves as additional tasks are incorporated. This suggests that fine-tuning on a broader task set facilitates transfer learning, yielding improvements even on tasks where the model was already competitive.

Optimal Tablet Size
Since training on 4M examples is resource-intensive, and tasks in Tablet-L exhibit substantial variation in size (see (b) pie chart in Table [4](https://arxiv.org/html/2509.21205v2#S5.T4.fig1)), we evaluate whether a smaller but more balanced dataset might perform comparably.
We create Tablet-M (a shorthand for Tablet-Medium) by capping each task at 140k examples. For benchmarks exceeding this cap (Column Type Annotation, Entity Linking, WikiBio), a random 140k subset is sampled and included as representative for that task. This results in a dataset with 1,117,217 training examples. We exclude HybridQA, InfoTabs, and Table Recognition from the training set to allow for held-out evaluation, leaving 1,031,082 examples in Tablet-M and 3,419,176 in the full dataset of our experiments. The capped distribution is shown in pie chart (b), Table [4](https://arxiv.org/html/2509.21205v2#S5.T4.fig1) and further details on training sets are in Appendix [7.1](https://arxiv.org/html/2509.21205v2#S7.SS1).

The model trained on Tablet-M performs competitively and in some cases, even better than Tablet-L (see Table [4](https://arxiv.org/html/2509.21205v2#S5.T4.fig1), rows in gray), offering a compelling trade-off between dataset size and model effectiveness. That said, the full dataset still yields slightly better performance overall and remains a valuable resource for future work that can benefit from large-scale, lossless training.

The Benefit of Training on Table Interpretation Tasks
Table Interpretation tasks represent 54.2% of the training examples in Tablet-L, making them the largest category by volume and the second longest in instruction length, following HybridQA (see Appendix [8.1](https://arxiv.org/html/2509.21205v2#S8.SS1)). While deng2020turl demonstrated the benefits of these tasks for textual TU, it is unclear whether this carries over to VTU. If these tasks prove unhelpful, removing them would save significant resources.

We therefore remove all Table Interpretation tasks from Tablet-M and fine-tune on this smaller version (we use Tablet-S as a shorthand for Tablet-Small) which excludes any tasks related to Column Type Annotation, Entity Linking, and Relation Extraction resulting in a dataset with a total of 690,467 training examples (67% of the example count in Tablet-M). As shown in Table [4](https://arxiv.org/html/2509.21205v2#S5.T4.fig1), there are benefits to be gained from including these tasks. Models trained on Tablet-M outperform those trained on Tablet-S on six out of 8 held-in benchmarks (Table [3](https://arxiv.org/html/2509.21205v2#S5.T3)) and achieve comparable performance on the remaining two. Similar gains are observed in four out of five held-out tasks (Table [4](https://arxiv.org/html/2509.21205v2#S5.T4.fig1)). These findings demonstrate that the benefits observed in the textual domain (deng2020turl) extend to multimodal VTU.

## 6 Conclusions

In this work, we introduced Tablet, a large-scale dataset for Visual Table Understanding that aggregates 4 million examples across 20 tasks and 2 million unique tables. Unlike prior resources, Tablet preserves original table visualizations whenever possible, provides both HTML and image representations, and maintains full traceability to the source datasets. Extensive experiments showed that (1) training on Tablet increases robustness to real-world tables compared to synthetic renderings; (2) models fine-tuned on Tablet consistently outperform those trained on related VTU datasets across held-in and held-out tasks; and (3) Tablet brings improvements on unseen benchmarks, which suggests that its diversity supports transfer across VTU tasks. We hope Tablet will foster further research on Visual Table Understanding, including the design of new tasks and the evaluation of models in settings that reflect real-world tabular data.

## 7 Appendix

### 7.1 Tablet Training Set Statistics

While the full released version of Tablet includes all available data, our experiments used different partitions. Specifically, to allow for held-out evaluation, we excluded HybridQA, InfoTabs, and Table Recognition from the training set; in addition, we created three training sets varying in size (Tablet-L, Tablet-M, Tablet-S) to explore various research questions. The statistics for these training sets are reported in Table [1](https://arxiv.org/html/2509.21205v2#S3.T1). The development and test sets remain identical to the ones mentioned in the paper.

**Table 5: Training set statistics for Tablet-L, Tablet-M, and Tablet-S used to fine-tune Qwenn2.5-VL 7B in our experiments. We report the number of examples (Examples), the tasks used (Tasks), and the number of examples with original table visualizations.**
|  | Examples | Tasks | Original Images |
| --- | --- | --- | --- |
| Tablet-S | 690,467 | 14 | 521,032 (75.5%) |
| Tablet-M | 1,031,082 | 17 | 812,748 (78.8%) |
| Tablet-L | 3,419,176 | 17 | 2,795,485 (81.8%) |

### 7.2 Image source per task

Figure [3](https://arxiv.org/html/2509.21205v2#S7.F3) shows the distribution of images per task included in Tablet. Images are sourced from Wikipedia, TabMWP, PubTabNet or are syntehtically rendered from information in the seed dataset.

Figure: Figure 2: Image source distribution in Tablet, broken down by task. That is, source of the image referred by each example in each task. While the distribution resembles that of the unique image pool, it is computed at the example level (e.g., if the same Wikipedia image appears in two examples of a task, it is counted twice). Wikipedia are original visualizations form Wikipedia. Seed render are synthetic images rendered form information in the seed dataset.
Refer to caption: x3.png

### 7.3 Dataset Crawling Dates

**Table 6: Data collection timeline for benchmarks included in Tablet.**
| Dataset | Date |
| --- | --- |
| TURL | 2013-11-01 |
| ToTTo | 2019-03-01 |
| TabFact | 2019-06-30 |
| HybridQA | 2020-01-31 |
| Infotabs | 2019-10-10 |
| WikiBio | 2015-09-01 |

We determined data collection dates for each dataset through various sources and methodologies. TURL originates from the TabEl dataset (sekhar2015tabel), which was crawled from the November 2013 English Wikipedia dump. For ToTTo, we obtained the collection date through direct correspondence with Ankur Parikh. TabFact’s timeline was established based on email correspondence with Wenhu Chen and the initial arXiv publication date.

For HybridQA, we conducted a comprehensive analysis of all dates present within the dataset tables and observed a significant decrease in data frequency from February 2020 onward, indicating the collection cutoff point. InfoTabs’ crawling year is documented at the bottom of their GitHub repository page, though the specific day represents our estimation. WikiBio’s GitHub documentation indicates that their tables are sourced from the Common Crawl snapshot enwiki-20150901.

### 7.4 Dataset example

For reference, we show an example from Tablet for the ToTTo table-to-text task. Note that examples for every other task follow the same format. We provide the instructions used in our experiments, together with the corresponding example metadata, and links to the corresponding example and table from the source dataset.

Figure: Figure 3: Example of a Tablet example for the ToTTo table-to-text task.
Refer to caption: x4.png

### 7.5 Definition of Tasks Represented in Tablet

As mentioned in Section [3.3](https://arxiv.org/html/2509.21205v2#S3.SS3), Tablet includes 20 TU tasks. We provide more detailed descriptions for each below.

#### Column Type Annotation

In this task, the model is required to identify the data type of the values in a highlighted table column. The model selects from a set of 255 randomly shuffled candidate data types. There can be multiple correct types per column. The column is visually highlighted to ensure that the model attends to the table itself rather than relying solely on the textual instruction to infer the answer. Examples for this task were obtained from TURL. Our dataset includes 602,406/13,188/12,802 (train/dev/test) instructions for this task. An example of this task is shown in Figure [17](https://arxiv.org/html/2509.21205v2#S8.F17).

#### Entity Linking

For this task, the model is given a table with a visually highlighted cell along with a set of up to 100 candidate entities and descriptions and must identify which entity and description corresponds to the entity in the selected cell. This task was aggregated from the TURL dataset. Our dataset includes 1,236,128/74,282/213,494 (train/dev/test) instructions for this task. An example of this task is shown in Figure [5](https://arxiv.org/html/2509.21205v2#S8.F5).

#### Relation Extraction

This task requires the model to select appropriate relations between two visually highlighted columns of a table from a set of candidates. This task is derived from the TURL dataset. Our dataset includes 60,615/2,145/2,030 (train/dev/test) instructions for this task. An example of this task is shown in Figure [7](https://arxiv.org/html/2509.21205v2#S8.F7).

#### Structure Aware Parsing

In this TSR task, the model needs to parse the table into markdown format. This task comes from Docstruct4M, using tables from PubTabNet, TabFact, and WikiTableQuestions. Our dataset includes 513,482/9,115/1,102 (train/dev/test) instructions for this task. An example of this task is shown in Figure [9](https://arxiv.org/html/2509.21205v2#S8.F9).

#### Free-form Table Question Answering (FeTaQA)

In this task, the model generates free-form answers to questions about Wikipedia tables, often requiring integration of information from discontinuous sections of the table. Unlike datasets with shorter text spans, FeTaQA emphasizes higher-level understanding through long-form answers. Examples for this task come from FeTaQA dataset. Our dataset contains 3,006/577/1,079 (train/dev/test) instructions for this task. An example of this task is shown in Figure [6](https://arxiv.org/html/2509.21205v2#S8.F6).

#### Hierarchical Table QA (HiTabQA)

This question-answering task involves hierarchical tables (with different headers across the table) and sometimes includes numerical reasoning, such as sums, averages, maximum, minimum, and counting, among others. Examples for this task were obtained from HiTabQA, with tables from ToTTo, StatCan, and NSF seed datasets. Our dataset contains 7,417/1,670/1,584 (train/dev/test) examples. An example of this task is shown in Figure [8](https://arxiv.org/html/2509.21205v2#S8.F8).

#### Table Fact Verification (TabFact, Infotabs)

Also known as Table Entailment, this task involves classifying statements as supported or refuted based on table content. Examples for these two tasks come from TabFact and Infotabs seed datasets. Our dataset includes 87,717/ 12,389 / 12,326 (train / dev / test) TabFact examples and 16,538 / 1,800 / 5,400 (train / dev / test) Infotabs examples. An example of one of these tasks is shown in Figure [11](https://arxiv.org/html/2509.21205v2#S8.F11).

#### Table Numerical Reasoning (TabMWP, TAT-QA)

Given a table and a mathematical question, the model must answer using mathematical reasoning over table values. Instructions were based on examples from TabMWP and TAT-QA. Our dataset includes 23,059 / 7,686 / 7,686 for TABMWP and 2,201 / 278 / 277 (train / dev / test) examples. An example of one of these tasks is shown in Figure [10](https://arxiv.org/html/2509.21205v2#S8.F10).

#### Table-to-Text (ToTTo)

In this task, the model needs to generate a description based on the visually highlighted cells in a given table. Instructions were generated based on examples from ToTTo. Not all examples from the original dataset were retrieved, as we could not trace all highlighted cells from the dataset to the retrieved table for 8.1% of the examples. These examples were discarded to avoid adding noise to the dataset. Our dataset contains 110,934/7,077/7,084 (train/dev/test) examples. An example of this task is shown in Figure [12](https://arxiv.org/html/2509.21205v2#S8.F12).

#### Hybrid QA (HybridQA)

This multi-hop question-answering task requires integrating structured table data with unstructured hyperlinked passages. Given a Wikipedia table and texts linked to the table’s entities, the model must answer multi-hop questions by reasoning across both modalities. Instructions were generated using HybridQA’s examples, resulting in a dataset containing 62,670/3,466/3,463 (train/dev/test) examples. An example of this task is shown in Figure [14](https://arxiv.org/html/2509.21205v2#S8.F14).

#### Table QA (WikiTableQuestions)

Given a Wikipedia table and a question, the model must answer based on the table’s content. For this task, WikiTableQA’s examples are phrased as instructions in our dataset. Our dataset includes 14,152/3,537/4,344 (train/dev/test) instructions. An example of this task is shown in Figure [16](https://arxiv.org/html/2509.21205v2#S8.F16).

#### Wikipedia Biography Generation (WikiBio)

Given a Wikipedia infobox of an entity, the model is prompted to generate a concise biography of this entity using the information in the infobox. This task’s examples are aggregated from WikiBio. Our dataset includes 582,659/72,831/72,831 (train/dev/test) instructions. An example of this task is shown in Figure [13](https://arxiv.org/html/2509.21205v2#S8.F13).

#### MMTab ’s Structure Understanding tasks

Tablet includes all tasks from MMTab that are meant to instill table structure understanding in the model. These include: Merged Cell Detection, Row & Column Extraction, Table Cell Extraction, Table Cell Location, Table Recognition, Table Size Detection, and instruction following pre-training tasks. We refer to zheng-etal-2024-multimodal’s work for a description of these tasks. Instructions for these tasks are directly aggregated from MMTab and tables come from the following seed datasets: InfoTabs, NSF, StatCan, TabMWP, TAT-QA, TabFact, ToTTo, WikiBIO, WikiTableQuestions. Tablet maintains the same instructions as in MMTab but uses our visualizations for the table images. As instructions originate from MMTab, and no dev set is provided in this dataset, Tablet does not include any example for these tasks in the development set. Our dataset includes the following examples: Merged Cell Detection (7,500/0/950), Row & Column Extraction (7,721/0/957), Table Cell Extraction (7,727/0/966), Table Cell Location (7,708/0/956), Table Recognition (6,927/0/843), Table Size Detection (7,800/0/950), and instruction following pre-training tasks (136,944/0/0). An example of one of these tasks is shown in Figure [17](https://arxiv.org/html/2509.21205v2#S8.F17).

### 7.6 MMTab vs Tablet

Table [7](https://arxiv.org/html/2509.21205v2#S7.T7) provides a detailed comparison between MMTab and Tablet across tasks and example counts (in training, development and test sets).

**Table 7: Comparison of MMTab and Tablet: tasks and examples across training, development, and test splits.**
|  | Train | Dev | Test |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Task | MMTab | Tablet | MMTab | Tablet | MMTab | Tablet |
| wikibio | 4,994 | 582,659 | 0 | 72,831 | 1,000 | 72,831 |
| wikitq | 17,689 | 14,152 | 0 | 3,537 | 4,344 | 4,344 |
| totto | 15,000 | 110,934 | 0 | 7,077 | 7,700 | 7,084 |
| tabmwp | 30,745 | 23,059 | 0 | 7,686 | 7,686 | 7,686 |
| tabfact | 31,321 | 87,717 | 0 | 12,389 | 6,845 | 12,326 |
| hitab | 11,941 | 7,417 | 0 | 1,670 | 3,160 | 1,584 |
| infotabs | 18,338 | 16,538 | 0 | 1,800 | 5,400 | 5,400 |
| fetaqa | 8,327 | 3,006 | 0 | 577 | 2,003 | 1,079 |
| tat-qa | 5,920 | 2,201 | 0 | 278 | 772 | 277 |
| table_instruction | 37,204 | 136,944 | 0 | 0 | 0 | 0 |
| table_cell_extraction | 8,000 | 7,727 | 0 | 0 | 1,000 | 966 |
| table_cell_location | 8,000 | 7,708 | 0 | 0 | 1,000 | 956 |
| table_size_detection | 8,000 | 7,800 | 0 | 0 | 1,000 | 950 |
| merged_cell_detection | 8,000 | 7,500 | 0 | 0 | 1,000 | 950 |
| row_column_extraction | 8,000 | 7,721 | 0 | 0 | 1,000 | 957 |
| table_recognition | 8,000 | 6,927 | 0 | 0 | 1,000 | 912 |
| rotowire | 3,400 | 0 | 0 | 0 | 334 | 0 |
| col_type | 0 | 602,406 | 0 | 13,188 | 0 | 12,802 |
| ent_link | 0 | 1,236,128 | 0 | 74,282 | 0 | 213,494 |
| rel_extraction | 0 | 60,615 | 0 | 2,145 | 0 | 2,030 |
| hybridqa | 0 | 62,670 | 0 | 3,466 | 0 | 3,463 |
| struct_aware_parse | 0 | 513,482 | 0 | 9,115 | 0 | 1,102 |
| OOD | 0 | 0 | 0 | 0 | 1,250 | 0 |
| TabMCQ | 0 | 0 | 0 | 0 | 1,029 | 0 |
| AIT-QA | 0 | 0 | 0 | 0 | 511 | 0 |
| PubHealthTab | 0 | 0 | 0 | 0 | 1,942 | 0 |
| All | 232,879 | 3,505,311 | 0 | 210,041 | 49,976 | 351,193 |

## 8 Supervised Fine-Tuning Details

All supervised fine-tuning (SFT) experiments were carried out with the same hyperparameters across models; the only varying factor was the dataset used for training. We fine-tuned Qwen2.5-VL-7B-Instruct using their official implementation.(^5^55[https://github.com/QwenLM/Qwen2.5-VL/tree/main/qwen-vl-finetune](https://github.com/QwenLM/Qwen2.5-VL/tree/main/qwen-vl-finetune)) Our code will be released alongside the dataset in their project repository.

#### Hyperparameters

All runs used the following common configuration:

- •
Training setup: DeepSpeed ZeRO-3, bf16 precision
- •
Epochs: 3
- •
Batch size: 2 (per device), gradient accumulation steps: 4
- •
Optimizer: AdamW, learning rate: 2e-7, weight decay: 0.01
- •
Scheduler: cosine decay with warmup ratio 0.03
- •
Gradient clipping: 1.0
- •
Sequence length: 8192 tokens
- •
Vision input size: max_pixels = 50,176, min_pixels = 784
- •
Other: data_flatten = False, data_packing = False, tune_mm_vision = False, tune_mm_mlp = True, tune_mm_llm = True

#### Compute Usage

Table [8](https://arxiv.org/html/2509.21205v2#S8.T8) reports the total GPU hours consumed by each experiment. All runs were conducted on clusters equipped with NVIDIA A100 GPUs.

**Table 8: GPU hours for supervised fine-tuning and prediction runs. For the predictions, the 8 models are the 7 SFT models and the baseline model.**
| Dataset | Setup | GPU Hours |
| --- | --- | --- |
| Tablet-B${}_{\text{org}}$ | 15h $\times$ 16 GPUs | 240 |
| Tablet-B${}_{\text{synth}}$ | 15h $\times$ 16 GPUs | 240 |
| Tablet-B${}_{\text{mix}}$ | 21h $\times$ 16 GPUs | 336 |
| MMTab | 21h $\times$ 16 GPUs | 336 |
| Tablet-S | 28h $\times$ 32 GPUs | 896 |
| Tablet-M | 40h $\times$ 32 GPUs | 1280 |
| Tablet-L | 125h $\times$ 32 GPUs | 4000 |
| Inference (Tablet-B${}_{\text{org}}$) | 21h $\times$ 8 models | 168 |
| Inference (Tablet-B${}_{\text{synth}}$) | 21h $\times$ 8 models | 168 |
| Inference (Tablet-B${}_{\text{mix}}$) | 24h $\times$ 8 models | 192 |
| Inference (MMTab) | 24h $\times$ 8 models | 192 |
| Inference (Tablet-B${}_{\text{L}}$) | 34h $\times$ 8 models | 272 |

### 8.1 Instruction Sequence Length

Figure [4](https://arxiv.org/html/2509.21205v2#S8.F4) shows how instruction length (measured in terms of tokens) varies across tasks in Tablet. As can be seen, HybridQA has the longest instructions, followed by Column Type, Entity Linking, and Relation Extraction.

Figure: Figure 4: Instruction length distribution across tasks in Tablet.
Refer to caption: figures/seq_length_log.png

### 8.2 Performance Degradation: Original vs Synthetic Images

Let each task $t$ belong to one of the metric families
$\mathcal{B}$ (BLEU), $\mathcal{A}$ (accuracy), or $\mathcal{F}$ (F1).
For a given model $m$, we compute the raw difference:

$$ $\Delta_{m,t}=\text{Score}^{\text{org}}_{m,t}-\text{Score}^{\text{synth}}_{m,t}$ $$

Negative values indicate that the model performs worse when evaluated on the original visualizations compared to the synthetic ones. Because the metrics differ in scale, we normalize each by the corresponding synthetic score to report results in terms of percentage change:

$$ $\delta_{m,t}=\frac{\Delta_{m,t}}{\text{Score}^{\text{synth}}_{m,t}}\times 100.$ $$

Next, for each metric family $F\in{\mathcal{B},\mathcal{A},\mathcal{F}}$, we compute the family mean ($M$). Finally, the Degradation Score for model $m$ is defined as the unweighted average of the family means:

$$ $\text{DegScore}=\frac{1}{3}\Big(M_{m,\mathcal{B}}+M_{m,\mathcal{A}}+M_{m,\mathcal{F}}\Big).$ $$

This guarantees that BLEU, accuracy, and F1 contribute equally, regardless of how many tasks are included in each family. Our
results are summarized in Table [9](https://arxiv.org/html/2509.21205v2#S8.T9) below where we show the Degration Score per dataset and overall. Results are reported only for tasks with synthetic *and* original images. InfoTabs is held-out, all other tasks are held-in. As can be seen in the table, the 0-shot model performs worse when evaluated on original visualizations, followed by Qwen2.5-VL-7B fine-tuned on Tablet-B${{}_{\text{synth}}}$. Models which have seen original images during fine-tuning perform best (see Tablet-B${{}_{\text{org}}}$ and Tablet-B${{}_{\text{mix}}}$.

**Table 9: Comparison of Qwen2.5-VL-7B-Instruct models fine-tuned on different Tablet-B variants and in zero-shot mode; models are evaluated on original (lossless) table visualizations and synthetic ones across held-in benchmarks. Totals are summed separately for BLEU, accuracy (acc), and F1; the average percentage-point change (DegScore) is a unit-less normalized aggregate (more negative is worse).**
| Model | WikiBio | ToTTo | WikiTQ | HiTab | TabFact | FeTaqa | Infotabs | (BLEU) | (acc) | (F1) | DegScore |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 0-Shot | +0.5 | -1.9 | -6.9 | -8.6 | -11.0 | -0.1 | -4.6 | -1.5 | -22.5 | -8.6 | -28.90 |
| Tablet-B${}_{\text{synth}}$ | -6.5 | +2.0 | -1.9 | -17.5 | +2.4 | -0.7 | -0.6 | -5.2 | -0.1 | -17.5 | -22.35 |
| Tablet-B${}_{\text{org}}$ | +1.9 | +2.2 | -2.3 | -10.4 | +1.6 | +0.0 | -2.7 | +4.1 | -3.4 | -10.4 | -6.63 |
| Tablet-B${}_{\text{mix}}$ | +1.7 | +1.6 | -2.4 | -11.6 | +1.8 | -0.1 | -4.8 | +3.2 | -5.4 | -11.6 | -7.87 |

### 8.3 Tablet - B Held-out Evaluation Results

In Table [10](https://arxiv.org/html/2509.21205v2#S8.T10), we present results on held-out portions of Tablet-B, with Qwen2.5-VL-7B fine-tuned on original table images (Tablet-B${}_{\text{org}}$), synthetic ones (Tablet-B${}_{\text{synth}}$), and a mixture (Tablet-B${}_{\text{mix}}$).

**Table 10: Comparison of Qwen2.5-VL-7B fine-tuned on different MMTab variants evaluated on purely lossless (org) table visualizations and a mix of both across held-out benchmarks. Results for TabMWP, AIT-QA, and PubHealthTab do not include mix because the tables in these tasks are original visualizations. We report accuracy for all benchmarks.**
|  | Infotabs | TabMCQ | AIT-QA | PubHealthTab |  |
| --- | --- | --- | --- | --- | --- |
| Model | mix | org | org | org | org |
| Tablet-B${}_{\text{synth}}$ | 52.4 | 51.8 | 87.8 | 64.4 | 57.6 |
| Tablet-B${}_{\text{org}}$ | 51.4 | 48.7 | 87.7 | 63.8 | 55.0 |
| Tablet-B${}_{\text{mix}}$ | 60.3 | 55.5 | 88.3 | 59.5 | 61.6 |

### 8.4 Open-weight Model Evaluation on Tablet

Finally, in Tables [11](https://arxiv.org/html/2509.21205v2#S8.T11) and  [12](https://arxiv.org/html/2509.21205v2#S8.T12), we report results on Tablet for open-weight VLMs other than Qwen2.5-VL-7B, in the zero-shot setting.

**Table 11: Open-weight VLMs evaluated on Tablet held-in datasets (zero-shot setting).**
|  | WikiBio | ToTTo | WikiTQ | TabMWP | HiTab | TabFact | FeTaQA | TAT-QA |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Table-LLaVA 7B (zheng-etal-2024-multimodal) | 5.7 | 16.3 | 17.7 | 47.5 | 10.6 | 53.6 | 7.5 | 1.8 |
| Qwen2.5-VL 7B (qwen2025qwen25technicalreport) | 2.5 | 9.1 | 53.4 | 59.1 | 31.2 | 73.9 | 7.0 | 6.9 |
| DocOwl2 8B (hu-etal-2025-mplug) | 1.8 | 1.2 | 18.9 | 0.5 | 9.5 | 54.3 | 9.6 | 2.9 |
| InternVL3 8B (zhu2025internvl3exploringadvancedtraining) | 4.6 | 11.9 | 45.3 | 83.9 | 40.8 | 64.3 | 3.8 | 13.0 |
| InternVL3 14B (zhu2025internvl3exploringadvancedtraining) | 6.3 | 12.4 | 51.6 | 86.8 | 52.0 | 75.1 | 6.6 | 11.9 |
| Gemma-3 12B (gemmateam2025gemma3technicalreport) | 5.1 | 5.4 | 39.1 | 72.3 | 24.2 | 69.5 | 20.3 | 2.5 |

**Table 12: Open-weight VLMs evaluated on Tablet held-out datasets (zero-shot setting). Results for HybridQA are not reported for Table-LLaVA 7B and DocOwl2 8B due to their VRAM requirements, which do not scale well to the long contexts needed for our dataset.**
|  | Infotabs | TabMCQ | AIT-QA | PubHealthTab | HybridQA |
| --- | --- | --- | --- | --- | --- |
| Table-LLaVA 7B (zheng-etal-2024-multimodal) | 61.5 | 59.1 | 4.7 | 50.2 | — |
| Qwen2.5-VL 7B (qwen2025qwen25technicalreport) | 64.5 | 84.4 | 51.7 | 64.7 | 34.2 |
| DocOwl2 8B hu-etal-2025-mplug | 15.5 | 63.0 | 36.8 | 14.6 | — |
| InternVL3 8B (zhu2025internvl3exploringadvancedtraining) | 59.6 | 88.5 | 52.4 | 64.2 | 33.1 |
| InternVL3 14B (zhu2025internvl3exploringadvancedtraining) | 67.4 | 88.8 | 52.3 | 74.9 | 46.7 |
| Gemma-3 12B (gemmateam2025gemma3technicalreport) | 65.5 | 88.9 | 48.7 | 64.7 | 35.2 |

Figure: Figure 5: Example for Entity Linking task based on highlighted table cell.
Refer to caption: figures/examples/e4f64d3973a0668fe6a612308104476b.png

Figure: Figure 6: Example for Free-form Table Question Answering task based on highlighted table cells (FeTaQA).
Refer to caption: figures/examples/fa363115ea7fd7aa7e2576ff03dab6d9.png

Figure: Figure 7: Example for Relation Extraction task based on highlighted table columns.
Refer to caption: figures/examples/dae811e15867b8ed713abae4d8b82fd4.png

Figure: Figure 8: Example for Hierarchical Table QA task (HiTabQA).
Refer to caption: figures/examples/efdca3255f70c6f10d205d82f3baf8dd.png

Figure: Figure 9: Example for Structure Aware Parsing task.
Refer to caption: figures/examples/4b78eb5480421e1f67b6d167746a9f1f.png

Figure: Figure 10: Example for Table Numerical Reasoning task (TabMWP).
Refer to caption: figures/examples/60590e15a9285ed4633e72482b4d0367.png

Figure: Figure 11: Example for Table Fact Verification task (Infotabs).
Refer to caption: figures/examples/18a0b2448f8aa7a07ae54a5f6a7f7beb.png

Figure: Figure 12: Example for Table-to-Text task based on highlighted table cells (ToTTo).
Refer to caption: figures/examples/a19d4be7ecd6f1c1394459d318e20caf.png

Figure: Figure 13: Example for Wikipedia Biography Generation task (WikiBio).
Refer to caption: figures/examples/c4cee7cafa426cfeddff0a705158426a.png

Figure: Figure 14: Example for Hybrid QA task (HybridQA).
Refer to caption: figures/examples/617200ff2ad801c322aa7d2c958ebc9d.png

Figure: Figure 15: Example for Table Cell Extraction.
Refer to caption: figures/examples/195ae1be7c2834f53d8135b727a72c7a.png

Figure: Figure 16: Example for Table QA task (WikiTableQuestions).
Refer to caption: figures/examples/16a82a8fb434b9aff36e64e9eb22e011.png

Figure: Figure 17: Example for Colum Type Annotation task based on highlighted table columns.
Refer to caption: figures/examples/dae811e15867b8ed713abae4d8b82fd4.png

## 9 Limitations

While Tablet is the largest resource for Visual Table Understanding to date, it has several limitations. Firstly, some original visualizations could not be retrieved due to missing or changed Wikipedia pages, and some embedded resources (e.g., images) were inaccessible. Secondly, we did not conduct a full ablation of the contribution of each task due to computational constraints, focusing instead on broader questions such as task balancing and dataset inclusion. Thirdly, most seed datasets are from Wikipedia, which, despite differing in visual format, are common in pretraining corpora, raising the possibility of data contamination. Finally, our fine-tuning focused on a single model (Qwen2.5-VL 7B), with zero-shot evaluation on others; future work should explore a broader range of architectures, especially those with reasoning or self-reflection capabilities.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 2
presentation: 3
questions: Please refer to the weaknesses section above. 
It seems that the author merely proposed a dataset and fine-tuned qwen with it, without providing comparisons to other models. I believe the paper still needs further polishing.
soundness: 2
strengths: +It collects a large-scale dataset combining multiple VTU sources. 
+It provides both HTML and image representations with metadata, which facilitate the document understanding community. 
+Experiments on the Qwen2.5-VL models demonstrating the effectiveness of the collected dataset.
summary: This paper presents TABLET, a large-scale dataset for UTV, containing 4M examples. The key contribution is the collected TABLET from sources from Wikipedia and PubTabNet. The dataset are split into train, dev, and test sets and comprising comprehensive table-related tasks. Experiments with Qwen2.5-VL-7B show good performance gains on several benchmarks, and the authors argue that TABLET improves robustness on real-world table images.
weaknesses: 1. The author presents a new dataset, TABLET, and the author claims that the dataset contains 4M examples. However, nearly 30% are existing, the author should describe this more rigorously.
2. No new model, algorithm, or theoretical insight in the paper. The paper mainly merges existing datasets and collected new table data on the network. 
3. One of the author’s main motivation is that synthetic data are of lower quality than real-world data. However, the majority of the dataset they collected still consists of screenshot-based images, many of which are taken from sources like Wikipedia. This seems somewhat inconsistent with the author’s stated motivation?
4. Poor experiments. In Tab.2 and 3, the performance improvement by the proposed dataset appears to be limited.
5. What confuses me is that the paper only reports results for qwen and the results fine-tuned on qwen, without any comparison to other models. This is completely unreasonable for a research paper.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: None
soundness: 3
strengths: 1.The motivation is real: current VTU data is mostly synthetic or visually simplified, which causes a distribution gap for vision-language models that must operate on real web-like tables.

2.The dataset is large, diverse, and traceable back to the original sources, which is helpful for reproducibility and later task extension.

3.Experiments are fairly comprehensive (synthetic vs. real-style vs. mixed) and the results are consistent with the stated motivation.
summary: The paper introduces TABLET, a large-scale dataset for visual table understanding (VTU). Compared to existing resources (e.g., MMTab), TABLET (i) reconstructs tables in their realistic visual form from web/Wikipedia sources, (ii) unifies them into an image + HTML + metadata format, and (iii) covers 20 VTU-related tasks. The authors fine-tune Qwen2.5-VL-7B on this data and show improvements on both seen and unseen VTU tasks. The main value of the paper is on the data/engineering side and on providing a more realistic training distribution.
weaknesses: 1. The paper does not introduce a new learning objective, a new model architecture, or a principled framework for unifying VTU tasks. The core message is essentially “better/realistic data → better performance,” which is valuable but closer to a dataset/empirical paper than to a method/theory paper.

2. The paper repeatedly argues that preserving realistic visual styles leads to better generalization, but the current evaluations mostly show dataset-level gains, not an analysis of which visual factors (borders, multi-row headers, fonts, layout noise) the model actually exploits.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. Given that the core experiment in Table 2 fails to show a clear superiority for TABLET-B_org over TABLET-B_synth, and you attribute this to flawed benchmarks (e.g., ToTTo), does this not imply that the primary contribution of this work is a resource for which the tasks do not yet exist?

2. The strong performance of the TABLET-B_mix model seems to suggest that a combination of synthetic and original data is optimal. Does this not contradict the paper's central premise that synthetic renderings are inherently flawed and should be replaced by original visualizations?

3. Why were new, visually-demanding tasks (e.g., "find the cell with the red background," "what is the value in the merged cell?") not introduced and evaluated to actually prove the hypothesis that the visual fidelity of TABLET is its key advantage?
soundness: 2
strengths: 1. **Problem Identification:** The paper's core premise is strong and well-articulated. It correctly identifies the "train-test mismatch" as a fundamental problem for the VTU field as it shifts towards pixel-only VLMs. This is a significant and practical issue.

2. **Methodological Rigor on Data Collection:** The primary contribution is the dataset's "visual fidelity." The engineering effort to achieve this is non-trivial and highly commendable. The authors detail a rigorous process of tracing seed datasets to their original crawl dates and using Wikipedia's archiving API to retrieve historical snapshots, followed by Levenshtein matching to find the correct table. This high-quality execution is a major strength.

3. **Strong Robustness Argument:** The paper's most compelling evidence is in Table 9. It clearly shows that models trained on synthetic data suffer a massive performance degradation (-22.35 points) when evaluated on original visualizations. In contrast, models trained on TABLET's original data are far more robust (only -6.63 points). This single finding strongly validates the paper's core hypothesis and the necessity of this dataset.

4. **Resource Value and Extensibility:** The paper delivers more than just a static benchmark; it provides a large-scale, multi-task resource. By including HTML representations, metadata, and traceability links to source datasets, the authors enable future research, task reformulation, and extensibility.
summary: This paper identifies a critical "train-test mismatch" in the field of Visual Table Understanding (VTU). The authors argue that existing benchmarks predominantly use "synthetic renderings" of tables, which lack the visual complexity (e.g., merged cells, colors, fonts, embedded images) of real-world tables. This causes models trained on them to fail when generalizing to real-world visual data.

To solve this, the authors introduce TABLET, a new large-scale dataset of 4 million examples over 20 tasks. The dataset's primary contribution is that 88% of its 2 million unique tables are "original visualizations" meticulously retrieved from historical web snapshots (primarily Wikipedia), preserving their true visual fidelity. The authors demonstrate through experiments that while performance on traditional tasks is mixed, training on TABLET dramatically improves a model's robustness to visual domain shift (synthetic vs. original) and enhances generalization to unseen tasks.
weaknesses: 1. **Core Claim Undermined by Own Results:** The paper's most significant weakness is that its primary experiment (Table 2) fails to support its central claim. When comparing models trained on TABLET-B_org (original) vs. TABLET-B_synth (synthetic), the performance is roughly equivalent across most tasks. This result is underwhelming and directly contradicts the motivation that visual fidelity improves performance on these tasks.

2. **Circular Argument:** The authors' explanation for the failure of Table 2 is that the benchmarks themselves (e.g., ToTTo, WikiTQ) are flawed and were designed for text, not visual cues. This is a "catch-22" and a form of circular reasoning: the paper claims its dataset is the solution but simultaneously admits that the tasks it uses for validation are incapable of proving it. They have built a high-fidelity testbed but failed to provide tasks that actually require that fidelity.

3. **Diluted Conclusion on "Mixed" Data:** In several experiments (Table 2, Table 10), the TABLET-B_mix (original + synthetic) model performs best. This dilutes the paper's central premise. The takeaway risks becoming "more data is better" (a trivial conclusion) rather than the intended, stronger claim that "high-fidelity data is better."

4. **Limited "Real-World" Diversity:** The paper claims to solve the "real-world" visualization problem, but the dataset's diversity is questionable. 88% of the data (61.5% of tables) is sourced from Wikipedia. While better than synthetic data, Wikipedia tables are still highly structured and relatively uniform. This is not representative of the true "in-the-wild" chaos of scanned PDFs, financial reports, or product pages.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: see the weakness
soundness: 3
strengths: TABLET addresses prior single-task/synthetic data limitations with large-scale, lossless real-world table visualizations and 20 diverse tasks, while its high quality lies in 4M examples, image-HTML pairings, and traceable metadata. Its clear unified format and extensible design, coupled with significant contributions to boosting VLM cross-task performance (including SOTA results) and enabling practical VTU research, underscore its field value.
summary: Current visual table understanding (VTU) benchmarks rely on synthetic data lacking real-world complexity and omit underlying serialized data, while TABLET is a large-scale dataset with 4 million examples from real visualizations, paired image-HTML representations, and 20 tasks. Fine-tuning vision-language models on TABLET boosts their performance on seen/unseen VTU tasks and robustness to real-world tables, establishing a foundation for future VTU model training and evaluation.
weaknesses: 1. Whether all tables are presented in English? As far as I am aware, Wikipedia provides original HTML files in multiple languages.
2. Each example in TABLET is accompanied by the HTML version of the corresponding table. Can such a large volume of data be used to train table parsing models? Is the annotation format of HTML consistent? For instance, how are formulas within the tables annotated? Additionally, do these HTML files contain certain formatting information, such as row widths and font colors?


Decision: Accept (Poster)