Same Content, Different Representations: A Controlled Study for Table QA

## Abstract

Abstract Table Question Answering (Table QA) in real-world settings must operate over both structured databases and semi-structured tables containing textual fields. However, existing benchmarks are tied to fixed data formats and have not systematically examined how representation itself affects model performance.
We present the first controlled study that isolates the role of table representation by holding content constant while varying structure. Using a verbalization pipeline, we generate paired structured and semi-structured tables, enabling direct comparisons across modeling paradigms. To support detailed analysis, we introduce a diagnostic benchmark with splits along table size, join requirements, query complexity, and schema quality.
Our experiments reveal consistent trade-offs: SQL-based methods achieve high accuracy on structured inputs but degrade on semi-structured data, LLMs exhibit flexibility but reduced precision, and hybrid approaches strike a balance, particularly under noisy schemas. These effects intensify with larger tables and more complex queries.
Ultimately, no single method excels across all conditions, and we highlight the central role of representation in shaping Table QA performance. Our findings provide actionable insights for model selection and design, paving the way for more robust hybrid approaches suited for diverse real-world data formats.

## 1 Introduction

Tables are a fundamental medium for storing and communicating information across domains such as finance (Chen et al., 2021), scientific communication (Ghosh et al., 2024), biomedical records (Ghosh et al., 2024), and the web (Chakrabarti et al., 2020). Unlocking the knowledge contained in these tables has motivated extensive research on Table Question Answering (Table QA), where models answer natural language queries grounded in tabular data (Pasupat & Liang, 2015; Iyyer et al., 2017).
In practice, tables appear in both structured formats with rigid schemas and executable SQL queries (Zhong et al., 2017; Li et al., 2023; Wu et al., 2025a), and semi-structured formats where columns are irregular and cells contain free text (Chen et al., 2021; 2020; Singh et al., 2025). Structured tables enable precision and symbolic reasoning, while semi-structured tables offer robustness to noise and incomplete metadata. Since both formats coexist in real-world settings, understanding their impact on Table QA methods is crucial.

Despite substantial progress, this key question remains unanswered: how do different modeling paradigms handle variation in table representation? Existing benchmarks focus on query domain or complexity (Li et al., 2023; Wu et al., 2025a; Zhu et al., 2025), but fix the table format itself. As a result, models are optimized for a single representation, leaving their robustness to representation shift unclear. Current approaches fall into three main families: NL2SQL methods (Dong & Lapata, 2016; Zhong et al., 2017; Liu et al., 2022),
LLM-based methods (Zhang et al., 2025c; a), and hybrid methods (Zhang et al., 2024; Ye et al., 2023; Abhyankar et al., 2025; Khoja et al., 2025). NL2SQL methods scale and reason precisely but are brittle under noisy or irregular schemas. LLM-based methods can handle semi-structured inputs containing free text but struggle on complex queries and long tables. Hybrid methods combine symbolic execution with neural reasoning to balance flexibility and precision. Without a controlled evaluation of representation effects, practitioners lack guidance on method selection, and researchers risk overfitting to narrow benchmark conditions.

Figure: Figure 1: Structured vs. semi-structured formats of the same table pose challenges for Table QA methods that assume a fixed data format.
Refer to caption: x1.png

To address this gap, we present the first controlled study of table representation in Table QA. Our framework generates paired structured and semi-structured tables using a verbalization pipeline, holding content constant while varying representation (Figure [1](https://arxiv.org/html/2509.22983v1#S1.F1)). It enables systematic comparisons across modeling families, task conditions, and evaluation dimensions. Our contributions include:

- •
Controlled comparison of representations: we introduce a verbalization pipeline to convert structured tables into semi-structured forms, isolating the effects of representation while preserving semantics.
- •
Fine-grained diagnostic benchmark: we decompose task difficulty into four dimensions including table size, table joins, query complexity, and schema quality. This enables targeted analysis of method strengths and weaknesses.
- •
Comprehensive evaluation: we assess NL2SQL, LLM-based, and hybrid methods on BIRD (Li et al., 2023), MMQA (Wu et al., 2025a), and TableEval (Zhu et al., 2025) datasets, revealing trade-offs that inform both model design and practical deployment.

Our experiments show that representation is a major driver of performance. NL2SQL excels on structured inputs but drops $30$–$45\%$ on semi-structured ones. LLMs are more stable (only $3.5\%$ decline) but struggle with long or compositional queries. Hybrids fall by under $5\%$ and are generally robust across conditions. These trade-offs intensify with larger tables, complex queries, and noisy schemas. Long tables reduce accuracy for all methods, though NL2SQL remains strong on long structured tables (62.9% accuracy). In contrast, hybrids outperform LLMs on long semi-structured tables, demonstrating better scalability. Multi-table reasoning and query complexity remain challenging across all methods. Schema quality proves especially influential: noisy schemas severely hurt NL2SQL, while verbalization often improves LLMs and hybrids by embedding schema cues in natural language.

Ultimately, no single paradigm excels across all conditions. Our findings establish representation as a central factor in Table QA, provide actionable insights for method selection, and motivate future hybrid systems to operate robustly across diverse real-world table formats. We will release our framework and benchmark to support ongoing research.

## 2 Exploring Potential Factors of Reasoning

Table QA is challenging not only due to the diversity of natural language queries but also due to how information is represented. A model’s performance depends jointly on table representation (structured vs. semi-structured) and additional factors such as table size, schema quality, and query complexity. These challenges are salient in real-world settings, where tables are often large, noisy, and irregularly organized. We first contrast structured and semi-structured representations, then introduce four core dimensions that govern reasoning difficulty and ground our diagnostic evaluation.

### 2.1 Table Representations: Structured vs. Semi-structured

Structured tables. Structured tables adhere to fixed schemas. Each column encodes a predefined attribute and each row conforms to the schema. This consistency enables efficient storage, precise querying, and symbolic approaches such as NL2SQL (Ramakrishnan & Gehrke, 2003). However, most NL2SQL datasets assume idealized schemas, overlooking real-world issues such as missing attributes or misaligned semantics. For instance, a query about the ‘standing of Alice’ becomes invalid if that information is embedded in free text rather than in a dedicated column (Figure [1](https://arxiv.org/html/2509.22983v1#S1.F1)).

Semi-structured tables.
Semi-structured tables relax these constraints. Columns may be merged or repurposed (e.g., combining grade and standing), rows may summarize entities, and cells may contain long-form text, lists, or multimodal content. Web tables (Wang et al., 2024) and spreadsheets (Ma et al., 2024) exemplify this format. While they capture rich real-world information, their irregular structure complicates parsing, schema alignment, and reasoning.

### 2.2 Core Dimensions of Reasoning Difficulty

Prior studies have shown that representational aspects such as table length (Chen et al., 2024), multi-table reasoning (Wu et al., 2025a), and query complexity (Zhu et al., 2025) substantially affect performance. Yet, these insights remain fragmented across benchmarks, and no framework has systematically examined their interaction with representation. We close this gap by analyzing four core dimensions: *table size*, *table joins*, *query complexity*, and *schema quality*.
By varying these conditions under both structured and semi-structured representations, we disentangle their individual contributions and analyze robustness.

Table size. Large tables strain token windows and increase attention cost quadratically, leading to truncation, sparse supervision, and difficulty locating relevant rows Liu et al. (2023); Hsieh et al. (2024). LLMs also approximate numbers and struggle with exact comparisons across many values, causing errors in factual and financial QA Chen et al. (2019); Zhu et al. (2021). In this work, we categorize *short tables* as under 100 rows and *long tables* as over 100 rows.

Table joins.
Joins require aligning schemas and linking rows across multiple tables. This introduces challenges of ambiguous keys, irrelevant columns/rows and compositional reasoning. Semi-structured representations exacerbate these issues since explicit keys are often absent. We vary the number of tables per query and the presence of explicit key constraints to probe join difficulty.

Query complexity.
Queries range from simple lookups to multi-step reasoning involving aggregation, filtering, comparisons, and arithmetic. Existing datasets often target narrow slices (e.g., SQL lookups (Zhong et al., 2017), logical forms over web tables (Pasupat & Liang, 2015), or discrete reasoning (Zhu et al., 2021)). It remains unclear how methods scale from lookups to multi-step reasoning, particularly across table representations.

Schema quality.
Schema encodes column names, data types, keys, and relationships. While curated databases maintain consistent schemas, real-world tables often have missing headers, inconsistent formatting, and absent metadata. Such noise can severely affect symbolic methods.

Together, these factors define the major dimensions of Table QA difficulty. By systematically varying them under both structured and semi-structured settings, we disentangle their individual contributions and assess model robustness.

## 3 Fine-grained Diagnostic Benchmark

To systematically study the impact of table representations and disentangle the effects of the four factors outlined in Section [2](https://arxiv.org/html/2509.22983v1#S2), we construct a fine-grained diagnostic benchmark. Our design follows two guiding principles: (1) Representation pairing. Comparing structured and semi-structured Table QA requires identical semantics; otherwise, differences may reflect content mismatches. We achieve this by transforming structured tables into semi-structured ones via a controlled verbalization strategy. (2) Factor isolation. We partition existing benchmarks into controlled subsets that vary along the core factors. This allows us to assess each factor’s impact independently.

### 3.1 Verbalization Pipeline

Given a QA pair over a structured table, our verbalization pipeline produces a semi-structured variant of the same table. Each question is thus paired with two versions of its table: structured and semi-structured.
The pipeline consists of three steps, illustrated in Figure [2](https://arxiv.org/html/2509.22983v1#S3.F2):

- 1.
Column selection. We first determine which columns to verbalize into free text. GPT-4o identifies suitable candidate columns, from which we sample *random* combinations to introduce diversity across instances. We experimented with three alternative selection strategies (Appendix [B](https://arxiv.org/html/2509.22983v1#A2)) and found comparable performance trends. We, hence, adopt the random strategy as the default.
- 2.
Template construction. For the selected columns, we generate natural language templates conditioned on the table schema, into which column values are inserted to form descriptive sentences. To reduce annotation cost, GPT-4o produces candidate templates, which are lightly corrected for errors. For each column combination, we create five diverse templates to increase variation.
- 3.
Serialization.
Finally, templates are instantiated with table values, merging verbalized content into a single free-text column while removing the original structured columns. The resulting semi-structured table mirrors the semantics of the original but encodes them in a text-heavy, less rigid format. Both versions are retained for side-by-side evaluation.

Figure: Figure 2: Verbalization pipeline for transforming structured tables into semi-structured representations while preserving semantics.
Refer to caption: x2.png

### 3.2 Diagnostic Splits

To isolate the effects of representational and structural factors, we partition benchmarks into controlled subsets rather than treating them as monolithic. Each split targets one of four dimensions: table size, number of tables, query complexity, and schema quality. This design allows us to identify settings where different paradigms succeed or fail and provides a fine-grained view of robustness not captured by benchmark-level scores.

We construct diagnostic splits from three complementary benchmarks:
1) BIRD (Li et al., 2023): a large-scale, curated dataset with clean schemas and reliable evaluation protocols, representing the best-case setting for structured Table QA.
2) MMQA (Wu et al., 2025a): a benchmark explicitly designed for multi-table reasoning, where questions require retrieving and integrating evidence across multiple relational tables.
3) TableEval (Zhu et al., 2025): a collection of real-world web tables with noisy schemas, incomplete metadata, and inconsistent formatting, reflecting practical deployment challenges. Together, these datasets span clean to noisy conditions and single- to multi-table reasoning, providing a broad testbed for our diagnostic evaluation. Table [1](https://arxiv.org/html/2509.22983v1#S3.T1) summarizes the resulting fine-grained diagnostic splits.

**Table 1: Diagnostic splits for factors: table length, no. of tables, query complexity, schema quality.**
| Subset ID | Data Source | Table Joins | Length | Query Complexity | Schema | #Samples |
| --- | --- | --- | --- | --- | --- | --- |
| S1 | BIRD | ✗ | short | lookup | clean | 95 |
| S2 | TableEval | ✗ | short | lookup | incomplete | 169 |
| S3 | BIRD | ✗ | short | compositional reasoning | clean | 435 |
| S4 | BIRD | ✗ | long | lookup | clean | 78 |
| S5 | BIRD | ✗ | long | compositional reasoning | clean | 258 |
| M1 | MMQA | ✓ | short | compositional reasoning | clean | 427 |
| M2 | MMQA | ✓ | long | compositional reasoning | clean | 166 |

## 4 Experimental Setup

We now describe our evaluation protocol for isolating representation effects, including research questions, baselines, and metrics.

### 4.1 Research Questions

We organize our comparison across method families around five key questions:
RQ1. How does performance vary across paradigms when comparing structured and semi-structured inputs with identical content?
RQ2. How do methods scale from short ($<$100 rows) to long ($\geq$100 rows) tables?
RQ3. How do multi-table alignment and join operations affect performance?
RQ4. How does performance change from simple lookups to multi-step compositional queries (filters, aggregations, joins, comparisons)?
RQ5. How robust are methods under noisy or incomplete schemas?

### 4.2 Baselines and Setups

We evaluate three method families by selecting representative methods from each family.

- •
LLMs. We use GPT-4o(^1^11gpt-4o-2024-11-20) (Hurst et al., 2024), Gemini-2.5-flash(^2^22gemini-2.5-flash) (Anil et al., 2023), and Qwen3(^3^33qwen3-235b-a22b-thinking-2507)(Yang et al., 2025) as baselines for directly predicting the answer without any SQL execution.
The detailed prompts are shown in Appendix [A](https://arxiv.org/html/2509.22983v1#A1).
- •
NL2SQL methods.
We include two approaches that generate SQL queries. a) LLM-NL2SQL baseline: a two-stage pipeline where an LLM generates SQL from the question and schema, executed via SQLite. We provide schema information (tables, columns, keys) and a small set of example values per column via value previews. The detailed prompts are shown in Appendix [A](https://arxiv.org/html/2509.22983v1#A1). b) XiYan (Gao et al., 2024): a recent framework that improves SQL generation via a multi-generator ensemble design. It includes modules for schema linking via column/value retrieval, candidate generation with diverse LLM-based generators, and candidate selection/ranking before execution.
We use GPT-4o as backbone across generators and rankers for fair comparison.
- •
Hybrid methods.
We consider two hybrid approaches that combine SQL-based retrieval with LLM reasoning.
a) H-STAR (Abhyankar et al., 2025): a hybrid method that first performs table extraction (relevant columns/rows via SQL and LLM filtering), then routes numeric/aggregation tasks to SQL and relational/descriptive reasoning to the LLM. This allows H-STAR to combine precise symbolic computation with flexible natural-language inference.
b) Weaver  (Khoja et al., 2025): executes a stepwise workflow assigning operations to SQL or LLM, feeding intermediate tables back into the plan for integrated reasoning. This tight integration allows it to combine the precision of SQL with the flexibility of LLMs, improving robustness on complex queries that require both symbolic computation and semantic interpretation.

### 4.3 Evaluation

Previous work (Deng et al., 2022; Wu et al., 2025a; Zhu et al., 2021) has largely relied on traditional metrics such as Exact Match (EM) and Partial Match (PM).
However, these metrics suffer from well-known limitations: they depend on strict string-level comparison with the gold answer. Consequently, they penalize semantically correct predictions due to paraphrasing, formatting, or minor rounding differences. To address these limitations, we adopt LLMs as judges, a strategy shown to correlate strongly with human evaluation in a variety of reasoning tasks (Maekawa et al., 2025). Concretely, GPT-4o is given both the gold answer and the model’s prediction and prompted to determine semantic correctness. This evaluator tolerates surface variation while enforcing strict semantic equivalence. Unless stated otherwise, all results reported follow this evaluation protocol.

## 5 Results and Analysis

### 5.1 RQ1: Model Performance across Paradigms

We first examine how model performance varies across paradigms when comparing structured and semi-structured inputs with identical content, using weighted averages over all diagnostic splits.

NL2SQL models achieve the highest accuracy on structured tables, while hybrid models perform best on semi-structured tables.
Table [2](https://arxiv.org/html/2509.22983v1#S5.T2) shows that input representation has a substantial and consistent effect on performance, with semi-structured inputs posing greater challenges across the methods. On structured inputs, SQL-based methods outperform both hybrids and LLMs, reflecting their reliance on clean, explicit schemas. On semi-structured inputs, this ranking reverses: hybrids lead, LLMs degrade modestly. SQL methods suffer dramatic drops since irregular or implicit structures disrupt symbolic execution.

LLMs are the most robust to representation changes, due to their training on free-text, though they do not attain peak accuracy in either setting. These trends highlight the central role of representation in shaping model performance.

**Table 2: Comparison of model paradigms on structured vs. semi-structured tables (RQ1). Weighted averages across diagnostic splits highlight the strong impact of input representation. Bold denotes the best, italic the second-best result within each block.**
| Model | Structured Acc. (%) | Semi-structured Acc. (%) | Drop (%) |
| --- | --- | --- | --- |
| GPT-4o | 45.37 | 41.93 | 3.44 |
| Gemini-2.5-flash | 52.07 | 50.78 | 1.29 |
| Qwen3-235B | 38.20 | 36.70 | 1.50 |
| LLM-NL2SQL | 69.14 | 38.65 | 30.49 |
| XiYan | 69.55 | 24.08 | 45.47 |
| H-STAR | 49.48 | 47.14 | 2.34 |
| Weaver | 62.19 | 57.70 | 4.49 |

### 5.2 RQ2: Impact of Table Size

We next examine how table size affects model performance across paradigms.
To isolate the effect of table length, we compare short tables S1 and S3 against long tables S4 and S5 (see Table [1](https://arxiv.org/html/2509.22983v1#S3.T1)).

Figure: Figure 3: Short tables vs. long tables (RQ2). All models struggle on long tables. LLMs are most sensitive, NL2SQL excels on short structured tables, and hybrids remain relatively stable.
Refer to caption: x3.png

NL2SQL pipelines scale best on structured data but collapse on semi-structured tables.
Table [3](https://arxiv.org/html/2509.22983v1#S5.F3) indicates that table size has a substantial effect on accuracy across all families. Performance declines monotonically as tables grow.
The rate of degradation, however, depends on the paradigm. NL2SQL methods perform best on short, structured tables and remain competitive on longer structured ones, leveraging execution engines that scale efficiently. However, semi-structured inputs expose their reliance on clean schemas. Accuracy drops sharply at scale (e.g., XiYan drops to 25.0%).

LLMs are highly sensitive to table length, whereas hybrids offer a balanced trade-off. LLMs perform reasonably on short tables but degrade substantially on long tables. For example, GPT-4o falls to 28.9% on structured and 27.1% on semi-structured data; with Gemini falling to $<$15%. Hybrid models, while less competitive on short structured tables, maintain relative robustness across table sizes, particularly on semi-structured data. For example, Weaver achieves 57.2% on short semi-structured tables and 35.4% on long ones, while H-STAR drops to $<$13% on long tables regardless of representation.

### 5.3 RQ3: Impact of Table Joins

To assess the effect of multi-table reasoning, we compare single-table subsets (S1–S5) with multi-table subsets (M1–M2).

NL2SQL models benefit from executable joins on structured tables but lose this advantage when schema signals are weak.
Table [4](https://arxiv.org/html/2509.22983v1#S5.F4) indicates that multi-table reasoning has a significant impact on performance, with effects varying by paradigm.
On structured multi-table inputs, NL2SQL models leverage joins to integrate information across tables. For instance, LLM-NL2SQL improves from 71.5% (single-table) to 82.3% (multi-table), outperforming GPT-4o by over 35 points. However, this advantage disappears under semi-structured conditions, where implicit structure hinders grounding, resulting in sharp performance drops.

LLMs are largely insensitive to join structure.
They show minimal differences between single- and multi-table inputs, reflecting limited ability to exploit relational dependencies. Gemini exhibits modest improvements on multi-table structured inputs, but gains are small relative to SQL-based methods. Hybrid models maintain relatively stable performance on structured multi-table inputs but currently offer limited support for multi-table reasoning under semi-structured conditions, constraining their effectiveness in realistic deployments.

Figure: Figure 4: Single- vs. multi-table (RQ3). NL2SQL benefits from structured joins but fails on semi-structured tables. LLMs are largely insensitive to joins, while hybrids lack multi-table support.
Refer to caption: x4.png

### 5.4 RQ4: Impact of Query Complexity

We assess how query complexity affects performance by comparing lookup (subsets S1, S4) and compositional (subsets S3, S5) queries. These subsets target single tables. To isolate impact of query complexity we exclude M1, M2 since they focus on multi-table settings.

NL2SQL models excel on structured queries but collapse under semi-structured conditions. Figure [5](https://arxiv.org/html/2509.22983v1#S5.F5) shows that NL2SQL models achieve high accuracy on structured compositional queries, confirming the advantage of executable SQL for complex reasoning. However, their performance drops sharply under semi-structured conditions, reflecting brittleness to representational noise.

LLMs perform well on lookup queries but struggle with multi-hop reasoning. They achieve  70% accuracy on simple lookups but degrade significantly on compositional queries, highlighting limitations in symbolic and multi-step reasoning without execution support.

Hybrid models occupy the middle ground. They experience moderate drops from lookup to compositional queries, balancing flexibility and precision. In particular, Weaver sometimes outperforms structured inputs on semi-structured lookup queries, suggesting that natural-language verbalizations can better align with LLM pretraining and improve reasoning.

Figure: Figure 5: Lookup vs. compositional queries (RQ4). Accuracy drops across all models on multi-hop queries. NL2SQL excels on structured but fails on semi-structured inputs. LLMs and hybrids also degrade, though hybrids often benefit from semi-structured lookup queries.
Refer to caption: x5.png

### 5.5 RQ5: Impact of Schema Quality

We assess the effect of schema quality by comparing clean-schema tables (S1) with incomplete-schema tables (S2). We focus on lookup queries to isolate schema effects.

NL2SQL models are highly sensitive to schema quality, while LLMs and hybrids are more robust.
Figure [6](https://arxiv.org/html/2509.22983v1#S5.F6) shows that NL2SQL performance drops under incomplete schemas, reflecting their dependence on well-defined metadata. In contrast, LLMs maintain relatively high accuracy on clean schemas and degrade only moderately on incomplete ones, leveraging surface patterns even in noisy tables. Hybrid models are more resilient to schema noise. H-STAR mitigates irregularities via row/column pruning and alternative table views. Weaver remains stable by automatically renaming columns during SQL execution, reducing schema mismatches.

Figure: Figure 6: Clean-schema vs. incomplete-schema with lookup queries (RQ5). NL2SQL degrades under schema noise, while LLMs and hybrids remain robust.
Refer to caption: x6.png

### 5.6 Case Study

We now present illustrative examples that capture the main failure modes when shifting from structured to semi-structured representations.
Figure [7](https://arxiv.org/html/2509.22983v1#S5.F7) shows a representative case from the “supplier” table, where the query asks for the top ten suppliers by account balance. On the structured version, NL2SQL methods (LLM-NL2SQL and XiYan) execute correctly by leveraging explicit fields such as “s_suppkey” and “s_acctbal”.
On the semi-structured version, however, these attributes are embedded into free-text descriptions. The same models either output incorrect results or raise execution errors due to missing column references, illustrating the brittleness of NL2SQL pipelines under representational shifts.
A similar failure is observed for the hybrid method Weaver, which also relies on SQL execution and cannot recover when key attributes are verbalized into text.
This example highlights the challenges that semi-structured representations pose for models that depend on explicit schema linking and executable queries. We also provide additional case studies in Appendix [D](https://arxiv.org/html/2509.22983v1#A4) that highlight scenarios where different model families diverge in behavior.

Figure: Figure 7: An illustrative case from the RQ1 experiments. A check mark or cross indicates that all models in the series answered correctly or incorrectly, respectively, while the warning symbol denotes mixed outcomes. Example also compares structured and semi-structured inputs.
Refer to caption: x7.png

## 6 Related Work

Answering questions over tables requires models to reason over information that can appear in both structured database-style formats (Li et al., 2023; Lei et al., 2025) and semi-structured natural-language-like representations (Chen et al., 2021; 2020). Approaches to this problem can be broadly grouped into three paradigms, including translating questions into SQL queries (Wang et al., 2025), directly reasoning over table representations with large language models (Wu et al., 2025b; Zhang et al., 2025b), and hybrid methods (Khoja et al., 2025; Abhyankar et al., 2025) that combine symbolic and neural reasoning.
Existing benchmarks (Zhong et al., 2017; Zhu et al., 2025; Wu et al., 2025a) for table QA typically categorize data by query type or task category, but pay little attention to representation factors such as table size, table joins, or schema quality together. Crucially, no prior work has isolated the effect of semi-structured representations
under the same information conditions.
Meanwhile, most analytical studies (Ashury-Tahan et al., 2025; Singha et al., 2023; Sui et al., 2024) evaluate a single family of methods in isolation,
without systematically comparing how different paradigms respond to representation shifts. Our work fills these gaps by conducting a controlled, same-information study of structured and semi-structured inputs, analyzing their interaction with key reasoning factors across multiple method families.

## 7 Conclusion

We performed a controlled, same-information comparison of structured and semi-structured table representations across LLM, NL2SQL, and hybrid paradigms using a fine-grained diagnostic dataset constructed via an information-preserving verbalization pipeline. Our findings confirm that representation has an first-order effect. NL2SQL attains peak accuracy on structured inputs yet is brittle under semi-structured or noisy schemas. LLMs are comparatively robust across formats but trade off peak accuracy. Hybrids mediate this trade-off and often lead under semi-structured inputs. Task factors consistently influence these trends. Long tables degrade all methods (most for LLMs), explicit joins benefit NL2SQL on structured data, multi-hop queries challenge all paradigms (especially NL2SQL on semi-structured inputs), and schema incompleteness severely impacts SQL-based systems. Overall, our results highlight the need for representation-aware benchmarks and systems, and for matching method families to data conditions in deployment.

## References

- Abhyankar et al. (2025)
Nikhil Abhyankar, Vivek Gupta, Dan Roth, and Chandan K. Reddy.
H-STAR: llm-driven hybrid sql-text adaptive reasoning on tables.
In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), *Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Papers, Albuquerque, New Mexico, USA, April 29 - May 4, 2025*, pp. 8841–8863. Association for Computational Linguistics, 2025.
doi: 10.18653/V1/2025.NAACL-LONG.445.
URL [https://doi.org/10.18653/v1/2025.naacl-long.445](https://doi.org/10.18653/v1/2025.naacl-long.445).
- Anil et al. (2023)
Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul Ronald Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, George Tucker, Enrique Piqueras, Maxim Krikun, Iain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, and et al.
Gemini: A family of highly capable multimodal models.
*CoRR*, abs/2312.11805, 2023.
doi: 10.48550/ARXIV.2312.11805.
URL [https://doi.org/10.48550/arXiv.2312.11805](https://doi.org/10.48550/arXiv.2312.11805).
- Ashury-Tahan et al. (2025)
Shir Ashury-Tahan, Yifan Mai, Rajmohan C, Ariel Gera, Yotam Perlitz, Asaf Yehudai, Elron Bandel, Leshem Choshen, Eyal Shnarch, Percy Liang, and Michal Shmueli-Scheuer.
The mighty torr: A benchmark for table reasoning and robustness.
*CoRR*, abs/2502.19412, 2025.
doi: 10.48550/ARXIV.2502.19412.
URL [https://doi.org/10.48550/arXiv.2502.19412](https://doi.org/10.48550/arXiv.2502.19412).
- Chakrabarti et al. (2020)
Kaushik Chakrabarti, Zhimin Chen, Siamak Shakeri, and Guihong Cao.
Open domain question answering using web tables.
*CoRR*, abs/2001.03272, 2020.
URL [https://arxiv.org/abs/2001.03272](https://arxiv.org/abs/2001.03272).
- Chen et al. (2024)
Si-An Chen, Lesly Miculicich, Julian Eisenschlos, Zifeng Wang, Zilong Wang, Yanfei Chen, Yasuhisa Fujii, Hsuan-Tien Lin, Chen-Yu Lee, and Tomas Pfister.
Tablerag: Million-token table understanding with language models.
In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), *Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024*, 2024.
URL [http://papers.nips.cc/paper_files/paper/2024/hash/88dd7aa6979e352fda7c4952ca8eac59-Abstract-Conference.html](http://papers.nips.cc/paper_files/paper/2024/hash/88dd7aa6979e352fda7c4952ca8eac59-Abstract-Conference.html).
- Chen et al. (2019)
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang.
Tabfact: A large-scale dataset for table-based fact verification.
*arXiv preprint arXiv:1909.02164*, 2019.
- Chen et al. (2020)
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang.
Hybridqa: A dataset of multi-hop question answering over tabular and textual data.
In Trevor Cohn, Yulan He, and Yang Liu (eds.), *Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020*, volume EMNLP 2020 of *Findings of ACL*, pp. 1026–1036. Association for Computational Linguistics, 2020.
doi: 10.18653/V1/2020.FINDINGS-EMNLP.91.
URL [https://doi.org/10.18653/v1/2020.findings-emnlp.91](https://doi.org/10.18653/v1/2020.findings-emnlp.91).
- Chen et al. (2021)
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Kenneth Huang, Bryan R. Routledge, and William Yang Wang.
Finqa: A dataset of numerical reasoning over financial data.
In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021*, pp. 3697–3711. Association for Computational Linguistics, 2021.
doi: 10.18653/V1/2021.EMNLP-MAIN.300.
URL [https://doi.org/10.18653/v1/2021.emnlp-main.300](https://doi.org/10.18653/v1/2021.emnlp-main.300).
- Deng et al. (2022)
Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu.
TURL: table understanding through representation learning.
*SIGMOD Rec.*, 51(1):33–40, 2022.
doi: 10.1145/3542700.3542709.
URL [https://doi.org/10.1145/3542700.3542709](https://doi.org/10.1145/3542700.3542709).
- Dong & Lapata (2016)
Li Dong and Mirella Lapata.
Language to logical form with neural attention.
In *Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers*. The Association for Computer Linguistics, 2016.
doi: 10.18653/V1/P16-1004.
URL [https://doi.org/10.18653/v1/p16-1004](https://doi.org/10.18653/v1/p16-1004).
- Gao et al. (2024)
Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, et al.
A preview of xiyan-sql: A multi-generator ensemble framework for text-to-sql.
*arXiv preprint arXiv:2411.08599*, 2024.
- Ghosh et al. (2024)
Akash Ghosh, B. Venkata Sahith, Niloy Ganguly, Pawan Goyal, and Mayank Singh.
How robust are the tabular QA models for scientific tables? A study using customized dataset.
*CoRR*, abs/2404.00401, 2024.
doi: 10.48550/ARXIV.2404.00401.
URL [https://doi.org/10.48550/arXiv.2404.00401](https://doi.org/10.48550/arXiv.2404.00401).
- Hsieh et al. (2024)
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg.
Ruler: What’s the real context size of your long-context language models?
*arXiv preprint arXiv:2404.06654*, 2024.
- Hurst et al. (2024)
Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Baek, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang, Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon
Eastman, Camillo Lugaresi, Carroll L. Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czarnecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, and Dane Sherburn.
Gpt-4o system card.
*CoRR*, abs/2410.21276, 2024.
doi: 10.48550/ARXIV.2410.21276.
URL [https://doi.org/10.48550/arXiv.2410.21276](https://doi.org/10.48550/arXiv.2410.21276).
- Iyyer et al. (2017)
Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang.
Search-based neural structured learning for sequential question answering.
In Regina Barzilay and Min-Yen Kan (eds.), *Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers*, pp. 1821–1831. Association for Computational Linguistics, 2017.
doi: 10.18653/V1/P17-1167.
URL [https://doi.org/10.18653/v1/P17-1167](https://doi.org/10.18653/v1/P17-1167).
- Khoja et al. (2025)
Rohit Khoja, Devanshu Gupta, Yanjie Fu, Dan Roth, and Vivek Gupta.
Weaver: Interweaving SQL and LLM for table reasoning.
*CoRR*, abs/2505.18961, 2025.
doi: 10.48550/ARXIV.2505.18961.
URL [https://doi.org/10.48550/arXiv.2505.18961](https://doi.org/10.48550/arXiv.2505.18961).
- Lei et al. (2025)
Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, and Tao Yu.
Spider 2.0: Evaluating language models on real-world enterprise text-to-sql workflows.
In *The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025*. OpenReview.net, 2025.
URL [https://openreview.net/forum?id=XmProj9cPs](https://openreview.net/forum?id=XmProj9cPs).
- Li et al. (2023)
Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chen-Chuan Chang, Fei Huang, Reynold Cheng, and Yongbin Li.
Can LLM already serve as A database interface? A big bench for large-scale database grounded text-to-sqls.
In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), *Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.
URL [http://papers.nips.cc/paper_files/paper/2023/hash/83fc8fab1710363050bbd1d4b8cc0021-Abstract-Datasets_and_Benchmarks.html](http://papers.nips.cc/paper_files/paper/2023/hash/83fc8fab1710363050bbd1d4b8cc0021-Abstract-Datasets_and_Benchmarks.html).
- Liu et al. (2023)
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang.
Lost in the middle: How language models use long contexts.
*arXiv preprint arXiv:2307.03172*, 2023.
- Liu et al. (2022)
Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou.
TAPEX: table pre-training via learning a neural SQL executor.
In *The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022*. OpenReview.net, 2022.
URL [https://openreview.net/forum?id=O50443AsCP](https://openreview.net/forum?id=O50443AsCP).
- Ma et al. (2024)
Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang.
Spreadsheetbench: Towards challenging real world spreadsheet manipulation.
In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), *Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024*, 2024.
URL [http://papers.nips.cc/paper_files/paper/2024/hash/ac840df270ac537dd74530a15c332684-Abstract-Datasets_and_Benchmarks_Track.html](http://papers.nips.cc/paper_files/paper/2024/hash/ac840df270ac537dd74530a15c332684-Abstract-Datasets_and_Benchmarks_Track.html).
- Maekawa et al. (2025)
Seiji Maekawa, Hayate Iso, and Nikita Bhutani.
Holistic reasoning with long-context LMs: A benchmark for database operations on massive textual data.
In *The Thirteenth International Conference on Learning Representations*, 2025.
URL [https://openreview.net/forum?id=5LXcoDtNyq](https://openreview.net/forum?id=5LXcoDtNyq).
- Pasupat & Liang (2015)
Panupong Pasupat and Percy Liang.
Compositional semantic parsing on semi-structured tables.
In *Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers*, pp. 1470–1480. The Association for Computer Linguistics, 2015.
doi: 10.3115/V1/P15-1142.
URL [https://doi.org/10.3115/v1/p15-1142](https://doi.org/10.3115/v1/p15-1142).
- Ramakrishnan & Gehrke (2003)
Raghu Ramakrishnan and Johannes Gehrke.
*Database management systems (3. ed.)*.
McGraw-Hill, 2003.
ISBN 978-0-07-115110-8.
- Singh et al. (2025)
Anshul Singh, Chris Biemann, and Jan Strich.
Mtabvqa: Evaluating multi-tabular reasoning of language models in visual space.
*CoRR*, abs/2506.11684, 2025.
doi: 10.48550/ARXIV.2506.11684.
URL [https://doi.org/10.48550/arXiv.2506.11684](https://doi.org/10.48550/arXiv.2506.11684).
- Singha et al. (2023)
Ananya Singha, José Cambronero, Sumit Gulwani, Vu Le, and Chris Parnin.
Tabular representation, noisy operators, and impacts on table structure understanding tasks in llms.
*CoRR*, abs/2310.10358, 2023.
doi: 10.48550/ARXIV.2310.10358.
URL [https://doi.org/10.48550/arXiv.2310.10358](https://doi.org/10.48550/arXiv.2310.10358).
- Sui et al. (2024)
Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang.
Table meets LLM: can large language models understand structured table data? A benchmark and empirical study.
In Luz Angelica Caudillo-Mata, Silvio Lattanzi, Andrés Muñoz Medina, Leman Akoglu, Aristides Gionis, and Sergei Vassilvitskii (eds.), *Proceedings of the 17th ACM International Conference on Web Search and Data Mining, WSDM 2024, Merida, Mexico, March 4-8, 2024*, pp. 645–654. ACM, 2024.
doi: 10.1145/3616855.3635752.
URL [https://doi.org/10.1145/3616855.3635752](https://doi.org/10.1145/3616855.3635752).
- Wang et al. (2024)
Maria Wang, Srinivas Sunkara, Gilles Baechler, Jason Lin, Yun Zhu, Fedir Zubach, Lei Shu, and Jindong Chen.
Webquest: A benchmark for multimodal QA on web page sequences.
*CoRR*, abs/2409.13711, 2024.
doi: 10.48550/ARXIV.2409.13711.
URL [https://doi.org/10.48550/arXiv.2409.13711](https://doi.org/10.48550/arXiv.2409.13711).
- Wang et al. (2025)
Tianshu Wang, Xiaoyang Chen, Hongyu Lin, Xianpei Han, Le Sun, Hao Wang, and Zhenyu Zeng.
Dbcopilot: Natural language querying over massive databases via schema routing.
In Alkis Simitsis, Bettina Kemme, Anna Queralt, Oscar Romero, and Petar Jovanovic (eds.), *Proceedings 28th International Conference on Extending Database Technology, EDBT 2025, Barcelona, Spain, March 25-28, 2025*, pp. 707–721. OpenProceedings.org, 2025.
doi: 10.48786/EDBT.2025.57.
URL [https://doi.org/10.48786/edbt.2025.57](https://doi.org/10.48786/edbt.2025.57).
- Wu et al. (2025a)
Jian Wu, Linyi Yang, Dongyuan Li, Yuliang Ji, Manabu Okumura, and Yue Zhang.
MMQA: evaluating llms with multi-table multi-hop complex questions.
In *The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025*. OpenReview.net, 2025a.
URL [https://openreview.net/forum?id=GGlpykXDCa](https://openreview.net/forum?id=GGlpykXDCa).
- Wu et al. (2025b)
Zhenhe Wu, Jian Yang, Jiaheng Liu, Xianjie Wu, Changzai Pan, Jie Zhang, Yu Zhao, Shuangyong Song, Yongxiang Li, and Zhoujun Li.
Table-r1: Region-based reinforcement learning for table understanding.
*CoRR*, abs/2505.12415, 2025b.
doi: 10.48550/ARXIV.2505.12415.
URL [https://doi.org/10.48550/arXiv.2505.12415](https://doi.org/10.48550/arXiv.2505.12415).
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jian Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu.
Qwen3 technical report.
*CoRR*, abs/2505.09388, 2025.
doi: 10.48550/ARXIV.2505.09388.
URL [https://doi.org/10.48550/arXiv.2505.09388](https://doi.org/10.48550/arXiv.2505.09388).
- Ye et al. (2023)
Yunhu Ye, Binyuan Hui, Min Yang, Binhua Li, Fei Huang, and Yongbin Li.
Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning.
In Hsin-Hsi Chen, Wei-Jou (Edward) Duh, Hen-Hsen Huang, Makoto P. Kato, Josiane Mothe, and Barbara Poblete (eds.), *Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023*, pp. 174–184. ACM, 2023.
doi: 10.1145/3539618.3591708.
URL [https://doi.org/10.1145/3539618.3591708](https://doi.org/10.1145/3539618.3591708).
- Zhang et al. (2025a)
Junwen Zhang, Pu Chen, and Yin Zhang.
Tablemoe: Neuro-symbolic routing for structured expert reasoning in multimodal table understanding.
*CoRR*, abs/2506.21393, 2025a.
doi: 10.48550/ARXIV.2506.21393.
URL [https://doi.org/10.48550/arXiv.2506.21393](https://doi.org/10.48550/arXiv.2506.21393).
- Zhang et al. (2024)
Siyue Zhang, Anh Tuan Luu, and Chen Zhao.
Syntqa: Synergistic table-based question answering via mixture of text-to-sql and E2E TQA.
In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), *Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024*, pp. 2352–2364. Association for Computational Linguistics, 2024.
doi: 10.18653/V1/2024.FINDINGS-EMNLP.131.
URL [https://doi.org/10.18653/v1/2024.findings-emnlp.131](https://doi.org/10.18653/v1/2024.findings-emnlp.131).
- Zhang et al. (2025b)
Xiaokang Zhang, Sijia Luo, Bohan Zhang, Zeyao Ma, Jing Zhang, Yang Li, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, Jifan Yu, Shu Zhao, Juanzi Li, and Jie Tang.
Tablellm: Enabling tabular data manipulation by llms in real office usage scenarios.
In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), *Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025*, pp. 10315–10344. Association for Computational Linguistics, 2025b.
URL [https://aclanthology.org/2025.findings-acl.538/](https://aclanthology.org/2025.findings-acl.538/).
- Zhang et al. (2025c)
Xuanliang Zhang, Dingzirui Wang, Keyan Xu, Qingfu Zhu, and Wanxiang Che.
Rot: Enhancing table reasoning with iterative row-wise traversals.
*CoRR*, abs/2505.15110, 2025c.
doi: 10.48550/ARXIV.2505.15110.
URL [https://doi.org/10.48550/arXiv.2505.15110](https://doi.org/10.48550/arXiv.2505.15110).
- Zhong et al. (2017)
Victor Zhong, Caiming Xiong, and Richard Socher.
Seq2sql: Generating structured queries from natural language using reinforcement learning.
*CoRR*, abs/1709.00103, 2017.
URL [http://arxiv.org/abs/1709.00103](http://arxiv.org/abs/1709.00103).
- Zhu et al. (2021)
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua.
TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance.
In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021*, pp. 3277–3287. Association for Computational Linguistics, 2021.
doi: 10.18653/V1/2021.ACL-LONG.254.
URL [https://doi.org/10.18653/v1/2021.acl-long.254](https://doi.org/10.18653/v1/2021.acl-long.254).
- Zhu et al. (2025)
Junnan Zhu, Jingyi Wang, Bohan Yu, Xiaoyu Wu, Junbo Li, Lei Wang, and Nan Xu.
Tableeval: A real-world benchmark for complex, multilingual, and multi-structured table question answering.
*CoRR*, abs/2506.03949, 2025.
doi: 10.48550/ARXIV.2506.03949.
URL [https://doi.org/10.48550/arXiv.2506.03949](https://doi.org/10.48550/arXiv.2506.03949).

## Appendix A Prompts

## Appendix B Effect of Column Selection Settings

We test whether our conclusions are sensitive to how content is chosen for verbalization.
On the S3 split, we keep the questions and gold answers fixed and vary *only* the selection setting. We consider four settings:
*query-related*, where only the columns directly referenced in the question are verbalized;
*non–query-related*, where only the columns not referenced in the question are verbalized;
*random*, where GPT-4o is prompted to randomly select a subset of columns to verbalize; and
*full*, where all columns in the table are verbalized into free-text. For the *full* strategy, directly verbalizing all rows would collapse the table into a single unstructured text block. To avoid this, we randomly verbalize only an $\alpha$% subset of rows, while keeping the remaining rows in their original structured form. The verbalized rows are then dropped, and the resulting semi-structured table contains both the structured part and the generated free-text column. We report
$\alpha=50$ and $\alpha=10$ below.

For tables with fewer columns than the target verbalization count, we discard those cases; all results are computed on the intersection of examples available under every selection setting to ensure strict comparability.

**Table 3: Effect of verbalization strategy on the S3. We vary only the selection policy while keeping questions and gold answers fixed.**
| Settings | Acc (%) |
| --- | --- |
| *Column verbalization* |  |
| Random (1 column) | 44.68 |
| Random (3 columns) | 43.97 |
| Random (6 columns) | 42.20 |
| Random (unconstrained count) | 42.20 |
| *Query-aware* |  |
| Query-Related | 46.45 |
| Non–Query-Related | 43.26 |
| *Full verbalization* |  |
| $\alpha{=}50\%$ rows | 43.97 |
| $\alpha{=}10\%$ rows | 42.91 |

#### Findings.

(1) Across selection settings, GPT-4o accuracy spans 42.20%–46.45%, indicating low sensitivity to the specific choice.
(2) *Column verbalization* exhibits a clear downward trend as the number of verbalized columns increases, with the unconstrained/random setting producing the lowest score (42.20%).
(3) *Query-aware* selection is the only variant that yields a consistent gain: verbalizing only the query-relevant columns gives the best result (46.45%), while excluding them hurts (43.26%).
(4) *Full verbalization* with $\alpha=50\%$ or $10\%$ behaves similarly to random column verbalization and offers no additional advantage. Given the comparable accuracy across all settings, we adopt *random column selection* as the default to maximize coverage and comparability in the main experiments.

## Appendix C Fine-grained Results

Beyond the overall trends summarized in Section [5](https://arxiv.org/html/2509.22983v1#S5), we highlight several noteworthy findings from the fine-grained results in Table [4](https://arxiv.org/html/2509.22983v1#A3.T4):

- •
NL2SQL models remain advantageous at scale.
Even with semi-structured inputs, NL2SQL sometimes outperforms LLMs and hybrids on long tables
(e.g., LLM-NL2SQL $41.1\%$ vs. Weaver $35.4\%$ and GPT-4o $27.1\%$ on S4),
showing that SQL execution can still offset representation mismatch when contexts are large.
- •
NL2SQL excels with joins.
On multi-table structured inputs, LLM-NL2SQL rises from $71.5\%$ (single-table) to $82.3\%$, exceeding GPT-4o by more than $35$ points.
This highlights that executable joins are a unique strength of NL2SQL, whereas GPT-4o is largely insensitive to join structure.
- •
Verbalization can improve simple lookup queries.
For simple lookup queries with clean schemas, aligning headers and content with natural language boosts hybrid models
(e.g., Weaver $\sim\!+3$ points on S1), demonstrating that verbalization not only enables semi-structured evaluation
but can also yield accuracy gains.

**Table 4: Structured vs. Semi-Structured accuracy across all splits (S1–S5 from BIRD/TableEval, M1–M2 from MMQA). $\Delta$ (pp) = semi-structured minus structured. Positive = improvement, negative = drop.**
| Split | Model | Structured Acc. (%) | Semi-Structured Acc. (%) | $\Delta$ (pp) |
| --- | --- | --- | --- | --- |
| S1 | GPT-4o | 89.47 | 90.53 | +1.06 |
|  | Gemini-2.5-flash | 90.53 | 88.42 | -2.11 |
|  | Qwen3-235B | 85.26 | 89.47 | +4.21 |
|  | LLM-NL2SQL | 87.37 | 62.11 | -25.26 |
|  | XiYan | 78.95 | 26.32 | -52.63 |
|  | H-STAR | 75.79 | 80.00 | +4.21 |
|  | Weaver | 88.42 | 91.58 | +3.16 |
| S2 | GPT-4o | 71.60 | 63.31 | -8.29 |
|  | Gemini-2.5-flash | 73.96 | 67.46 | -6.50 |
|  | Qwen3-235B | 66.86 | 57.40 | -9.46 |
|  | LLM-NL2SQL | 24.26 | 15.98 | -8.28 |
|  | XiYan | 30.18 | 18.94 | -11.24 |
|  | H-STAR | 88.17 | 81.66 | -6.51 |
|  | Weaver | 86.39 | 76.33 | -10.06 |
| S3 | GPT-4o | 46.90 | 43.58 | -3.32 |
|  | Gemini-2.5-flash | 61.24 | 63.07 | +1.83 |
|  | Qwen3-235B | 37.62 | 38.07 | +0.45 |
|  | LLM-NL2SQL | 74.77 | 48.18 | -26.59 |
|  | XiYan | 73.79 | 34.94 | -38.85 |
|  | H-STAR | 48.74 | 47.82 | -0.92 |
|  | Weaver | 57.47 | 49.66 | -7.81 |
| S4 | GPT-4o | 53.85 | 48.72 | -5.13 |
|  | Gemini-2.5-flash | 46.84 | 44.30 | -2.54 |
|  | Qwen3-235B | 45.57 | 44.30 | -1.27 |
|  | LLM-NL2SQL | 64.10 | 51.28 | -12.82 |
|  | XiYan | 55.13 | 16.67 | -38.46 |
|  | H-STAR | 28.21 | 23.08 | -5.13 |
|  | Weaver | 53.85 | 55.13 | +1.28 |
| S5 | GPT-4o | 21.32 | 20.54 | -0.78 |
|  | Gemini-2.5-flash | 5.41 | 5.81 | +0.40 |
|  | Qwen3-235B | 18.92 | 17.05 | -1.87 |
|  | LLM-NL2SQL | 62.55 | 37.98 | -24.57 |
|  | XiYan | 62.79 | 27.52 | -35.27 |
|  | H-STAR | 8.11 | 8.14 | +0.03 |
|  | Weaver | 31.78 | 29.46 | -2.32 |
| M1 | GPT-4o | 53.40 | 50.82 | -2.58 |
|  | Gemini-2.5-flash | 75.88 | 71.19 | -4.69 |
|  | Qwen3-235B | 41.45 | 41.22 | -0.23 |
|  | LLM-NL2SQL | 83.37 | 33.96 | -49.41 |
|  | XiYan | 84.54 | 18.97 | -65.57 |
| M2 | GPT-4o | 19.88 | 16.87 | -3.01 |
|  | Gemini-2.5-flash | 21.69 | 21.69 | 0.00 |
|  | Qwen3-235B | 15.66 | 13.25 | -2.41 |
|  | LLM-NL2SQL | 71.69 | 30.72 | -40.97 |
|  | XiYan | 76.51 | 19.88 | -56.63 |

## Appendix D Additional Case Studies

We present additional case studies to further illustrate the distinctive behaviors of different methods.

### D.1 Model Behavior on Long Tables

Figure [8](https://arxiv.org/html/2509.22983v1#A4.F8) shows a long user profile table with hundreds of rows and a query “What is the id of the youngest user?”. LLM-NL2SQL correctly answers the query on the structured version, which exposes the Age column and allows SQL execution to identify the minimum value (id = 7828).
In contrast, LLMs (e.g., GPT-4o) fail to reliably scan the full column and return arbitrary large ids (e.g., 35942).
Hybrid methods behave similarly. Weaver returns an arbitrary large id (e.g., 10215), while H-STAR concludes that the task is impossible because of numerous NaN values.
This example illustrates that while NL2SQL with execution remains competitive on structured long tables, both LLMs and current hybrid methods are highly sensitive to table length and may produce spurious results when the input context becomes too large.

Figure: Figure 8: Case study: NL2SQL correctly finds the youngest user on a long structured table, while LLMs and hybrids produce spurious results.
Refer to caption: x8.png

### D.2 Model Behavior on Incomplete Schemas

Figure [9](https://arxiv.org/html/2509.22983v1#A4.F9) presents two representative examples illustrating the impact of incomplete schemas on model performance. In the first example (left, RoboTaxi fare table), the Rest Day cell for 17:00–20:00 is left blank, although by business rules it should inherit the weekday rate (7 RMB). In this case, LLMs succeed by leveraging contextual cues to infer the missing value, while NL2SQL and hybrid pipelines fail due to their inability to capture implicit inheritance. The second example (right, biomedical citation table) involves a clinical reference for Alectinib that is embedded only within a free-text description column.
In this case, NL2SQL methods return empty outputs, and LLMs hallucinate incorrect citations. In contrast, both hybrid methods (H-STAR and Weaver) successfully locate and extract the correct reference, demonstrating their ability to combine schema-guided retrieval with flexible reasoning over unstructured content. These examples illustrate that while NL2SQL pipelines are highly sensitive to missing schema entries, LLMs can sometimes compensate via contextual reasoning, and hybrid approaches offer a middle ground capable of handling certain forms of schema incompleteness when textual cues are present.

Figure: Figure 9: Incomplete schema example. LLMs can infer missing values while NL2SQL and hybrid methods fails (left). Hybrid methods can correctly extract the information while NLSQL and LLMs fail (right).
Refer to caption: x9.png

## Appendix E Use of Large Language Models

We made limited use of large language models—specifically ChatGPT and GitHub Copilot—for language polishing and routine coding assistance (autocompletion, refactoring, and boilerplate generation). LLMs were not used to generate research ideas, experimental results, or claims. No non-public or sensitive data was shared with these tools. All suggested text and code were independently reviewed, tested, and edited by the authors, who remain responsible for the content.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: See W1.
soundness: 3
strengths: S1. The paper explores various facets of the table representation, which are reasonable and comprehensive directions.
S2. The paper is generally well structured and organized.
summary: The authors perform a controlled study to examine how table representation affects model performance. The diagnostic benchmark they created has structured tables, but also has semi-structured variants, which are produced by selecting columns to change into free-text, reconstructing templates and serialize the updated table. They look into five key factors: structured vs semi-structured inputs, short vs long tables, multi-table alignment vs join operation tasks, simple lookups vs multi-step compositional queries and noisy schemas. The method families they test on are LLM-based, NL2SQL methods and hybrid methods. They generally find NL2SQL models to be better with structured tables and hybrid models to be more balanced with robustness across formats and peak accuracy.
weaknesses: W1. Some of the claims don't seem to match the actual results. For instance, on lines 284-286, the claim is that "LLMs degrade modestly" and "hybrids lead" for RQ1. However, based on Table 2, the drop % for LLM methods is similar or even much lower than the Hybrid ones. It doesn't seem that hybrid models lead in that case.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: See the Weaknesses above.
soundness: 3
strengths: - The study introduces a novel, well-controlled evaluation framework that isolates representation effects with strong methodological rigor. It provides comprehensive empirical evidence across datasets (BIRD, MMQA, TableEval), method families, and conditions, revealing consistent, interpretable trends. 

- The verbalization pipeline and diagnostic splits are technically elegant and reproducible, offering a valuable benchmark for future Table QA research. The analyses are thorough, uncovering key trade-offs and failure cases that inform model and benchmark design.
summary: This paper presents a controlled study on how table representation, structured vs. semi-structured, affects model performance in Table Question Answering (Table QA). Using a verbalization pipeline that preserves table semantics, the authors compare NL2SQL, LLM-based, and hybrid approaches across varied settings of table complexity and query difficulty. Results show that structured inputs favor symbolic methods, semi-structured forms favor LLMs, and hybrid systems achieve the best balance, highlighting how representation choice fundamentally shapes reasoning accuracy and robustness.
weaknesses: The paper is largely empirical, offering extensive measurements but limited analytical explanation for why representational shifts yield performance divergence: What internal representations or attention dynamics contribute to this behavior? While the benchmark is comprehensive, it focuses mainly on static accuracy; are there analyses on error propagation, confidence calibration, or reasoning trace fidelity under different table formats?

The verbalization pipeline relies on LLM paraphrasing—how consistent is this process across seeds, and how might linguistic variation bias outcomes? The study primarily evaluates medium-scale models and standard datasets; can the results generalize to larger instruction-tuned LLMs or real-world multi-table enterprise data?

Finally, while comparisons are broad, statistical significance testing and variance reporting across multiple runs are limited. Could more rigorous uncertainty quantification strengthen empirical claims?

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1.  **Additional Experiments on Schema Restoration**: Consider incorporating experiments that utilize methods like NameGuess[1] to restore damaged schemas, evaluating how such restoration impacts model performance.  
2.  **Table Reconstruction Prior to Testing**: It may be beneficial to experiment with table reconstruction as a preprocessing step before applying the evaluation methods.  
3.  **Mitigating LLM Judge Bias**: To address potential biases introduced by LLM-based evaluation, it is advisable to sample a subset of data for analysis—comparing LLM judgments with human scoring—to enhance the validity of the evaluation approach.

[1] Zhang J, Shen Z, Srinivasan B, et al. NameGuess: Column name expansion for tabular data[J]. arXiv preprint arXiv:2310.13196, 2023.
soundness: 3
strengths: 1. **Substantial Innovativeness**: The impact of table representation formats on model performance is an important yet systematically understudied topic.  
2. **Comprehensive Evaluation**: The assessment covers multiple approaches—including NL2SQL, LLMs, and hybrid methods—across various datasets and evaluation dimensions.
summary: This paper presents the first controlled experimental study on a key yet underexplored problem in Table QA: the impact of table representation on model performance. It provides a comparative analysis of NL2SQL, LLM, and hybrid methods across different representation formats. The research is rigorously designed and supported by extensive experiments.
weaknesses: 1.  **Limited Table Representations Explored**: Although this study systematically investigates the impact of table representation formats on model performance, the variety of representations examined remains limited. Other transformations—such as row/column permutation, transposition, and modality conversion (e.g., to visual representations)—could be considered.
2.  **Room for Enhanced Table Diversity**: The current definition of short and long tables based solely on row count could be expanded. For instance, tables with very high column counts—common in fields like medical research—represent another important dimension of diversity.
3.  **Over-Simplified Table Transformation Methods**: Even when considering only structured and semi-structured tables, the transformation methods applied (e.g., textual serialization and cell merging) remain relatively basic. More advanced structural transformations could be explored.
4.  **Potential Data Insufficiency in Some Experiments**: The sample size in certain experiments (e.g., S4) may be insufficient for robust conclusions.
5.  **Recommendation: From Analysis to Decision Support**: Beyond data analysis, we recommend developing a decision tree to guide practitioners in selecting the most suitable method for specific scenarios.

---

confidence: 2
contribution: 2
initial_rating: 4
presentation: 2
questions: see weakness
soundness: 2
strengths: 1  The pipeline that generates semantically equivalent structured and semi-structured table pairs is reasonable.

2  The four-dimensional diagnostic framework (table size, joins, query complexity, schema quality) provides good insights into how different factors interact with representation formats.

3  The case studies and error analysis provide concrete examples
summary: This paper presents the controlled study examining how table representation (structured vs. semi-structured formats) affects Table QA performance across different modeling paradigms. The paper introduces a pipeline that generates paired structured and semi-structured tables with identical content, enabling direct comparison of NL2SQL, LLM-based, and hybrid methods under controlled conditions. The study reveals consistent trade-offs.
weaknesses: 1 The verbalization process may not fully capture the complexity of naturally occurring semi-structured tables. Real-world tables often exhibit more irregular patterns and noise than the systematically generated semi-structured versions used in this study.

2 The paper provides limited theoretical analysis of why different representations affect model performance. More fundamental insights into the cognitive or architectural reasons for these differences would strengthen the contribution.

3 The paper could discuss the scale limitations in evaluation, for example, long tables with more than 100 rows.


Decision: Accept (Poster)