Beyond Text-Only: Towards Multimodal Table Retrieval in Open-World
Authors: Da Li
Keping Bi
Jiafeng Guo
Wei Yuan
Fan Yang
Tingting Gao
Xueqi Cheng
Venue: ICLR 2026 Poster (2026)
Keywords: Table Retrieval
MultiModal Retrieval
Primary area: datasets and benchmarks

Abstract: Open-domain table retrieval aims to retrieve semantically relevant structured tables from a large-scale corpus in response to natural language queries. Unlike unstructured text, tables store information not only through their textual or numerical content but also through their structural properties, including hierarchical relationships between headers and cells, as well as complex spatial arrangements within the table layout. Existing methods predominantly treat table retrieval as a variant of text retrieval. They struggle to accurately preserve the rich structural semantics of diverse table formats during text serialization. Existing methods typically flatten tables into linear text sequences through row-wise or column-wise serialization, inadvertently discarding structural information. The problem becomes particularly acute when processing complex table layouts containing merged cells or irregular alignments, ultimately compromising retrieval performance. Moreover, existing methods struggle to handle embedded images within table cells. Notably, visual representations inherently preserve both structural and content information while being format-agnostic. This insight motivates our exploration of image-based table retrieval, as it can naturally overcome the challenges faced by existing methods. In this paper, we introduce TaR-ViR (Table Retrieval via Visual Representations), a new benchmark that reformulates table retrieval as a multimodal task by treating tables as images. Experiments on TaR-ViR show that this paradigm shift achieved more effective and efficient retrieval performance. Crucially, it eliminates the need for error-prone text conversion, enabling scalable collection and utilization of open-world tables. Our data are available at https://github.com/Trustworthy-Information-Access/Tab-ViR.

Published as a conference paper at ICLR 2026
BEYOND TEXT-ONLY: TOWARDS MULTIMODAL TA- BLE RETRIEVAL IN OPEN- WORLD
Da Li Keping Bi B233333 Jiafeng Guo 1.233 Guo1.233
Wei Yuan"
Fang Yang' Tingting Gao
Xueqi Cheng! 1.2.3
State Key Laboratory of AI Safety
2 Institute of Computing Technology, Chinese Academy of Sciences
3 University of Chinese Academy of Sciences 4 Kuaishou Technology (lida21s, bikeping. guojiafeng, exq] @ict.ac.cn (yuanwei05, yangfan) @ kuaishou.com
ABSTRACT
Open-domain table retrieval aims to retrieve semantically relevant structured ta- bles from large-scale corpus in response to natural language queries. Unlike unstructured text, tables store information not only through their textual or nu- merical content but also through their structural properties, including hierarchical relationships between headers and cells, as well as complex spatial arrangements within the table layout. Existing methods predominantly treat table retrieval a variant text retrieval. They struggle to accurately preserve the rich structural semanties of diverse table formats during text serialization. Existing methods typi- cally flatten tables into linear text sequences through row-wise or column- se- rialization, inadvertently discarding structural information. The problem becomes particularly acute when processing complex table layouts containing merged cells or irregular alignments, ultimately compromising retrieval performance. More- over, existing methods struggle to handle embedded images within table cells. Notably, visual representations inherently preserve both structural and content in- formation while being format-agnostic This insight motivates our exploration of image-based table retrieval, as it can naturally overcome the challenges faced by existing methods. In this paper, introduce TaR-ViR (Table Retrieval via Visual Representations), a new benchmark that reformulates table retrieval as a multimodal task by treating tables as images. Experiments TaR-ViR show that this paradigm shift achieved more effective and efficient retrieval performance. Crucially, it eliminates the need for error-prone text conversion, enabling scal- able collection and utilization of open-world tables. Our data are available at https://github.com wrruttoorthy--foooaaiooooo
INTRODUCTION
Open-domain table retrieval has emerged critical challenge in information systems (Cafarella et al. 2008: Zhang & Balog 2020). focusing on the accurate and efficient retrieval of structured tables from extensive corpora satisfy diverse user information needs. As a fundamental data storage format, tables offer unique advantages for knowledge organization and dissemination due to their dual capacity for structured data storage and human-readable presentation. Their inherent spatial efficiency and logical arrangement make them indispensable for applications ranging from business intelligence to scientific research. Recent studies reveal that approximately 27% of web search queries implicitly or explicitly target tabular data (Kwiatkowski et al. 2019), highlighting the critical need for effective table retrieval systems. This substantial demand elevates table retrieval to critical function in contemporary information systems to bridge the gap between unstructured natural language queries and structured knowledge repositories.
Table retrieval remains significantly less explored compared to conventional plain text retrieval. Ex- isting methods predominantly treat table retrieval as a specialized variant of text retrieval (Chen
Corresponding authors

---

Published as a conference paper at ICLR 2026
et al. 2023). Unlike plain text, tables have their corresponding row-column structure that cannot be ignored during retrieval, suggesting that table retrieval presents unique challenges that distinguish it from standard text retrieval tasks (?). This is particularly the case in complex table layouts like merged cells, multi-level headers, or irregularly aligned tables. As an example, scientific datasets often merged cells to represent logical groupings. Existing approaches foeus optimizing the understanding of table structures during the encoding process, thereby enhancing performance. Some text retrieval techniques have been effectively repurposed for table retrieval based on exist- ing system practices. TAPAS (Herzig et al. 2020) and DTR (Herzig et al.] (021) introduce row embedding and column embedding, similar to position embedding, to represent tabular structures. UTP (Chen et al., 2023) and THYME (Li et al. 2025b) optimize the retrieval training process to enhance performance. ECAT eLi al. 2025a) improves the performance of existing table retrievers
through entity matching.
These methods have facilitated the development and application of table retrieval, but they share limitation in practical applications. These methods inherently assume that tables are stored ex- clusively in textual format. However, in real-world scenarios, tables often exist in diverse formats such spreadsheets, databases, PDFs, or embedded in web pages, rendering text-only approaches insufficient for comprehensive retrieval (Zheng et al.) 2024). The limitations of text table retriev- are primarily manifested in two aspects: (1) They struggle express the structural complexity commonly found in real-world tables, including merged cells, partitioned layouts, and hierarchical organization. Such deficiencies significantly impair the adaptability and practical utility of table retrieval systems. (2) They focus solely on textual content, disregarding multimodal components, including but not limited visual markers and embedded images. This oversight fundamentally compromises the comprehensive understanding of tables. The text-only representation paradigm substantially restricts the generalizability and practical deployment of table retrieval systems.
Notably, visual presentation is inherently format- inneedddnde while preserving structural and con- tent information. Given the characteristics of tables, Image-based tabular representations and mul- timodal retrieval methods can naturally overcome the challenges encountered by text retrievers. Unfortunately, the absence of standardized benchmarks specifically designed for visual table re- trieval has significantly hindered progress in this promising research direction. To fill the gap in the field of table retrieval, we introduce TaR-Vit (Table Retrieval via Visual Representations), a pioneering benchmark that conceptualizes table retrieval as multimodal problem by representing tables in visual images. Based on NQ-TABLES (Herzig et al.] (20TT) a text-based table retrieval dataset, we transformed it into multimodal table retrieval benchmark. Specifically, we collect the webpage screenshots corresponding to the tables in the NQ-TABLES dataset from Wikipedia. Due to Wikipedia's 's dynamic and continuously updated characteristics, the crawled table images may no longer precisely match their original counterparts in the NQ-TABLES dataset. Consequently, the relevance between the queries and tables may have degraded over time. To address such issues, we designed an annotated pipeline that leverages the image understanding and reasoning capabilities of multimodal large models to assist annotators correcting relevance annotations. Furthermore, we regenerate and check the answers corresponding to users' time-sensitive queries in TaR-ViR.
We conduct extensive experiments on TiR-ViR to compare the performance of various text retriev- ers and multimodal retrievers under different settings. We find that multimodal retrievers achieve competitive performance with text retrievers on table retrieval. Multimodal retrievers even outper- form text retrievers in the metric of recall. These results validate the viability of image-centric table retrieval as an effective paradigm. Multimodal retrievers enable direct, large-scale retrieval of tab- ular data from images, eliminating the need for table extraction and data organization processes. This addresses persistent challenges in collecting and standardizing structured data across diverse domains. TiR-ViR indicates promising paradigm of able retrieval, which can shed light on future research on this topic.
2 RELATED WORK
2.1 OPEN-DOMAIN TABLE RETRIEVAL
Open-dome doma-d table retrieval involves extracting tables from large corpus and matching them with queries. Tables store information not only through their textual or numerical content but also through their inherent structural properties. A large amount of structurally similar related information is
2

---

Published as a conference paper at ICLR 2026
stored in tables. Due to the value of tables and their structural characteristics that differ from plain text, the research community has constructed specialized benchmarks tailored to tables to promote development in this field. NQ-TABLES (Herzig et al. (02I) OTT-QA (Chen et al.] 2020) introduces table retrieval from Wikipedia to support open-domain question answering. H2F- WTQ (Pan et al.) (02T) extends WikiTTaleeQussionss into retrieval setting and proposes Cell-Level Table Retrieval (CLTR), focusing on cell-level semantic matching, The single table sometimes fails to satisfy the user's information needs. MMQA (Wu et al., (025) introduces Multi-Table Retrieval, which trans- forms multi-table retrieval tasks into multi- mouti-round single-table retrieval tasks by decomposing the multi-hop problem into a series of subproblems. Open- WikiTable (Kweon al.) 2023) expanded WikiSQL and WikiTableeQestiioss by incorporating a larger table corpus.
With the development of benchmarks, methods specifically designed for table retrieval have been proposed. Since pre-trained language models are primarily trained on textual corpora, they cannot comprehend structured data like tables. To improve the comprehension of tables, TAPAS (Herzig) et al. 2020). TaBERT (Yin et al. 2020), and StruBERT Trabelsi et al. (0222) improve retrieval per- formance by adding special neural network layers based on BERT (Devlin et al. 2019). In addition to the special network layers, training based on the table corpus improves the pre- pre-trained model's understanding of tables (Herzig et al.) 2020; Chen et al. (023). There are also some works (Li a 2005at Wang et al.) 2022a Li al., (025b) that demonstrate the matching characteristics of tables that differ from plain text. These methods have promoted the development of table retrieval technology, but due the limitations of text storage, there are limitations in handling complex table structures such as merged cells, hierarchical, and nested tables. In addition to complex structures, table content may contain images that cannot be stored in text format. The limitation of text storage restricts the practical application of table retrieval in real life.
2.2 VISION-LANGUAGE MODELS FOR MULTIMODAL RETRIEVAL.
Multimodal retrieval has long been a significant research challenge. Early works like CLIP (Rad- Ford et al., (02T). BLIP (Li et al. 2022), Align (Jia et al. 202T), SigLIP (Zhai et al. 2023), SimVLM (Wang al., 2022b), and CoCa (Yu al. 2022) primarily focused on learning univer- sal representations from large-scale, weakly supervised image-text pairs. These models generally encode images and text separately, projecting them into a shared space. Most research on univer- sal multimodal embeddings involves fine-tuning models like CLIP or BLIP, typically using simple fusion mechanisms to combine visual and language information. For instance, UnilR (Wei et al. 2023) creates multimodal embeddings by simply adding text and visual features. Furthermore, Mag- ielens (Zhang et al. 2024) employs shallow self-attention layers to integrate these features more effectively. These approaches have laid the groundwork for more recent multimodal large language models such as VL (Liu et al. 2023) and Qwen-Vl. et al. 2025). Several pioneering works like VLM2VEC (Jiang et al. (2025) and GME (Zhang al. (025a) have demonstrated that through extensive training on large-scale, well-aligned image-texi datasets, these models can learn to gener- ate unified multimodal embeddings. The success of these methods has opened new possibilities for various downstream tasks, including retrieval, visual grounding, and so on.
3 BENCHMARK CONSTRUCTION
3.1 TASK DEFINITION
Let D = [(a, } be labeled dataset, where q denotes query and T is set of tables { relevant to q. The number of elements 1+ in T is different per query. A table contains an image and textual title. The objective of table retrieval is to train retriever that can identify relevance patterns of a query g and its relevant tables T Based on this task definition, our data construction process is illustrated in Figure I We employ an annotation pipeline that combines MLLMs with annotators to ensure annotation quality while improving efficiency.
3.2 DATA COLLECTION
Through comprehensive review of prior work, we observe the lack of multimodal table retrieval datasets. To validate the feasibility of an image-centric table retrieval, we introduce the first spe- cialized dataset designed for this paradigm. Existing table retrieval benchmarks can be roughly
3

---

Published as a conference paper at ICLR 2026
Collection
Assistance
Human
Annotation
System
Annotation
Top-n-
Fup-1
Top-3
O
90
Relevance
Top-2
Releranne Judgement
DO Brambell
Ranking
Answer Generation
Fix
Who played the son steptoe
Seepoor and Son, (Croared by, Sharing Arilcco) (Ray Gollen and Alon Siryoo. Dacenbher 1974)
Who played the sow steptor
Corbett
Figure I: Construction pipeline for TaR-ViR, By combining automated annotation with an annotator, optimize both the effectiveness and efficiency of the annotation process.
divided into two types based on their construction methodology. The first type derives from ex- isting table guestion-answeringg (QA) benchmarks, where contextual information is removed from the original questions to generate context-independent queries, thereby repurposing the dataset for table retrieval. The second type is constructed by extracting table-related search queries from web logs, leveraging real-world user interactions to create retrieval-oriented benchmark. The first type produces overly specific queries that poorly reflect real-world search behavior, whereas the second demands substantial annotation efforts, resulting in high development costs. However, compared to the former, it reflects the information needs of the real world and thus holds greater practical value. Therefore, we mine multimodal table retrieval datasets from web logs. Benefiting from previous work, we don't t need to mine web logs from scratch. We decided to adapt a textual table retrieval dataset. Based on the NQ-TABLES (Herzig al. 2021 dataset, we constructed TaR-ViR (Table Retrieval via Visual Representations) NQ-TABLES is a widely used benchmark for textual table retrieval, with tables collected from Wikipedia. To extend into the multimodal domain, we system- atically collect screenshots of the webpages where the tables in NQ-TABLES are embedded from Wikipedia, resulting collection of approximately 2 million images.
3.3 ANNOTATION PIPELINE
Through systematic analysis of the crawled images, identify two critical issues requiring resolution: redundant table occurrences and temporal shifts in query-table relevance. The redundancy issue stems from two main sources: multiple URLs referenc- ing identical webpage content, and instances where different webpages. This results in significant duplication within the cor- pus. To address this, we employ CLIP (Chen al.) 2025) for filtering. We consider two images dupli- cates if their similarity score exceeds 0.9 and they share identical URL prefixes. For each set of dupli- cate images, we retain only a single representative instance and perform corresponding annotation con- solidation to maintain data integrity. addition, we sort the multiple images associated with table in NQ-TABLES and pick the Top-1 result as the image representation for this table.
<1/4 Browsr Window
36.34%
26.02%
the same table appears
Image Size Distribution
35.75%
67.97%
1/4-1 Browser Window
>1 Browser Window
Figure Distribution of table image size. The outer circle shows the distribution across the entire corpus, while the inner circle dis- plays the corresponding tables in the test.
Given the observed temporal shifts in query-table relevance, correct this through re-annotation. Annotation requires large number of annotators, which is both time-consuming and costly. To address this challenge, developed cost-efficient
4

---

Published as a conference paper at ICLR 2026
annotation pipeline leveraging MLLMs. Specifically, employ Qwen2.5- (Bai et al. 2023 for automated annotation generation, with human annotators performing streamlined ver- ification and correction process. The human annotation is divided into two parts: (1) Relevance Annotators need to judge whether the provided pseudo-relevance results are correct. (2) Answer Annotation: For query-table pairs laheled relevant by annotators, further verification is required to determine whether the answers align with the questions and reference table images. Due to the scalability constraints of our benchmark and the costs associated with human annotation, we limit manual annotation to the test set. For the training set, we retain only those instances anno- tated as relevant by Dwen2.5-VL-72B We collected 1,550 query-table pairs for human evaluation, of which I 1,249 (~ 80%) were validated as correct. This indicates that our automated annotation generation achieves an accuracy of approximately 80%. Although the training set was not manually refined, the automated filtering ensures its high quality.
Annotation:
3.4 DATASET STATISTICS
The statistical information of TaR-ViR are summarized in Table Based on the relative size com- pared to the browser window, we classified tables in both the corpus and test set into three difficulty levels: Easy (< 25% window coverage). Medium (25% 100%). and Hard (tables exceeding the browser window). Figure 2 shows the distribution of these categories. This classification reflects progressively challenging scenarios for MI.LMs, as larger tables require more advanced visual un- derstanding, spatial reasoning, and adaptability to increasing size and complexity.
We also provide a detailed analysis of TaR- ViR with multiple previous datasets in Table We con- duct comprehensive comparison of various benchmarks by analyzing five key dimensions: dataset origin, context independence, corpus modality, data scale, and downstream tasks. Existing datasets typically originate from close-domain table questtio--nsserring datasets or web logs. Based on their provenance, we classify the context independence of different datasets into three distinct levels. In closed-domain QA benchmarks, user queries are inherently context-dependent, as they assume familiarity with the underlying tables. This type of data poses challenges for retrieval due to its abun- dance of directive phrases, which are often ambiguous and difficult for retrieval systems to interpret. While some datasets attempt to decontextualize queries in closed-domain QA benchmarks, they can not be representative of real-world user queries. Therefore, the user queries in this part of the context-
Table 1:
Statistics of TaR-ViR. The table statistics are based on the images and the text obtained through OCR.
dataset range from context-independent dependent. Many existing datasets are derived from web user logs, which capture real-world query pat- terns. The user queries within them are context- independent, but such datasets are typically costly to collect and relatively small in scale. Com- pared to these datasets, TaR- ViR is derived from a high-quality textual table retrieval benchmark, NQ- TABLES, which reflects user information needs real-world scenarios. Among image-centric table benchmarks, TaR- ViR is the largest in scale and simultancously supports both retrieval and QA evaluations.
Train
Test
Count
7,377
1.249
Query
Avg. Tokens.
10.47
10.44
Count
81,839
81,839
Avg. Width.
765.23
765.23
Table
Avg. Height.
1,087.35
1,087.35
Avg. Tokens.
738.18
738.18
# Relevant Tables
1.00
1.02
Table 2: Comparison between TaR- ViR and existing datasets related to table. Modality is the storage format of the table. And q-t Pairs refers to the number of query and table pairs contained in all sets (train and test).
Source
Cux-nndependent
Modality
#q Pairs
Tables
Reuneval
Q&A
WikiSQL Zhong 2017
Dataset
T
80,654
24,241
NQ-TABL LBS (Herzig al.
A021
Web Log
High
T
11,628
169,898
OTT-QA Chen et 2020
QA Dataset
Middle
43,683
419,183
F2F. (Pa
QA Dataset
Middle
T
1,216
2,108
E2E-GNO Pan ct 2021
Web Log
High
T
789
74,224
StatCan al 21243
Web Log
High
4,468
1,244
Open-Wiujaale weee et
2023
QA Dataset
Middle
T
67.023
24,680
ComTQA (Wicchao Zilao
QA Dataset
Low
9,000
9,000
TabFQuAD (Fayse
Web Log
High
210
210
TaR-ViR
Web Log
High
8,646
81,839
5

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 BASELINES
To evaluate the effectiveness of different table representations (text or image), we employed two distinct types of retrievers: Text Retrievers, which process tabular content as plain text, and Mul- timodal Retrievers, which leverage both text and image. We compared their performance on table retrieval tasks to assess the impact of representation choice. In TaR-ViR, where tables are stored as images, we first extracted their textual content using Qwen2.5-V -LL77 (Bai et al. (025) to convert the image-based tables into structured HTML representations. These tables then expanded row by row and transformed into sequential text format as input for the text retrievers.
4.2 EXPERIMENTAL SETUP
For the different baselines, we conducted training on TaR-ViR using open-source weights and frame- work and compared their performance differences. We implemented distinct training protocols according to model architectures: full-parameter fine-tuning was applied to BERT and CLIP-based retrievers, while LLM and MLLM-based retrievers were optimized using LoRA with rank of 16. To ensure consistency across experiments, all models were trained for 20 epochs with fixed batch size of 144.
4.3 EVALUATION METRICS
For evaluation, we employ three standard metrics: Recall, normalized discounted cumulative gain (NDCG), and mean reciprocal rank (MRR). Since the retrieved tables will be used in downstream tasks such as question answering, we use 50 as the maximum cutoff, following established prac- tices in table retrieval (Herzig et al. 2020 Li et al. 2025a). Specifically, we report Recall @ 10 and @ 50 to measure the coverage of relevant tables in top-k results. For ranking quality assess- ment, adopt NDCG@5 @5 and NDCG @ 10 to evaluate the graded relevance of results, accounting for the fact that different tables may contribute variably to downstream tasks. We also use MRR @ and MRR @ 10 supplementary metrics to measure how rapidly the first relevant table is retrieved, which is a crucial consideration for time-sensitive applications.
5 OVERALL PERFORMANCE
We extensively evaluated representative text and multimodal retrievers on the TaR-ViR. Detailed results are summarized in Table 3] Based on how data collected and used, we categorize these retrievers into three distinct groups By comparing the performance of retrievers within the same group and across different groups, we have summarized some interesting conclusions.
Firstly, retrievers incorporating both table titles and content exhibit superior performance, regardless of whether the input is OCR text or raw images. However, in real-world applications, table titles may not always exist. Even when present, they cannot always be extracted accurately and converted into text for storage. Considering the absence of table titles during data collection, the performance of multimodal table retrievers can reach approximately 70% of what they would achieve with com- plete data. When table content is presented as images, multimodal retrievers that incorporate title and content image (such as UniME and VLM2Vec) even outperform traditional OCR-based text re- trievers. UniME significantly outperforms the state-of-the-an text retriever BGE on metrics such as NDCG and MRR. However, this performance advantage comes with increased model complexity. Multimodal retrievers achieving comparable performance to text-based approaches typically require substantially more parameters. Multimodal retrievers with fewer parameters struggle to integrate information from different modalities.
Comparative observations of model parameters and retrieval metrics across different retrieval ap- proaches highlight distinct asymmetry in how parameter scaling affects text versus multimodal retrievers. In text retrievers, BGE outperforms Embedding, demonstrating that model optimization, such pretraining objectives and data quality,
models like GTE and Qwen3-
https://gi com/ /T GERR- Labb /V.M2ve

---

Published as a conference paper at ICLR 2026
Table 3: Performance comparison of retrievers under different input modalities. R. N. and M cor- respond to Recall, NDCG, and MRR. Bold and underline indicates the optimal and suboptimal performance, respectively.
Performance
Models
Backbones
Params
R@
R
N@5
N@ 10
M@5
M@
Tide (Text)
Content (OCK Text)
BM25 Robertson
2009
20.50
26.49
21.70
23,40
19.79
20.50
BIBERT Devim al 2019
BERT-Base
109M
89.08
95.64
68.82
70.67
63.92
64.71
GTE ara A144
Qwen2
1.54B
87.46
96.84
64.68
67.58
59.99
61.68
BGB
BERT-Base
109M
92.15
98.23
72.82
74.69
68.18
68.95
Qwen3- Embedding Zhiang et al.
2025b
Qwen3
4.05B
89.30
98.15
71.93
73.83
68.04
68.84
Title (Text) + Content (Image)
CL.IP Radiond et al. 2021
CL.IP-L.rge
428M
36.11
$1.69
26.15
27.47
24.14
24.69
SigLIP
SOViT-100m
878M
48.61
63.53
33.43
35.89
31.75
GME Zhhng at al. RUA53
Qwen2 VL.
2.218
59.80
68.80
43.68
45.84
40.15
41.06
UniSE-MLLM (Liu et 2025
Qwen2-VI VL
2.21BB
92.23
98.53
68.01
70.18
62.17
63.09
VeM2 (jang el 2025
Qwen2-VL
2.21B
90.69
97.53
70.33
62.72
63.68
Ve.M2 Drang
2025
Qwen2- VI.
7.07B
94.23
98.92
70.56
72.81
64,80
65.76
UniME (Gü et m.. 20255
116
7,57B
93.38
98.76
75.62
77.39
71.44
72.20
Content (Image)
DSE Ma et 2024
Phi-3-V
4.15B
66.45
77.61
51.44
53.76
48.59
49.54
ColPan sysse
2025
Qwen2-VL
2,21B
63.59
72.89
48.93
51.15
46.15
47.05
more critical than parameter scale in text embedding. However, in multimodal retrievers, results demonstrate a consistent improvement in performance with increasing model parameters and ca- pabilities. This finding indicates that joint visual-text representation learning requires models with strong foundational capabilities, making larger parameter scales indispensable.
The above results reveal paradigms for addressing multimodal table retrieval. One utilizes MLLMs to extract text from images, transforming the multimodal table retrieval task into a pure text-based table retrieval task. Relevant tables are then retrieved through small text retriever like BGE. The other directly employs an MLLM-base M-beedd multimodal retriever to retrieve the relevant tables. Compared
text retrievers, multimodal retrievers do not require preprocessing step to convert images into text to be input into the retrievers. Multimodal retrievers can directly process text and images with simplicity and efficiency. The direct utilization of both text and images offers streamlined and effective approach. However, text retrievers based on OCR content are more flexible in use. The OCR content of the table can be obtained by calling MLLMS via the API. In practical applications, choosing between these two methods requires balancing resource and efficiency.
6 FURTHER ANALYSIS
6.1 IMPACT OF DIFFERENT TABLE FORMAT
Multimodal retrievers have demon- strated competitive performance in table retrieval. However, because of differences in the training data used and optimization procedures across retrieval systems, the performance comparisons in Table 3 not en- tirely fair. To eliminate performance discrepancies arising from different training processes, we independently trained dedicated text retrievers and multimodal retrievers using Qwen2-
Table 4: Performance comparison of retrievers with dif- ferent input formats. All retrievers are implemented using Awen2-VL-2B. The only difference lies in the format.
Input Format
R@ 10
N@5
M@5
Title (Text) Content (OCR)
89.00
65.91
61.46
Title (Text) + Content (Web)
90.30
65.20
60.64
Title (Text) + Content (Image)
91.69
67.42
62.19
Title (Image) + Content (Image)
82.30
58.09
53.32
VL-2B as the base architecture. For rigorous experimental control, we maintained identical hyper- parameter configurations across all models, modifying only the input data format to accommodate different modalities. The result is shown in the Table MLLMs can parse table information directly from images while learning the relevance between queries and tables. We also observed variations between different input contents. When using the original table content from web pages instead of OCR results input for the text retriever, it showed different trends across different metrics. The
7

---

Published as a conference paper at ICLR 2026
metric of recall has increased, while NDCG and MRR have shown downward trend. Through case observation, we have identified the causes behind this phenomenon. For retrieval, more input information does not necessarily yield better results. Given the presentation of tables in images, MILMMs demonstrate a systematic preference for extracting information that has been visually em- phasized through color contrast, font variations, and other typographic cues. This behavior mirrrrs human information-seekinn patterns. As a result, OCR- based input leads to superior performance in ranking-related metrics, including NDCG and MRR.
6.2 IMPACT OF INFORMATION VOLUME ON TABLE RETRIEVERS
Tables can store large amounts of data through a simple row and column structure. In practice, tables vary in size from single cells to multi-page layouts. To investigate the performance differences of tables with varying information under different retrievers, we divided the test set into three groups based the size of the corresponding relevant tables for each query, following the classification standards outlined in Section 3.4 The results are shown Table 5
Table 5: Performance comparison for tables with different information volumes. The best perfor- mance in each group is denoted by *, t, and o, respectively.
Metries
Title
Content
R@5
R@ 10
N@5
N@10
M@5
M@ 10
Text
OCR Text
87.65*
96.29*
71.86*
74.39*
67.07
67.80*
Easy
Text
Web Text
81.81
92.20
68.19
71.33
63.42
64.71
Text
Image
85.71
92.20
68.85
70.82
63.20
63.87
Text
OCR Text
84.59
91.23
72.66
74.801
68.661
69.53T
Medium
Text
Web Text
88.07
92.66
71.73
73.24
66.18
66.81
Text
Image
90.21
92.96*
73.841
74.75
68.32
68.69
Text
OCR Text
76.12
87.50
62.82
66.51
58.23
59.77
Hard
Text
Web Text
80.93
89.65
62.92
65.77
57.07
58.20
Text
Image
80.81
91.51
65.29
68.76
60.04
61.53
When tables are small, text retrievers using OCR or original text content from the webpage per- form better than multimodal retrievers. However, as tables grow in both structural complexity and physical dimensions, MLLMs struggle to accurately extract and parse textual content appear- ing in tabular images. The image-centric table retrieval paradigm is gradually demonstrating its advantages. The multimodal approach demonstrates particular effectiveness preserving the com- plete informational integrity of tabular images, thereby enabling more precise query-table matching. Comparison results using OCR text and original text from the webpage, we observed the same phenomenon in Section 6.1 OCR-based text retrievers outperformed the original text in ranking- related metrics. However, as the table information in the image increases, this advantage gradually diminishes. When table contains information spanning more than one browser window, informa- tion highlighted through visual cues becomes difficult for MLLMs to accurately extract, leading to decline in retrieval performance.
RAG APPLICATION
The advent of large language models has transformed retrieval systems into auxiliary knowledge providers that primarily supply input LLMs, rather than operating as standalone user interfaces. This functional transition is most apparent in applications such as question answering and text sum- marization systems. The TaR- ViR we proposed not only includes annotations for relevance but also annotations for the answers to user questions given the relevant tables. In this section, demon- strate the evaluation of the RAG system to highlight the value of TaR-ViR. For retrievers, we employ text retriever and multimodal retriever trained Qwen2-VL-2B. VL-2B. For generators, we evaluate both LLM and MLLM under two distinct retriever configurations. These diverse combinations can illustrate the modal preferences of RAG systems when processing tables.
The experimental results, obtained using Mistral (Jiang et al.] 2023), Llama3 (Grattation] I.LaVA-OneViiin
al. 2024 Qwen3 (Yang al. 2025).
(Li al. 2024 and
8

---

Published as a conference paper at ICLR 2026
Qwen2.5-VL (Bai al. 2025) presented in Table 6 large-scale collection
Our data is based
NQ-TABLES, a dataset comprising a Following standard practice in question- answering evaluation, we measure accuracy by verifying whether the annotated ground-truth answers appear in the models' generated responses. Table [4] demonstrates the supe- rior performance of the multimodal retriever (trained on Dwwn2--L-225 compared to its text-based counterpart. Similar conclusions were also demonstrated in downstream QA evaluations. Qwen2.5-VI. achieved better performance when tables were input as images. However, overall, text-based LLMs exhibit superior upper-bound performance. To answer questions corresponding to image-based tables, MLLMs must simultaneously possess precise visual parsing, structured semantic under- standing, and symbolic reasoning capabilities. Current MLLMs, however, have not yet achieved complete integration of these essential competencies. This also reflects the potential value of TaR- ViR from another perspective. It demonstrates the limitations of current MLLMs.
of factual questions.
Table Performance evaluation of the RAG sys-
tem across different configurations. Gray indi- cates multimodal input.
Retriever
Generator
Avcuracy
Mistral-TB
31.43
36.15
35.92
Llamaš.1-88
31.27
35.17
37.21
Qwen2-VL-228 (Teat Retrieval)
Qwen2.5-VL
37.78
38.51
Owen3-S8 BB
46.57
55.35
58.38
LLaVA CLaVA-OneeVe Viston
21.25
20.84
18.21
Qwen2 5-V1.77
40.34
40.87
Mistral-2B
30.61
36.56
36.97
Llamaš.1-8
15.83
Owen2-VL-2B (MulriModal Revrieal)
Owen2 5-VL-BB
31.59
38.02
39.33
Qwen SB
44.05
55.86
58.55
LnaVA-One Visiiio
20.68
20.35
Qwen2 5-VL.7B
32,82
41.10
42.46
CASE STUDY
To visually analyze how image-centric table retrieval paradigms impact performance, we present a concrete case study in Figure [ The figure compares the results of the same query processed through both text-only and multimodal retrieval systems. Notably, while the textual content of the table image was accurately extracted via OCR, the multimodal retriever achieved superior ranking performance by effectively leveraging visual-semantic features. This improvement can be attributed to the additional visual cues-such as photographs, colors, and layout-embedded in the image. These visual elements emphasize key information of the table, enhancing the multimodal retriever's ability to identify and prioritize specific content.
Aulrii
Pupe
MultiModal
sings want to be rockstar ?
Retriever
Retriever
ligi Aoomm.
Expes 2000
OCR
Image
Zegal
611
Rank:
18
Rank:
Predew
Figure 3: Retrieval result comparison between different storage formats. The text retriever and multimodal retriever are Qwen2- VL-BB trained using OCR and multimodal data.
CONCLUSION
In this work, we propose TaR-Vi ViR, a multimodal table retrieval benchmark, and demonstrate the effectiveness of multimodal retrievers in table retrieval. Our experiments show that multimodal retrievers achieve competitive performance compared to text retrievers on this task. Multimodal retrievers can directly analyze table content within images while learning the semantic relevance between the table's visual representation and user queries. Furthermore, unlike text retrievers, mul- timodal retrievers can directly process images without converting them into text, offering significant efficiency advantages. This result also demonstrates that adopting more flexible storage method, like images, can effectively organize and manage the massive amounts of tabular data encountered in real-world applications, thereby better addressing users' information needs.

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
TaR-Vit we propose is derived from existing open-source datasets. Additionally, the data we have scraped based on publicly available content from Wikipedia and does involve any privacy information belonging to organizations individuals. We paid the annotators their corresponding compensation for their work in constructing the dataset.
REPRODUCIBILITY STATEMENT
We propose a novel multimodal table retrieval dataset TaR-ViR, leveraging publicly accessible Wikipedia data throughout the process. Furthermore, our training process is based on open-source model weights and code implementations. Our data will also be made open-source to the public, promoting the advancement of the field of table retrieval.
ACKNOWLEDGEMENT
This work funded by the National Natural Science Foundation of China (NSFC) under Grants No. 62302486 and No. 62441229, the Innovation Project of ICT CAS under Grants No. E361140, the CAS Special Research Assistant Funding Project, and the project under Grants No. JCKY202213C033,
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang. Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang. Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
abs /2t//2502.00 13923
Michael J Cafarella, Alon Halevy, Daisy Zhe Wang, Eugene Wu, and Yang Zhang. Webtables: exploring the power of tables on the web. Proceedings of the VLDB Endowment, 1(1):338-549 2008.
Hong-You You Chen, Zhengfeng Lai, Haotian Zhang, Xinze Wang, Marcin Fichner, Keen You, Meng Cao, Bowen Zhang, Yinfei Yang, and Zhe Gan. Contrastive localized language-image pre- training, 2025. URLhttps: /tettps://arivv. org/ abs/aab//..0./ 02746
Jianly Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, multi-functionality, multi gramullrtyy text embeddings through ceff-knowledge dis- tillation, 2024. URL htttttttt//ax.tt.. org/ abs /2002 03216
Nuo Chen, Linjun Shou, Ming Gong. Jian Pei, Chenyu You, Jianhui Chang. Daxin Jiang, and Jia Li. Bridge the gap between language models and tabular understanding, 2023.
Wenhu Chen, Ming-Was Wei Chang, Eva Schlinger, William Wang, and William W Cohen. Open ques- tion answering over tables and text. arXiv preprint arXiv:2010. 10439, 2020.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. 2019.
Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models, 2025. URL
01449
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar. Artem Ko- renev, Arthur Hinsvark. Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux,
10

---

Published as a conference paper at ICLR 2026
Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Qlano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jong- 800 Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani. Kate Plawiak, Ke Li, Kenneth Hcafield. Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeat Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji, Ning Zhang. Olivier Duchemne. Onur Çelebi. Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic. Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong. Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do Vish Vogeti, Vítor Albiero. Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaelle Gold- schlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An- nie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim. Chao Zhou, Chester Hu, Ching- Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le. Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel. Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang. Hanwen Zha, Haroun Habeeb, Harri- son Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Flena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang. Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy
Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia
11

---

Published as a conference paper at ICLR 2026
Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L.. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Ro- driguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang. Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay. Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar. Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen. Steve Kehoe, Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk. Suraj Subramanian, Sy Choudbury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang. Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Ku- mar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiao- jian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li. Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang. Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Vitoto Zef Rosnbrick. Zhao- duo Wen, Zhenyu Yang. Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. URL httpps: / www.vvi..gg..oo.ooooo o abs/ 2407.21788
Tiancheng Gu, Kaicheng Yang, Ziyong Feng, Xingjun Wang, Yanzhao Zhang, Dingkun Long, Yingda Chen, Weidong Cai, and Jiankang Deng. Breaking the modality barrier: Universal embed- ding learning with multimodal Ilms, 2025. URL
wwwp://v..rrrbb//////ba/ 2250: 17432
Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Eisen- schlos. Tapas: Weakly supervised table parsing via pre-training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. doi: 12.18663/1/22222.cccmmmaaa
URL
18653/v1/2023 acl-main.
Jonathan Herzig, Thomas Müller, Syrine Krichene, and Julian Martin Fisenschlos. Open domain question answering over tables via dense retrieval, 2021. URL httptt://aaiii..oo/////////
2103. 12011
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe V. Le, Yunhsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision- language representation learning with noisy text supervision, 2021. URL w/tp://aavvvv///a////a//ea./.wav org/ abs /atp////12..mm 05918
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https:
//arxiv. www.:orgggaboo...o abs///////// 02000 06825
Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. Vlm2vec:
Training vision-language models for massive multimodal embedding tasks, 2025. URL https:
1/arxiv.org/abet /20e/24111 2410 05160
Sunjun Kweon, Yeonsu Kwon, Sconhee Cho, Yohan Jo, and Edward Choi. wikitabl:: Dataset for open domain question answering with complex reasoning over table, 2023. URL https:
12

---

Published as a conference paper at ICLR 2026
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le. and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452 466, 2019. doi:
00276. URL
009 1026/
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer, 2024. URL htttp:://aaxxii.rr/////ss/////es
03326
Da Li, Keping Bi, Jiafeng Guo, and Xuegi Cheng. Bridging queries and tables through entities in table retrieval, 2025a. URL
Da Li, Keping Bi, Jiafeng Guo, and Xueqi Cheng. Tailoring table retrieval from field-aware hybrid matching perspective, 2025b.
02251
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022. URL https:
/ wttpvv/..rg//s////1..... 12086
Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkum Long, Pengjun Xie, and Meishan Zhang. Towards
general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL https://arxiv. http:///axxxv..rr/aabe wttgg/bb//2300 08485
Ze Liu, Zhengyang Liang, Junjie Zhou, Zheng Liu, and Defu Lian. Any information is just worth one single screenshot: Unifying search with visualized information retrieval, 2025. URL. https:
7/ //axxiv. sazxiv.org/abs/25000 11431
Xing Han Lu, Siva Reddy, and Harm de Vries. The statcan dialogue dataset: Retrieving data tables through conversations with genuine intents. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 2799-2829, As-
sociation for Computational Linguistics, 2023. doi: 10.18655/v122233eaclmmmm waaee-mmiin2..
URL
http:/ / dx. www.ooo.oooooooo oro.orgg/1... 1...00.86633111//
Xueguang Ma, Cheng-Chieh Lin, Minghan Li, Wenhu Chen, and Jimmy Lin. Unifying multi- modal retrieval via document screenshot embedding, 2024. URL httts:://axiv...
abs/2405 11251
Feifei Pan, Mustafa Canim, Michael Glass, Alfio Gliozzo, and Peter Fox. Cltr: An end-to-end, transformer-based system for cell level table retrieval and table question answering, 2021. URL
https: orgg abss 2106.04447
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PmLR, 2021.
Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and be- yond. Foundations and Trends@ in Information Retrieval, 3(4):333-389, 2009.
Mohamed Trabelsi, Zhiyu Chen, Shuo Zhang, Brian D. Davison, and Jeff Heflin. Strubert: Structure- aware bert for table search and matching. In Proceedings of the ACM Web Conference 2022, WWW '22. ACM, April 2022. doi: 10. 10.1145/348547733511922 URL http://////// //d::///xxx /xx doi org/ 10.1145/3485447.1 3511972
Zhiruo Wang, Zhengbao Jiang, Eric Nyberg, and Graham Neubig. Table retrieval may not ne- cessitate table-specific model design. In Wenhu Chen, Xinyun Chen, Zhiyu Chen, Ziyu Yao, Michihiro Yasunaga, Tao Yu, and Rui Zhang (eds.), Proceedings of the Workshop on Struc- tured and Unstructured Knowledge Integration (SUKI), pp. 36-46, Seattle, USA, July 2022a. Association for Computational Linguistics. doi: 10.18653/v1/22 w/w.:/..uk2-211.2.2. URL https:
13

---

Published as a conference paper at ICLR 2026
Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, and Yuan Cao. Simvlm: Simple visual language model pretraining with weak supervision, 2022b. URL https://arxiv.
org/abs/ 2wtp//21.... 10904
Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. Uniir: Training and benchmarking universal multimodal information retrievers, 2023.
wttp:////////....1../. URL httltttts//aaxxtt
Qi Liu Jingqun Tang Binghong Wu Lei Liao Shu Wei Yongjie Ye Hao Liu Wengang Zhou Houqiang Li Can Huang Weichao Zhao, Hao Feng. Tabpedia: Towards comprehensive visual table under- standing with concept synergy. In Advances Neural Information Processing Systems, 2024.
Jian Wu, Linyi Yang, Dongyuan Li, Yuliang Ji, Manabu Okumura, and Yue Zhang. Mmqa: Eval- uating lims with multi-table multi-hop complex questions. In The Thirteenth International Con- ference on Learning Representations, pp. 1, 2025.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang. Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin. Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL https: /ttp://aaiiii wwaaiiv.ooo org/ abs /205./205..0 0 09388
Pengcheng Yin, Graham Neubig, Wen tau Yih, and Sebastian Riedel. Tabert: Pretraining for joint understanding of textual and tabular data, 2020.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung. Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models, 2022. URL /ttpss/tt////xxiv.
org//bs/ http//bb/2200.... 0ttp/22055019977
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre- tre-training, 2023.
Kai Zhang, Yi Luan, Hexiang Hu, Kenton Lee, Siyuan Qiao, Wenhu Chen, Yu Su, and Ming-Wes Wei Chang. Magiclens: Self-supervised image retrieval with open-ended instructions, 2024. URL
URL https:://axiv..or//aa/.
19651
Shuo Zhang and Krisztian Balog. Web table extraction, retrieval and augmentation: A survey, 2020.
wttp://ggg/g.aoooooo/oooo 00207
Xin Zhang, Yanzhao Zhang, Wen Xie, Mingxin Li, Ziqi Dai, Dingkun Long, Pengjun Xie, Meishan Zhang, Wenjie Li, and Min Zhang. Gme: Improving universal multimodal retrieval by multimodal llms, 2025a. Utt.htts::///aee www.aaaaiivv.........aa http::///a.xx..../a///////soss// 2412. 2485. 6855
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embed- ding: Advancing text embedding and reranking through foundation models, 2025b. URL.
als /bs.r///a////05../ 05176
Mingyu Zheng, Xinwei Feng, Qingyi Si. Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. Multimodal table understanding, 2024. URL
08100
Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning, 2017. URL https:: pettss://rriv... arxiv. org/abs. 1709 00103
14

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
In this paper, we employed a large language model (LLM) two auxiliary capacities: (1) to assist with wording refinement, and (2) to generate data visualization code templates. Importantly, all fundamental concepts, theoretical frameworks, experimental methodology, and the complete initial manuscript were independently developed and authored by the authors. All LLM- L---eerrrtted content was manually validated by the authors.
B PROMPT FOR RAG EVALUATION
Answer Generation:
You need to answer the user's questions based on the tables provided to you.
[User Question]: <Question>>,
Below is the information from <Table Count> tables.
[Table I]: <Titl>:>Conle Content>>
[Table 2}:
<Title>: Content>.
[Requirements]: You must output the answer directly. Outputting anything other than the answer is not permitted.
[Output]:
C PROMPT FOR TABLE OCR
Table Content Extraction:
You are an OCR table recognition expert with years of experience.
Goals: You need to recognize the content in the given image and output the results in CSV table format.
Constraints:
1. You need to recognize the content in the image and completely extract the content from each table cell, placing into CSV table structure.
2. There may be placeholders in the table cells in the image, such as "'" "_", '/", etc., which need to recognized.
3. The output table structure must follow the structure in the image exactly, with complete consistency.
4. Pay special attention to merged cells in the image ensure the structure is correct. For images with a lot of content, ensure the output is complete, without omitting or fabricating any information.
6. The final output needs to be CSV table format.
7. Extract the main content of the table and limit the content to within 2000 tokens. Initialization:
Please think carefully and output the CSV table result.
(Table J: <Tille>:<Iaage>>
[Output]:
15


Reviews:
[{"review_id": "WV7XVqrQMf", "reviewer": "Reviewer_g7XG", "summary": "This paper targets at open-domain table retrieval. Instead of using text as the source of retrieval, this paper proposes to use image as an alternative. This paper introduces a new benchmark TaR-ViR based on an existing text-based table benchmark. The authors have conducted some ablation studies that show how the table image would help in scenarios like RAG setup.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Though treating tables as images has been explored in various existing literature (see weakness 3), the authors introduce this setup in the table retrieval.\n\n- The authors have conducted experiments to show the potential of treating tables as images.", "weaknesses": "- I wonder if the dataset is comprehensive enough to cover diverse visual web table types. As stated by the authors, `...adapt a textual table retrieval dataset.`, the authors may ignore the types of tables explored in [11]. I believe this distinguishes the visual table understanding / retrieval from considering tables just from the text perspective.\n\n- [12] has conducted ablations on different resolutions of table images, [1] has proposed together with different table formats in text, different image formats for tables. It would be nice for the authors to conduct similar ablation studies to understand how these factors play in your setup. \n\n- In certain experiments, the size and the type of models used are limited. For instance, in the RAG setup, the authors mostly conducted their experiments on 7-8B sized LLMs with 2B retrievers.\n\n- Related works such as [1] is worth mentioning. In the related work section, it is worth mentioning more efforts from the table community, for instance, the ones working on architecture changes [2], the recent waves on instruction tuning foundational LLMs on tables, including [3, 4, 5, 6, 7, 9], and the line of research on investigating table representations [8, 10].\n\n### References\n\n[1] Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. 2024. Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs. In Findings of the Association for Computational Linguistics: ACL 2024, pages 407–426, Bangkok, Thailand. Association for Computational Linguistics.\n\n[2] Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, and Shachi Paul. 2022. TableFormer: Robust Transformer Modeling for Table-Text Encoding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 528–537, Dublin, Ireland. Association for Computational Linguistics.\n\n[3] Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. 2024. TableLlama: Towards Open Large Generalist Models for Tables. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6024–6044, Mexico City, Mexico. Association for Computational Linguistics.\n\n[4] Xiaokang Zhang, Sijia Luo, Bohan Zhang, Zeyao Ma, Jing Zhang, Yang Li, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, Jifan Yu, Shu Zhao, Juanzi Li, and Jie Tang. 2025. TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10315–10344, Vienna, Austria. Association for Computational Linguistics.\n\n[5] Naihao Deng and Rada Mihalcea. 2025. Rethinking Table Instruction Tuning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 21757–21780, Vienna, Austria. Association for Computational Linguistics.\n\n[6] Li, Peng, et al. \"Table-gpt: Table fine-tuned gpt for diverse table tasks.\" Proceedings of the ACM on Management of Data 2.3 (2024): 1-28.\n\n[7] Zha, Liangyu, et al. \"Tablegpt: Towards unifying tables, nature language and commands into one gpt.\" arXiv preprint arXiv:2307.08674 (2023).\n\n[8] Li, Liyao, et al. \"Table as a Modality for Large Language Models.\" The Thirty-ninth Annual Conference on Neural Information Processing Systems.\n\n[9] Su, Aofeng, et al. \"Tablegpt2: A large multimodal model with tabular data integration.\" arXiv preprint arXiv:2411.02059 (2024).\n\n[10] Long, Lin, et al. \"Bridging the Semantic Gap Between Text and Table: A Case Study on NL2SQL.\" The Thirteenth International Conference on Learning Representations. 2025.\n\n[11] Titiya, Prasham Yatinkumar, et al. \"MMTBENCH: A Unified Benchmark for Complex Multimodal Table Reasoning.\" arXiv preprint arXiv:2505.21771 (2025).\n\n[12] Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. 2024. Multimodal Table Understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9102–9124, Bangkok, Thailand. Association for Computational Linguistics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lVXekY1iYM", "reviewer": "Reviewer_5rq1", "summary": "This paper introduces TaR-ViR, a multimodal benchmark that redefines open-domain table retrieval by treating tables as images rather than text sequences. The work argues that text-only approachesfail to capture the rich structural and spatial semantics of real-world tables. TaR-ViR extends the NQ-TABLES dataset by collecting approximately 2 million table screenshots from Wikipedia and aligning them with natural-language queries via a semi-automated annotation pipeline that leverages MLLMs.\nComprehensive experiments compare text-based retrievers against multimodal ones, showing that multimodal retrievers achieve competitive or superior performance, particularly in recall and large-scale retrieval efficiency.", "questions": "- Would performance degrade if the tables included handwritten or low-quality scanned data, given that all current images are digital screenshots?\n\n- Could multimodal retrievers trained on TaR-ViR generalize to document-level retrieval tasks where tables coexist with charts or paragraphs?\n\n- The benchmark focuses on retrieval efficiency but omits latency and compute cost comparisons between OCR-based and pure visual retriever?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The benchmark scale is large. \n\n- TaR-ViR provides a full framework integrating visual annotation, OCR-based comparison, and RAG-based downstream tasks.", "weaknesses": "- Although the paper claims open-world applicability, TaR-ViR relies entirely on Wikipedia-sourced tables. These are relatively clean, consistently formatted, and visually homogeneous.\n\n- The reliance on Qwen-VL for pseudo-labeling and 80% correctness in human verification introduces potential label noise. The test set’s limited manual validation raises concerns about bias propagation, especially since multimodal retrievers were trained on partially machine-labeled data.\n\n- While Section 7 integrates TaR-ViR into a RAG QA pipeline, the downstream results show only marginal accuracy improvements and rely primarily on recall.\n\n- The paper’s primary contribution is dataset construction and evaluation rather than a new retrieval architecture. Its impact may hinge on community adoption rather than algorithmic innovation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T0Kpa7Qrfy", "reviewer": "Reviewer_U5D7", "summary": "The authors extend the existing open-domain table retrieval paradigm, framing it as a visual retrieval task rather than the previous text retrieval task. Based on this, they construct TaR-ViR, the first multimodal table dataset that supports both QA and retrieval.", "questions": "See weakness part", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The authors present their ideas clearly with fluent logic, and the overall experiments are comprehensive.\n1. Novel Paradigm for Table Retrieval: The author fills a critical gap in existing research by being the first specialized benchmark that reformulates table retrieval as a multimodal task, treating tables as images instead of relying solely on text. Experiment results demonstrating that image-centric table retrieval can outperform text-based methods (especially in recall and handling complex structures) while eliminating error-prone OCR/text conversion, the work proposes a more efficient and flexible paradigm for real-world table retrieval. It also supports both retrieval and QA evaluations, enhancing its utility for diverse downstream tasks.\n2. High-Quality Data Construction: Leveraging Wikipedia’s rich table resources and NQ-TABLES’ foundation, the dataset ensures real-world relevance while resolving key issues (redundancy, temporal relevance shifts) via a cost-efficient annotation pipeline combining MLLMs (Qwen2.5-VL-72B) and human verification.", "weaknesses": "1. During the data construction process, the authors initially collected 2 million table screenshots based on NQ-TABLES, which contains approximately 100,000 document data entries. Why is there such a large discrepancy in the data collection process? It is necessary to specify how the data volume changes at each subsequent step of data processing, as well as the data splitting method and whether it is consistent with that of NQ-TABLES.\n2. The authors also mention in the paper that the comparisons in the experiments of Section 5 are unfair, and I agree with this view. I believe the conclusions derived here are not sufficiently compelling—they only demonstrate that table retrieval tasks can be completed using visual elements instead of text. Additionally, I am concerned about the results of \"title + content (web)\", which the authors have not provided here.\n3. In the experiments of Section 6, the authors observe that performance degrades due to OCR limitations when tables become complex, but performs better on simple tables. I am curious whether combining image and text content would yield better results; furthermore, regarding the definition of complex tables, I believe relying solely on size is insufficient, and the authors could attempt to derive tables of different complexities by decomposing table parsing structures.\n4. In Table 6, the authors should present the improvements of multimodal retrieval compared to text retrieval. Currently, for RAG applications, it does not show significant advantages over text retrieval, which raises doubts about whether this paradigm is superior to text-based approaches. Moreover, the performance disadvantage should not be solely attributed to MLLMs.\n5. The authors also state that there are images in tables that existing methods cannot handle. If MLLMs are utilized to embed images into text sequences, what would the overall performance be?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E3nebd1F54", "reviewer": "Reviewer_LgPw", "summary": "This paper introduces TaR-ViR, a vision-based benchmark for table retrieval, providing an in-depth analysis of the fundamental limitations of text-based table retrieval methods when handling complex table structures and embedded images. It also proposes a novel paradigm that treats tables as images for retrieval.\nThe benchmark is constructed with a rigorous methodology that balances both quantity and diversity, and offers valuable insights for future research through comprehensive experiments.", "questions": "1. As a benchmark for RAG, it is recommended to incorporate more table-specific performance metrics, such as robustness to row/column permutation (for permutable tables), rendering method robustness, and image resolution robustness.\n2. The data deduplication phase employed the CLIP model, while CLIP is also used in subsequent comparisons. This may introduce potential bias.\n3. This paper identifies limitations of text-modal tables, such as difficulty in handling visual elements (e.g., national flags, emojis). However, the benchmark statistics do not specify the proportion of such tables, making the benchmark less targeted.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work provides an in-depth summary of the fundamental limitations of text-modal tables in handling complex table structures and embedded images.  \n2. The benchmark construction methodology is sound, cost-effective, and exhibits sufficient volume and diversity.  \n3. The experiments are reasonably comprehensive and offer a thorough comparison of various retrieval configurations.", "weaknesses": "1. Some expressions are overly strong. For example, the text modality is not entirely incapable of representing certain heterogeneous tables, and it must be admitted that text-based retrieval can be simple and fast (especially when the original data is already in text form). Similarly, table-modal tables are not perfect either — for instance, visual modalities also face challenges when dealing with very large database tables.\n2. RAG also includes reranking. Although more combinations would significantly increase experimental complexity, it is still recommended to add some related experiments.\n3. Are the criteria for difficulty classification somewhat oversimplified?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pzGMUnEyJi", "reviewer": "Reviewer_EvmC", "summary": "This paper proposes TaR-ViR, a new benchmark that reframes open-domain table retrieval as a multimodal task by treating tables as images rather than serialized text. The authors crawl Wikipedia screenshots for tables and derive a corpus with 81,839 tables and 8,646 query–table pairs. They use a hybrid MLLM-assisted annotation pipeline: using Qwen2.5-VL-72B proposes labels which are then partially verified by humans. The evaluation of the paper compares text retrievers with multimodal retrievers, and reports that large multimodal retrievers can match or surpass strong text retrievers on several ranking metrics. They also include a RAG study showing modest gains when using multimodal retrievers for table-QA inputs, while text-only LLMs still achieve a higher upper bound overall.", "questions": "Check the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "**1. Well-motivated**: Directly operating on table images addresses notorious issues with serialization losses, such as merged cells, hierarchical headers, embedded figures that harms text-only pipelines. \n\n**2. Scale & coverage**: The pipeline begins from 2M screenshots and yields a large image-centric retrieval benchmark with 81,839 tables; clear dataset statistics are provided. \n\n**3. Practical annotation design**: The MLLM-assisted relevance and answering workflow with human verification is cost-aware and reports empirical quality (80% held rate), with manual curation reserved for the test set. \n\n**4. Evidence of benefits**: On TaR-ViR, strong multimodal models (e.g., VLM2Vec-7B) can outperform the best text retrievers (e.g., BGE) on ranking metrics when using title+content images.", "weaknesses": "**1. Benchmark scope**: The corpus is Wikipedia centric. Table images, such as scanned documents, enterprise spreadsheets, and PDFs beyond Wikipedia layouts is not evaluated, limiting external validity. \n\n**2. Novelty**: The novelty of hybrid data collection pipeline of TaR-ViR is limited. \n\n**3. RAG gains**: The RAG table shows modest benefits from multimodal retrieval, while text LLMs still dominate, leaving the practical payoff ambiguous for QA pipelines that can rely on OCR and text LLMs. \n\n**4. Annotation noise**: The auto-labeled training set is reported with 80% precision suggests non-trivial label noise that could bias model training. More rigorous robustness analysis, such as different MLLMs' label-noise sensitivity, would strengthen claims. \n\n**5. Reliance on titles**: Several best results assume title and image are both available. In the wild, titles may be missing or noisy. Extra results on how performance degrades when titles are unavailable are helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)