Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing
Authors: Jinsong Li
Xiaoyi Dong
Yuhang Zang
Yuhang Cao
Jiaqi Wang
Dahua Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Large Vision-Language Models
Chart Parsing
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: While Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities for reasoning and self-correction at the textual level, these strengths provide minimal benefits for complex tasks centered on visual perception, such as Chart Parsing. Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and hallucination. Inspired by the human strategy of using a finger as a ``visual anchor'' to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR). The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors. We instantiate the VSR paradigm in the domain of Chart Parsing by proposing ChartVSR. This model decomposes the parsing process into two stages: a Refine Stage, where it iteratively uses visual feedback to ensure the accuracy of all data points' Pixel-level Localizations, and a Decode Stage, where it uses these verified localizations as precise visual anchors to parse the final structured data. To address the limitations of existing benchmarks, we also construct ChartP-Bench, a new and highly challenging benchmark for chart parsing. Our work also highlights VSR as a general-purpose visual feedback mechanism, offering a promising new direction for enhancing accuracy on a wide range of vision-centric tasks.

Published as a conference paper at ICLR 2026
VISUAL SELF-REFINE: A PIXEL-GUIDED PARADIGM FOR ACCURATE CHART PARSING
Jinsong Li Xiaoyi Dong! 1.2 Yuhang Zang² Yuhang Cao? Jiaqi Wang2.4 Dahua Lin
1 The Chinese University of Hong Kong 2 Shanghai Al Laboratory
3 CPII under InnoHK Shanghai Innovation Institute
1j0240ie buhk.edu. .hk Github: https: /ttp//ggttu///tt IonmInnternLM/VR
ABSTRACT
While Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities for reasoning and self-correction at the textual level, these strengths provide minimal benefits for complex tasks centered on visual perception, such as Chart Parsing. Existing models often struggle with visually dense charts, leading errors like data omission, misalignment, and hallucination. Inspired by the human strategy of using a finger as "visual anchor" to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR). The core idea of VSR to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it intuitively inspect and correct its own potential visual perception errors. We instantiate the VSR paradigm in the domain of Chart Parsing by proposing ChartVSR. This model decomposes the parsing process into two stages: a Refine Stage, where it iteratively uses visual feedback to ensure the accuracy of all data points' Pixel-level Localizations, and a Decode Stage, where it uses these verified localizations as precise visual anchors to parse the final structured data. To address the limitations of existing benchmarks, we also construct ChartP-Bench, a new and highly challenging benchmark for chart parsing. Our work also highlights VSR as a general-purpose visual feedback mechanism, offering a promising new direction for enhancing accuracy on wide range of vision-centric tasks.
INTRODUCTION
Large Vision-Language Models (LVLMs) have undergone rapid development (Bai et al., 2023b; Wang et al., 2024a; Bai et al., 2025), evolving from handling simple tasks to tackling complex and challenging ones. The emergence of models with "thinking" capabilities (Wei et al., 2022), such as OpenAl ol (Jaech et al., 2024) and Gemini-2.5-Pro (Comanici et al., 2025), allows them identify and rectify errors present in their direct responses, thereby achieving superior performance. However, their reasoning and deliberation processes are predominantly conducted at the textual level. This paradigm is highly effective for purely text-based tasks, such mathematical problem-solving (Muennighoff et al., 2025; Chen et al., 2025), where textual feedback is well-suited for reflection and correction. In contrast, for tasks where the challenge lies in "visual perception", such as Chart Parsing, this text-centric self-correction yields minimal benefits, as illustrated in Figure 2. This observation motivates us explore a critical question: for tasks centered visual perception, can we introduce "visual feedback" to enable an effective form of visual reflection and refinement?
The Chart Parsing task serves as prime example of "visual perception" heavy challenge. As crucial visual medium for presenting structured information, charts contain data that is often precise and critical, demanding accurate extraction and utilization. Even minor errors data extraction can have significant impact on downstream tasks. Early Chart Parsing methods relied heavily on OCR and hand-crafted rules (Savva al., 201 I; Jung et al., 2017). which struggled to generalize to diverse chart types. More recently, approaches in the era of LVLMs typically train end-to-end on data pairs cnartaanootation format. While these models achieve promising results on simple
"This work is done during an internship in Shanghai AT Laboratory. Corresponding author

---

Published as a conference paper at ICLR 2026
Chart 1ite
"Lobel ar, "Label ||a.00/0...222....00//0303333333
"Label
"Label A".. "Label (o0_,,
"Lahel.
(298,57)
e""
"Label (S8, (68, (75, (125, 64) missiin (225,55)(255 (2755,)))
"Lobel 6%_ "Lobel 6%
(b) Error Cases on Chartt Parsing Task
Input
Chart Tile
"title": Title", "L_title": Label", "y_title"
Axis Label",
Pixel-level Localizations (291,475[[21,,100
"Label
"35":
ChartV5R
Image Editing
"sa":
Check
"138": 60.
"1767:
Input
"225":
"290": 57
ChartVSR
impreccde / indialgnedd / amassions Image Editing
Wrong
"Labe1 87:
"25":
/ Antivcinorioe
"po":
"so":
Refined Pixel-level Localizations [291, [1291, 786]
"139": 55,
"225": 17,
"299": 34
Check
or
"Label c":
"35": 48,
Input
"sa": 10,
"78": 27,
Correct
"130": 68.
Chartv5R
"376": 49,
Parse the chart based on these piscels: (29,, 706]
Input
"325": 55,
Chartv5R
"298": 57
(b) An Example of Visusi Seff-Refinementt
Parsing Result
Figure 1: Overview of Chart Parsing and Visual Self-Refine (VSR). (a) Even strong models often fail to produce entirely correct results in single pass of Chart Parsing. (b) A schematic illustration of the proposed VSR method. The process follows the sequence indicated by the gray arrows.
charts, their performance degrades significantly charts with high visual information density and no explicit numerical labels (Han et al., 2023; Xia al., 2024; Chen al., 2024). They frequently exhibit issues such large numerical errors, misaligned data correspondences. data omissions, and even hallucinations of non-cxistent data points. (Zhang et al., 2025; Liu et al., 2025)
For charts with high visual density and no explicit numerical labels, such the shown in Figure (a), even strong closed-source models like GPT and Gemini perform unsatisfactorily, with their responses containing multiple errors. For humans, however, while meticulously extracting all information from such a chart might be a slow process requiring careful inspection, straightforward task. Although humans find it challenging read such charts accurately single glance and are prone to misreading or confusion, common human strategy is to use a finger to point each data point sequentially. The position of the finger acts as a "visual anchor". enabling more accurate value readings while effectively preventing issues like reading duplicate points, omitting points, or misaligning correspondences.
Inspired by this human strategy, we propose new paradigm named Visual Self-Refine (VSR). The core idea is for model generate pixel-level localization outputs, visualize them. and then feed these visualizations back to itself as visual input. This process allows the model to see its own previous work, giving it an opportunity to identify and correct potential visual perception errors such mislocalizations, omissions, hallucinations.
We instantiate VSR paradigm in the domain Chart Parsing by proposing the ChartVSR. As shown in Figure 1 (b), for an input chart, the model first generates pixel-level localization outputs. These outputs are then visualized and fed back into the model as visual feedback for it to inspect. Compared to purely textual feedback, visual feedback allows the model to its prior visual perception results. Once the model confirms that the visualized output is correct, uses these precise, "finger-pointing" locations parse the final structured data from the chart.
intuitively identify and correct
2

---

Published as a conference paper at ICLR 2026
Read all coordinates in this chart.
"Order Coue712021 (00.9600022---000 9200) (2020-05 000-0008899900000000 7900 7700) (3080-06 7000)
"hodutte"2022-55 www.g0000-000000000000000 wwwp:///2........oo 6900) (2020-07 5200022200000000.. 5100)
"Total Sale Price"2202 05 00.9:00g2021 01,8800 .. (202) 01,5100) (2020 07, 270000022000 06.0)
GPT-40
Check the correctness of the coordinates and refine the errors.
original
Image
Perform coor are correct chart.
GPT-40
(a) Direct Self Check & Refine
Insight of Visual Self ReffRe
the visualization coordinate chart. Check the coordinntes and refine the errors.
Some coordinates ore incorrect
Incomeet Entriies (values do not match the visual bons):
* 2020-07
Totul Sale Price is visuslly oround 800, but listed os 1700
correction
2020 Tahol Sate Price is weully around 4000, listed 2400
Recormancedd correction: 4000
2020 69
Price wauslly 3000,
3000
2020-11
Total Sale Price is visulll around 5000, but listed os 7400 Recommenced correction: 5000
Perform better
(b) Visual Self Check & Refine
Figure 2: Insight of Visual Self-Refine (a) Without visual feedback, strong LVLM (GPT-40) fails to identify its own parsing errors. (b) In contrast, when the model's 's output is visualized onto the chart (yellow markers). With this explicit visual feedback, the same model can now readily spot discrepancies and identify its mistakes. This highlights the ineffectiveness of direct self-correction for vision-centric tasks and motivates our proposed Visual Self-Refine approach.
Furthermore, beyond existing Chart Parsing benchmarks, have constructed ChartP-Bench. new and highly challenging benchmark, created by manually selecting and annotating complex charts from multiple data sources. Crucially, VSR, as a visual feedback paradigm, is not limited to Chart Parsing. We also demonstrate its potential applicability on other tasks, including Visual Counting and Visual Grounding. In summary, our contributions are threefold:
a) We introduce the concept of Visual Self-Refine (VSR), novel paradigm that utilizes visual feedback for self-correction, offering a promising new direction for tackling vision-centric tasks.
b) We instantiate the VSR paradigm in the Chart Parsing domain with our ChartVSR model, experimentally demonstrating the paradigm's 's effectiveness in improving parsing accuracy. c) We present Chartt-BBnnhhe high-quality and challenging benchmark, to foster future research in the Chart Parsing domain.
2 METHODS
2.1 CHART PARSING
Chart Parsing is fundamental visual perception task aimed at extracting structured numerical and textual information from chart image, such as converting line plot into its underlying data json. This process is crucial step for enabling precise downstream reasoning, like chart-based question answering. Early work such as DePlot (Liu et al., 2022a) framed the task as plot-to-table generation. ChartVLM (Xia et al., 2024) enhance performance through supervised fine-tuning chart-specific datasets. OneChart (Chen et al., 2024) introduce numerical loss to explicitly supervise numerical accuracy. Existing models still struggle with high-visual-densio charts, frequently exhibiting data omission, hallucination, or numerical deviations. Further related works are provided Appendix C.
2.2 CHARTVSR
Our ChartVSR model is built upon Swen2.55VL-33. (Bai et al., 2025), an open-source LVLM. Its architecture consists of three components: Vision Transformer based vision encoder, Qwen2.5 language model as the reasoning core, and an MLP-based merger to connect them. The vision encoder divides the input image into 28 28 non-overlapping visual patches and generates dense
3

---

Published as a conference paper at ICLR 2026
visual embeddings that preserve the spatial granularity essential for Chart Parsing. We finetune this model using a specialized data curation and training pipeline (detailed in Section 3).
2.3 VISUAL SELF- RELFNE FOR CHART PARSING
The core of ChartVSR is the Visual Self-Refine (VSR) paradigm, which decomposes the traditional one-shot parsing process into two logical stages: the Refine Stage and the Decode Stage. This design mimics the human behavior of using a finger to aid in reading chart, aiming to ensure perceptual accuracy through explicit visual feedback.
Refine Stage: Iterative Refinement via Visual Feedback The goal of this stage is for the model to "point out" all key data elements in the chart, much like a human using finger. We provide the model with the original chart image. The model's 's task is read the values directly, but output list of Pixel-level Localizations (a series of [x, y] pixel coordinates corresponding each data point).
(0,0)
Pixel Coordinate System
541
lendtng dectling
17.5
mabiii
12.5
This key step deconstructs the complex chart pars- ing task into simpler visual grounding problem, where the model can focus solely on "where" the data points are, without the immediate need to parse "what" they are or "how much" they represent. The generated Pixel-level Localizations serve as interme- diate, structured visual anchors. Subsequently, we vi- sualize these Pixel-level Localizations on the original chart image using predefined markers. This edited im- age is then fed back into the model as explicit visual feedback. By observing its own previously generated markers, the model can identify imprecise localiza- tion, omissions, or hallucinations, and subsequently output a set of corrected Pixel-level Localizations. This "generate-feedbar -eereeeecccckceeree"""" loop can be per- formed iteratively until the model indicates that the visualize localizations are correct or a preset maximum number of iterations reached.
10.0
458
8.9 7.5
5.0
2.5
Chart Coordinate System
Pixel-levell Localization: [541, 458]
Chart Coordinates: (88.0, 8.9)
Figure 3: An illustrative case of ChartVSR.
Decode Stage: Parsing from Visual Anchors Once the Refine Stage is complete, obtain set of high-precision Pixel-level Localizations that have been verified by the model itself. In the Decode Stage, the model receives the original chart image along with this final set of Pixel-level Localizations as input. At this point, the model's 's task shifts from "localization" to "interpretation".
Guided by these precise visual anchors, the model correlates them with the chart's visual elements. For instance, as shown in Figure 3, for a data point on a bar chart, the model utilizes its Pixel-level Localization [ [541, 458] to infer its value within the chart's coordinate system (i.c., Chart Coordi- nates (88.. 0, 8) and associates it with the corresponding legend label. By decoupling the two sub-tasks of perception (localization) and parsing (interpretation), and using Pixel-level Localizations as the bridge, CSR effectively mitigating common errors such as data misalignment, omissions, and numerical deviations.
TRAINING DATA
3.1 ANALYSIS OF EXISTING CHART DATASETS
Existing chart datasets can generally be divided into two categories: (1) non-parsable QA datasets and (2) fully annotated parsable datasets. Non-parsable Question Answering (QA) datasets (Xu et al., 2023; Wang et al., 2024b) typically present tasks in a question-based format that probes specific aspects of a chart. Consequently, they require only partial chart-related information to answer the questions, rather than full parsing of the chart's s entire contents. This characteristic allows
4

---

Published as a conference paper at ICLR 2026
Others
Chart
High Quality Data
Quality
Line Chart
1
Human Check
Data
Stack Chart
matplctllb
ChartVRR
Bor Chart
Render
Hybrid Canfig
Specific Annnntst
Clart Cooed
Generator
Templates
*ax.trofstott""
Model Check
Filicred CharVVSR Datn
Pisel Coord
Figure 4: Overview of our Data Engine. (a) The ChartVSR data generation pipeline. (b) Example data generated by our data engine, showcasing high diversity and high quality.
them to be conveniently collected from publicly accessible sources such as websites and academic papers. In contrast, parsable chart datasets (Masry et al., 2022; Methani et al., 2020; Xia et al., 2024) necessitate comprehensive annotations, such the original drawing code, tables, JSON, or CSV files corresponding to the charts. Such detailed annotations are rare in real-world sources, making it difficult to acquire large-scale sal-worldd data. Consequently, almost all parsable chart datasets are synthesized using plotting tools such as Matplotlib, Pyecharts, or MATLAB. Existing synthesized parsable datasets commonly suffer from limited diversity, implicit regularities, and data homogeneity,
a) Lack of Stylistic Variety: Existing datasets typically apply adjustments to only few visualiza- tion parameters (e.g., color, pattern, or fonts), resulting in similar visual styles across generated charts. This stylistic homogeneity significantly hinders the model's generalization capability, leading to poor performance when applied to visually diverse real-world charts.
b) Implicit Regularities and Data Homogeneity: Many existing parsable datasets sourced from the web or specific statistical databases. These datasets often contain monotonic trends, recurring strings, or implicit regularities (c.g., specific company names frequently co-occur, or numeric labels around the year 2000 typically represent incremental year sequences). As shown Appendix D, models trained on them often overfit these patterns, showing fast convergence but poor real-world generalization, leading to hallucinations and degraded performance.
3.2 DATA ENGINE
To overcome these limitations, we meticulously designed and developed highly diverse, robust, and scalable data engine. This engine consists of two core components: Parameter-Agnostic Templates and Hybrid Configuration Generators. As illustrated in Figure (a), for each chart type, first construct a Garameter-Agnostee Template, which is a universal plotting script capable of generating both chart and its corresponding annotations simultaneously. These templates comprehensively cover all relevant plotting parameters available in Matplotlib for the specific chart types.
Building upon these templates, developed Hybrid Configuration Generator that automatically produces a wide range diverse chart configurations. It samples across Matplotlib's full visualization space, covering dimensions such color schemes, fonts, and layouts. To synthesize chart elements
5

---

Published as a conference paper at ICLR 2026
(e.g., labels, titles, data points), it combines real-world content with randomly generated content. This process maximizes stylistic diversity and avoids the implicit regularities common in existing datasets, thereby promoting more robust Chart Parsing capability in the model. To quantitatively demonstrate the superiority our data, we propose four key metrics for evaluating dataset quality. As shown in Table dataset excels these metrics. (Detailed calculation methods and explanations for each metric are in Appendix E)
In the initially generated dataset, we observed that small portion of charts suffered from poor visual quality due to unusual layout configura- tions suboptimal combinations of plotting parameters produced by the Hybrid Configu- ration Generator. To address this, we manu- ally reviewed and categorized charts of various types, identifying and labeling low-quality sam- ples. Based these positive and negative ex- amples, we finetuned Qwen2-VI. VL-BBB model (Wang et al., 2024a) to serve data filter. This model was then applied to the raw dataset to discard luw-quality charts. Through this entire pipeline, we constructed a final high-quality dataset containing approximately 800K samples.
Table 1: Quantitative comparison across four chart datasets. (1) indicates higher is better, while (L) indicates lower is better. Best values are in bold and second-best -est are underlined.
Metric
Ours
ChartQA
PlotQA
ChartX
Avg. Data Points/ham (T)
22.69
15.04
12:48
10.95
Unique Numerical Ratio (1)
59.30% %
22. 22.77%
50.70%
4.06%
Arg. Abs, Conveation (1)
0.38
0.76
0.77
0.71
Avg. Top (1)
Study (4-poknr) (1)
3.15
2.68
3.3 TRAINING DATA FORMAT
The format designed to align with the two stages of ChartVSR. For the Refine Stage, we construct three types of training samples to teach the model to: a) generate locations from scratch, b) correct erroneous markers (e.g., missing, shifted, or hallucinated), and c) confirm correct markers to terminate the refinement loop. For the Decode Stage, the objective is to train the model to parse structured data, given accurate visual anchors. training sample consists of the original chart image and set of verified-as-accurate pixel-level coordinates. The model's task is parse the chart into structured ISON object containing numerical values and metadata, guided by these precise visual anchors.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
We conduct our training on a server equipped with 8 NVIDIA H200 GPUs, each with 144GB memory. To prevent mismatches between our Pixel-level Localizations and the model's internal coordinate system due resizing, we resize the longer side of each input image 1036 pixels, multiple of 28. The model is trained for one epoch with total batch size of 128. We use the Adam W optimizer with a learning rate of x 10-7, warmup ratio of 0.03, and a weight decay of 0.01. All three components of the model are unfrozen and trained end-to-end.
4.2 BENCHMARKS AND METRICS
Existing Benchmarks Although many existing benchmarks are related charts, most focus on Chart Question Answering tasks. Only a few include fully annotated, parsable chart structures suitable for evaluating Chart Parsing, such as ChartQA-SE (Chen et al., 2024: Masry al., 2022), PlotQA-SE (Chen et al., 2024; Methani et al., 2020), and ChartX-SE (Chen et al., 2024; Xia et al., 2024). ChartQA-SB contains real-world charts with natural layouts and visual designs; PlotQA-SE consists of charts synthesized from real-world data; and ChartX-SE is fully synthetic, rendered using Matplotib. All three benchmarks are converted from Chart Question Answering datasets, and some contain explicit numeric annotations the chart images.
During a manual spot-check on ChartQA-SE, we discovered several mislabeled samples, which prompted full inspection of all 1,509 test samples. We found that over 100 samples had incorrect ground-trute annotations (see Appendix G for details). To ensure evaluation reliability, we removed these erroneous cases and created cleaned subset, named ChartQA-SSE-Clean.

---

Published as a conference paper at ICLR 2026
Table 2: Results on general Chart Parsing benchmarks. We report Standard SCRM Average
Precision (AP): Strict (Jthr 0, Cthr 0), Slight (Jehr 2, Cthr 0.05), and High (Jthr = 5. Cthr = 0.1 The best results are highlighted in bold and second- are underlined. Model sizes are listed for fair comparison.
ChariQA-SE -EE-Clc
PloIQA-SE
ChartX-SE
Model
Size
AP-Strict
AP-Slight
AP-High
AP-Strict
AP-Slight
AP-High
AP-Strict
AP-Slight
AP-High
DePlot
1.3B
55.61
66.89
70.89
3.11
16.49
26.50
14.84
33.42
42.42
ChartAst
13B
33.81
63.82
71.23
5.18
48.67
56.08
18.78
36,76
47.28
CLM
77.17
82.11
3.81
46.83
34.92
OneChart
0.2B
66.20
78.89
83.92
34.56
84.18
86.10
44.55
53.12
59.72
CharVVSR
3B
65.70
83.69
85.64
34.99
84.61
88.10
43.99
53.48
62.89
Table 3: Results
ChartP-Bench. The benchmark is divided into Easy subset (charts with < data points) and a Hard subset (charts with >18 data points). We report Standard SCRM Average Precision (AP): Striet (Jehr O, Ethr = 0), Slight (Jehr 2, ethr 0.05), and High (Jth-
ethr = 0.10). The best results are highlighted in bold and second-best are underlined. Model sizes are included for fair comparison.
Easy Subset
Hard Subset
Model
Size
Avg.
AP-Strict
AP-Slight
AP-High
Avg.
AP-Strict
AP-Slight
AP-High
Avg.
Strong Closed-souree LVLMs
GPT-4o Gemini-2.5-Flash
0.00
1.69
6.73
2.81
0.00
0.99
4.14
1.71
2.09
0.00
37.00
55.57
30.86
0.00
23.84
40.22
21.35
24.62
Gemini-2.5-Pro
0.06
36.51
46.05
27.54
0.00
46.11
66.39
37.50
34.07
Chart L.V.Ms
DePlot
1.3B
0.00
9.73
15.35
8.26
0.00
1.27
4.19
1.82
ChartAst
13B
0.00
1.56
5.79
2.45
0..00
LLM
7.3B
0.00
1.74
6.34
2.70
0.00
0.48
3.56
1.35
1.81
OneChart
0.2B
0.00
4.75
18.89
7.88
0.00
1.98
9.48
3.82
5.22
ChartVSR
3B
0.11
51.95
67.44
39.83
0.02
49.30
63.67
37.66
38.41
ChartP-Bench (Chart Parsing Benchmark) To address the limitations of existing benchmarks, we and highly challenging benchmark for chart parsing. We manually selected, filtered, annotated, and cleaned 1,200 high-quality chart images from multiple sources. This benchmark is characterized by its high difficulty and visual information density, with each chart containing over 20 data points on average. Unlike existing datasets, ChartP-Bence has undergone rigorous quality control to ensure there are no annotation errors and avoid issues such as stylistic homogeneity and implicit regularities.
introduce ChartP-Bence
Evaluation Metrics We adopt the Structuring Chart-oriented Representation Metric (SCRM) (Xia al., 2023) for evaluation. SCRM quantifies the structural consistency between a model's parsed result and the ground truth at both the image and dataset levels. It first computes the Intersection over Union (IoU) between predicted and ground-truth data triplets based on a predefined string similarity threshold, Jahr- and a relative numerical error threshold, Ctr- The overall performance is then evaluated by calculating the Average Precision (AP) over range of loU thresholds. For more details on SCRM, please refer to Appendix F.
4.3
MAIN RESULTS
ChartVSR demonstrates robust and compelling performance on existing Chart Parsing bench- marks. As shown in Table 2, ChartVSR achieves competitive performance across three existing Chart Parsing benchmarks. It is noteworthy that these benchmarks vary in data origin and style: ChartQA-SE--CIe is sourced from real-world charts, PlotQA-SE is synthesized from world data, and ChartX-SB is entirely synthetic. The excellent results of ChartVSR across these datasets with different distributions, particularly under the more lenient Slight and High settings, demonstrate the strong generalization capabilities of our proposed method.
7

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study on ChartVSR components. The benchmark is divided into an Easy subset (charts with <18 data points) and Hard subset (charts with > 18 data points). We report Average Precision (AP) under three evaluation criteria: Strict (Jthr 0, ethr 0), Slight (Jurr
Ethr 0.05), and High (Jehr 5, Ethr 0.10). The best results are highlighted in bold.
Easy Subset
Hard Subset
Model
Avg.
AP-Strict
AP-Slight
AP-Iligh
Avg.
AP-Strict
AP-Slight
AP-Iligh
Avg.
ChartVSR
0.11
51.95
67.44
39.83
0.02
49.30
63.67
37.66
38.41
w/o VSR
0.10
50.83
66.67
39.20
0.01
47.50
57.93
35.14
36.54
VSR Pixel
66.19
38.97
57.50
34.71
36.17
Table 5: Per-round analysis of VSR's error detection Table 6: VSR's computational cost. and correction capability. We report the total number of The cost is measured by the total num- charts with visual perception errors initially (Round 0) and ber of model forward inference calls, after each refinement round. We also calculate the model's where Nmaz denotes the maximum recall on errors from the previous round and its precision in number of allowed refinement rounds. confirming correct localizations.
Refine Round
# Eror Samples
Error Recall
Correct Confirmation
Method
Inference Calls
Baseline (w/o VSR)
51
92.3%
88.2%
VSR (Nmazz 1)
3
54
85.5%
76.0%
VSR (N max
3
52
85.8%
76.6%
VSR (N Max 12, 2 1)
to (n+2)
ChartVSR excels on the challenging ChartP-Bence On the more challenging ChartP-Benchh ChartVSR exhibits a significant performance advantage over other models, detailed in Table 3. Among the powerful closed-source models, the Gemini series (especially Gemini-2.5-Pre demon- strates strong capabilities, far surpassing other baselines, while GPT-4o struggles to handle this complex task (GPT-4o's outputs are provided in Supplementary Material), On both the Easy and Hard subsets, SSRrtVSR substantially outperforms all competing models, including the strongest Gemini model. As ChartVSR performance over general-purpose model like Gemini is expected. Furthermore, these results corroborate our earlier assertion that existing chart parsing models perform poorly on charts with high visual information density and stylistic diversity, which truly test visual perception capabilities. Furthermore, nearly all models score close to zero on the AP-Strict metric. This is because the numerical labels in ChartP-Bench are highly precise (e.g., 17.12 instead of integers like 10 or 20), making the strict requirement of zero numerical error (Carr 0) extremely demanding.
specialized model designed specifically for chart parsing, its superior
4.4 ANALYSIS ON VSR
VSR significantly boosts chart pars- ing performance, especially on com- plex charts. Table 4 shows that the full ChartVSR model achieves a clear perfor- mance gain over the version without the VSR module (w/o VSR). This improve- ment particularly pronounced on the Hard subset, which contains more data points and is visually denser. This sug- gests that for simple charts, errors primarily stem from minor numerical deviations. For complex charts, however, the main perfor- mance bottlenecks are structural errors like data point omissions or misalignments, and VSR's visual feedback mechanism is specifically designed correct these visually salient errors. Furthermore, comparing the "w/o VSR" and "w/o VSR & w/o Pixel" variants reveals that merely introducing Pixel-level Localization without the accompanying VSR yields minimal performance gains.
VSR Round
VSR Round 2
Figure 5: A multi-round refinement of ChartVSR. The initial prediction (left) contains multiple errors, which are fully corrected after two rounds of refinement.
8

---

Published as a conference paper at ICLR 2026
How mony binds one the pictune?
Pisel-lanii Lecdisst ons Visudlastion on for VSR
Ground the white face sheep.
Sounding Box Visusiiatioo for VSR
Wrong
(6) Visual Counting
Visucl Grounding
Figure 6: The potential of the VSR paradigm on Visual Counting and Grounding. (a) For Visual Counting. the model generates pixel-level localizations and the visualization check for errors like omissions
duplicates. (b) For Grounding, the model visualizes its predicted bounding box self-assess whether the localization is accurate and requires refinement. This illustrates the broad applicability of VSR as a general- purpose visual feedback mechanism for self-correction
VSR shows highly effective error correction in the first round, with diminishing returns sub- sequently. As detailed in Table 5, the model successfully identifies 92.3% of initial errors in the first refinement round, reducing the number of error samples from 1 110 to 51 and correcting over half of the errors. In subsequent rounds, however, the total number of error samples does not decrease significantly, even though the model maintains an error recall rate of over 85%. This suggests that the remaining, more stubborn errors may stem from the model's deeper perceptual limitations, which are difficult to resolve through simple multi-round iteration. potential direction for improvement could be to adopt a bootstrapping-lit strategy: use the trained model for large-scale inference, collect cases where refinement fails, and add these hard cases to the training data for iterative finetuning. This could potentially enhance the model's ability to resolve complex errors in later rounds.
VSR enhances performance at the cost of increased inference overhead. As shown in Table 6, the VSR paradigm introduces additional forward inference calls. While baseline model requires only a single call, VSR requires at least three (one for initial localization, one for visual feedback confirmation, and one for final decoding). This a deliberate trade-off, analogous to the strategy employed by recent "thinking" large language models like OpenAl ol and Gemini 2.5 Pro. which invest more computational resources during inference for deliberation and refinement to achieve higher accuracy and reliability.
4.5 POTENTIAL OF VSR ON OTHER TASKS
VSR is a general visual feedback paradigm, with applications extending beyond Chart Parsing. As illustrated in Figure 6, VSR can be seamlessly adapted to other tasks that demand high-fidelity visual perception, such as visual counting and visual grounding. For risual counting, a model can first output Pixel-level Localizations for all target objects. These localizations the original image and fed back to the model, giving it an opportunity to inspect for errors such as omissions or superfluous counts and then refine its answer. The application of VSR to visual grounding is even more natural, as the task's native output is a bounding box in pixel coordinates. The predicted bounding box can be drawn directly onto the image and returned the model, allowing it to self assess whether its localization is accurate and if the box requires adjustments in position or scale. VSR holds significant potential as a versatile tool for self-correction vision-centric tasks.
then visualized
CONCLUSION
In this work, we addressed the limitations of existing Large Eision-Language Models on visually intensive perception tasks like Chart Parsing by introducing Visual Self-Refine (VSR), novel paradigm. Inspired by the human strategy of using a finger to aid in chart reading, VSR enables model to generate pixel-level outputs, visualize them, and use this visualization as direct visual feedback to iteratively inspect and correct its own perceptual errors. We instantiated this paradigm with our ChartVSR model, which decouples the task into Refine Stage and Decode Stage, significantly improving the accuracy of data extraction. Our contributions are threefold: a) we proposed the general VSR paradigm new self-correction mechanism for vision-centric tasks; b) we implemented the ChartVSR model and experimentally demonstrated its sfate-of-the art performance, especially on our new, challenging benchmark, ChartP-Bench where it substantially outperforms

---

Published as a conference paper at ICLR 2026
existing methods; and c) we introduced the high-quality ChartP-Bende to facilitate future research in this domain. Looking forward, the VSR paradigm holds sigmificant potential for broad range of other tasks requiring precise visual perception, such visual counting and visual grounding. Future work could also explore methods to enhance the efficiency of the refinement process or leverage refinement failures hard negatives for iterative model improvement.
ACKNOWLEDGMENTS
This project is funded in part by Shanghai Artificial Intelligence Laboratory, Shanghai Innovation Institute, the Centre for Perceptual and Interactive Intelligence (CPII) Ltd under the Innovation and Technology Commission (ITC)'s s InnoHK. Dabua Lin a PI CPII under the InnoHK.
REFERENCES
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: frontier large vision-language model with versatile abilities. arXiv preprint arXiv:238.. /22300....96,.. 2023b.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 13X1v:2502.13923, 2025.
Victor Carbune, Hassan Mansoor, Fangyu Liu, Rahul Aralikatte, Gilles Baechler, Jindong Chen, and Abhanshu Sharma. Chart-based reasoning: Transferring capabilities from Ilms to vims. arXiv preprint arXiv:2403. 12596, 2024.
Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Onechart: Purify the chart structural extraction via one auxiliary token. In Proceedings of the 32nd ACM International Conference on Multimedia, pp- 147-155, 2024.
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4y: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12799 2023.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision- vision-language models? In The eight Annual Conference on Neural Information Processing Systems, 2024a.
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, al. Sharegpt4video: Improving video understanding and generation with better captions. arXiv preprint arXiv:2406.04325, 2024b.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang. Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of- thought for reasoning large language models. arXiv preprint arXiv:2503. 09567, 2025.
Wei-Lin Chiang. Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* a chatgpt quality. See https://vviuna. Imsys. org (accessed 14 April 2023), 2023.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint 2rXiv:2204.02311. 2022.
Gheorghe Comanici. Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.0626/ 2025.
10

---

Published as a conference paper at ICLR 2026
Wenliang Dai, Junnan Li. Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose models with instruction tuning, 2023.
purpose vision- -lanuala
Shuangrui Ding, Rui Qian, Haohang Xu, Dahua Lin, and Hongkai Xiong. Betrayed by attention: A simple yet effective approach for self-supervised video object segmentation. In European Conference on Computer Vision, pp. 215-233. Springer, 2024.
Chaoyoo Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. Mme: comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 13394, 2023.
Yucheng Han, Chi Zhang, Xin Chen, Xu Yang. Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: A multimodal Ilm for chart understanding and generation. arXiv preprint arXiv:2311.16483 2023.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey,, al. Openai ol system card. arXiv preprint arXiv:24/2.16720, 2024.
Daekyoung Jung, Wonjae Kim, Hyunjoo Song, Jeong-in Hwang, Bongshin Lee, Bohyoung Kim, and Jinwook Seo. Chartsense: Interactive data extraction from chart images. In Proceedings of the 2017 chi conference on human factors in computing systems, pp. 6706-6717, 2017.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping tanguage-image pre- training for unified vision- understanding and generation. In International Conference Machine Learning, pp. 12888-12900. PMLR, 2022.
Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual language reasoning by plot-to-table translation. arXiv preprint arXiv:2212. 2022a.
Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Martin Eisenschlos. Matcha: Enhancing visual language pretraining with math reasoning and chart derendering. arXiv preprint arXiv:2212.00660 2022b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023a.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IÊEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26296-26306, 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, oer, and world knowledge, January 2024b. URL https: v/ www...
01-30- 10000-llavee llava-mann next//.
Junteng Liu, Weihao Zeng, Xiwen Zhang. Yijun Wang, Zifei Shan, and Junxian He. On the perception bottleneck of vims for chart understanding. arXiv preprini arXiv: 18433 2025.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? 2023b.
arXiv preprini 2307
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203. 10244, 2022.
Fanging Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. Chartassisstant: A universal chart multimodal language model via chart-to-table pre- training and multitask instruction tuning. arXiv preprint arXiv:2401.0223 2024.
11

---

Published as a conference paper at ICLR 2026
Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. Plotqa: Reasoning scientific plots. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1527-1536, 2020.
Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501. 2025.
OpenAl. Chatgpt. https: http://hhh..........././o com/, 2023b.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 2022.
Rui Qian, Shuangrui Ding, and Dahua Lin. Rethinking image-to-video adaptation: An object-centric perspective. In European Conference on Computer Vision, pp. 329-348 Springer, 2024a.
Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. Advances in Neural Information Processing Systems, 37:119333-119360, 2024b.
Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video Ilms with active real-time interaction via disentangled perception, decision, and reaction. In Proceedings of the Computer Vision and Pattern Recognition 2025.
Conference, pp.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, pp. 8748-8763. PMLR, 2021.
Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pp. 393 2011.
InternLM Team, Internim: A multilingual language model with progressively enhanced capabilities, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09280 2023a.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision- language model's 's perception of the world at any resolution. arXiv preprint 2rXiv:2409.12/11 12191, 2024a.
Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal lls. Advances in Neural Information Processing Systems, 37:1 13569-1 13697, 2024b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of thouhht prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24831 2022.
Renqiu Xia, Bo Zhang, Haoyang Peng. Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Yu Qiao, and Junchi Yan. Structchart: Perception, structuring, reasoning for visual chart understanding. arXiv preprint arXiv:2309. /268, 2023.
Renqiu Xia, Bo Zhang. Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Peng Ye, Min Dou, Botian Shi, et al. Chartx chartylm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint 422 2024.
12

---

Published as a conference paper at ICLR 2026
Long Xing, Qidong Huang, Xiaoyi Dong. Jiajie Lu, Pan Zhang. Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision- langunla models via pyramid visual redundancy reduction. arXiv preprint arXiv:2410.172477 2024.
Zhengzhuo Xu, Sinan Du, Yiyan Qi. Chengjin Xu, Chun Yuan, and Jian Guo. Chartbench: A benchmark for complex visual reasoning in charts. arXiv preprint arXiv:2312.15913 2023.
Zhengzhuo Xu, Bowen Qu, Yiyan Qi Sinan Du, Chengjin Xu, Chun Yuan, and Jian Guo. Chartmoe: Mixture of expert connector for advanced chart understanding. arXiv preprint arXiv:2409.03277, 2024.
Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang. et al. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309. 10305, 2023.
Xiang Yue. Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint arXiv:2311. 2023.
Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, et al. Map-neo: Highly capable and transparent bilingual large language model series. arXiv preprint arXiv:2405.19322 2024a.
Kai Zhang, Jianwei Yang, Jeevana Priya Inala, Chandan Singh, Jianfeng Gao, Yu Su, and Chenglong Wang- Towards understanding graphical perception in large multimodal models. arXiv preprini arXiv:2503.10855 2025.
Pan Zhang. Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Haodong Duan, Songyang Zhang, Shuangrui Ding, et Internln-xcomposer:: A vision-language large model for advanced text-image comprehension and composition. arXiv preprint ar arXiv:2309.55// 2023.
Pan Zhang, Xiaoyi Dong, Yuhang Cao. Yuhang Zang, Rui Qian, Xilin Wei, Lin Chen, Yifei Li, Junbo Niu, Shuangrui Ding, al. Internim-xcompoeer 5-omnilive: A comprehensive multimodal system for long-term streaming video and audio interactions. arXiv preprint 14Xiv:2412.095599 2024b.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
13

---

Published as a conference paper at ICLR 2026
A REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, we provide comprehensive details and resources in the main paper and the Supplementary Material. In Section 3, we have detailed the format of our training data, model implementation specifics, hyperparameter settings, and the full training pipeline. Furthermore, in the Supplementary Material, we have included key resources required reproduce our core findings. These include: (1) ChartP-Benchl containing images and their corresponding ground-truth annotations; (2) visualization results from the ChartVSR inference process; (3) the example evaluation code for result assessment. We will open-source the data engine used for generating training data and model weights, upon the acceptance of this paper to facilitate further research within the community.
B THE USE OF LARGE LANGUAGE MODELS
We utilized Large Language Models (LLMs), such as Google's ' Gemini, as a writing assistance tool. The use of the LLM was strictly limited to improving the clarity, conciseness, and grammatical cor- rectness of the text. Specific applications included proofreading for typographical errors, rephrasing complex sentences enhance readability, and ensuring consistent terminology throughout the paper.
C RELATED WORK
Chart-Specific Large Vision-Language Models. With the rapid advancement of large language models (LLMs) (Chiang et al., 2023; Touvron et al., 2023a; Yang et al., 2023; Team, 2023; OpenAl, 2023b; Zhang et al., 2024a; Ouyang et al., 2022; Chowdhery et al., 2022; Chen et al., 2024a;b), researchers have developed powerful multimodal systems by integrating visual encoders with lan- guage backbones and aligning them using large-scale image-text data (Chen et al., 2023). From early works such as CLIP (Radford et al., 2021), BLIP (Li et al., 2022), InstIUct-BLIP (Dai et al., 2023) to more recent models like MiniGPT-4 (Zhu et al., 2023), LLaVA series (Liu al., 2023a; 2024a;b), and Qwen- VL series (Bai et al., 2023b; Wang et al., 2024a; Bai et al., 2025), general- purpose large vision-language models (LVLMs) have achieved impressive performance across wide range of vision-language tasks (Liu et al., 2023b; Fu et al., 2023; Yue et al., 2023; Xing et al., 2024). Concur- rently, recent advancements have significantly enhanced LVLMS complex visual perception (Zhang et al., 2023; Qian et al., 2024b; Zhang et al., 2024b; Qian et al., 2025; 2024a; Ding et al., 2024), establishing robust baselines for handling intricate visual details. However, applying these general LVLMs directly chart-related tasks often leads to suboptimal results. To bridge this gap, several chart-specific LVLMS have been proposed, including LaaM (Han et al., 2023), ChartVLM (Xia et al., 2024), ChartPaLI (Carbune et al., 2024), and ChartAst (Meng et al., 2024), which rely on supervised fine-tuning using chart-specific datasets to align visual and linguistic modalities. More recently, ChartMoE (Xu et al., 2024) introduced a powerful mixture-offexxert aligned chart-text pairs in formats like JSON, tables, and CSV, achieving strong performance on multiple chart QA benchmarks. Overall, most existing chart-specific LVLMs focus primarily on high-level understanding tasks such as chart question answering (Masry et al., 2022: Methani et al., 2020; Xu et al., 2023; Wang et al., 2024b), while paying limited attention to the low-level fundamental task chart perception.
model trained on
Chart Parsing Task. Chart Parsing involves extracting structured numerical and textual information from visual charts, crucial step for enabling precise downstream reasoning. Early work DePlot (Liu et al., 2022a) framed the task as plot-to-table generation: a MATCHA (Liu et al., 2022b) is fine-tuned to parse the chart's underlying table. Similarly. ChartAST (Meng et al., 2024) also trains models directly on large-scale chart-to-table datasets. Recently, OneChart (Chen et al., 2024) proposes an integrated solution that enhances LVLMs with the number loss. By explicitly supervising numerical accuracy and using an auxiliary token to assess answer reliability, OneChart achieves more precise chart parsing than other models. Despite these efforts, existing chart parsing models still struggle with complex, high-density charts, frequently producing missing data, hallucinated data, or severe numerical deviations. Moreover, some of these models can offer the confidence score of their own answer but lack the ability to refine or self-correct incorrect outputs, which limits their applicability in high-accuu nigh-accuracy scenarios.
14

---

Published as a conference paper at ICLR 2026
D CASES FROM REAL-WORLD CHARTS IN CHARTP-BENCH
Real-World Chart Image
ChartVSR "Point" Visualization
micarrSpp
mtcarssmpp
00
R
0
00
10
Index
Index
Human-annotated
Gemini-2.0-Fllsh
OneChart
ChartVSR
GT
"y_title":
"wEcorsses"",
"x.titl:": "utcarstepg".
"x_title": "Indox",
"y_titl":: "stcorsips",
"valurs":
"velues":
"y.title": "cturm"" "churne'sping"
"values":
12": 23,
"series2""
"valums":
12" 120.0"
12"1 21,
1gh: 21.
1gh: 169"
'3" 120-57
25,
23.
"5"1 "60"
23-5,
"28": "80"
120-7"
"11"1
311 "38.6"
114" 15,
22" 220.0
116" 10,
"37 "36.6" 66"
113" 17.5.
137" 10,
10,
199 31,
15"
3367
331% 13.
21 22.5 5,
'27* 19,
222" 21
22 15
199": 15.
23" 114
"25 18.5.
110": 19,
26" -20
26" 13.
132" 21
25" -22
26"
"38": 19.
"11": "29.1"
"31": 10,
"12": "26.6"
"32"
"35") "20.9"
Figure 7: case 1
15

---

Published as a conference paper at ICLR 2026
Real-World Chart Image
ChartVSR "Point" Visualization
Poylls
Deoupp les BIa
pesoppis BIG
uCepor-thel
sCepchshaph
700
700
Oveest Trrugtout (Nrsise))
Queest Thrustout (oreise))
600
600
500
500
400
300
300
200
200
100
100
0
0
YCSB workliad
YCSB workload
Pigure 9: Ovenall throughput when using 256 YCSB client threads for different cey-value
Pigure 9: Overall throughput when using 256 YCSB client threads for different cry-valoe
Human-annotated
Semini--20-Fllast
OneChart
ChartVSR
GT
"tstie": "None",
"titte": "Overall theoughout when 256 YESS illest chroads for difrerent key-value stores." "i.titl": "yesa worklows", "ytitle"": "Overoll Throughout (kops/secc", "valuss":
"title": "None",
"title":
"_title": "vesa acrillad",,
"source": "Mase", "x "itiiie": "Honz",
"r_titl":: "VSm vorkiced"
"_title": "Overall
thruggput (saps/sec)",
"y.Hirle:: "Dverall
"y_title: "Overall throughout (Kops/sec)", "volues". "scylla":
"values"
Throughout (kowp/SEC)", "voluss" "scycla":
"scylta":
"A": 25,
"att 45,
"28.3"
"A":
"o":
"c": 229.5"
*P"
"C": "51.2"
"chs
"ch: "34.0"
"aorooptie":
rg": 59
"C": "39.8"
ra": 250.
"P": 30
<C": 162.4"
"serosptke":
"a": 350
1, "serospike" [ 280,
"A8-2"
"A": "200"
"c": 300.
"59.5"
'n":
330,
Acrospace"
"uDepnt-trt--iol'
"a": 538,
"P":
"Ch: 169.5"
"Jpepot tnt-odo" to TATls "520.0 matt "6c0" nent 1607. An
"at': 630.
"c"s *45
"C": 625,
".Oapot cre-alo": "A"T 530,
"Bh: son,
"P": 379
"g"t 500,
147.3"
"onu "sce" "Se9"
600,
"45-9"
160,
"S2.3"
"JDepot spdeet
spoi7s
'D":
"A": 590.
*58.9"
nems 1687.
"P": 198
Hano 670.
"630"
"Cns 599.
"cmm *58.7"
"P") "382"
"D"1 020.
"G3.0""
"p"" 190
"sa-""
"C": "53.3""
"sa.0"
"53.2"
Figure 8: case 2
16

---

Published as a conference paper at ICLR 2026
Real-World Chart Image
ChartVSR "Point" Visualization
Multiple Lines Plot
Multiple Lines Plot
Line
Use 2
10
Human-annotated
Gemini-2.0-Flasi.
OneChart
ChartVSR
GT
"tstle": "Multiple Lines
"title": 'Nuitiple Lines
"title": "Multiple
"title": "Multiple Lines
Plot", 'x",
Plot".
LinesPlot", "source":
Plot
"X" "_title":
"yritte";
"_tiitle::
"y.tiele"::
LLin 1":
"Line
"Line 11:
tat: 3.5.
<3m
Hatt 22
rat: 5.5.
"o": *7.90"
14"
"g": "9.95"
75" "5.0",
1671
"a":
"Line {
'18":
"18":
"1071 "2.01"
"Line 2ns
"Line 2":
"g"t *9.96"
"Line 21:
myt: 7,
"1e": "3.00"
mam 19.00 27.07
tat:
ngtt 4,
13":
"gn: 5.
LLnn at:
19":
"4":
"7.97
"y":
"g":
"3.00"
tq":
Hots *2.8"
"18": 1
"18":
"10%: "1.15"
"Line at1
"Lioe st:
"Line at:
"1":
"3" 3,
Hant "3.807.
'2":
61
"2": "6.0"
"3":
16"1
"7" "0.07"
'g":
"a": "2.00"
"9":
"9":
"g" 66.007
'18": 1
"10":
120": "2.es"
Figure 9: case 3
17

---

Published as a conference paper at ICLR 2026
Real-World Chart Image
ChartVSR "Point" Visualization
Loe sande (16 letaa
- Lon bolle (P lelaa
Eo shy
Decosiing thee calli 2010
lecousiyy do
Decoating valich 2510
localiy dccoaaing
Human-annotated
Gemini-2.0-Flasi.
OneChart
ChartVSR
GT
"titie": "None",
"title": null. "x_title:: null. "itilee:: sull,
"title": "None", "source": "Mose",
"title": "Wone"
"Kttttle:: "Nore",
"x. "x_title": "None" "y title:"
"y title":
"y_title": stable
(79.344 (99.340)"
(79.348)"
"ans 5.8, "ant 5.9.
(79.348",
taf: 5.5.
*2" "5.07" "gn: 16 "6.01") "5.35"
"371
"3mh 6-1.
"4": 5.2
5.4
1, "forreasing (8.018)T: 5.8, 6.6, 25.2
1.
"Trereastigg (9.618)": "1"1
"Incrwering (4.018)
*d":
"ah: "14.4"
natt "24.8"
"Increasing then decreasiing (3.0m)" {
"Increasing then decreasing (3.885)
"focrwaeing then- decreantig (3.88 5)":
"s"c 12.5,
12-4, 22.5. 26.7,
22.5 17,
"12.0" "22.25" "3p.0",
"Decreasing "a": *2"1 12.5,
"Desrsasing (R.208)" "3"
"becrcasing
stable (8.258)":
"2%h *22.6"
"3") 12.7,
"2TS 12.5, 12.8. 11.2
"2% "12.27)
"4": 13
"3"2 "12-27
"10.0""
Figure 10: case 4
18

---

Published as a conference paper at ICLR 2026
Real-World Chart Image
ChartVSR "Point" Visualization
102
x
X
Number of ice cee-creams sold
Number of ice ice-crea re-cremms sold
94
94
90
90
86
86
82
82
70
66
66
62
62
58
58
54
54
50
50
1416188 1416182002222622830
14 1416182022242622833
Temperature ("C)
Temperature ("C)
Human-annotated
Gemini-2.0-Flash
OneChart
ChartVSR
GT
"titie": "None",
"titte": noll, "r_title": "Teaperature (T))
"title": "None", "sourre": "Mase", titiit"" "Tanserature
"title": "1"
"u_title": "Tesperature
"_title": "cenperature (c)"
(*E)"
"y_ittle:; "lamber of tea-
">Hitle": "Naaber of Ice-
(?C)",
"y_title:: "Nusher
creams sols".
creams unld"
"y. >ititl":: "Number of IC-
iceucreems sold", "volucs":
"values": 10,
"voluss":
cerons sold".
"valums"
"30":
228"
"18":
"11": "18
"63.4"
"287:
"99" "CB.2"
211" 74.
"22" '70",
"13": "35"
"2"" "66.2",
225" 75.
222% 'Tc"
"1o" "40"
"21" '78.7",
*26"5 53.
226" "22"
225" "45"
"23" "75.6",
2251
2261
"16"
"32.7"
03.
228": 26,
"18"
228" 778"
228" "55"
"20": "55"
288 "05"
29" "55"
29" "55"
Figure case
19

---

Published as a conference paper at ICLR 2026
E EXPLANATIONS FOR DATASET COMPARISON METRIC
Average Data Points per Chart (1) This metric measures the average visual complexity and information density the charts. A higher value indicates that the dataset contains more intricate charts with a larger number of data elements, which poses greater challenge for chart parsing models and is more representative of complex real-world visualizations.
Unique Numerical Ratio (1) This ratio is calculated as the number of unique numerical values divided by the total number of numerical values across the entire dataset. It serves as direct measure of data value diversity. A low ratio suggests high data homogeneity and value repetition, which can cause models to overfit to specific numerical patterns. Our dataset's high ratio demonstrates a significant diversity in the underlying data values.
Average Absolute Correlation Coefficient (1) To measure the diversity of data trends, we calculate the Pearson correlation coefficient for all multi-series charts and average their absolute values. A score close to 1.0 indicates that the dataset is dominated by monotonic trends (strong positive negative correlations), common issue in synthesized datasets. A lower score, as achieved by our dataset, signifies richer variety of data relationships, including weak correlations, non-linear patterns, and random distributions, thus preventing models from developing biases toward simple linear trends.
Average PMI of Top-K Label Pairs (Q) This metric quantifies the prevalence of "implicit reg- ularities" by measuring the Pointwise Mutual Information (PMI) between the most frequent label pairs. A high PMI score reveals that certain labels (e.g.. '2020' and '2021', 'USA' and 'China') co-occur far more frequently than by chance, creating predictable patterns. While our dataset's PMI competitive and avoids the extreme regularities seen in ChartX, ChartQA exhibits the lowest PMI. indicating the least predictable label associations among the compared datasets.
User Study To assess the perceived visual diversity and realism of the generated charts, we conducted comprehensive user study with 14 volunteers. For each of the four datasets (Ours, ChartQA, PlotQA, and ChartX), we randomly selected and compiled 16 chart images into a single 4x4 grid. In each of the 50 trials presented to volunteer, they were shown the four grids side-by-side. The volunteers were then asked to rate the overall stylistic diversity and visual quality of each grid 4-point Likert scale (1: Poor, 2: Fair, 3: Good, 4: Excellent). The order of the grids was randomized in each trial to prevent positional bias. The results, summarized in Table 1, show that our dataset was rated significantly higher, confirming its superior visual variety and quality from human perspective.
20

---

Published as a conference paper at ICLR 2026
F STRUCTURING CHART-ORIENTED REPRESENTATION METRIC
We adopt SCRM (Structuring Chart-oriented Representation Metric) (Xia et al., 2023) as the eval- uation metric. SCRM quantifies the structural consistency between model's chart-parsing result and the ground truth at two levels, image and dataset. Given the p-th predicted triplet and the q-th ground truth (GT) triplet, entity similarity [p. q) is measured by the edit distance, and value error e(p, is measured by the relative difference. A predicted/GT pair is regarded as matched when the two conditions J (p,4) q) S Jehr and e(p, q) ethr are both satisfied:
if ((p. S Jeh A e(p, q) s Edhr E(p.9) = (0, otherwise
11,
Let P and Q be the numbers of predicted and GT triplets in the image, respectively. The Image-level structural IoU is
P
(p. ②)
IoU
(2)
P+Q Q
{pp 4)
Then, for the i-th chart in dataset of charts, define
if IoU(t) 2 d(i, 1) 0, otherwise
1,
(3)
The Dataset-level precision is then:
AP(IoU))r) [IoU L
Σ
Σ d(i,t)
(4)
sEToUi 1=1
In our experiments, use the standard SCRM, which adopts the threshold set IoUnt 0.50, 0.55.... ... 0.95.
21

---

Published as a conference paper at ICLR 2026
G ERRONEOUS SAMPLES IN CHARTQA-SE
As shown in Figure 12 and Figure 13, we present some representative emror samples ChartQA-SE. The complete list of erroneous samples is provided in the "chartqasse.wrong josna.oonnnnn.n..............."
Chart Image
Incorrect Label
"Leages": "3899339266758.0o
"ets":
"titie":
"beaths fros eating disorders, 1998",
"source":
"Wone",
"x title": "Mone".
"y. title":; "None",
"values": {
"Ctee d'Ivoine": 1d"",
"Teeladd"; "<l")
"all":
Mouritius": "d""
Grades, test scores top list of factors Americans say should be considered in college admissions
"Inages": "1921.pang,
"gts":
"Litle": "Grades,
Americans should considared college adsiasions
saying
following
scytnn each follnwing abould
adiins
udmizzione decisione
"scurce": "hone"
titie": "None",
Major
factor
"y_title"; "None",
"values":
High grades
"Major factor":
"High school grades": '67", "Scandardized test Scores": "47",
Standardlaed
"Comuniity service Involveent" "21",
"Being first person in fanily to collage":
Communiyy service
invovemmnt
21
48
30
228
"athletic ability":: "g", "Whether relative attended the school": ret, "Bace athnicity': '7",
Being first in family collage
20
27
53
"Gonder": "g"
"Hinor factor":
"High school grades": '26",
Whether reletive attended the schoo
"Standardi
"es",
"Community service involvement
person fantly collage":
Race or athricity
29
78
"Achlatic abilityy:
attanded school"; "24', "Race or ethnicity": "19",
"Whether
81
"Gender": "14"
Source:
22
2010.
"kot factor":
"Righ school grades": '7" "Standandizzed test Scores': "11",
PEW RESEARCH CENTER
"Commnnity service involvenent". "38",
"Being first person fanily to collage":
53"
"athletic ability": "57",
"whether relative attended school": "98",
"Race ethnicity":
"Gandar":
"irages":
"DECD: AGAICULTURAL LAND, CAN KAZ KOR, 009817.p'8",
"ats": [
"title": "Agricultural land",
"sounce": "None".
"x.title": "Hone",
"y citie""" "Home",
"values":
"Lueebourg" "ak", "Karca": "25k".
"Spain": "62K"
"Canada": "213k"
Figure 12: Erroneous samples in ChartQA-SE
22

---

Published as a conference paper at ICLR 2026
Chart Image
Incorrect Label
"Ieages": "two col___2_277..pp8"
"ets":
"titie": "Non=",
"source": "None",
"x_tite':: "Mkon".
"y_title": "Musber of strears in willions",
"values":
13-3an": "1"4.25",
"17-Jan": "138.2"
"31-Jan": "347.47",
"14-Feb": "161.91",
"28-fab":
"13-Mar":
"27-Mar": "145.19',
"10-Apn": "151.63"
24--pp": '145.15',
"05-lay" '142.77"
222-Pay": '143,91"
"65-Jun" '133.51"
110-um" '148-18"
"85-361 ": "134.59"
"17-3u1": "138.98"
"31-301": "135. 84",
"14-Aug": "132.27',
"28-Aug": '131.72',
"11-5ep" "136.09',
225-Sep".
099-0ct" '127.01"
"trages": "two_col_S31P.pgg
"sts": "None";
"title":
"source":
"xutitle": "Wore"
"yLtitle": "Wore",
"values":
"Vencooon":; "24. 78.,
"Carrefour"; "8.6"",
"Lidi": "6.30"
Rissing data
More worry than enthusiasm about the idea of gane editing for bables
"Images": "8587.png", "WES":
"More werry enthuslase whout the
adsits who possiviity gene editing healtty bables
gene editing possibility of gone editing give healthy bobies much reduced risk of sersous diseases and conditions makes then.
U.S. who say
much redaced risk serious disenzes crealtiinn maker them
Somewhat Not
"source": "Mone"
"x.title": "hone".
"y_title": "hone",
"values":
"Vory":
Respondents who retgveeve amaner arenstatcen
"Enthusiactic": "15",
coeduceeddlcch 2098
"Norried": "22"
S. Putric Mhyy old oredeeal tnchnologiss to behanss Maman
Mariees Raiites"
"Soownat": "Inthasiastic": "Marcied": "as"
"Not too": "Enshuslastic": "30", "Warried": "23"
Net all': Ennhhsiasstic": 719",
Worried"
Figure 13: Erroneous samples in ChartQA-SE
23

---

Published as a conference paper at ICLR 2026
H STUBBORN ERRORS AFTER THE FIRST REFINEMENT ROUND
VSR Refine Round 0
VSR Refine Round
Missing
Refinement Finled Poun Soil Missing
Figure 14: High Visual Density In charts with extreme data density, the model struggles to distinguish individual elements within the dense clusters, leading to persistent verification failures.
VSR Refine Round 0
VSR Refine Round
-1.
Misidemiifia
Wyenburg
Wynnburg
Figure 15: Error Misinterpretation and Shifting The model sometimes misidentifies the type of error, causing "correction" that shifts the mistake rather than solving it. For instance, missing point might be misinterpreted as a large deviation of nearby point. The model then "corrects" " this by moving the existing point the missing location, effectively fixing the omission but creating a new vacancy where the original point stood.
VSR Refine Round 0
VSR Refine Round 1
3500
2000
2500
2500
7000
2000
1500
3500
500
Shall Viaua Devition
Numencal Eimor
Figure Visual Insensitivity Relative Precision This typically occurs charts with large data disparities (e.g.. a range containing both 10,000 and 10). A deviation of merely a few pixels is visually imperceptible in the context of the large scale. However, for the small data values (e.g.. 10), this tiny pixel shift translates to massive error (e.g., 50%), causing the model to incorrectly confirm the visual position "accurate" while it fails the numerical evaluation metric.
24


Reviews:
[{"review_id": "IsOjqdJqTX", "reviewer": "Reviewer_NmPr", "summary": "This paper proposes a methodology to improve chart parsing by including an additional step of providing pixel-level annotations on the charts in order to help the model improve precision. The refine step involves generating the pixel locations which are then used to annotate the image. Both the image and the pixel values are fed as input to the model which then makes a prediction based on this additional information. Experiments involve evaluating on a number of existing datasets as well as a new dataset introduced in this work. VSR results in nice improvements in some settings but the results aren’t consistent on existing datasets.", "questions": "see above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The new dataset seems nice and useful for chart parsing evals.  \nVSR is an interesting recipe and focusing on pixel-level annotations is a nice instantiation of this setup for chart parsing.  \nSome of the improvements seem compelling, particularly the information dense charts.  \nIf the extra calls are prohibitive for an inference pipeline, this recipe can probably be used to create distillation data.", "weaknesses": "While the recipe is interesting, it’s not very general and will probably become outdated for this task as models’ visual understanding improves over time.  \nIt seems strange that performance doesn’t improve much after a step or two of refinement even though there’s so much headroom. Why is this? Maybe annotations should be adjusted or focused on incorrect ones? Or doing step-by-step correction is necessary? Either way, it seems like the feedback and the recipe need some…refinement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wIN9kCdtql", "reviewer": "Reviewer_c4Zx", "summary": "This paper proposes Visual Self-Refine (VSR), a novel paradigm that introduces visual feedback as a self-correction mechanism for LVLMs in visually intensive tasks such as chart parsing. The method, instantiated as ChartVSR, decomposes the parsing process into two stages: the Refine stage and the Decode stage. The authors also present ChartP-Bench, a new and challenging benchmark featuring visually dense and stylistically diverse charts. Experimental results demonstrate consistent improvements over both chart-specific and general-purpose LVLMs.", "questions": "1. I am curious about the scalability and generalization ability of the proposed approach, particularly regarding its performance on unseen chart types and the relationship between the number of localized regions and the extent of performance improvement.\n2. The open-source models achieve performance comparable to ChartVSR on existing datasets such as ChartQA-SE-Clean, but their performance drops significantly on the newly proposed ChartP-Bench. Could the authors clarify the reason for this discrepancy? Is it possible that the training data distribution of ChartVSR is similar to that of ChartP-Bench, giving it an advantage? If this is the case, I would suggest that the authors conduct an additional experiment in which the training set explicitly excludes data similar to ChartP-Bench in order to more convincingly demonstrate the model's out-of-domain generalization ability.\n3. How does the proposed model perform on chart-related question-answering tasks such as ChartQA and PlotQA?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents a novel and interesting paradigm for chart parsing, introducing a visually grounded self-correction mechanism that enhances interpretability and addresses an existing gap in LVLM perception.\n2. The authors introduce a high-quality dataset, ChartP-Bench, which is carefully curated, diverse in style, and fills an important gap in chart parsing evaluation.\n3. The ablation studies are comprehensive, providing thorough analyses of the effects of pixel localization and refinement, and consistently demonstrating the benefits of the proposed approach.", "weaknesses": "1. The chart parsing paradigm proposed in this paper can be viewed as a type of reasoning paradigm. However, the experimental section lacks comparisons with other recent visual reasoning models, such as o1, Qwen3-VL, and InternVL-3.5. I understand that some of these models might not have been publicly available at the time of submission, but I recommend that the authors include such comparisons in future revisions to strengthen the solidity and comprehensiveness of the work.\n2. Around line 405, the paper explains why the AP-Strict scores are so low. According to the authors, this metric only rewards models that output exactly correct numerical values. I find this requirement excessively strict, as it is practically impossible to infer such highly precise numbers (sometimes up to two or three decimal places) from a single image. Therefore, I have some reservations about the practical significance and interpretability of this metric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YUcaDE8s03", "reviewer": "Reviewer_jBBM", "summary": "The authors first had the model generate pixel-level localization results, visualized these localizations and plotted them back onto the original image, then fed this labeled image back to the model, allowing the model to self-check and correct visual perception errors as if \"checking point by point with a finger,\" and finally performed structured parsing based on the confirmed pixel anchors.", "questions": "Will the model trained after refine + decode experience a performance degradation on refinement tasks? Quantitative results are needed.\n\n\nThere are limited benchmarks for evaluating papers, lacking authoritative datasets like Chart-Pro and ChartXiv.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "A visual self-refinement paradigm, Visual Self-Refine (VSR), is proposed: the model first generates localization points, visualizes them, and then feeds them back to the model for self-checking and error correction.\n\nIn the graph parsing task, the process is divided into two stages: Refine and Decode.\n\nA challenging benchmark, ChartP-Bench, is constructed, and ChartQA-SE is cleaned to obtain ChartQA-SE-Clean. Significant performance is reported on multiple benchmarks, especially outperforming strongly closed-source models (such as Gemini-2.5-Pro) and existing dedicated models on ChartP-Bench.", "weaknesses": "There are limited benchmarks for evaluating papers, lacking authoritative datasets like Chart-Pro and ChartXiv.\n\nThis method has limited nooverty, and its two-stage design is very similar to the design philosophy of SoM. Many previous works on visual prompts have demonstrated that such visual prompts can improve performance.\n\nIt lacks a crucial baseline, such as a comparison of the localization ability of the first-stage model with that of other grounding models on chart localization tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KL09IlI8yd", "reviewer": "Reviewer_tCHt", "summary": "This paper introduces Visual Self-Refine (VSR), a novel paradigm to address the poor performance of LVLMs on vision-centric tasks like chart parsing, where text-based self-correction fails. Inspired by the human strategy of using a finger as a \"visual anchor\" , VSR enables a model to generate pixel-level localizations, visualize them on the image, and then feed this visualization back to itself to iteratively inspect and correct visual perception errors like omissions or hallucinations. The authors instantiate this paradigm in a model called ChartVSR, which uses a Refine Stage to verify pixel locations and a Decode Stage to parse the chart using these verified anchors. This work is supported by two additional contributions: a new challenging ChartP-Bench benchmark and a robust data engine for generating a large-scale , diverse training dataset.", "questions": "Could you provide more qualitative or quantitative analysis on the \"stubborn errors\" that VSR fails to correct after the first round? Are they of a specific type (e.g., heavily occluded points, ambiguous labels, tiny markers)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is very well-written and very clear.\n- The paper conducted abundant analysis.\n- Meaningful contributions in both data and method.\n- The potential applications to other precision-oriented tasks like Visual Counting and Visual Grounding are clearly articulated (Figure 6) , opening a promising new research direction.", "weaknesses": "- In Table 2 and 3, the authors are comparing open-source models of different sizes. The author should clearly list the sizes of these models for fairer comparison. Additionally, to better show the effectiveness of the proposed training data, the author can consider training the MatCha model with their data and do direct comparison with DePlot.\n- For chart parsing evaluation, the paper should also compare with the Chart-to-Text [2] and AskChart [3].\n\n\n\n\n[1] MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering. ACL 2023\n\n[2] Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning. ACL 2024 Findings\n\n[3] AskChart: Universal Chart Understanding through Textual Enhancement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)