Charts Are Not Images: On the Challenges of Scientific Chart Editing
Authors: Li Li
Ryan A. Rossi
Sungchul Kim
Sunav Choudhary
Franck Dernoncourt
Puneet Mathur
Zhengzhong Tu
Yue Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Scientific Chart Editing
Benchmark
Primary area: datasets and benchmarks

Abstract: Generative models, such as diffusion and autoregressive approaches, have demonstrated impressive capabilities in editing natural images. However, applying these tools to scientific charts rests on a flawed assumption: a chart is not merely an arrangement of pixels but a visual representation of structured data governed by a graphical grammar. Consequently, chart editing is not a pixel-manipulation task but a structured transformation problem.
To address this fundamental mismatch, we introduce \textit{FigEdit}, a large-scale benchmark for scientific figure editing comprising over 30,000 samples. Grounded in real-world data, our benchmark is distinguished by its diversity, covering 10 distinct chart types and a rich vocabulary of complex editing instructions. The benchmark is organized into five distinct and progressively challenging tasks: single edits, multi edits, conversational edits, visual-guidance-based edits, and style transfer.
Our evaluation of a range of state-of-the-art models on this benchmark reveals their poor performance on scientific figures, as they consistently fail to handle the underlying structured transformations required for valid edits. Furthermore, our analysis indicates that traditional evaluation metrics (e.g., SSIM, PSNR) have limitations in capturing the semantic correctness of chart edits. Our benchmark demonstrates the profound limitations of pixel-level manipulation and provides a robust foundation for developing and evaluating future structure-aware models. By releasing \textit{FigEdit}, we aim to enable systematic progress in structure-aware figure editing, provide a common ground for fair comparison, and encourage future research on models that understand both the visual and semantic layers of scientific charts.

Published as a conference paper at ICLR 2026
CHARTS ARE NOT IMAGES:
ON THE CHALLENGES OF SCIENTIFIC CHART EDITING
Shawn Li', Ryan Rossi Sungehul Kim", Sunav Choudhary", Franck Dernoncourt Puneet Mathur?, Zhengzhong Tu Yue Zhao' Universsity of Southern California, 'Adobe Research, Teexa A&M University (1i.1i02, yue. 2) @usc.edu (ryrossi, sukim, schoudha, dernonco, puneetm) @adobe. com tzz@tamu. edu
ABSTRACT
Generative models, such as diffusion and autoregressive approaches, have demon- strated impressive capabilities in editing natural images. However, applying these tools to scientific charts rests on a flawed assumption: a chart is not merely an arrangement of pixels but visual representation of structured data governed by graphical grammar. Consequently, chart editing not maxellmmnipulation task but structured transformation problem. To address this fundamental mismatch, we introduce FigEdit, a large-scale benchmark for scientific figure editing com- prising over 30,000 samples. Grounded in real-world data, our benchmark is dis- tinguished by its diversity, covering 10 distinct chart types and a rich vocabulary of complex editing instructions. The benchmark is organized into five distinct and progressively challenging tasks: single edits, multi edits, conversational edits, visual- iisual-guidance- based edits, and style transfer. Our evaluation of a range of state- of-the-ar models on this benchmark reveals their poor performance scientific figures, as they consistently fail to handle the underlying structured transforma- tions required for valid edits. Furthermore, our analysis indicates that traditional evaluation metrics (e.g., SSIM, PSNR) have limitations in capturing the seman- tic correctness of chart edits. Our benchmark demonstrates the profound limita- tions pixel-level manipulation and provides a robust foundation for developing and evaluating future structure-aware models. By releasing FigEdit (https: //gi thub. com/ adobe research wwt.reiguueeeeeeee iigg. we aim to en- able systematic progress structure-aware figure editing, provide common ground for fair comparison, and encourage future research on models that un- derstand both the visual and semantic layers of scientific charts.
INTRODUCTION
Vision-language models (VLMs) have advanced rapidly, showing strong results in recognition, cap-
tioning, and Dnstructioo-following image editing (Radford al., 2021; Schuhmann al.) 2022
Rombach et al. [022 Brooks et al. 2023al Zhang et al.) 2023 Team et al. 2023 OpenAl 2024
Chen et al. | 2024b) Wang et 2024al Lu et al. 2024; Liu et al. 2024b Li al. 2024a] Yao al. 2024 Xu et al.) 2024). Beyond natural images, chart editing focuses on the precise modification of charts and graphs from natural-language instructions, which central scientific communica- tion and data analysis. Typical workflows include updating figures when upstream tables change, adapting layouts for publication, aligning styles across related plots, and converting encodings to highlight specific trends. In collaborative environments, edits often arrive as multi-turn requests with references to earlier messages, related figures, or localized visual cues. Such use cases require outputs that remain faithful to underlying data, consistent with visualization rules, and auditable for provenance (Belouadi et al., 2024).. At the same time, instruction tuning and dialogue- centric edit- ing continue expand the ability of modern systems to follow multi-turn turn control (Li al. 2024d
Huang et al.
2024a Ma et al. 2025| Wei et al. 2024b; Hahn et al. 2024 Deng et al. 2025 Zhang
al. 2025
Li al. 2024c
2025bb Shawn et al. 2025 Li al. 2025a).

---

Published as a conference paper at ICLR 2026
Despite these advances, figure editing differs fundamentally from natural image manipulation. A chart is the rendering of structured data through graphical grammar, and valid edits are structured transformations
marks, scales, encodings, and legends rather than pixel changes. Instructions such as "add a bar for category X with value 42" require coherent updates to data schema and vi- sual mappings, yet current models often treat them as visual rearrangements, producing outputs that appear plausible but violate semanties. This exposes a persistent problem-method mismatch: instruction-followin editors and multi-turn generation systems (Brooks al. 2023aa] Zhang al.)
2023 Huang et al. 2023] Wang et al., 2024b) are optimized for perceptual alignment under open- ended goals, whereas figure editing is constrained by data fidelity and visualization rules. Models trained on web-scale natural images (Schuhmann et al. 2022 Radford et al. (022) lack inductive bias to preserve value-encoding consistency, axis coherence, and legend integrity. While dialog-
driven clarification (Andukuri et al. 2024] [Chen et al.] 22024a, Zelikman et al.) (024) or OCR
augmentation (Rodriguez et al. 2023a b) can mitigate ambiguity locally, they do not guarantee pencttre-preserving edits, leaving the core mismatch unresolved.
Current approaches and benchmarks. On the approach side, diffusion editors and multimodal
LLMs have been extended to multi-turn control and aatrivval-augmenter interaction (Li et al.. (002dd)
Huang et al.
2024a; Ma et al., [025 Wei et al. 2024b; Wang et al. 2025; Hahn et al. [2024] Deng
jt al. 2025] Li et al., |026][ Liu et al., [204cc] Taneja & Goel| 2025;
Zhao et al. al.] 2025b). ). Yet, these
systems rarely operate executable specifications or enforce semantic constraints, which makes
them unsuitable for structured figure editing. On the benchmark side, prior chart-related datasets
have mainly targeted captioning. QA, table extraction, or chart-to-code generation (Hsu al.
2021
Kantharaj et al. 2022 Masry et al. 2023 HHan et al., 2023] Zhang et al.) 2024c Xia et al.
2024
Shi et al. 2024a Masry et al.) [2024] Zhang et al. 2024bb) As shown in Tab. 1 these resources leave
several gaps. Some lack real underlying data altogether (e.g., Xiaa et al. 2024 Zhang et al. 2024d),
(eig.,(Xia
reducing their grounding in authentic visualization workflows. Coverage of edit categories is also narrow: data-level updates, layout transformations, and style changes are often missing. Interactive scenarios such as visual guidance or style transfer are almost entirely absent, despite being common in real practice. Even the recent ChartEdit benchmark (Zhao et al., 2025a), while closer to editing, only partially spans instruction types and lacks paired figure outputs for direct comparison. Overall, existing benchmarks fall short of representing the breadth of figure editing and still depend heavily on pixel-level similarity metrics, which do not reflect semantic correctness. This highlights the need for task-structured, semantics-aware and scale-ready benchmark dedicated to figure editing.
Our benchmark. We introduce FigEdit, a large-scale benchmark for scientific chart editing with over 30,000 instances collected from realistic sources (Fig. []. It spans 10 chart types and a diverse set of instructions, as summarized in Tab. 2 and is organized into five evaluation settings: single edits, multi edits, conversational edits, visual-guided edits, and style transfer edits. The benchmark also covers wide range of operation categories, including data-centric edits, layout adjustments, style modifications, and text updates, detailed in Tab. 3 Unlike prior benchmarks that lack real data or paired chart outputs, FigEdit grounds edits in authentic charts and provides both charts and specification references. To address the absence of interactive scenarios, it includes conversational editing for multi-turn consistency, visual-guided editing with localized cues, and style transfer for cross-chart alignment. Finally, beyond SSIM and PSNR, FigEdit introduces semantics-aware eval- uation that verifies transformations at the level of data and encodings, with executable targets or programmatic specifications where possible (Li et al. 2024b) Zhang et al. 2024a) Zheng et al.) 2023 Wei et al. 2024a] Guo et al. 2024: Shi al. 2024a). These design choices directly address the limitations of existing benchmarks and shift evaluation from pixel similarity toward semantic correctness in structured editing.
Our contributions are summarized as follows:
We define chart editing structured transformation task gov- erned by a graphical grammar, clarifying required invariants such as data-encoding align- ment, axis coherence, and legend integrity.
Problem formalization:
Task-structured benchmark:
We present FigEdir, benchmark with 30K+ instances and 10 chart types, spanning single, multi, conversational, visual- -guide,, and style transfer with diverse instruction set.
2

---

Published as a conference paper at ICLR 2026
Sclentiflc Figure Editing
Single
Malii
Refore
Afher
Accese
Add new datapoout for CCanadi with of $10 at the and the dataset
FigEdit
Conversational Edit
Style Transfer
Mad guii
Rchudgnen pay.
ViunalCGuddanc
Figure 1: FigEdit benchmark. Top-left: example figure illustrating the basic task. Bottom-left: radar chart comparing model performance on single edit task, highlighting the benchmark's ability to reveal differences in editing capabilities. Right: taxonomy of the benchmark covering five tasks (single edit, multi edit, conversational edit, visual guidance, and style transfer).
Comprehensive study: We systematically evaluate state-of of-thefr-te-rt editors and VLMs, showing that strong scores pixel metrics do not imply correct structured edits, and ana- lyze frequent failure modes.
2 RELATED WORK
Text-to- Image Generation. The rapid progress of diffusion-based models has revolutionized text-conditioned image generation, enabling results that are both high-fidelity and prompt-faithful (Ramesh et al. 2022 Rombach et (022)) ControlNet and related approaches expand controllabil- ity by incorporating structural or spatial priors (Zhang et al. (023). Yet these advances have focused primarily on natural imagery. Scientific figures remain relatively neglected, despite their demand for symbolic precision, calibrated spatial relationships, and embedded textual fidelity. Evaluations show mainstream systems often fail in data accuracy and layout coherence for scientific use cases (Zhang) ctt al. 2024b). In response, specialized methods such as OCR-aware generative frameworks (Ro- driguez et al. 2023b) and programmatic vector-graphic synthesis (Belouadi et al. 2024 highlight the need for tailored solutions.
Image Editing. Instruction-basee editing has evolved from GANs and encoder-based systems to- ward diffusion-drivees methods, which better balance realism with semantic aligument. A survey by Huang et al. (2025) provides comprehensive overview of this transition. Representative works include LEDITS ++ (Brack et al. 2024), which extends text-driven editing to unconstrained trans- formations: Emu Edit (Sheynin al. (2024), which integrates recognition for localized precision; and Liu et al. (Liu et al. 2024a)) which probe attention mechanisms to preserve semantic fidelity. More recent works push toward interactivity and compositionality: SmartEdit (Huang al. 2024b) employs multimodal LLMs to compose edits, ProxEdit (Han et al. (024) stabilizes transtormations without tuning, and DragDiffusion (Shi et al., (222b) enables point-based manipulation. AnyEdit
Yu et al.) 2025) exemplifies the broader trajectory toward unified, general-purpose editing frame- works.
Scientific Chart Editing. Unlike natural images, charts encode structured data, calibrated axes, and embedded text, requiring semantic consistency and readability throughout editing. While a broad literature addresses diffusion-based editing of natural scenes (Brooks et al. 2023a Huang et al.)
2024b Han et al. 2024; Sheynin et al., 2024 Brack et al. 2024; Shi et al. 2024b] Yu et al.) 2025 Huang et al. (225)) research specific to scientific figures is limited. Scimage investigates the limita-
tions of multimodal LLMs for figure generation (Zhang et al. (002b);; Automa explores text-to-
vector generation under programmatic constraints (Belouadi et al. 2024); and ChartEdit formulates
3

---

Published as a conference paper at ICLR 2026
Comparison of our proposed benchmark with existing chart-related benchmarks. While prior benchmarks mainly target captioning, QA, or chart-to-code generation, they provide limited coverage of editing operations and interactive settings. FigEdit the first benchmark designed for evaluation of figure editing, supporting diverse chart types, multiple instruction categories, and interactive scenarios such as visual guidance and style transfer edits.
Table I:
Output
Visual Guidance
Style Transfer
Name
Format
Data
Types
Editing Instruction
Layout
Style
ChariCrafi Yan al.
2024
Json
x
K
Ploc2Code
ChartX Xia ef dl
2014
Code
x
K
Acalemat bun 7
2024d
Code
K
ChartMimic Shi cl
2023
Codo
ChartEdit (Zho
20258
Code
FigEdii (Ours)
Figure
Table Benchmark data statistics across chart types and editing tasks. Each entry shows the number
of instances per task, with subtotals by chart family and overall totals.
Chart Type
Single Edit
Multi Edit
Conv. Edit
Style Transfer
Visual Guidance
Area
1463
586
369
406
299
Line
1649
593
398
424
399
Bar
1800
600
375
410
398
Stacked-bar
1800
600
398
498
396
Pie
1000
600
398
200
397
Donut
1000
600
399
200
398
Other
Box
1199
568
388
200
198
Violin
1000
500
300
200
150
Scatter
1398
598
324
400
322
Dot
1796
599
383
458
398
Totals
14105
6244
3732
3400
3355
All tasks combined: 30836
chart editing as a multimodal evaluation benchmark (Zhao et al.) 2025a). A common limitation in existing work is reliance intermediate code (e.g., matplotlib) as the target of modification. While this guarantees structural validity, it reduces evaluation code executability and neglects perceptual quality and user-facing usability. Thus, the field lacks benchmarks that jointly measure instruction adherence, semantic fidelity, and visual clarity in an end-to-end setting, motivating figure editing as distinct line of inquiry.
BENCHMARK
We introduce a figure-centric benchmark for scientific figure editing, Ground truth (GT) images obtained by applying deterministic edit functions to the results. Evaluation performed in image space. This design provides pixel-consistent supervi- sion across atomic edits, one-shot composite edits, multi-turn conversations, figure edits with visual guidance, and figure edits with referenced figures, without depending on package-specific code.
specifications and rendering
3.1 FORMAL DEFINITION OF CHART
A natural image I can be viewed function mapping 2D coordinates to color values, / R2 R In contrast, a chart is the rendered output of a structured specification. Formally, we define deterministic renderer R that maps specification Σ to an image 7 xx
=(()).
(I)
hotps://vgaa.gihuu.i/ https://veg..gihuuiioeeeeee
4

---

Published as a conference paper at ICLR 2026
Each specification can be decomposed into two components:
SC,
where Content (C) denotes dataset D. a chart type т, and a mapping function that encodes vari- ables in D to geometric marks. Style (S) denotes the visual configuration, including palettes, fonts, strokes/fills, gridlines, legend layout, spacing, and margins.
An atomic edit & is total function fe Σ + Σ, with pre-fpost-conditions on (C,S). Given an initial specification a with rendered image I = R((a) and an instruction u, a model M produces either an image 1 = (1, u) specification ō = (1,
3.2 TASKS
Task 1: Single Chart Edit. Given (T, u) where u specifies one atomic edit e, the updated specifi- cation is follows:
ot = felo),
-" R((").
Multiple Chart Edits. Given (I, u) where u specifies k 2 atomic edits {ei,...,ee, ek applied jointly, the updated specification is
Task 2:
a' (Se, a... (a))
1' R(o*).
For non-commutative edits, we adopt a fixed canonical order in the generator.
Task 3: Conversational Chart Edits. A session consists of T rounds. At round I, the input (I--1, Hi-1, 4e), where 4-1 is the previous image, Hi-1 is the dialogue history, and us is the current instruction. The updated specification is
of (Se, .... Ser (a),
T R(at).
Task 4: Style Transfer. Given a source chart I, = R(a,) and target content (D,, T2), the goal is to preserve the target content while adopting the source's ' style:
C(a²) = (Dt, 7x),
S(a*) PJ S(a.),
1" |= R(o").
Task 5: Visual-Guidance Edits.
Given (7, u, 9), where g is visual guidance, the goal is to apply the edit within the guided region while preserving other regions:
= Se,,,
I" R(oo).
3.3 BASE FIGURE SOURCING AND GENERATION
To construct base figures, define set of chart classes C and associate them with curated datasets A drawn from public sources (full list in Appx. F.. Each chart class E C is paired with a prefer- ence list P(c) encourage semantically coherent choices. We employ LLM to propose candidate specifications conditioned on class hints and dataset lists. A set of automatic validation and filter- ing rules ensures that generated charts satisfy schema requirements, avoid duplicates, and maintain semantic diversity. In addition, heuristic alignment between dataset domains and chart types further improves quality and coverage. All generations are logged with provenance information, and further implementation details are provided in Appx.
3.4 EDITING OPERATIONS
We build a suite of editing tasks derived from a canonical operation set O (See Appx. B for more details). Each element in O encodes atomic edit, covering text, style, layout, and data-centric manipulations. Invalid operations are filtered out depending on chart semantics (e.g., spacing edits require band/point scales).
From each chart we automatically produce (i) natural-langune instructions augmented with ma- chine-readable OP f (ii) edited specifications with inline data values, and (iii) corresponding
*OP operation; each OP tag encodes the intended atomic edit.

---

Published as a conference paper at ICLR 2026
Table 3: Distribution of editing operations by task. Operations are grouped into categories such as data-centric, text, style, and layout, with counts reported per task and overall totals.
Task
Category
Operation
Image Count
Add element
1941
Data-centric
Remove element
1892
Text
Add title
1942
Single Edit
Change background color
1944
Style Editing
Change data color
1729
Margin Adjustments
Adjust category spacing
1729
Font
Font Adjustment
2943
Dual-operation
Combine 2 edits
3370
Multi Edit
Triple-operation
Combine 3+ edits
2660
Conversational Edit
3575
Style Editing
Change data color
1666
Visual Guidance
Data-centric
Remove element
1819
Style Mapping
Transfer style
1511
Style Transfer
Style Editing
Change data color
1728
Margin Adjustments
Adjust category spacing
387
Overall Total
30836
rendered images. On top of these atomic edits, we derive (iv) conversational annotations that align multi-step edits with their constituent single edits, (v) visual-guidune assets where the target gion is circled on the original chart, and (vi) style-transfer annotations that pair a target edit with reference figure providing the desired style attribute. More details are provided in Appx. B
3.4.1 SINGLE AND MULTI EDIT GENERATION
For each chart we sample a feasible subset (cc) C O and realize the edits as natural instructions validated to preserve schema correctness,
with corresponding OP tags. Edited specifications ensure visible changes, and maintain consistent data accounting when adding or removing rows. These checks guarantee deterministic and reproducible supervision.
3.4.2 CONVERSATIONAL ANNOTATIONS
We further construct short multi-turn conversations by decomposing a two-step edit into its con- stituent single edits. Fach conversational sample provides the original chart, two turns of instructions with their intermediate ground truth states, and the final outcome. This setting evaluates whether models can maintain state and history across turns rather than only executing isolated edits.
3.4.3 TISUAL-GUIDANCS ASSETS
For a selected subset of operations, we create visually grounded variants by marking the target region directly the original chart. To generate the visual overlay, employ vision language model (GPT-Image) that is prompted to draw a thin red circle around the specified element while leaving chart content unchanged. Each sample provides both concise natural instruction and guidance image with the circled target. This variant enables evaluation of multimodal understanding, where the model must integrate textual instructions with explicit visual cues.
3.4.4 STYLE-TRANSEE TRANSFER ANNOTATIONS
Finally, we introduce style-transfer setting in which an edited chart paired with reference chart whose current style attribute matches the target of the edit. The model is asked reproduce the target chart while adopting the style of the reference. This task connects editing with cross-fiee style adaptation and highlights the challenge of disentangling content from stylistic attributes.

---

Published as a conference paper at ICLR 2026
Output Figure
Input Figure
Instruction
OmniCGen
Classic Metries
LLM Scores
SSIM: 0.49
Instr. Follow:
Change color of the datapoints 'Black' 'drkgreenn.
LPIPS: 0.68
Content Pres.:
0.70
Visual Quality:
PSNR: 3,80
OCR: 0.21
Overall:
SSIM: 0.85
Instr: Follow:
Inerease the of the axis titles to 16.
LPIPS: 0.53
Content Pres.:
CLIP: 0.77
Visual Quality: 2
PSNR: 8.88
SSIM: 0.89
Instr. Follow:
Remove the datapoint for 'May with excchunge' as 'Kakken' 'volume' 70000.
LPIPS: 0.46
Content Pres.:
Visual Quality:
PSNR: 10.67
OCR: 0.00
Overall:
Figure 2: Comparison of chart editing evaluation signals on three representative cases. The left block shows the Input Figure and the Instruction. The right block shows the Outpul Figure from OmniGen2, the Classic Metrics (e.g., SSIM and PSNR), and the L.M Scores. We observe that clas- sic pixel metrics can remain high while the edit is wrong. This reveals a gap between pixel similarity and semantic edit correctness, which motivates semantics-aware evaluation for figure editing,
3.5 DATASET STATISTICS
The final benchmark contains 30,836 edited figures, distributed across five task families. Tab. 3 sum- marizes the counts by operation type. Single edits form the largest portion of the dataset, covering basic manipulations such as element addition/removal, text and font changes, color and background modifications, and spacing adjustments, totaling 14, 14,105 figures. Multi edits contribute another 6,244 examples, split between dual edits and three-operation combinations. Conversational settings add 3,732 two-turn sequences, while the visual-guidance and trans-rransfer tasks contribute 3,355 and 3,400 figures, respectively. Together, these distributions provide balanced coverage of atomic ed- its, composite edits, multimodal guidance, and cross-style adaptation. A breakdown by chart type shown in Tab. 2 Importantly, all base figures derived from real-world datasets, spanning do- mains such as economics, climate, healthcare, sports, and social science. complete list of datasets used in figure generation is provided Appx.
3.6 EVALUATION PROTOCOL
We evaluate all models directly in image space. We compute six complementary metries: SSIM (Wang et al.) 2004). PSNR (Hore & Ziou| 2010). LPIPS (Zhang et al. 2018), CLIP simi- larity (Radford et al. 2022). OCR similarity (Smith) 2007], and an LLMM- instruction
The first five are classic metrics widely used in image generation and vision tasks, while the last directly evaluates whether edits satisfy the instruction, preserve chart content, and maintain visual quality. More details on implementations are provided in Appx.
4 EXPERIMENT
Baselines. We evaluate against four representative instruction-based editing models: GPT- Image (OpenAl 2025). Imagen 4 (Google) (025). OmniGen 2 (Wu et al.) (2025), and Instruct- Pix2Pix (Brooks al. 2023b). These span closed source commercial systems and open-source research frameworks, covering both diffusion- based editors and multimodal approaches. Further details each baseline provided in Appx.
Experiment Setup. We evaluate chart editing across five tasks. All methods operate on the same set of instructions and images. Prompts standardized to encourage strictly local modifications

---

Published as a conference paper at ICLR 2026
Oulput Figure
Input Figure
Instraction
Imagen
GPT-Image
OmniGien
Remove the Gatapoint for Segment I with avenago transaction value of 670.
Change the background color chart light gray.
new datapoint for the categooy 'neegyy with value 8.3.
Figure 3: Qualitative examples of figure editing with three representative instructions. For each case, the input figure and target instruction are shown on the left, and outputs from Imagen 4, GPT-Image, and OmniGen2 are shown the right.
Cfiimage
(a) Multi Edit Task
(b) Conversational Edit Task
Figure 4: Radar charts for different tasks (normalized with epsilon, LPIPS inverted). Each chart compares all models on SSIM, PSNR, OCR, LPPIS, and three L.M scores.
while maintaining axes, labels, and other contextual elements. Further implementation details are provided in Appx.
4.1
MAIN RESULTS
Overall performance across tasks. Tab. summarizes the performance of representative editing models across the five evaluation settings. Imagen 4 achieves consistently high scores on SSIM and PSNR, reflecting strong pixel-level resemblance to the input figures, but its cirruction-follwingg and preservation scores the lowest among all models. GPT-Image excels in conversational and transfer settings, showing the highest Instruction-following scores, but often sacrifices content fi- delity. OmniGen2 strikes balance, performing reliably across most tasks with solid LLM scores and relatively stable OCR accuracy. InstructPix2Pix remains competitive but generally underper- forms OmniGen2, particularly complex edits, while still clearly surpassing Imagen 4 on semantic alignment. These results highlight that strong performance on pixel-based similarity metrics does necessarily translate into correct faithful edits.
detailed comparison of multi-edit and conversational tasks. Classic metrics such as SSIM and PSNR exaggerate the performance of pixel- oriented models like Imagen 4, while M.L.-d scores and OCR accuracy reveal significant
Linitations of classic metrics. Fig. 4 provides
8

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison grouped by task. Higher is better for SSIM, CLIP, PSNR, OCR, and LLM Scores. Lower is better for LPIPS. Instr. denotes instruction following score. denotes content preservation score. Qual. denotes image quality score.
Preserv.
Task
Model
S LPIPS
CLIP
PSNR OCR f
MILM Score (1-5) f
Instr.
Preserv.
Qual.
Imagen
0.7726
0.4094
0.7781
13.04
0.0723
1.58
1.51
2.05
GPTImage
0.7295
0.5383
0.8099
10.32
0.2054
3.47
1.71
2.45
Single
InstructPix2Pix
0.7211
0.4811
0.8328
11.02
0.2568
3.27
2.50
2.77
OmniGen2
0.7350
0.4705
0.8350
11.30
0.2620
3.35
2.55
2.85
Imagen
0.6958
0.5549
0.7738
11.02
0.1069
1.26
1.32
2.15
GPTImage
0.7017
0.5787
0,8070
0,2185
2.51
1.63
2.34
Multi
InstructPix2Pix
0.6460
0.5204
0.8043
9.83
0.2584
2.48
2.00
2.51
OmniGen2
0.7100
0.5100
0.8220
10.15
0.2650
2.65
2.10
2.70
Imagen
0.7180
0.4923
0.7599
11.58
0.0698
1.35
1.23
2.11
GPTTmage
0.6732
0.5257
0.8525
10.66
0.1721
4.59
2.51
2.91
Cony.
InstructPix2Pix
0.6890
0.5075
0,8200
10.40
0.2540
2.90
2.25
2.65
OmniGen2
0.7050
0.4950
0,8280
10.80
0.2600
3.10
2.35
2.75
Imagen
0.8420
0.5050
0.7600
13.10
0.1200
1.40
1.35
2.20
GPTImage
0.8355
0.5207
0.8444
12.85
0,4665
2.39
3.16
3.95
Visual
InstructPix2Pix
0.7380
0,5220
0.8190
10.90
0.2200
1.85
2.20
2.80
OmniGen2
0.7508
0.5236
0.8187
8.98
0.1806
1.19
1.85
2.74
Imagen
0.8500
0,4800
0.7700
14.00
0.1300
1.30
1.25
2.15
GPTlmage 0.8438
0.4934
0.8054
13.81
0,5092
3.06
3.57
4.16
Transfer
InstructPix2Pix
0.7960
0.5020
0.8160
12.90
0.2400
2.20
2.60
3.10
OmniGen2 0,8246
0.4376
0.8127
12.08
0.3147
1.53
2.14
2.64
mantic errors. The radar plots make this gap visually explicit: models that appear strong under pixel similarity collapse when judged by whether the requested edits were actually applied. This finding is consistent with the qualitative evidence in Fig. 3 and reinforces the need for evaluation protocols that go beyond pixel resemblance.
Per-instruction breakdown. We further analyze performance at the level of individual instruc- tions in Appx. E These results confirm the same trend: models often achieve high SSIM or PSNR even when edits such as adding datapoints or changing axis labels are not correctly applied.
4.2 ANALYSIS
The gap between pixel-level similarity and semantic correctness. Fig. 2 and Tab. 4 highlight consistent limitation classic image metrics in the context of figure editing. Models such as Imagen 4 and OmniGen2 can obtain high SSIM and PSNR scores, yet their outputs often fail to apply the intended transformation. As illustrated in Fig. edits may preserve overall appearance while the in- struction ignored, the figure is distorted, or key content changed. Tab, [soows the same pattern across tasks: pixel-based metrics remain strong, but cestructtio-followiiii and content-preservatiin scores from LLM-bbsed evaluation drop sharply, especially for multi-step and conversational edits. These results indicate that similarity at the pixel level is not reliable indicator of semantic correct- ness. They also motivate the need for benchmarks that evaluate edits the level of data and visual encodings rather than image resemblance alone.
No single model dominates across tasks. Tab. shows that performance highly fragmented: no model achieves consistently strong results across all task types metrics. Imagen 4 tends to lead on low-level pixel fidelity metrics such as SSIM and PSNR, yet performs poorly on instruction- following and semantic preservation, indicating that its edits often look visually smooth but fail to reflect the requested change. GPT- Image shows the opposite trend: excels instruction scores, especially in conversational and transfer settings, but lags behind on PSNR and OCR accuracy, sug- gesting weaker robustness to text- eeat-heavy or sensui-ee edits. InstructPix2Pix performs compet- itively some semantic metrics but generally less reliable than OmniGen2, which offers more

---

Published as a conference paper at ICLR 2026
balanced profile. However, OmniGen2 also struggles with visual-guided and transfer edits, high- lighting its limitations in cross- cosssinntennce reasoning. These results reveal that current models overfit to specific task structures metrie types, and that strong performance classic pixel-level metrics does not guarantee reliable edit satisfaction in more challenging scenarios.
Qualitative study. Fig. 3 iillsstrates representative failure cases in figure editing. Across different instructions: removing a datapoint, changing a background color, or adding a new element, cur- rent models frequently produce outputs that appear visually similar yet fail realize the requested transformation. These cases mirror the quantitative results: classic pixel-level metrics often remain high even when semantic correctness is violated. The examples highlight how generative editors, optimized for perceptual similarity, struggle with structure- airueture-peeserving transformations, reinforcing the need for evaluation protocols and benchmarks that explicitly target semantic consistency. More cases can be found in Appx.
5 CONCLUSION
We introduced FigEdit, large-scale benchmark for scientific figure editing that treats editing as structured transformation problem grounded graphical grammar. The benchmark spans diverse chart types and task settings, and it provides both figure outputs and executable specifications to support reliable evaluation. Our experiments show that existing models perform poorly when edits require semantic consistency, which reveals a clear gap between current approaches and the needs of figure editing. By offering a task-structured and semantics-awarc evaluation protocol, FigEdit establishes foundation for developing future models that can perform faithful, data-aligned, and auditable edits.
6 ETHICS STATEMENT & REPRODUCIBILITY STATEMENT
This work adheres to standard academic research practices. All data used are either publicly avail- able or synthetically generated, and the study intended solely for scientific and educational pur- poses. We do not foresee any ethical concerns arising from the content or methodology presented. For reproducibility, we have included sufficient technical details in the paper to allow other re- searchers to replicate our experiments. The dataset statistics, task definitions, and evaluation pro- tocols are described detail, and we aim to facilitate further exploration and extension by the community.
7 ACKNOWLEDGEMENT
We sincerely appreciate the support from Adobe Research and the USC FORTIS Lab.
REFERENCES
Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, and Noah D Goodman. Star-gate: Teaching language models to ask clarifying questions. arXiv preprint arXiv:2403. 2024.
Jonas Belouadi, Anne Lauscher, and Steffen Eger. Automatikz: Text-guided synthesis of scientific vector graphics with tikz, 2024.
Manuel Brack, Felix Friedrich, Katharia Kornmeier. Linoy Tsaban, Patrick Schramowski, Kristian Kersting, and Apolinario Passos. Ledits++: Limitless image editing using text-to-image mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patten Recognition (CVPR), pp. 8861-8870 June 2024.
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning follow image editing instructions, 2023a.
Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-1 18402, 2023b. doi: 10. 10011///VVP522222200007777.,
10

---

Published as a conference paper at ICLR 2026
Maximillian Chen, Ruoxi Sun, Sercan Ô Ank, and Tomas Pfister. Learning to clarify: Multi- turn conversations with action-based contrastive self self-training. arXiv preprint arXiv:2406.00222, 2024a.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen. Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguisicc tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185-24195 2024b.
Yang Deng, Lizi Liao, Wenqiang Lei, Grace Yang, Wai Lam, and Tat-Seng Chua. Proactive conver- sational ai: A comprehensive survey of advancements and opportunities. ACM Transactions on Information Systems, 2025.
Google. Imagen 2025. URL
wwwp://ei..d.ooo//eeeeeee/eeeeee imaggen//
Daya Guo, Qihao Zhu, Dejian Yang. Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming- the rise of code intelligence. arXiv preprint arXiv:2401. 2024.
Meera Hahn, Wenjun Zeng, Nithish Kannen, Rich Galt, Kartikeya Badola, Been Kim, and Zi Wang. Proactive agents for multi-turn text-to-image generation under uncertainty. arXiv preprint arXiv:2412.06771, 2024.
Ligong Han, Song Wen, Qi Chen, Zhixing Zhang, Kunpeng Song, Mengwei Ren, Ruijiang Gao, Anastasis Stathopoulos, Xiaoxiao He, Yuxiao Chen, et al. Proxedit: Improving tuning-free real image editing with proximal guidance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4291-4301, 2024.
Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: multimodal 11m for chart understanding and generation. arXiv preprint arXiv:2311.16483, 2023.
Aljoscha Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 20th International Conference on Pattern Recognition, pp. 2366-2369. IEEE, 2010.
Henn'eettKehhh Huang. Scicap: Generating captions for scien- Ting-Yar Hsu, C Lee Giles, and Ting-Hao' Kenneth'
tific figures. arXiv preprint / 10. 1624, 2021.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 12i-compbench: A comprehen- sive benchmark for open-world compositional text-to-image generation. In Advances in Neural Information Processing Systems, volume 36, pp. 78723-78747. Curran Associates, Inc., 2023.
Minbin Huang, Yanxin Long, Xinchi Deng, Ruihang Chu, Jiangfeng Xiong, Cheng, Qinglin Lu, and Wei Liu. Dialoggen: Multi-modal interactive dialogue system for multi- turn text-to-image generation. arXiv preprint arXiv:2403.08857. 2024a.
Xiaodan Liang, Hong
Yi Huang, Jiancheng Huang, Yifan Liu, Mingfu Yan, Jiaxi Lv, Jianzhuang Liu, Wei Xiong. He Zhang, Liangliang Cao, and Shifeng Chen. Diffusion model-based image editing: A sur- vey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(6):4499-4433 2025.
doi: 10. 11119//TAA...2225554111660
Yuzhou Huang, Liangbin Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, et al. Smartedit: Exploring complex instruction-base image editing with multimodal large language models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 8362-8371, 2024b.
Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. Chart-to-text: A large-scale benchmark for chart summarization. arXiv preprint 22XXv22003.06486 2022.
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. mattteeext-ttteleeave Tackling multi-image, video, and 3d large multimodal models. arXiv preprint arXiv:2407.07895, 2024a.
11

---

Published as a conference paper at ICLR 2026
Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, Zhiyong Huang, and Jing Ma. Mmcode: Bench- marking multimodal large language models for code generation with visually rich programming problems. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 736-
783, 2024b.
Li, Wei Ji, Yiming Wu, Mengze Li, You Qin, Lina Wei, and Roger Zimmermann. Panoptic scene graph generation with semantis-prooooypp learning. Proceedings of the AAAI conference on artificial intelligence, volume 38, pp- 3145-3153, 2024c.
Shawn Li, Peilin Cai, Yuxiao Zhou, Zhiyu Ni, Renjie Liang, You Qin, Yi Nian, Zhengzhong Tu, Xiyang Hu, and Yue Zhao. Secure on-device video ood detection without backpropagation. In International Conference Computer Vision (ICCV), October 2025a.
Shawn Li, Huixian Gong, Hao Dong. Tianki Yang. Zhengzhong Tu, and Yue Zhao. Dpu: Dynamic prototype updating for multimodal ofitt--diitribution detection. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 10193-10202, 2025b.
Shawn Li, Chenxiao Yu, Zhiyu Ni, Hao Li, Charith Peris, Chaowei Xiao, and Yue Zhao. De- fenses against prompt attacks learn surface heuristics, 2026. URL https::/arxiv.or
Yanda Li, Chi Zhang, Gang Yu, Wanqi Yang, Zhibin Wang, Bin Fu, Guosheng Lin, Chunhua Shen, Ling Chen, and Yunchao Wei. Enhanced visual instruction tuning with synthesized image- dialogue data. In Findings of the Association for Compulational Linguistics ACL 2024, pp.
14512-14531, 2024d.
Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7817-7826, 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. next:-next: Improved reasoning, ocr, and world knowledge, January 2024b.
Yilun Liu, Minggui He, Feiyu Yao, Yuhe Ji, Shimin Tao, Jingzhou Du, Duan Li, Jian Gao, Li Zhang, Hao Yang, al. What do you want? user-centric prompt generation for text-to-image synthesis via multi-turn guidance. arXiv preprint arXiv:2408. /2910, 2024c.
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint 45355:2433055525, 2024.
Shichao Ma. Xinfeng Zhang, Zeng Zhao, Bai Liu, Changjie Fan, and Zhipeng Hu. Dialogdraw: Image generation and editing system based on multi-turn dialogue. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 24795-24803, 2025.
Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. arXiv preprint arxiv:2305. (47, 2023.
Ahmed Masry, Mehrad Shahmohammadi, Md. Rizwan Parvez, Enamul Hoque, and Shafiq R. Joty. Chartinstruct: Instruction tuning for chart comprehension and reasoning. In Annual Meeting of the Association for Computational Linguistics, 2024.
OpenAl. Gpt-4o, 2024. URL
Accessed:
2024-05-13.
OpenAl. Introducing our latest image generation model in the api, 2025. URL https://
openai . com/ http//idexxxiii httpae/////mneeegggeeeaiooooo
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PMLR, 2021.
12

---

Published as a conference paper at ICLR 2026
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022.
Juan A. Rodriguez, David Vazquez, Issam Laradji, Marco Pedersoli, and Pau Rodriguez. Ocr-vqgan: Taming text-within-ing image generation. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 3678-3687, 2023a. doi: 10.1 LL09/WAAC56688.2223.....
Juan A Rodriguez. David Vazquez, Issam Laradji, Marco Pedersoli, and Pau Rodriguez. Qgravq Taming text-within-image generation. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 3689-3698, 2023b.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models, 2022.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
Li Shawn, Jiashu Qu, Linxin Song, Yuxiao Zhou, Yuehan Qin, Tiankai Yang, and Yue Zhao. Treble counterfactual VLMs: causal approach hallucination. In Association for Computational Linguistics: EMNLP 2025, pp. 18423-18434, Suzhou, China, November 2025. Association for Computational Linguisties. ISBN 779-8-89176-335-1
Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8871- 2029.
Chufan Shi, Cheng Yang, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, et al. Chartmimic: Evaluating Imm's cross-modal reasoning capability via chart-to-code generation. arXiv preprint arXiv:2406.09900 099611 2024a.
Yujun Shi, Chuhui Xue, Jun Hao Liew, Jiachun Pan, Hanshu Yan, Wenqing Zhang, Vincent YF Tan, and Song Bai. Dragdiffusion: Harnessing diffusion models for interactive point-based image edit- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8839-8849, 2024b.
Ray Smith. overview of the tesseract ocr engine. Ninth International Conference on Document Analysis and Recognition, volume 2, pp. 629-633. IEEF, 2007.
Karan Taneja and Ashok K Goel. Mudoc: An interactive multimodal document-groundee conver- sational system. arXiv preprint arXiv:2502. 09843, 2025.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Baptitttstte Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.1 1805, 2023.
Jianhui Wang, Yangfan He, Yan Zhong, Xinyuan Song, Jiayi Su, Yuheng Feng. Hongyang He, Wenyu Zhu, Xinhang Yuan, Kuan Lu, al. Twin co-adaptive dialogue for progressive image generation. arXiv preprint arXiv:2504. 14868, 2025.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, model's perception of the
Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing world at any resolution. arXiv preprint arXiv:2409.12199 1219,, 2024a.
Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, and Tianyi Zhang. Promptcharm: Text-to-image generation through multi-modal prompting and refinement. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI '24, pp. 1-21, May 2024b. doi: 10.1145/3613944. 3642803.
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612, 2004.
13

---

Published as a conference paper at ICLR 2026
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Empowering code generation with oss-instruct. In Forty-first International Conference on Machine Learning, 2024a.
Zhaohui Wei, Lizi Liao, Xiaoyu Du, and Xinguang Xiang. Balancing visual context understanding in dialogue for image retrieval. Association for Computational Linguistics, 2024b.
Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang. Zhixuan Liang, Zeyu Lu, Ying Shan, and Ping Luo. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. arXiv preprint arXiv:2405.077 079900 2024.
Chenyuan Wu, Pengfei Zheng. Ruiran Yan, Shitao Xiao, Xin Luo. Yueze Wang. Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, 7e Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, Zhongyuan Wang, Tiejun Huang, and Zheng Liu. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506. 18871, 2025.
Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, al. Chartx & chartvlm: versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12183 2024.
Ruyi Xu, Yuan Yao, Zonghao Guo, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, Maosong Sun, and Gao Huang. Llava-uhd: an Imm perceiving any aspect ratio and high- resolution images. arXiv preprint
2024.
Pengyu Yan, Mahesh Bhosale, Jay Lal, Bikhyat Adhikari, and David Doermann. Chartreformer: Natural language-drin driven chart image editing. In International Conference on Document Analysis and Recognition, pp. 453-469. Springer, 2024.
Yuan Yao. Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Miniepm-v: gpt-4v level milm on your phone. arXiv preprint arXiv:2408.01 8000, 2024.
Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang. and Yueting Zhuang. Anyedit: Mastering unified high-quality image editing for any idea. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 26125-26135, 2025.
Eric Zelikman, YH Wu, Jesse Mu, and Noah D Goodman. Star: Self-taught reasoner bootstrapping reasoning with reasoning. In Proc. the 36th International Conference on Neural Information Processing Systems, volume I 1126, 2024.
Chen Zhang, Xinyi Dai, Yaxiong Wu, Qu Yang, Yasheng Wang, Ruiming Tang, and Yong Liu. A survey multi-turn interaction capabilities of large language models. arXiv preprint 2XXi:25001.0959, 2025.
Fengji Zhang, Linquan Wu, Huiyu Bai, Guancheng Lin, Xiao Li, Xiao Yu, Yue Wang, Bei Chen, and Jacky Keung. Humaneval-v: Evaluating visual understanding and reasoning abilities of large multimodal models through coding tasks. arXiv preprint arXiv:2410.1220 12381, 2024a.
Leixin Zhang, Steffen Eger, Yinjie Cheng, Weihe Zhai, Jonas Belouadi, Christoph Leiter, Si- mone Paolo Ponzetto, Fahimeh Moafian, and Zhixue Zhao. Scimage: How good multimodal large language models at scientific text-to-image generation?, 2024b.
Liang Zhang, Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, and Fei Huang. Tinychart: Efficient chart understanding with visual token merging and loogram-of-hhogge learning. arXiv preprint arXiv:2404.1666 2024c.
Lvmin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to icxt-to-image diffusion models, 2023.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586-595, 2018.
14

---

Published as a conference paper at ICLR 2026
Zhehao Zhang, Weicheng Ma, and Soroush Vosoughi. Is gpt-4v (ision) all you need for automating academic data visualization? exploring vision-lange -siionlanguage models' capability in reproducing aca- demic charts. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp- 8271-8288, 2024d.
Xuanle Zhao, Xuexin Liu, Haoyue Yang. Xianzhen Luo, Fanhu Zeng, Jianling Li, Qi Shi, and Chi Chen. Chartedit: How far are milms from automating chart analysis? evaluating milms'' capability via chart editing. arXiv preprint arXiv:2505.71953 2025a.
Zijia Zhao, Longteng Guo, Tongtian Yue, Erdong Hu, Shuai Shao, Zehuan Yuan, Hua Huang, and Jing Liu. Chatsearch: dataset and a generative retrieval model for general conversational image retrieval. Pattern Recognition, pp. 11169,, 2025b.
Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, et al. Codegeex: A pre-trained model for code generation with multilingual evaluations humaneval-x. arXiv preprint arXiv:2303. AXXi:23033.1766 17568, 2023.
15

---

Published as a conference paper at ICLR 2026
A BASE FIGURE SOURCING AND GENERATION
As discussed in Sec. 3.3 base figures are generated for chart classes (bar, stacked-bar, line, area, box, violin, donut, pic, dot, scatter) using dataset names from a curated whitelist A (see Appx. F) For each class preference list ((c) C guides the assignment toward semantically coherent themes.
LLM-guided spec proposal. A chat model M instructed output a single JSON object
(vega.spec o, dataset = d], a E, deA.
where is the set of allowed dataset names. Any mismatch with the requested dataset d triggers rejection and re-sampling. Each prompt includes a class hint (c), preferred dataset list P(c). an exemplar specification E. (style only), and an avoid-terms block derived from recent generations. The sampling temperature is fixed to 0.55 to balance validity and diversity. Detailed prompt template shown below:
System Message (Part 1)
You return ONLY valid JSON object with exactly two fields: vega_spec: VALID Vega vó specification JSON object dataset: ONE string chesen ONLY from the allowed set (see Appendix Table X)
Return exactly one JSON object. Do not add any pretix, suttix, code fences.
No Markdown, no explanations, no backticks.
vega_spec spec requirements:
Use "$schema";
http://wwwaaggt.....eeeg wwtssscceeaeeeevveeeeeeeee j Base data a popular public dataset; URLs disallowed
Include a small
inline sample in "values".
12
The chosen dataset MUST naturally support category-vvalue or
13
category x series->value aggregation suitable for bar/stacked/ grouped charts.
14
Match the requested chart class:
15
bar: one catogorical field one quantitative field stacked-bar: category series value (stacked) grouped-bar: category + series value (side-by-side)
17
Include all necessary components (data, scales, axes, marks)
it renders.
19
Do NOT include extra meta fields inside vega_apec.
16

---

Published as a conference paper at ICLR 2026
System Message (Part 2)
fe1f-check before responding:
Prefer a specific cataset name from the allowed set that
credibly aligns
with field/entity names (full whitelist in Appendix A, Table X, aapppx:alloxeddddee -ved-datasstts))
see
Use "unknown" only if credible alignment exists.
Output rules:
Return EXACTLY one JSON object containing "vega_spec": ... dataset": ). vega_ must be valid Vega v6 JSON object with inline values" (no "uri").
No Markdown, no commentary, no extra keys.
User Template (Bar, excerpt)
Task:
Chart class: (chart_class)
Instruction: (alass_hintt
Generate JSON object with two fields: vega_spec (Vega v6 spec)
and dataset
(string from the allowed set or 'unknown').
Hard constraint for this sample:
You MUST set "dataset" EXACTLY to: cattase_ttarget Field names and inline 'values' must be plausible for dataset_target.
10
Data requirements:
12 Include inline 'values' only (no 'url"); fit bar -> categery + value.
13
5..12 categories; numeric magnitudes should be plausible for the dataset.
14
15 Diversity controls:
16 Avoid reusing identical numeric multisets for the same field set.
Avoid terms seen recently:
18 terom]
20 Preferred catasets for this class (see Appendix A, Table for the full list):
pneterred_catasee.e
22
23
Output:
ONLYY one JSON object: "vega_spec": ..., "dataset": ...
Scheduling and validity. A scheduler balances dataset usage by always selecting the least-used candidate for each chart class, based on compatibility heuristics (e.g., time series + line/area; survey data + bar/pie/dot). Returned specifications are checked for Vega vó schema, completeness (data, rks, scal les, axes), and type-specific field patterns (e.g., bar requires (category, numeric); stacked-bar requires (category, series, numeric)). Invalid proposals are rejected and resampled.
Shape validation. Beyond generic schema checks, additional constraints enforce meaningful con- tent. For example, bar charts must contain at least categorical and numeric field, while
17

---

Published as a conference paper at ICLR 2026
stacked bar charts must include two categorical fields and one numeric field. Other chart types validated using generic rules.
Duplicate and near-duplicate control. For every o, we compute four signatures over inline data and structure:
Dexect (o) = SHA256
sorted Vega rows).
rows
(aulti(ar) SHA226(numeric multiset per field set),
Sval(o) = SHA2266pprr-fee histograms with b=6 and (p,o)).
Sstruct (o) SHA256((size buckets, mark types, scale types/flags, axis orients, legend presence),
where SHA256 is cryptographic hash function that produces fixed 256-bit digest with extremely low collision probability. A specification is rejected if hexact or hmultti has been observed previously, or if both vval and Sstruct have appeared before. This eliminates duplicates and near-duplicates while permitting controlled variability.
Semantic diversity via term overlap. Categorical fields are inferred from scales and encodings, forming token (o).. sliding window W of the last k samples (default k 16) maintained, and the Jaccard overlap ratio
T(6) nul (T()) uul
U= Us Sew
=
computed. A candidate is rejected m = 2). The current union U is injected back into subsequent prompts as an avoid-terms block, enforcing semantic diversity across generations.
0 (default 0 0.70) and (T(o) \ UU| < m (default
Additional mechanisms. Further enhancements improve robustness: malformed completions handled by stripping Markdown fences or extracting JSON blocks; provenance is logged into JSON index with raw outputs for debugging. Together, these mechanisms ensure quality, diversity, and reproducibility of the generated base figures.
B EDITING OPERATIONS
As briefly discussed in Sec. 3.4 this section provides extended details of the editing operations and annotation pipeline. We describe how we generated single and multi-edit supervision, conversational annotations, Disual-guidane assets, and style-trane trynsfrr pairs. Representative prompt excerpts are also included. first define canonical operation set:
increase.text.size change background color, decrease.category wawegory.spaccnggg...........a.. add_datapoint,
decrease text size, increase increase.categon spacing, add_title, remove.datapoint
0 =
B. SINGLE & MULTI EDIT GENERATION
For each chart specification select feasible subset Occ), filtering out inapplicable edits (e.g., spacing operations for charts without band/point scales, or removals when only one data row exists). An LLM is prompted to return exactly one sentence instruction followed by explicit OP tags, as well as the edited Vega vó specification. We canonicalize op names, infer missing keys (such as axis_labhe libelissee ze, new_color, new.bg, new.padding). and apply minimal but detenmin- istic edits to ensure the modification is visually effective. Validation includes schema conformance, key completeness, and visible effect realization. Detailed prompt is shown below:
18

---

Published as a conference paper at ICLR 2026
Prompt (Single/Multi, Part 1)
Return ONLY JSON object markdown) with keys:
"instruction": ONE English sentence that describes exactly N edits (N in (1,2,3)),
followed by EXACTLY tag lines order:
[#OP1 op=<...?; keyl=valuel; key2=value2; ...]
[#OP2 op=<...>; ....
[#OP3 op-<...>; ...] (only N==3)
"ops": array of length N; each item has "op" in:
("dhange_dattppim___olla,"" "it.coloo.,,,nncsssseeettt "increase_text_size""" "decrease_text_siiz
"cchnge_ bor__baacrround_ooorrrr
decrease srrrass__ateppry_ssccinn
10
"edited_spec"; VALID Vega vé JSON spec that keeps "Šschema" v6
and uses inline "values" only (strictly no "url").
13
14 Required keys per operation (inside each [#OPi ...] tag): baango_backgroundccolrrr new new.bosxxcss-or-ther
16
17
increase_text_size / decrease_text_size:
axis lal_llaaeeeeeeiinn in 6..30>; size_sizes (opticnal);
title_eize-<int> (optional)
19

---

Published as a conference paper at ICLR 2026
Prompt (Single/Multi, Part 2)
ineresse_aateepry.spcinn /
scale=autolxly: waw_paddiigg<fflott in 10,0,9]>
add_title:
< title_size=<ine (optional)
chargge__aatppoin__coorrr:
tazzett_cateooyy===aabbbb>
target_serie==<labbl (omit if not
applicable);
new_color==<es-or-tbee not used in original>
add_datapointt:
If singlc-scrics:
lend;
11
lacegory=<ne__laaeel>, value=<number>>
12
If multi-series:
positions...; catepory=<cat_labbll> series=<series__aabee
value=<number>
14
remove_dat :atoovedattaoint:
15
single-series:
16
cecepory<<axistinnglaabeell
If multi-series:
18
categoryscexistingg.aaa label> series=cexisting series>
19
Editing rules and checks:
21 Apply minimal, deterministic edits; ensure each step has a visible effect.
22 Maintain inline data only; never introduce "url".
23 Sizes must be >= 6; band/point padding must be within [0,0.9).
New colors must not collide with original literal colers.
ts For add/remove datspoint, edit exactly one rew and keep rew-count accounting censistent:
66
rows (edited) rows (original) adds removes
27 Preserve unrelated content (data, labels, titles) unless the step explicitly changes them.
28
ng
For Single set N-1; for Multi set N in (2, 3). Output exactly one JSON object.
B.2 CONVERSATIONAL ANNOTATIONS
To simulate multi-turn editing, we align each two-op edit with its corresponding single-op edits. Given two-op edit 02), we locate the two single edits with the same operations, generate inter- mediate ground truth images, and concatenate them into a two-round dialogue. Each conversational sample therefore contains: (i) the original figure, (ii) turn-1 with an instruction and intermediate ground truth, and (iii) turn-2 with follow-up instruction and the final ground truth. This design yields per-round supervision and enables evaluation of temporal consistency. Detailed prompt is shown below:
20

---

Published as a conference paper at ICLR 2026
Prompt (Conversation)
You are a strict formatter that assembles 2-turn conversation object
from provided edit annotations. You MUST return ONLY one JSON object.
Inputs (conceptual):
original: the unedited chart (spectimage). single-edits: two entries, each has: op (atomic op name),
instruction (one sentence + [#OP* ...] tags), edited_spec (Vega v6, inline values only), image.
10
11
multi-edit (2-step); has: ops-[opl, op2] in the exact execution order, edited_spec (final target),
13
14
edited edited_image, image,
15
instruction (one sentence tags).
16
Formatting rules:
18 1) Preserve execution order strictly as [opl, op2].
19
2) Use the single-edits whose matches opl and op2, respectively,
3) For each turn i in 10,1):
21
instruction: copy the corresponding single's instruction verbatim, trimming leading/trailing whitespace only,
22
23
gt: spec=image=the correspending single's ground truth intermediate).
24
op: the corresponding op (opl for turn 0, op2 for turn 1).
sppc 4) final: use the multi-edit's ecited spec and edited image.
06
5) Include the multi-edit's instruction as multi_instruction.
27
6) Do NOT invent or rewrite text; do NOT change specs.
18
7) Output must be single JSON object with the following fields:
29
30
"chart :ype": "<string>",
12
"figure_id": "<string>",
33 "ops": ["<opl>", "<op2>"1,
"original": ("spec": <json>, "image": "<path-or-id>"), "turns":
35
16
37
"surn_ids 0, "op": "<opl>",
18
"instruction"; "<singlel instrustion Instrustionnttiimedds
40
"gt": ("spec": <json>, "image": "<path-or-id>")
41
12
43
"turn_idx": 1, "op": "<op2>", "instruction":
45
16
"g=": ("spee"; <json>, "image": "<path-or-ids"]
18
1.
"final": ("spec": <json>, "image": "<path-or-id>"", "multi instruction:: "<multi int
50
51
93
Return exactly this one JSON object and nothing else.
21

---

and Published as a conference paper at ICLR 2026
B.3 VISUAL-GUIDANCE ASSETS
For selected atomic operations (notably datapoint color changes and datapoint removals), we con- struct visual-guided variants by highlighting the target region directly in the original chart. To pro- duce the overlays, we employ a vision-language model (GPT--mmae) instructed to draw a thin red circle around the specified element while leaving the of the chart untouched. This yields paired data: (i) a natural-lanuuage instruction referencing the circled element, and (ii) visually annotated chart. Such assets allow evaluation of multimodal understanding, where the model must integrate textual instructions with explicit visual cues. Detailed prompt is shown below:
Prompt (Visual Guidance)
You are image editor. Given chart image and target descripticn,
draw a thin red circle around exactly one target element. De not change
any chart content.
Inputs:
Image: the original chart may be letterboxed on plain background.
Chart noun: (bar|slice) ppoint mark)).
Target condition (optional but preferred): category "<CATEGORY>"
10 series = "<SERIES>"
12 Task:
13 Locate the single element that satisfies the target condition (if
given)
14 If no explicit condition is given, use the instruction sentence
prefix
15 as a hint to identify the most likely target element.
16 Draw exactly one circle that tightly encloses the target element.
18 Rendering constraints:
19 Stroke color: #FF0000 (pure rec).
00 Stroke style: thin line, no glow, no shadow.
21 Circle only; no arrows, no text, no highlights or masks.
Do not crop, scale, move the chart content.
23 the image letterboxed, ignere place
the chart only. 55 Preserve the original resolution and aspect ratio.
the circle
26
Do not alter colors, fonts, or data marks other than the added circle.
22
t8 Output:
29
Return a single edited image where the only modification is the
thin
red circle tightly around the target el element.
B.4 TRYEE-TRANSEER SINGLES
We further derive one-shot style-transfer supervision by linking existing single edits to style sources. For each single edit, we identify another original chart whose current style attribute already matches the target attribute of the edited chart. We construct a natural instruction such as "Make this bar chart the same background color the reference chart," and pair it with the corresponding OP
22

---

Published as a conference paper at ICLR 2026
tag. This produces style-transfer pairs across both same-type and cross cross-type chart classes, enabling
evaluation of style generalization. Detailed prompt is shown below:
Prompt (Style Transfer)
You are writing ONE natural-language instruction for style transfer (single attribute)
Inputs you are given (out of bana) include:
target chart class and image, a SINGLE-OP edit already chosen for the target (with its OP tag line),
reference chart ("the reference chart") that already exhibits the target style value.
Task:
Write exactly ONE concise Engl ish sentence that asks to make the target chart
use the SAME style attribute as the reference chart.
10
Use phrasing like:
"Make this <chart noun> use the background coler the reference chart."
"Make this <chart noun> use the same axis label font size the reference chart."
13
"Make this <chart noun> use the same category spacing as the reference chart." "Match the datapoint color used in the reference chart." "Add a title with the same font size as the reference chart." NOT mention internal ids. Say "the reference chart" "the example chart".
14
15
18
After the sentence, append EXACTLY ONE OP tag line, keeping the provided tag VERBATIM:
19
[#0P1
keyl=valuel; key2=value2; ...]
20
Allowed ops (single attribute only):
22
change_backgroun__cooo: increase_text_size decrease_text_s1ze
decrease_categorrsspooa add title change canbnee_datppint_eco
Increas__tteprrysppciii
24
25
Hard constraints:
27
Output ONLY the final instruction text (one sentence) followed by the single CP tag line.
include edited_spec. Do NoT invent new 8 NOT return JSON,
keys or values. Preserve the original OP tag exactly as given (verbatim).
Through these pipelines, each figure can appear as (i) atomic edits (single/multi), (ii) conversational trajectories, (iii) visually guided variants, and (iv) style-transfer pairs. All assets are designed to be reproducible, diverse, and machine-readable, while supporting multimodal evaluation settings.
C EVALUATION METRICS
As discussed in Sec. 3.6 semantic correctness.
report both classic image metrics and LLM-based score to capture
SSIM. Structural Similarity Index Wang al. (2004) is applied grayscale renderings with Gaus- sian weighting to emphasize local structure. This metric accounts for luminance, contrast, and structure, making it more perceptually meaningful than pixel errors.
23

---

Published as a conference paper at ICLR 2026
PSNR. Peak Signal-to Noise Ratio Hore & Ziou (2010) is computed with pixel values clipped to [0, 255] and averaged across RGB channels. It quantifies the logarithmic ratio between the maximum possible signal and mean squared error.
to-oise
LPIPS. Learned Perceptual Image Patch Similarity Zhang et al. (2018) is computed using the official framework, with AlexNet the default backhone. Images normalized to [-1, before feature extraction. LPIPS captures perceptual discrepancies such as texture or shape distortions.
CLIP similarity. We use CLIP ViT-L/14 Radford et al. (2021) to extract image embeddings and report cosine similarity between I and I*. This provides semantic-level measure of alignment beyond pixel similarity.
OCR similarity. We extract text from both images using Tesseract OCR Smith (2077. Similarity is measured as the normalized edit distance:
(si 81.)
Simocr =
max maax(177111)) (81.1)
where 8y and sy. are the concatenated OCR strings. This metric emphasizes correctness of labels, legends, and annotations.
LLM-based instruction score. To directly evaluate editing success, we prompt large language model OpenAl (2024) with (i) the original chart and instruction (I, u), (ii) the edited output and (iii) the ground truth 1*. The model issues binary judgments on:
Instruction satisfaction: whether the requested edit is applied.
Content preservation: whether the underlying chart data remain intact.
Visual quality: whether the rendering is artifact-free and coherent.
Responses are parsed into structured JSON objects, which are aggregated into per-instance and per- model scores. Trimmed prompt examples are provided below:
LLM Score (System Message, Part 1)
You are expert assistant specializing in data visualization evaluation.
Your task is to evaluate how well an AI-generated chart follows given text instruction.
You will be given an instruction, reference "Ground Truth" image, and the "Generated Image" to evaluate.
Evaluate the generated image based en the following three criteria:
1. Instruction Following the specific
How accurately was instruction exccuted? (c.g., asked to change coler to orange,
is it orange?)
2. Content Preservation (scoro_preservation)) Were all other elements of the
10
chart preserved correctly without unwanted changes? (e.g., data values,
11
labels, and titles are unchanged unless specificd).
12 3. Image Quality (score_quality) Is the generated image free of major artifacts, distortions, or unreadable text?
13
24

---

Published as a conference paper at ICLR 2026
L.M Score (System Message, Part 2)
For each of the above, assign from 1 (very poor) excellent).
Then compute total score (score) as the average of the three above, rounded to the nearest integer.
Your response MUST be JSON object with the following keys: "score_instruction Integer [1-5]
" "score_preserrationn: Integer [1-5]
"score_quallt" Integer [1-5]
"score": Integer [1-5], the average of the above
10
"reasoning": One-sentence explanation justifying the sccres
11
Example Response:
13
14
icorr_instrucctonn; 5,
15
"score_prooeevvtionn: 4,
16
"score_cuality"" 5,
"score": 5,
18
"reasoning": "The instruction was followed perfectly, content was mostly
preserved, and the image quality is excellent.*
00
LLM Score (User Template)
InIntruction:**
<instruction text>
**Reference Image (Ground Truth) :**
<data::image/pnn/baeeee
**Generated Image (to be evaluated) ***
D ADDITIONAL EXPERIMENTAL DETAILS
Pre- and Post-Processing. To ensure consistent inputs, all charts are letterboxed into a square can- before inference. After editing, outputs are mapped back to the original resolution using contain resizing, which preserves the full layout without cropping. This procedure guarantees that models are evaluated under identical geometric conditions while avoiding distortion of axes labels.
Prompt Construction. For all tasks, prompts explicitly instruct the model to make localized mod- ifications while leaving unrelated elements unchanged. In Visual tasks, prompts additionally em- phasize that only the circled region should be modified. For Transfer tasks, the prompt specifies two-panel setup, where only the left (base) panel editable and the right (reference) panel serves as style guide.
Baselines. For comparison, we include four representative baselines that capture the current state of instruction-driven image editing: (1) GPT-Image (OpenAl 2025). A commercial instruction-driven editing system provided by OpenAL. It supports free-form natural language instructions and has been widely used for general- purppse editing tasks. Although proprietary, it reflects the strongest available commercial option. (2) Imagen 4 (Google 2025). A proprictary diffusion-basse editor developed by Google and released via the Vertex Al platform. Imagen 4 is optimized for control-
25

---

Published as a conference paper at ICLR 2026
Output Figure
Input Figure
Instruction
Imagen 4
GPT-Image
OmniGen 2
datapoint for the year 2017 and
season precipitation value 140.
Add datapoint for 'Brazil' with an export the end of the figure.
Incrense text size of titles
Add title to the chart with the text "GP Growth Rates Over Years', and incrense the spacing between calogorics the
Remove the point in red circle.
Remove point circle.
Figure 5: Additional qualitative examples of figure editing results. Each shows an input figure (left), the corresponding natural language instruction (middle), and the output figures generated by Imagen 4, GPT-Image, and OmniGen (right). The cases ing data point removal, data point addition, axis text scaling, layout adjustments, and targeted point deletion. While the models sometimes produce visually consistent outputs, they often fail accu- rately execute the requested transformation, highlighting the limitations of current instruction- based figure editing systems.
representative edit types, includ-
lable, high-fidelity image generation and editing, though its design is primarily tuned for natural image content. (3) OmniGen (Wu 2025). An open-source multimodal model recently in- troduced for text-guided and image-guided editing. It supports multi-turn interaction and has shown promising results for chart and figure editing. We use the official released checkpoint and infer- ence pipeline. (4) InstructPix2Pix (Brooks et al., 2023b). An open-source approach that finetunes a diffusion backbone on paired instruction-image data. was among the first methods to explicitly align natural language instructions with image translation, and remains strong research baseline for instruction-condiie nnsruction-conditioned editing.
Together, these baselines span both closed and open ecosystems, diffusion and multimodal paradigms, and commercial and academic settings. They represent the strongest available instruction-driven editing approaches the time of writing.
26

---

Published as a conference paper at ICLR 2026
Table Per-instruction performance comparison (Part 1/2). Higher is better for SSIM, CLIP, PSNR, OCR, and LLM Scores. Lower is better for LPIPS. Instr. denotes instruction following score. Preserv. denotes content preservation score. Qual. denotes image quality score.
SSIM t LPIPS 1 CLIPT PSNR 1 OCR
LLM Score (1-5)
Model
Instr.
Preserv.
Qual.
Instruction: Change the colors of the data point
InstructPix2Pix
0.736
0.52
0.84
10.7
0.25
2.66
2.76
2.76
OmniGen2
0.748
0.48
0.85
11.1
0.26
3.63
3.32
3.66
GPT--mage
0.733
0.54
0.87
10.4
0.22
4.34
3.84
4.29
Imagen 4
0.772
0.41
0.80
13.1
0.08
2.09
1.84
2.72
Instruction: Add new title
InstructPix2Pix 0.741
0.47
0.83
11.0
0.17
1.09
2.91
3.07
OmniGen2
0.744
0.46
0.84
11.2
0.29
3.34
3.14
3.36
GPT-Image 0.728
0.53
0.88
10.5
0.36
4.41
Imagen
0.769
0.40
0.79
13.0
0.07
1.00
1.41
2.07
Instruction: Increase font
InstructPix2Pix
0.735
0.50
0.84
10.8
0.26
2.48
2.10
2.83
OmniGen2
0.747
0,85
11.1
0.30
2.12
3.27
GPT-Image
0.729
0.52
0.86
10.3
0.27
4.05
4.05
4.40
Imagen 4
0.771
0.39
0.81
13.2
0.26
1.74
1.50
2.26
Instruction: Decrease font size
InstruetPix2Pix
0.748
0.49
0.85
11.1
0.27
2.02
1.77
2.58
OmniGen2
0.752
0,46
0.84
11.4
0.31
2.10
3.05
3.38
GPT-Image
0.734
0.51
0.86
10.6
0.24
2.70
4.15
4.00
Imagen 4
0.773
0.38
0.81
13.2
0.18
1.61
1.50
2.18
E
MORE RESULTS
As discussed in Sec. 4. aggregate results already show clear gap between pixel similarity and semantic correctness. Tab. 5 and Tab. 6 provide a more fine-grained view, breaking down perfor-
mance by specific instruction types.
A recurring pattern is that edits involving numbers, such as adding or adjusting datapoints, are often the hardest to get right. Models may place a new bar point, but the actual value is off, the axis scale shifts incorrectly, type also tend expose structural weaknesses: grouped bars converted to stacked bars often result in overlapping marks, or the scales fail to adjust.
the legend does not update. Edits that change the overall layout or chart
By contrast, stylistic edits like changing background colors are sometimes handled better, though even here models often stop short of a full update. For example, the background changes, but the legend or axis elements remain inconsistent. Text edits such as axis labels or titles show the partial benefit of OCR, but issues like misplaced text, font mismatches, or truncated labels still appear.
F DATASETS USED FOR BASE FIGURES
Tab. and Tab. 8 list all datasets from which we sampled base figures. These sources span public machine learning repositories, official statistical agencies, open data portals, and journalism/sports archives. We include the identifier strings exactly as used in pipeline.
G USE OF LLMS
In addition to conventional data collection and analysis, made of LLMs at several stages of our work. First, LLMs were applied during the writing process to assist with polishing and improving the clarity of the manuscript. Second, LLMs also leveraged to support certain
27

---

Published as a conference paper at ICLR 2026
Table 6: Per-instruction performance comparison (Part 2/2). Higher is better for SSIM, CLIP, PSNR, OCR, and LLM Scores. Lower is better for LPIPS. Instr. denotes instruction following score. Preserv. denotes content preservation score. Qual. denotes image quality score.
SSIM t LPIPS + CLIP PSNR 1 OCR
LLM Score (1-5)
Model
Instr.
Preserv.
Qual.
Instruction: Increase margin
InstructPix2Pix
0.726
0.49
0.83
10.9
0.25
2.70
2.55
3.35
OmniGen2
0.739
0.47
0.84
11.1
0.29
2.75
2.90
3.75
GPT-Image
0.731
0.52
0.87
10.3
0.22
2.95
3.60
4.05
Imagen 4
0.769
0.41
0.79
13.0
0.08
2.37
1.68
2.42
Instruction: Decrease margin
InstructPix2Pix
0.728
0.48
0.84
11.2
0.27
2.60
2.60
3.55
OmniGen2
0.742
0.46
0.83
11.4
0.30
2.30
2.80
3.65
GmT-Image 0.733
0.51
0,86
10.5
0.23
3.15
3.50
Imagen 0.771
0.40
0.80
13.1
0.09
2.00
1.56
2.75
Instruction: Add new data point
InstructPix2Pix
0.724
0.50
0.82
10.8
0.24
1.21
2.66
3.14
OmniGen2
0.737
0.83
11.0
1.86
2.10
3.14
GPT-Image
0.730
0.53
0.87
10.4
0.21
3.07
3.69
4.07
Imagen 4
0.768
0.42
0.79
13.2
0.08
1.04
1.61
2.39
Instruction: Remove an existing data point
InstruetPix2Pix
0.727
0.49
0.83
11.1
0.25
1,59
1.83
2.83
OmniGen2
0.740
0.47
0.82
11.3
0.27
1.38
1.59
2,41
GPT-Image
0.732
0.52
0.86
10.5
0.22
3.10
3.34
4.28
Imagen 4
0.770
0.40
0.80
13.0
0.07
1.41
1.68
2.41
aspects of dataset construction, where they were used to generate and refine synthetic examples in controlled manner. These to auxiliary tasks such as language editing and expanding data diversity, without affecting the core experimental design evaluation.
complementary primary methodology and were limited
28

---

Published as a conference paper at ICLR 2026
Table 7: Allowed datasets (part A), See Table 8 for continuation.
Datasets (Part A)
Kaggle: Titanic
Kaggle: House Prices
Kaggle: Instocart Market Basket
Kaggle: NYC taxl Trip Euration
Kaggle:
Reggle: Yelp Reviexs
saggle:
Kaggle: Marcari Price Suggastion
Xaggle: Quoza Insinoere Queationa
Kaggle: Toxic Comment Clastinication
Kaggle: Porto Seguro Safe Driver
Kaggle:
Customer Transaction
keggle: Santandez Value Prediction
kaggle:
Temperature
Kaggle: COVTR-19 Globsl Dataset
Kaggle: World Bappiness Repozt
Kaggie: FIZA Flayer Statistics
Kaggle: Alr Quality UCI
Kaggle: Accidents
xzggle: Zomato destaurants Lataset
Kaggle: Video Game
Raggle: Netflix Movies and Shows
Kaggle: New York City Airbnb Open Data
<aggle: Google 2lay Store Apps
Kaggle: Bike Shazing Demand
Kaggle: Rossmana
xzggle: Store Item Demand Forecasting Challenge
Kaggle: Malmart Recrulting Stare Salea Forecasting
Raggle: Retailrocket Recommender System Dataset
Kaggle: 311 Service Requests NYC
kzggle: Chicago Crime
Keggle: Austin 3ikeshare Trips
saggle: Seattle weather
Kaggle: Daily Delhi Climote
Xaggle: Us Economic Indicators
Kaggle: S&2 500 Companies and Prices
Kaggle: Times Higher Education world University Kankings
Kaggle: Global Terrerrimm Dotabase
Kaggle: World Development Indicators
Kaggle: Airline On-Time Performance
Kaggle: Avita Demard Prediction
Xaggle: TalkingData Ad7zacking Fraud Detection
Kaggie: IE3B-CIS Fraud Detection
Kzggle: Home Credit Default 8isk
Kaggle: Give Some Credit
xzggle: Loan Frediction 111
Kaggle: Credit
Fraud Detection
Kaggle: Telco Customer Churn
Xaggle: Bank Marketing
kaggle: Student Performance
Kzggle: Heart Disease ucr
Kaggle: Breast
Klaconsin (Diagnosticc
Xzggle: Fima Indiars Disbetes Dacabase
Xaggle: Stroke Prediction natasat
Kaggle: FIFA Player Dataset
Keggle: NBA Player Stats
szggle: internatieral rootball Results
Kaggle: European Soccar natabasa
Raggle: years 0lympic history (athletes reauite]
Keggle: Serflix Stock Price
Xaggle: Bizectn Ristorical Data
Kaggle: Cryptocuztency Eistorical Prices
29

---

Published as a conference paper at ICLR 2026
Table 8: Allowed datasets (part B). Continuation of Table
Datasets (Part B)
DCI: Tris
UCI: Wine
UC1: Adule
UCE: Evaluatior
UCT: Ana
001: Studeat Ferformance
U0I: Heart Disease Dataset
UCI: Bank Marketing Dataset
UCL: forest tires Dataset
OCT:
World
oUCD 213A azores
US Census ACS
US Bureau Labor Statistics
us Bureau sconemic Analysis
UN COMTRADS.
MHC Mo:tality Database
Survey
FRED sccnomic Data
US Energy Informstion Administracton
Global Carbon Project
NOAA climate Bata
Berkeley COVID 19 Time Secies
Temperature
Johns lookkins
Price
USDA Crap Praduction Data
OpenFlighta Airport Routes
30


Reviews:
[{"review_id": "860jKRWFii", "reviewer": "Reviewer_rzRC", "summary": "The paper **“CHARTS ARE NOT IMAGES: On the Challenges of Scientific Chart Editing”** introduces **FigEdit**, a large-scale benchmark (30K+ samples, 10 chart types) for **scientific figure editing**. Unlike natural images, charts represent **structured data governed by graphical grammar**, so valid edits must preserve **data–encoding alignment**, **axis coherence**, and **legend integrity**—not just pixel similarity. FigEdit defines **five tasks**: single edit, multi edit, conversational edit, visual-guidance edit, and style transfer. It includes paired figures and specifications to evaluate **semantic correctness** rather than pixel similarity. Experiments on models like GPT-Image, Imagen 4, OmniGen 2, and InstructPix2Pix show that **high SSIM/PSNR scores often mask incorrect edits**, highlighting the failure of pixel-based metrics. Overall, FigEdit reframes chart editing as a **structured transformation problem** and provides the first **semantics-aware benchmark** for developing models that understand both the **visual and data layers** of scientific charts.", "questions": "- How realistic are the synthetic Vega-generated charts compared to real scientific figures? Could adding real-world data improve generalization?\n- How reliable are the LLM-based evaluation scores—were they compared with human judgments or alternative metrics?\n- Why not include code-level (Vega spec) comparison as an additional evaluation, since all charts are programmatically generated?\n- Are there plans to expand beyond 10 chart types to include more complex scientific visualizations (e.g., heatmaps, multi-panel figures)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Quality:** Builds a **large, well-controlled benchmark (30K+ charts)** generated via deterministic Vega rendering, with clear task taxonomy and reproducible evaluation.\n- **Clarity:** The paper is well-organized and visually clear, using intuitive figures and radar plots to illustrate the **gap between pixel similarity and semantic correctness**\n- **Significance:** Establishes the first **semantics-aware benchmark** for chart editing, providing a valuable testbed for evaluating multimodal and instruction-following models on structured, data-grounded visual tasks", "weaknesses": "- **Synthetic generation bias vs. real-chart curation.** FigEdit’s base figures and edits are produced via **LLM-guided Vega/Vega-Lite specs** and rendered images, which can drift from real publication practices; several prior sets rely on **human-curated real charts** and manual validation (e.g., ChartEdit’s 1,405 instructions on 233 real charts).\n- **Subjectivity/noise in LLM-based scoring.** The paper’s “semantics-aware” evaluation relies on **LLM judgement** for instruction following/content preservation, potentially variable across prompts/seeds.\n- **Baseline breadth.** The paper evaluates **four** editors; competing benchmarks often assess **more models** (ChartEdit: 10 MLLMs), providing a stronger landscape read.\n- **Complex figure phenomena underrepresented.** From tables/figures, FigEdit does not clearly cover **multi-axes, error bars, subfigures, or dense annotations**, which are common in scientific graphics and present failure modes captured in broader reasoning/analysis sets like ChartX.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bj5VVJeSl3", "reviewer": "Reviewer_mXb8", "summary": "This paper introduces FigEdit, a large-scale benchmark for scientific chart editing. The authors compellingly argue that chart editing is a structured transformation problem, fundamentally different from pixel-level image manipulation. By providing a benchmark with over 30,000 instances, spanning 10 chart types and 5 distinct tasks, the work systematically exposes the limitations of state-of-the-art image editing models. The paper's core contribution lies in its rigorous problem formalization, the high-quality benchmark itself, and a critical analysis of evaluation metrics, advocating for a shift towards semantics-aware evaluation. This is a valuable and timely contribution that lays a solid foundation for future research in this important yet underexplored area.", "questions": "**Scalability to Complex, Multi-Element Charts:** The examples presented in the paper (e.g., Figure 1, 3, 5) appear to involve relatively simple charts with a moderate number of data points and clear visual separation between elements. It is unclear how the proposed tasks and evaluation would scale to highly dense or complex visualizations, such as scatter plots with thousands of points, intricate network diagrams, or multi-panel figures where inter-panel consistency is critical.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.  **Excellent Problem Formulation:** The paper's primary strength is its clear and insightful formulation of scientific chart editing as a \"structured transformation\" problem governed by a graphical grammar. This conceptual shift from pixel-manipulation to structure-awareness is crucial and correctly identifies a fundamental mismatch in current approaches.\n\n2.  **High-Quality, Comprehensive Benchmark:** The introduction of FigEdit is a significant contribution. The benchmark is large-scale, diverse (10 chart types, 5 task categories), and grounded in real-world data. Its design, which includes ground-truth specifications (Vega/Vega-Lite), enables precise and reproducible evaluation, setting a high standard for future work.\n\n3.  **Critical Evaluation and Insightful Analysis:** The paper provides a thorough empirical study that reveals the shortcomings of existing models. More importantly, it demonstrates the inadequacy of traditional pixel-based metrics (SSIM, PSNR) for this task and convincingly argues for semantics-aware evaluation, notably through the novel use of an LLM-based score. This challenges the community to rethink evaluation standards.", "weaknesses": "1.  **Limited Coverage of Edit Operations:** The set of atomic edits, while canonical, appears somewhat limited. The paper focuses on operations like `add_datapoint`, `change_background_color`, and `increase_text_size` (Table 3, Appendix C). However, real-world chart editing often involves more complex structural changes, such as changing chart type (e.g., bar to line), reordering categories, grouping/ungrouping data, or modifying axis scales (e.g., linear to log). The current operation set may not fully represent the breadth of transformations required in practice.\n\n2.  **Lack of Subjective and Holistic Edit Instructions:** The benchmark is constructed around objective, atomic instructions. In reality, user requests can be more subjective and holistic, such as \"make the layout more compact,\" \"use a more professional color palette,\" or \"highlight the most important trend.\" While these edits are harder to formalize and evaluate, their absence limits the benchmark's applicability to more creative and user-centric editing scenarios. For a dataset-focused paper, addressing the challenge of collecting and evaluating such instructions, even if labor-intensive, would have significantly strengthened the work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VecJuR2Wyj", "reviewer": "Reviewer_PWEt", "summary": "This paper addresses scientific chart editing, specifically proposing a new evaluation benchmark and metrics for this task and an analysis of existing editing and vision-language models on this task.\n\nTheir contributions include:\n- The novel problem formulation of chart editing.\n- A benchmark (FigEdit) specifically designed for scientific chart editing, containing over 30,000 instances. The dataset covers 10 chart types and various edit instructions.\n- LLM-based evaluation techniques to measure structural correctness over pixel similarity, in contrast to metrics like SSIM and PSNR.\n- An evaluation of state-of-the-art models on chart editing that shows severe limitations of existing methods.", "questions": "- lines 204-207: Are Content (C) = {D, $\\tau$, mapping function} and Style (S) = {palettes, fonts, etc.} standard in related literature? It seems arbitrary. For example, why is chart style included in C but not S?\n- (Task 2, lines 216-221) Why not denote multiple atomic edits as $u_1$, $u_2$, $u_3$, etc. instead of just $u$?\n- line 225: Why is $H_{t-1}$ not included in the equation for $\\sigma^*$?\n- (292-294) Was the reliability of GPT-Image evaluated? As in, can one trust that the correct element, and only the correct element, is always circled?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The paper proposes a novel problem formulation (figure editing) that could have significant and practical real-world impact across a variety of industries, like research and business.\n- The paper rethinks traditional pixel-based image metrics (LPIPS, PSNR, SSIM, etc.) for better evaluating the specific task of figure editing.  This is an important research direction for other generative vision tasks, as well.\n- The dataset is constructed from diverse real-world data across different fields, which prevents bias to certain distributions of data, and therefore creates diverse chart appearances.\n- The paper figures are helpful at clarifying and exemplifying the different categories of editing tasks.", "weaknesses": "- There is limited evaluation / justification for why the LLM-based metric is better or more reliable than traditional metrics. A comparison between all these scores and human evaluation would be beneficial for showing whether or not the LLM metric is actually better for evaluating figure edits.\n- Similarly, it would be informative to compare the LLM score per-category (Instr., Preserv., Qual.) to average human score per-category.\n- Fig. 1. The fonts are quite small, especially the axes of the charts and the legend for the radar chart.\n- A brief description of Vega / Vega-Lite would be helpful.\n- The conversational edits are essentially just two-step multi-edits (lines 266-267)?. This seems too short of a conversation to be able to actually determine ability to maintain conversation history. Also, if they are just derived from multi-step edits, how can one be sure that the edits are actually progrssive (e.g. \"make the bars red\" --> \"make the bars a darker red\") and are not completely unrelated (e.g. \"make the bars red\" --> \"make the bars blue\")?\n- The paper might consider adding several more-recent state-of-the-art editing models, such as Gemini/Nano Banana, Kling, and GPT-4o.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)