Panoptic Pairwise Distortion Graph
Authors: Muhammad Kamran Janjua
Abdul Wahab
Bahador Rashidi
Venue: ICLR 2026 Poster (2026)
Keywords: distortion analysis
low-level vision
iqa
graphs
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: In this work, we introduce a new perspective on comparative image assessment by representing an image pair as a structured composition of its regions. In contrast, existing methods focus on whole image analysis, while implicitly relying on region-level understanding. We extend the intra-image notion of a scene graph to inter-image, and propose a novel task of Distortion Graph (DG). DG treats paired images as a structured topology grounded in regions, and represents dense degradation information such as distortion type, severity, comparison and quality score in a compact interpretable graph structure. To realize the task of learning a distortion graph, we contribute (i) a region-level dataset, PandaSet, (ii) a benchmark suite, PandaBench, with varying region-level difficulty, and (iii) an efficient architecture, Panda, to generate distortion graphs. We demonstrate that PandaBench poses a significant challenge for state-of-the-art multimodal large language models (MLLMs) as they fail to understand region-level degradations even when fed with explicit region cues. We show that training on PandaSet or prompting with DG elicits region-wise distortion understanding, opening a new direction for fine-grained, structured pairwise image assessment.

Published as a conference paper at ICLR 2026
PANOPTIC PAIRWISE DISTORTION GRAPH
Muhammad Kamran Janjua, Abdul Wahab, Bahador Rashidi Huawei Technologies, Canada
(muhammad. kamran. jan jua wwwwwwieewwwe...acm com com
a smamtpercpptioon wtt.:/oooodoooooo hos.hioooo....tttttttoooggaaaa
ABSTRACT
In this work, we introduce a new perspective on comparative image assessment by representing image pair structured composition of its regions. In con- trast, existing methods focus on whole image analysis, while implicitly relying on region-Ievel understanding. We extend the intra-image notion of scene graph to inter-image, and propose novel task of Distortion Graph (DG). DG treats paired images as a structured topology grounded in regions, and represents dense degradation information such as distortion type, severity, comparison and quality score in compact interpretable graph structure. To realize the task of learn- ing distortion graph, we contribute (i) a region-level dataset, PANDASET, (ii) a benchmark suite, PANDABENCH, with varying region-level difficulty, and (ii) an efficient architecture, PANDA, to generate distortion graphs. We demonstrate that PANDABENCH poses a significant challenge for state-of-the-art multimodal large language models (MLLMs) they fail to understand region-level degradations even when fed with explicit region cues. We show that training on PANDASET or prompting with DG elicits region-wise distortion understanding, opening new direction for fine-grained, structured pairwise image assessment.
Anchor
Target
Distortion Graph (0G)
Legend
Ancher
Target
Distortion
attributes
Distortion
Reietiors
Scene
darken
Clean
clo
Ficid
Fixld
Jaaceet
Racket
eversharpen
Cleas
Figure 1: DG Task Overview. Top: Given two images, PANDA learns the proposed Distortion Graph (DG).
Bottom: Grounded Subgraphs illustrate how DG grounds regions in distortion information.
INTRODUCTION
In humans, perceptual decisions are often cognitively involved, deliberate, and contextual (Ding & Gold, 2013). Studies have argued that any model of such perceptual decision making should consider the representation the relevant sensory input and how that representation is formed (Gold & Ding, 2013). In the case of visual stimuli, one example of such perceptual decisions is distortion analysis. Yet, when it comes to computational perceptual decision making, often, current design choices favor a top-down approach considering global view of the input for analyses tasks (Zhang et al., 2025; Li et al., 2025; Wu et al., 2024c; You et al., 2024b; Wu et al., 2023).
any categorical decision about the presence or identity of sensory stimulus

---

Published as a conference paper at ICLR 2026
Anchor
Target
Distortion Graph (0G)
Vanaare reppy
Beelos Queny
Person Suburaph
Sherp Subgrach
large language
voda
regiono eeggh.c to
rllgt, an ciegoo In treer lnc.
wiich lagg
(o-Instruct)
Is the severity of the distertion [by] are anaagn quility acars...
DE Suurary
Answer (Bepins Queryl;:
qallity it surrage.
appeer te unssed ous, and thene ls a ioss of
Persen: Lnage sedenate lavel
with chustariie bluedin.
clotting
The tirst Inage coepro The bas botter quality, sty?
SPT Query
Desils: tupel a user natinokie Aicto copressie, ratery
sn
hartass supoiaccety ketter tut tu tirr distor ertificts.
can meensce les of Nattel five anitl besionatly
teagos.
2. Shopp The imags aignoficant oiliiee'
with discem tire detalis, The cozore are also spreciat say vicced
1he firet lmage bot scouenee repecars, eich colore, aut aiffictest Lighting reculetin in cose dishertie and hlureleees, The woln cirsec cttll recepizaatle On detacles. Highring,
Lagend
Anchin
Target
Houmaads arage sdrinat distersion, cles
sise crratls we lotr, ssktigg dieliialt idveriiy seject.
the
Distartion
quility is good with no reticoanle distortien.
Answer (SFT Query)
Scene Rols
Comparative Assessment/Score Mrong Inswer!
No Grounding Regton Inforeation!
Figure 2: Motivation. Current MLLMs (c.g., Co-Instruct (2024c)) fail at region-level understand- ing, struggling even when given explicit region details (name, description, bounding box). DG
grounds assessment in regions, relating distortions and attributes to provide a structured view. Op- tionally, the graph can be fed to MLLM for region- region-wise language descriptions. Scene relations (yellow) are not predicted. Best viewed zoomed-in.
Such design choices are inherently limiting because they do not lend themselves to fine-grained un- derstanding (Rahmanzadehgervi et al., 2024). Further, in multimodal language models (MILMs), they restrict distortion-specifie visual question answering (VQA), ranking, descriptive understand- ing, and even quality scoring to image-level (Li et al., 2025; Jiang et al., 2024; You et al., 2024b). With instruction tuning (Liu et al., 2023) on a limited instruction set as the learning paradigm, more often than not, the outcome is rigid multi- sponses (Zeng et al., 2023; Chu et al., 2025), see fig. One reason for the prevalence of a tow-down approach is the lack of a structured representation that is grounded in image regions.
parameter model that parrots template re-
In this work, we offer novel perspective on learning structured representation between image pairs for assessment, and introduce the task of a Distortion Graph (DG). DG is a general-purpose pairwise graph structure, with regions as atomic components. Each node corresponds to a region, while inter-region edges capture comparative relationships (predicates). Nodes encode the local distortion and severity type as well as a region-level quality score (attributes), enabling region-first reasoning over paired images, fig. 1. We argue that DG is a pertinent structured approach for pairwise comparative purposes since such information aggregates to image-level judgments, while vice-versa is often not true. We position DG such that it can complement MILMS in offering region-wise distortion analysis in natural language, see fig. 2 for illustration.
To realize the task of learning a distortion graph, we introduce a region-level distortion dataset, termed as PANDASET. By design, PANDASET comprises over 500K image pairs degraded by 15 different distortions, ranging from sensor-induced and equipment failure distortions weather dis- tortions, with four different severity levels. Each region has an associated quality score indicating on what end of the distortion spectrum (from clean to severely degraded) it lies. We show that the pro- posed task is indeed computationally tractable and design an efficient architecture, termed PANDA, that learns to predict region-level attributes and predicates to generate DG.
We introduce PANDABENCH, benchmark derived from PANDASET with three splits of increasing region-level difficulty to enable systematic evaluation. We evaluate both open-source and closed- source frontier MILMs on the proposed benchmark under zero-shot and fine-tuned setups. We empirically show that iistortion-ssecifii MLLMs suffer greatly when reasoning over image regions and often resort to template responses, even when explicitly prompted with region- region-wise wise visual mark- ers. Further, such methods are limited by the context length in terms of processing variable number of regions. On the other hand, frontier MLLMS are less rigid and have superior instruction fol-
2

---

Published as a conference paper at ICLR 2026
lowing abilitiis,, yet their performance is not much better than random chance. Additionally, showcase application, we demonstrate that DG in thain-of-thouggee prompting encourages emergent capabilities of MLLMs for distortion understanding.
RELATED WORK
Distortion MILMs. One of the earliest efforts towards enabling low-level vision understanding in MILMs is Q-Instruct, wherein Wu et al. (2024b) introduced a new dataset, Q-Pathways, and instruction-tuned LLaVA-v1.5 (Liu et al., 2024a) for distortion identification and VQA. Several works followed suit, introducing improved benchmarks, training recipes, and methods. Zhang et al. (2025) proposed an extension to Q-Pathways, and unified the task of image quality assessment in terms of numerical scores and descriptive analysis in MLLMs. Wu et al. (2024a) introduced benchmark Q-Bench comprising low-level attribute and descriptive tasks, along with quality score regression to evaluate MLLMs' ability on low-level vision. However, all of these works focus on single image analysis. Towards comparative assessment, You et al. (2024b) proposed a new dataset, M-BAPPS, and fine-tuned Vicuna-vl.5 (Chiang et al., 2023) on quality comparison task in the full- reference setting (i.e., a clean reference image should be available). DepiciQA (You et al., 2024a) introduced general-purpose dataset, DQ495K, to let MLLMs perform similar comparative tasks but even without any reference image. Co-Instruct (Wu et al., 2024c) introduced MICBench, a bench- mark to evaluate MLLMs performance on comparative tasks. but the introduced method allowed multiple images to be compared. Note that specialist models in pre-MLLM era often focused on learning numerical score for image quality assessment (IQA), and several works exist in literature along this direction (Chen et al., 2024a; Agnolucci et al., 2024; Wang et al., 2023a).
Region Understanding. None of the above mentioned work region-first by design. Further, the MLLMs fine-tuned on these datasets can not extend to regions since that is an out-of-domain setting (Rajani et al., 2025), also see fig. 2. Literature, however, has made efforts to enable general- purpose region-level understanding in MLLMs. Set-of-Mark (SoM) prompting (Yang et al., 2023) introduced visual markers overlaid on regions to prompt MLLMS to reason about regions. Wang et al. (2023b) proposed to utilize ofgion-of-interest (Rol) features to generate region-level tokens for ML.LMs. Omni-RGPT (Heo et al., 2025) introduced token markers for region-level comprehen- sion in images and videos. One partieular work, Seagull (Chen al., 2024c), explored region-level descriptive distortion analysis, but only for single image setting. Seagull utilized mask pooling to generate region-level tokens for MLLMs, and introduced dataset for the analysis task. Q- Ground (Chen et al., 2024b) introduced a single-image dataset of image, textual descriptions, and region-level segmentation and trained an MLLM to jointly provide the explanation, and pixel-level distortion masks for 5 distortion types. Its grounding thus phrased as mapping quality descriptions onto segmentation masks within one im- age. Similarly, Grounding-IQA (Chen et al., 2024d) operates in a single-image setting, and defined two sub-tasks: GIQA-DES which considers quality descriptions with bounding boxes and GIQA- VQA which refers to region-wis quality QA. It introduced a dataset GIQA-160K plus aIQA-Bench to fine-tune and evaluate MILMSs on grounding quality attributes to local regions.
(ifround100K. built on top of Q-Instruct (Wu et al., 2023),
Note that none of these works simultancously (i) comparative in nature, (ii) region firsst and (iii) provide dense distortion annotations, for a diverse set of distortions, at the region level (distortion type, severity, quality scores) plus region-wise comparative labels between two images.
DISTORTION GRAPH
Consider a pair of images denoted by IA and Ir referred as anchor and target, respectively. A Distortion Graph (DG) is defined as 4-tuple, i.e.,
G (OAA 04 ED, Es),
where 0+,, Arr are sets of object (or regions) nodes in images IA (anchor), and IT (target), respec- tively. Ep and Es are sets of distortion and scene edges denoting relations among the objects. Given set of distortion relations denoted by Rp, can formally say Ep C 0h x RD x Of, Similarly, given a set of scene relations denoted by Rs, we can write Eg C (Oth x Rs x Ob) (O Rş X (1rr), ), where O= = 44 U 04.
3

---

Published as a conference paper at ICLR 2026
Let AD denote the set of distortion attributes, and As denote the set of scene attributes, then each object of E O takes the form of = I AD.., As,i).
where j E {A, T). of is the class of the object, I denotes the image the object belongs to, AD S &D. and As, - As. Let y denote map written as y : O M with M denoting set of binary masks and M = M U M In other words, Y maps each object of O to its binary mask mi M. i.e., 7(0²) m²/ effectively grounding the object in its image. Note that, in eq. (1), Es is optional, and subsequently, Iss and Ag are also optional, since distortion graph generalizes scene graph (Johnson al., 2015; Li al., 2024) to distortions and scene information (relations or attributes) is orthogonal to DG semantics. For the sake of completeness, however, we define DG with scene information.
3.1 PROPERTIES OF DISTORTION GRAPH
A Distortion Graph (DG) obeys three important properties to meaningfully describe an image pair in terms of its regions, namely the validity, ordering, and functional comparison.
Preliminaries. There exists a finite index set II and two injective enumerations )er S 04 and (oF)ses S Oh, such that for each i E J. o²* and of denote the same object (or region) across two images (anchor and target). By convention, if i + j then of A of and of + of. We refer to (o, as the ith matched region (or object) pair.
Definition 1 (Validity of Distortion Edges). For every (o,r,o') E ED, there exists i E with of E 0 o' = of E O and P.D In particular, no intra-image triplets belong to Ep. Formally, we define validity as:
ED S (loA, i I RD) S O RD 0h.
Definition 2 (Ordering of Distortion Relations). The distortion relation set (or comparative relation set) R, is interpreted as anchor relative to larget. Accordingly, distortion edges are always ordered and written as (of, r, o..) Formally,
ED S 0 x RD x O and Vi J E RD (of of ) )& E.
(3)
Definition (Functional Comparison). For every matched region pair (o,, o!)) where E J, exactly one distortion relation E En compares them. Formally, we can write:
E : |[r E RD (o,,,, EED> =1
(4)
3.2 GENERATING DISTORTION GRAPH
We propose a simple and efficient method, termed as PANDA to learn Panoptic Pairwise Distortion Graph for image pair. PANDA is a neural network parametrized by 0 that takes input pair of images, referred to as anchor and target, and predicts for each region, distortion relationship (comparative relation), type of distortion afflicting the region, severity of the distortion, and a quality score through multiple heads in DETR-like (Carion et al., 2020) fashion, see fig. 3 for illustration of the architecture.
We treat comparative relation, distortion and severity type as categorical values with categorical cross-entropy as the loss function of choice for their respective heads, while L,, loss function penal- izes the score regression head. Each head is simple 3-layer MLP. PANDA is trained for total of 30 epochs with AdamW (Loshchilov & Hutter, 2017) as the optimizer, and a learning rate of le with weight decay of 0.01. The total loss function is C = X₂ Leel + >g dist + As + A We search for the optimal values of the learning rate, and each X: see details appendix
3.2.1 PANDA ARCHITECTURE
Given an image pair IA and Ir. we feed both to a pretrained encoder (c.g., DINOv2 (Oquab et al., 2023)) to get feature map Fj E R XXX'' where H, W denote the spatial dimensions, C. is the number of channels, and j E Ir, A panoptic segmentation method (e.g.. SAM (Kirillov et al., 2023)) acts as the map function (7) segment each region into corresponding binary masks mf M. Let NR denotes the number of regions in each image. We make sure that all regions align
4

---

Published as a conference paper at ICLR 2026
Region Attributes
Rusagxi
Degradation Decoder
Candidate Images
Region
Rel
Sev Head
Score Read
NLP
Relationship
Head
R Rosagsttorre
Image Encoder
Frane
proj
Degradation
Cross Attention
ScIF Attention
Features
becoder
xN
(A/T)
Projection
Hi
F
Panoptic
Segnentation
Regions
Token
Proj- Fextures
Pool
R R
R
Figure 3: Architecture Diagram. Illustration of the proposed PANDA architecture to learn Distor- tion Graph (DG). A pair of image is fed as input, and for each region in the pair, their comparative relationship (predicates), distortion type, severity type and quality score (attributes) are predicted.
across both images in the pair for one-to-one correspondence in regions, i.e., Nn N No- For exposition, we only use N₂ here onward.
Token Pool. To associate each region with the image, we maintain token pool comprising learnable vectors called tokens of same spatial shape as the binary masks, similar in spirit to Heo et al. (2025). We define a token pool set of learnable vectors for each image in the pair Th and T wherein each E! E xxx11
|T K where K is the total number of tokens. From their respective pools, we sample NR indices uniformly and without replacement to Nu obtain The {+/ Jiel and The {t. Na Nn
E [IA, IT}, and [Th]
Ji=l or more generally with abuse of notation, we can write I2 Tw E XX Every ith region mn one-to-one matched with ith token E. We then compute the Hadamard product h! m, @ t² where hy E H' and H E RNaxHx and project with a convolutional layer to match the dimensions and combine with the respective image features, i.e., H = Conv(H)) @ F E X
Additionally, let the pretrained features be learnable through obtain fv Conv(F'). This procedure allows variable number of regions to borrow information from respective images with minimal compute.
convolutional layer and
Degradation Decoder. Given the feature map for a batch B of image pairs F where k E {LA, IT). and the region features H where j {IA, Ir) and j * k, we feed them through L Transformer layers (Dosovitskiy et al., 2020) followed by four prediction heads to decode each region into rela- tions and distortion attributes. At I-th layer, we first reshape F E Bx xwx xXxC to F E R Bx DxC where D = H x W and denotes the number of patches, and add positional embedding to each patch, i.e., F F* PE. We, then, project for to obtain Opz: Kpe: Vps f*w/F* matrices, and compute multi-head attention (MHA) (Vaswani et al., 2017) followed by skip connection to obtain
SA F
MHA (MH , Kge. Vpa)+ F*
RBDxX.
(5)
We then let each region in image attend to the image features and learn its correspondence with its matched region in the other image k. In other words, we compute cross- cross-attention where query comes from Hi, and key and value matrices come from F*. Similar to F*, we reshape Hi e X to (( by combining regions in the batch dimension and denotes the number of patches of each region. For each region in batch dimension and for each patch associated with the region, add positional embedding, i.e., H = H' PEN, + PE. We, then, project H' to obtain the query matrix QH + H' and key and value matrices come from SAA i.e., K Kys,, Vyn ySAWy For brevity, drop the subseript f from SAA Since Qn E DxC we repeat both Kysa and V yoar Ng times and compute multi-head cross- attention followed by skip connection, i.e..
CA 3
MHA (HAA(QE Kysn, Vysa) + HD] E (BxxN))xDXC
(6)
5

---

Published as a conference paper at ICLR 2026
Region First
Comparative Nature
Diverse Distortions
Severity Levels
Quality Score
Benchmark
Q-Bench (Wu et al., 2024a)
x
x
X
DQ495K (You et al., 2024a)
x
Seagull- 100w (Chen al., 2024c)
V
X
V
Q-Patwways (Wu et al., 2023)
X
X
X
V
V
MICBench (Wu et al., 2024c)
X
X
Q- Ground100K (Chen al., 2024b)
V
X
X
GIQA Bench (Chen et al., 2024d)
X
X
PANDABENCH
Table I: Benchmark Summary. A comparison of PANDABENCI with prior distortion benchmarks in literature. Note that, none of these benchmarks are both region- ferist and comparative by design.
The output feature map goes through MLP and obtain yj-k (y which summa- rizes how each region in image compares with image k. In other words, such a procedure lets each region in one image find corresponding region in the other image in a pair. Note that, before attention and MLP layers, we project the input through layernorm (Ba et al., 2016).
Prediction Heads. simple global average pool (GAP) averages the spatial dimension of the ob- tained output feature map, i.c., Gz- GAP (y)=k) BBXC We feed G, to four 3-layer MLPs with layernorm (Ba et al., 2016) and GELU activation (Hendrycks & Gimpel, 2016), i.c., y CELU
followed by & FC(CLLU(IM(FC())) E BxNxx where LN is layernorm, FC is a fully-connected layer, ý is the output, and c is the output dimension which can either be the number of classes single score accordingly. We omit scene prediction heads, as scene information is out of scope, though the architecture trivially accommodates them.
DATASET & BENCHMARK
Given the lack of dataset for the purpose of region-level pairwise comparative distortion anal- ysis, see table 1, we propose new dataset, termed as PANDASET, and benchmark, termed as PANDABENCH. We build our dataset on two publicly available datasets, namely PSG (Yang al., 2022), and Seagull-100s (Chen et al., 2024c). PSG (Yang et al., 2022) is an intersection of Vi- sual Genome (Krishna et al., 2017) with COCO (Lin et al., 2014), i.e., combining scene information with region-level panoptic segmentation. While Seagull- 100w contains images with real distortions, simulated through varying the parameters of an ISP, and region- region-wise wise segmentation maps.
4.1 PANDASET
We sample 2, 200 high-quality unique images depicting diverse set of scenes in both indoor and outdoor settings captured various lighting settings with different camera angles. Around 1, 592 images are taken from PSG, and 608 images come from Seagull-100ws We divide the dataset into train, validation and test sets with 2, 000 images in train, 50 images validation, and 150 images in test set. Each image has variable number of regions with maximum of 112, and mean of 18. In total, PANDASET contains 528K image pairs across train, validation and test sets.
Distortions. We extend 11 categories of distortions from DepictQA (You et al., 2024a) with three weather-induced distortions, namely, rain, snow and haze, yielding total of 14 distortion cate- gories: blur, brightness, compression, contrast strengthen, contrast weaken, darken, haze, noise, oversharpen, pixelate, rain, saturation strengthen, saturation weaken, and snow. Each distortion is further sub-categorized (different types of noise, blur, compression methods, etc.), giving total of 32 sub-types. We also consider the mixed distortion setting, where each region is degraded differ- ently by uniformly sampling from the list of distortions. In case of Seagull-100w however, we keep the ISP degradation wherever an overlap exists with the chosen distortion for a particular region, i..., ISP noise or blur for a region is picked over synthetic noise or blur.
We use pre-norm residual blocks.

---

Published as a conference paper at ICLR 2026
Methods
Comparison
Distortion
Severity
R
FT
P
R
F1
A
R
F1
SR
PL
Q-SIT (2025)
0.18
0.09
0.07
0.05
Q-Insight (2025)
0.16
0.16
0.00
0.07
0.19
0.19
0.24
0.11
DepictQA (2024b)
0.15
0.11
0.13
0.10
0.27
0.12
0.20
0.11
Seagull (2024c)
0.23
0.25
0.20
0.18
0.32
0.331
0.26
0.26
Gemma-3 27B (2025)
0.31
0.24
0.20
0.30
0.31
0.30
0.27
DepiciQA (2024b)
0.49
0.48
0.38
0.42
0.75
0.82
0.71
0.76
0.55
0.53
0.45
0.48
0.78
0.77
GPT-5 Nano (202))
0.34
0.26
0.28
0.26
0.37
0.37
0.29
0.28
0.29
0.28
0.27
0.21
0.39
0.44
GPT-5 Mini (2025)
0.31
0.32
0.31
0.26
0.49
0.54
0.44
0.44
0.36
0.32
0.31
0.29
0.52
0.54
GPT-1o (2024)
0.26
0.29
0.26
0.23
0.60
0.11
0.11
0.27
0.51
Gemini 2.5 Pro (2025)
00.22
0.29
0.25
0.18
0.39
0.59
0.36
10.1
0.29
0.32
0.25
0.26
0.59
0.60
Random
0.20
0.20
0.20
0.19
0.07
0.07
0.07
0.06
0.25
0.25
0.25
0.25
0.00
0.00
Linear Probe
0..77
0.35
0.22
0.15
0.20
0.16
0.09
1,00
0.27
0.22
0.26
0.15
0.12
0.14
Atlentive Probe
0.47
0.47
0.42
0.43
0.40
0.38
0.42
0.39
0.29
0.26
0.27
0.26
0.37
0.44
PANDA
0.58
0.61
0.54
0.56
0.78
0.79
0.81
0.79
0.59
0.61
0.58
0.59
0.79
0.83
Table 2: PANDABENCH Easy, Results of different MILMS on the Easy set. E indicates open source/open-wett weight, and denotes closed-source MILMs. @ stands for baselines. f indicattss method is trained on PANDASET. A: Accuracy, P: Precision, R: Recall, SR: SRCC & PL: PLCC.
This forms the basis of pairs wherein sample two images with different distortions but same scenes forming a total of " = 240 permutations, and, hence, 480K pairs for training. 12K pairs for validation, and 36K pairs for testing. We add distortions region-wise by uniformly sampling from 14 distortions with 80% probability that a region is degraded, and 20% probability that it is clean. A region can either be degraded with one of the 14 distortions or can be clean giving a total of 15 different distortion types; samples are shown in fig. 8.
Severity & Quality Scores. For each region, the chosen distortion is added with one of three severity levels: minor, moderate, and severe. In case of no distortion, clean with 20% probability, the severity is set to none, giving a total of four severity types. The intensity of each distortion varies with each severity level, and we follow You et al. (2024a) to vary the intensity of non- non-weather distortions. For weather-induced distortions such as rain, and snow, we utilize various rain and snow overlays (Garg & Nayar, 2006; Liu et al., 2018), 8), while for haze, we vary the atmospheric light and haze density parameters in the atmospheric scattering model following Guo et al. (2024). For quality scores, we compute full-reference TOPIQ (Chen et al., 2024a) score (E [0. 1)) between the distorted region and the ground -tuth region to serve as a quantitative indication of region quality. We present visual summary of the entire PANDASET in fig. 6 (appendix). Regions are uniformly distributed among different distortions, around R 3.5%, and each severity category spans RJ 15% of regions.
Comparative Relationships. In DG, inter-region edges are labeled with relationships (or predi- cates) that compare them. We find that TOPIQ (Chen et al., 2024a) accurately indicates the severity of a distortion in terms of a numerical score. For simplicity we adopt TOPIQ as the basis of com- parative relationships, however, more complex preferences can be used. For every region pair, we define a threshold on the difference between scores of the region in the anchor and the target image. If the difference is less than 10.1/, we label the region as same, while it is slightly better or worse in the interval [0..1, 0.3). Similarly, if the difference more than 0.3, we label it as significantly better worse depending on which pair (anchor or target) scores higher.
4.2 PANDABENCH
Three representative splits from the test set of PANDASET, termed Easy, Medium, and Hard, com- prise the proposed benchmark, PANDABENCH. In Easy, we only consider pairs where all the regions in an image are degraded by single type of distortion, but with either same or different severity levels. In Medium, one of the images in the pair is from the mixed setting, i.c., each region exhibits different degradation and level of severity. In case of Hard, both images are degraded with mixed distortions and severity. In each setting, we randomly sample 300 image pairs. This spectrum of splits, with increasing difficulty from Easy to Hard, enables thorough evaluation of the methods to benchmark region-level understanding for distortion analysis. A few illustrative examples from each setting are presented in the appendix, see fig. 15.
7

---

Published as a conference paper at ICLR 2026
Methods
Comparison
Distortion
Severity
R
FT
R
F1
A
R
F1
SR
PL
Q-SIT (2025)
0.16
0.07
0.05
0.04
Q-Insight (2025)
0.21
0.05
0.06
0.04
0.24
0.16
0.21
0.12
DepictQA (2024b)
0.06
0.08
0.05
0.24
0.17
0.20
0.10
Seagull (2024c)
0.13
0.10
0.13
0.08
0.25
0.22
0.21
0.18
Gemma-3 27B (2025)
0.09
0.09
0.07
0.25
0.26
0.25
0.22
DepiciQA (2024b)
0.33
0.21
0.18
0.19
0.22
0.12
0.00
0.00
0.30
0.24
0.23
0.22
0.18
0.17
GPT-5 (2025)
0.21
0.17
0.16
0.16
0.19
0.09
0.07
0.06
0.25
0.20
0.18
0.17
0.02
0.04
GPT-5 Mini (2025)
0.18
0.19
0.20
0.15
0.17
0.10
0.10
0.09
0.27
0.22
0.21
0.20
0.09
0.13
GPT-1o (2024)
0.16
0.19
0.16
0.11
0.07
0.08
0.18
0.06
Gemini 2.5 Pro (2025)
0..1
0.20
0.14
0.12
0.11
0.10
0.07
D.D8
0.17
0.21
0.13
0.16
0.10
0.14
Random
0.21
0.20
0.20
0.19
0.07
0.07
0.07
0.06
0.25
0.25
0.25
0.21
0.0
0.0
Linear Probe
0.43
0.28
0.21
0.14
0.30
0.11
0.09
1.000
0.24
0.29
0.26
0.14
0,22
0.23
Atlentive Probe
0.39
0.23
0.21
0.18
0.10
0.08
0.08
0.07
0.24
0.25
0.25
0.23
0.02
0.02
PANDA
0.40
0.31
0.25
0.24
0.27
0.22
0.18
0.19
0.33
0.34
0.33
0.33
0.36
0.38
Table 3: PANDABENCH Hard. Results of different MLLMs on the Hard set. E indicates open source/open- weight, and denotes closed-source MILMs. @ stands for baselines. f 'indicates method is trained on PANDASET. A: Accuracy, P: Precision, R: Recall, SR: SRCC & PL: PLCC.
5 EXPERIMENTS
Given that we propose the task of learning a Distortion Graph (DG), there exist no specialist meth- ods in literature. Hence, we consider several open-soure and closed-source (frontier) MLLMS to conduct thorough experiments on the proposed PANDABENCH. Several current MILMs such as Q-Instruct (Wu et al., 2023), Co-Instruct (Wu et al., 2024c). Janus-Pro-77 7B (Chen et al., 2025), and LLaVA v1.5 (Liu et al., 2024a) fail to reliably' perform comparative analysis at region-level. Meth- ods like Q-Instruct (Wu et al., 2023) are not suited for multi-image tasks and are biased to the order in which multiple images are fed. While Co-Instruct (Wu et al., 2024c), Janus-Pro-785 (Chen al., 2025), and LLaVA v1.5 (Liu et al., 2024a) can accept multiple images, but, if they are instruction tuned for distortion tasks such as Co-Instruct (Wu et al., 2024c) is, they have trouble following new instructions (Chu et al., 2025); see fig. 2 for example.
General purpose open-source MIL.Ms, the other hand, suffer in distortion analysis tasks, likely due to their lack of exposure to degraded images as well as differences in training data, objectives, and scale compared frontier models. We present a few case studies, including failure cases and discussion on their behavior, for each of these methods in appendix C.
Open-Source MLLMs. A few open seee--suurceopen-wwege 2025). Insight (Li et al., 2025), DepictQA (You et al., 2024a). Gemma-3 27B (Team et al., 2025), and Seagull (Chen et al., 2024c) understand distortions, where Gemma-3 is general-purpose, and others are distortion-specciee MI..Ms. While they still can not do region-wise comparative tasks, they can classify distortion type and level of severity. We prompt open-source methods with a single region at time overlaid with visual marker in the form of a bounding box covering region of interest (Rol), while the rest of the image is dimmed', see appendix fig. 9 for sample prompt. We find that only in this manner, the output responses be parsed and scored.
MLLMs such as Q-SiT (Zhang et al.,
Closed-Source MLLMs. We also evaluate four frontier LLMs GPT-5 Nano (OpenAL, 2025), GPT-5 Mini (gpt-5-mins wwwpnii--2----2 08 08-07) (OpenAI, 2025), GPT-4o (gpt-44-2024-11-22 (Hurst et al., 2024), and Gemini 2.5 Pro (Comanici et al., 2025). Unsurprisingly, these frontier L.MMs un- derstand regions better than open-source MILMs, and are able perform region-wise comparative assessment. For closed-source MLLMs, we prompt them with one image pair at a time, providing each region's description and bounding box, and ask for region-wise outputs including comparison, distortion type, severity level, and a quality score within the specified range, see appendix fig. 10 for sample prompt.
Lastly, we evaluate three baselines: Random, Linear Probe, and Attentive Probe. In Linear Probe, linear heads on top of DINOv2 (Oquab al., 2023) backbone predict relations and at- tributes, while in Attentive Probe, a Transformer block with cross- tross-attention layer using a learnable query acts as the output head.
Baselines.
We use "reliably" to mean the output responses from these methods can not be scored. *Passing in just the cropped region renders these methods blind due variable size of regions.
8

---

Published as a conference paper at ICLR 2026
Comparison Accuracy
Distartion Accuracy
Severity Accuracy
Qua Lity Score
1.0
1.0
0.8
0.8
0.8
0.8
0.78
4PISPin--55
0.67
0.6
0.6
0.49
0.4
0.31
0.4
0.4
0.36
0.31
0.4
0.36
0.18
8.17
0.09
0.0
0.0
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Figure 4: Emergent Results. Feeding predicted DG in prompt as chain-of-thought (CoT) results in improvement of = 15% (accuracy) region-wise distortion understanding of GPT-5 Mini.
5.1 RESULTS & DISCUSSION
In tables 2, and 4, we present results on the Easy, Hard, and Medium settings of PANDABENCH For comparison, distortion, and severity type, we measure accuracy, precision, recall, and F1 score. While for quality score, report SRCC/PLCC following Mean Opinion Score (MOS) litera- ture (Chen et al., 2024a). In all the cases, higher is better. Across all three settings and all four tasks, our proposed method PANDA achieves the best performance. DepictQA (You et al., 2024a), despite being a much larger model (7B parameters, Vicuna v1.5 (Chiang et al., 2023) backbone), lags significantly behind PANDA, which attribute to the absence of region-firss design considera- tion. DepictQA, like any other open-source MILM,, suffers from context limitations, and frequently hallucinates or omits regions.
As result, these models often fail to provide complete and faithful region- assessments. 5 This limitation reinforces that structure is necessary compactly represent pairwise information. We show that fine-tuning DepictQA| on PANDASET encourages region-first distortion understanding, resulting in second- best performance. A consistent trend across methods performance decline from Easy to Hard settings, highlighting the difficulty of fine tne-grained distortion understanding under complex degradations. Notably, PANDA exhibits the smallest performance drop Hard setting suggesting its efficacy. We conduct ablation studies on design choices, and discuss it in appendix B.
Distortion Severity Performance. In terms of distortion and severity classification, the evalua- tion reveals two consistent trends. First, closed-source MLLMs achieve notably higher accuracy on distortion classification than open-source counterparts in Easy setting, indicating a substantial performance gap. This gap, however, diminishes the task difficulty increases, with all models converging to about a difference less than 12% under the Hard setting. Second, in severity clas- sification, several models, including strong closed-source ones, degrade to the point of performing worse than random baseline the Hard setting.
Comparative & Quality Score Performance. All of the open-source methods compare, in ta- bles 2, 3 and 4, struggle on region-wise comparison and quality score prediction task, including Gemma 3 (Team al., 2025) which 27B parameter model. While closed-source frontier MLLMS perform better, their performance trends mirror those of distortion and severity classification. Across Easy, Medium, and Hard, every method suffers a consistent drop in accuracy, with several MLLMs degrading to near-chance performance under the Hard setting. These results underscore both the current limitations of satt--of-theass models and the value of PANDABENCH in highlighting failure modes that remain hidden under other simpler evaluation benchmarks.
5.2 SHOWCASE APPLICATION
In principle, a distortion graph can be learned for any task where comparative assessment informs downstream cases. On the application front, consider the downstream task of distortion understanding, We follow the experimental setup of Mitra et al. (2024), and adopt predicted DG in chain-of-thought to prompt GPT-5 Mini (gpt-5-ml i- -2255008-77) (OpenAI, 2025). Our findings on easy and hard splits PANDA PANDABENCH, in fig. 4, indicate that coupling DG chain-of- thought elicits emergent capabilities of LLMs in region-wise distortion understanding.
Because of this limitation, prompt open-source MLLMs per-region.

---

Published as a conference paper at ICLR 2026
Ground Truth
Anchor
Target
Region Ancher
Predicted (Hint GPT-5 Min1)
("anctor": ("elstertien"' "clean"
"soverity": 8, "scome": 0.5079), "target"
"arightness" "scorel:: 0.9263), "cssparitos": "ssur")
GPT-5 Kind Prediction
>aactorr: ("Gistortion" "darken" 3. 0.0913), ("distortion") "brightness" "severity" "score": 0.9243), "saparison" "saee")
(Hint
Mini)
("anchor":: ("distortion":
"clean",...) "target":
("distortion": "clean",
"corparison"s "sare")
GPT-5 Mini Prediction
["anchor":: ("distortion":
"oversharpen"", (..), "target";
("distortion"; "clean" ....),
"coeparison": "iinntiicantly "orrs""))
Figure 5: Distortion Graph as Context. Illustrative figure analyzing showcase application wherein predicted DG is fed as context GPT-5 Mini. Top: Sample taken from PANDA PANDABENCH BENCH Easy, Bottom: Sample taken from PANDABENCH Hard. GPT-5 Mini indeed overrides the predicted DG when the pixels disagree with DG.
Analysis of Distortion Graph as Context. We analyze the improvement in fig. 4 and evaluate whether GPT-5 Mini simply copies the DG predictions verbatim. We explicitly instruct GPT-5 Mini to treat DG as hint and fall back the input pixels whenever a conflict arises, and we find that it often does override DG when the visual evidence disagrees, see fig. 5. For example, in the PANDABENCH Easy sample (top row of fig. 5), DG incorrectly predicts the distortion type on the anchor region as clean, while GPT-5 Mini correctly identifies it as darken; this requires actually comparing the anchor and target images rather than copying the graph. Likewise, in the PANDABENCH Hard sample, DG mislabels the bag region clean, but GPT-5 Mini correctly infers oversharpen. We observe a few such cases where GPT-5 Mini corrects DG using pixel evidence.
On the flip side, when there is little trust DG. In the Easy example, for instance, the target sky region is predicted as brightness by DG, and GPT-5 Mini repeats this label even though the ground-truth degradation is clean. We emphasize that this behavior is precisely the intended usage: DG acts as an additional structured cue that the MLLM may either leverage directly or override when its own visual understanding disagrees.
contradictory signal in the pixels, GPT-5 Mini tends to
6 CONCLUSION
In this work, we introduced the task of Distortion Graph (DG), a grouuded topological repre- sentation for pairwise image assessment. We argued that structured representations like DG provide an efficient, compact, and interpretable means for comparative evaluation. To support this task, contributed PANDABENCH, benchmark for assessing the region-level distortion understanding of MLLMs, and demonstrated that current open-source models exhibit clear gaps in region-aware analysis. Our experiments showed that either training PANDASET or prompting with DG part of reasoning chains substantially improves region-wise assessment, We hope this work motivates further exploration of region-first representations for distortion understanding and establishes DG as useful data structure for fine-grained comparative reasoning.
REFERENCES
Lorenzo Agnolucci, Leonardo Galteri, Marco Bertini, and Alberto Del Bimbo. Amiqa: Learning distortion manifold for image quality assessment. In Proceedings of the IEEE/CVF Winter Con- ference Applications of Computer Vision, pp. 189-198, 2024.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv: 1e00106070055 2016.
10

---

Published as a conference paper at ICLR 2026
James Burgess, Xiaohan Wang, Yuhui Zhang, Anita Rau, Alejandro Lozano, Lisa Dunlap, Trevor Darrell, and Serena Yeung-Levy. Video action differencing. arXiv preprint arXiv:2503.07860, 2025.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229. Springer, 2020.
Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics to distortions for image quality assess- ment. IEEE Transactions on Image Processing, 33:2404-2418 2024a.
Chaofeng Chen, Sensen Yang, Haoning Wu, Liang Liao, Zicheng Zhang, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-ground: Image quality grounding with large multi-modality models. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 486-495, 2024b.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811 2025.
Zewen Chen, Juan Wang, Wen Wang, Sunhan Xu, Hang Xiong, Yun Zeng, Jian Guo, Shuxun Wang, Chunfeng Yuan, Bing Li, et al. Seagull: No-reference image quality assessment for regions of interest via vision-language instruction tuning, arXiv preprint arXiv:2411. 2.1111111111,,, 2024c.
Zheng Chen, Xun Zhang, Wenbo Li, Renjing Pei, Fenglong Song, Xiongkuo Min, Xiaohong Liu, Xin Yuan, Yong Guo, and Yulun Zhang, Grounding-iqa: Multimodal language grounding model for image quality assessment, arXiv preprint arXiv:241..17237 2024d.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https: //wmsys. http:////////loo////////////
30 30-vicuna/.
Meng-Jiun Chiou. Learning Structured Representations of Visual Scenes. PhD thesis, National University of Singapore (Singapore). 2022.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501. 2025.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Long Ding and Joshua I Gold. The basal ganglia's contributions to perceptual decision making. Neuron, 79(4):640-645 2013.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929, 2020.
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models but not perceive. In European Conference on Computer Vision, pp. 148-166. Springer, 2024.
Kshitiz. Garg and Shree K Nayar. Photorealistic rendering of rain streaks. ACM Transactions on Graphics (TOG), 25(3):996-100) 2006.
Joshua Gold and Long Ding. How mechanisms perceptual decision-makee affect the psycho-
metric function. Progress in neurobiology, 103:98-111 2013.
11

---

Published as a conference paper at ICLR 2026
Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. Conceptgraphs: Ooen-vocab 3d scene graphs for perception and planning. In 2024 IEEE International Con- ference on Robotics and Automation (ICRA), pp. 5021-5028. IEEE, 2024.
Yu Guo, Yuan Gao, Yuxu Lu, Huilin Zhu, Ryan Wen Liu, and Shengfeng He. Onerestore: A universal restoration framework for composite degradation. In European conference on computer vision, pp. 255-272. Springer, 2024.
Dan Hendrycks and Kevin Gimpel. Gaussian error arXiv: 08415, 2016.
linear units (gelus). ar preprint
Miran Heo, Min-Hung Chen, De-An Huang, Sifei Liu, Subhashree Radhakrishnan, Seon Joo Kim, Yu-Chiang Frank Wang, and Ryo Hachiuma. Omni-rgpt: Unifying image and video region-level understanding via token marks. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3919-3930, 2025.
Carl J Hodgetts, James OE Close, and Ulrike Hahn. Similarity and structured representation in human and nonhuman apes. Cognition, 13::10549,, 2023.
Eva JI Hoeijmakers, Bibi Martens, Babs MF Hendriks, Casper Mihl, Razvan L Miclea, Walter H Backes, Joachim E Wildberger, Frank M Zijta, Hester A Gietema, Patricia J Nelemans, et al. How subjective image quality assessment becomes surprisingly reliable: pairwise comparisons instead of likert scale. European Radiology, 34(7):4494-4503 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21277 2024.
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max W.F. Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi- -mmge instruction tuning. Transactions on Machine Learning Research, 2024, 2024. URL https:
openreview / net forum? www.oooddddstddduaaooooooo
Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. Image retrieval using scene graphs. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), June 2015.
Alexander Kirillov, Fric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan-Yen Lo, et Segment anything. Proceed- ings of the IEEE/CVF international conference computer vision, pp- 4015 2023.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting lan- guage and vision using crowdsoureed dense image annotations. International journal of computer vision, 123(1))32-73, 2017.
Hongsheng Li, Guangming Zhu, Liang Zhang, Youliang Jiang, Yixuan Dang, Haoran Hou, Peiyi Shen, Xia Zhao, Syed Afaq Ali Shah, and Mohammed Bennamoun. Scene graph generation: A comprehensive survey. Neurocomputing, 566: 127052, 2024.
Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, and Jian Zhang. Q-insight: Understanding image quality via visual reinforeement learning. arXiv preprint arXiv:2503.22679, 2025.
Hanhe Lin, Vlad Hosu, and Dietmar Saupe. Kadid-1Ok: A large-scale artificially distorted iqa database. In 2019 Tenth International Conference Quality of Multimedia Experience (QoMEX), pp. 1-3. IEEE, 2019.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
12

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16296-26300 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llavanext: Improved reasoning, ocr, and world knowledge, 2024b.
Yun-Fu Liu, Da-Wei Jaw, Shih-Chia Huang, and Jenq-Neng Hwang. Desnownet: Context-aware deep network for snow removal. IEEE Transactions on Image Processing, 27(6):3064-3777 2018.
Julian Lorenz, Mrunmai Phatak, Robin Schön, Katja Ludwig. Nico Hörmann, Annemarie Friedrich, and Rainer Lienhart. Copa-sg: Dense scene graphs with parametric and proto-relations. arXiv preprint arXiv:2506.21357 2025.
lya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 171.5500 1.0, 2017.
Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig, Compositional chain-of- thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14420-14431, 2024.
OpenAl. Gpt-5 system card 8 2025. (ee---e--------- -ww----88------- 9ebb--6eb---- 2awe299ffff2222 httpffffttttttptttee/g gpt hate----syytem---r-----a .pdf.
URL https: cdn http:////dddo.p......o wwtppppenaa....ppaaaeeeeeee.emee / /
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fermandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprini arXiv:2304.07193, 2023.
Nikolay Ponomarenko, Lina Jin, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazarian, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al. Image database tid2013: Peculiarities, results and perspectives. Signal processing: Image communication, 30:57-77, 2015.
Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind: Failing to translate detailed visual features into words. arXiv preprint arXiv:2407.06581, 2024.
Neel Rajani, Aryo Pradipta Gema, Seraphina Goldfarb- Tarrant,, and Ivan Titov. Scalpel vs. hammer: Grpo amplifies existing capabilities, sft replaces them. arXiv preprint arXiv:2507. 1066 2025.
Sakib Reza, Xiyun Song, Heather Yu, Zongfang Lin, Mohsen Moghaddam, and Octavia Camps. Reef: Relevance-aware and efficient 11m adapter for video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2592-2603, 2025.
Milton A Saffir. A comparative study of scales constructed by three psychophysical methods. Psy- chometrika, 173):179-198, 1937.
Ke Sun, Shen Chen, Taiping Yao, Ziyin Zhou, Jiayi Ji, Xiaoshuai Sun, Chia-Wen Lin, and Rongrong Ji. Towards general visual-linguistic face forgery detection. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19576-19588 2025.
Chris Swoyer. Structural representation and surrogative reasoning. Synthese, 87(3):4495508, 1991.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, al. Gemma 3 technical report. arXiv preprint arXiv:2503. 2025.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz. Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- tion processing systems, 30, 2017.
Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 2563, 2023a.
13

---

Published as a conference paper at ICLR 2026
Weiyun Wang, Min Shi, Qingyun Li, Wenhai Wang, Zhenhang Huang, Linjie Xing. Zhe Chen, Hao Li, Xizhou Zhu, Zhiguo Cao, et al. The all-seeing project: Towards panoptic visual recognition and understanding of the open world. arXiv preprint arX.v:2308.0099:0 2023b.
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jachong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for Ilm reasoning on long videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3272- 3283, 2025.
Hao Wu, Jiayuan Mao, Yufeng Zhang, Yuning Jiang, Lei Li, Weiwei Sun, and Wei- Ying Ma. Unified visual-semantic embeddings: Bridging vision and language with structured meaning representa- tions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6609-6618, 2019.
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312. 2023.
Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, and Weisi Lin. Q-bench: A benchmark for general- purpose foundation models on low-level vision. ICLR, 2024a.
Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang. Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, et al. Q-instruct: Improving low-level visual abilities for multi-modality foundation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 25490-25500 2024b.
Haoning Wu, Hanwei Zhu, Zicheng Zhang. Erli Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Annan Wang, Wenxiu Sun, Qiong Yan, et al. Towards open-ended visual quality comparison. In European Conference on Computer Vision, pp. 360-377. Springer, 2024c.
Ying Xu, Kiran Raja, Luisa Verdoliva, and Marius Pedersen. Learning pairwise interaction for gen- eralizable deepfake detection. In Proceedings the of computer vision, pp. 672-682, 2023.
the IEEE/CVF winter conference on applications
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mart prompting unleashes extraordinary visual grounding in gpt-4v, arXiv preprint arXiv:2310. 11441, 2023.
Jingkang Yang, Yi Zhe Ang. Zujin Guo, Kaiyang Zhou, Wayne Zhang, and Ziwei Liu. Panoptic scene graph generation. In ECCV, 2022.
Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug-ow13: Towards long image-sequence understanding in multi-mode large language models. arXiv preprint arXiv:2408.04844 2024.
Zhiyuan You, Jinjin Gu, Zheyuan Li, Xin Cai, Kaiwen Zhu, Chao Dong, and Tianfan Xue. Descrip- tive image quality assessment in the wild. arXiv preprint 1888: 2024a.
Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. Depicting beyond scores: Advancing image quality assessment through multi-modal language models. In European Conference Computer Vision, pp. 259-277 259-276. Springer, 2024b.
LU Yunfan, Yanlin Qian, Ziyang Rao, Junren Xiao, Liming Chen, and Hui Xiong. Rgb-event isp: The dataset and benchmark. In The Thirteenth International Conference on Learning Represen- tations, 2024.
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu. Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, and Dawei Yin. Exploring memorization in fine-tuned language models. arXiv preprint arXiv:2310.06 14, 2023.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11988 2023.
14

---

Published as a conference paper at ICLR 2026
Juexiao Zhang, Gao Zhu, Sihang Li, Xinhao Liu, Haorui Song, Xinran Tang, and Chen Feng. Mul- tiview scene graph. Advances in Neural Information Processing Systems, 37:17761-17788, 2024.
Zicheng Zhang, Haoning Wu, Ziheng Jia, Weisi Lin, and Guangtao Zhai. Teaching Imms for image quality scoring and interpreting. arXiv preprint 19X19 2025.
15

---

Published as a conference paper at ICLR 2026
TECHNICAL APPENDICES
Region Diversity
Rain
Weather
plant
woman
a -
noow
bax Spsed
tra
Leaves SA
b
a
3:1h
3.35
3.35
Seow
Comeray
Equipment
redyy
3.29
I
POLLUS
flower ca
3.25
floore
road
Evershorpes
war
staater
green
g! ht
män Stat
Minor
Severities
23.15
Decrense
betnball
M PANDASET
2.34
Iscroase
ence
e
Hork In Pavennn
rock
becrease
book
painting
eetoryycee
table
water cei
peopi
chair
Rustatzerde
bottle
a S
ceiling
oup
ing
D
bridee
house
ssone
Nose
3.0%
Savere
15.25
2.09
3.0%
Clean
6.7%
6.0%
Brighteess
3.4%
Turken
Zinglete
Destortion
Light
Compression
Satarate reade
Distortion
Figure 6: PANDASET Summary. Left: A word cloud of region names indicating diversity of the objects images. Right: A region-wise summary of PANDASET in terms of distortions severity. All 15 of the distortions are uniformly distributed across the regions, and we broadly categorize the distortions in super categories: weather, cameralequipment digital, & light, and clean.
Comparison
Distortion
Severity
Scores
R
FI
R
FI
FT
SR
PL
Q-SiT (2025)
0.18
0.09
0.07
0.05
Q-Insight (2025)
0.19
0.08
0.07
0.05
0.21
0.19
0.22
0.12
DepiciQA (2024b)
0.13
0.09
0.11
0.07
0.27
0.114
0.20
0.11
Seagull (2024c)
0.19
0.18
0.16
0.13
0.28
0.27
0.22
0.20
Gemma-3 (2025)
0.21
0.21
0.17
0.13
0.26
0.27
0.26
0.23
DepiciQA/ (2024b)
0.32
0.30
0.25
0.27
0.47
0.56
0.41
0.46
0.43
01.40
0.34
0.36
0.44
0.42
GPT-5 Nano (2025)
0.25
0.21
0.22
0.20
0.26
0.20
0.16
0.16
0.28
0.26
0.23
0.21
0.18
0.21
GPT-5 Mini (2025)
0.29
GPT--4o (2024)
0.19
0.20
0.21
0.17
0.25
0.21
0.22
0.27
0.26
0.22
0.22
0.28
0.33
Gemini 2.5 Pro (2025)
0.14
0.21
0.18
0.13
0.24
0.34
0.20
0.23
0.24
0.29
0.19
0.23
0.31
0.34
Random
0.20
0.20
0.19
0.07
0.07
0.06
0.25
0.25
0.00
Lincar Probe
0.38
0.29
0.21
0.11
0.21
0.16
0.09
0.06
0.28
0.27
0.26
0.15
0.17
0.20
Attentive Probe
0.39
0.34
0.29
0.29
0.24
0.25
0.24
0.223
0.26
0.25
0.26
0.24
0.16
0.21
0.44
0.38
0.55
0.50
0.52
0.49
0.47
0.48
0.56
Table 4: PANDA BENCH Medium. Results of different MLLMS on the Medium set. indicates open source/open-weitt weight, and @ denotes closed-source MLLMs, @ stands for baselines. indicates method is trained on PANDASET. A: Accuracy, Precision, R: Recall, SR: SRCC PL: PLCC.
A MOTIVATION
It is well-studied and understood that a proper structured representation enables many of the visual intelligence tasks in humans (Swoyer, 1991) and machines (Chiou, 2022) alike: spatial understand- ing (Zhang et al., 2024; Lorenz et al., 2025), visual planning (Gu et al., 2024), video reasoning (Wang al., 2025), and even similarity comparisons in visual input in the case of humans (Hodgetts al., 2023). One of the fundamentals of human decision making are pairwise comparisons. Classic psy- chophysics has established that pairwise comparison produces reliable and interpretable perceptual scales compared to absolute ratings (Saffir, 1937), and enable principled selection. It is, thus, natural to consider a structured representation can also aid such comparative decisions. We argue that our proposed Distortion Graph (DG) offers general -pupose structure and acts scaffold towards these decisions.
general-purpoe
DG as General Comparative Formalism. Unlike conventional approaches that rely holistic embeddings or scalar quality scores, DG decomposes perceptual differences into object-anchored nodes, attribute descriptors, and explicit comparative relations across paired inputs. This formal- ism provides several advantages. First, DG offers a general abstract: the same formalism that en- codes region-wise distortions in images naturally extend other setups and modalities, such
16

---

Published as a conference paper at ICLR 2026
as pose differences in paired videos for video action differencing task (Burgess al., 2025), region grounded differences for forgery detection (Sun et al., 2025; Xu et al., 2023), pairwise CT scan as- sessment (Hoeijmakers et al., 2024), benchmarking image signal processors (ISPs) (Yunfan et al., 2024), compressing redundant frames memory hased on similarity (Reza et al., 2025), etc. Sec- ond, DG is inherently interpretable, i.e., each edge explicitly localizes where and how two inputs diverge, enabling fine-grained analysis that opaque embedding distances can not provide. Third, by relying on fixed predicate (such as for comparisons in image quality assessment), DG encour- ages compositional reasoning. Finally, the structured format of DG makes it a natural scaffold for multimodal large language models (MLLMs), which often benefit from symbolic context to reduce hallucination and improve grounding. DG also lifts the requirement of Supervised Finetuning (SFT) the MILMs on a fixed distortion set, and general purpose LLMs, with better instruction following abilities, can be coupled as is. We illustrate this in fig. 4.
Taken together, these properties suggest that DG offers structured reasoning over differences in the visual input, and is step toward a unifying comparative formalism for assessment tasks.
B ABLATION STUDIES
Linear and Attentive probes in tables 2, 3 and 4 serve as ablations of proposed decoder in PANDA architecture. Their performance drop shows that DINOv2 (Oquab et al., 2023) features alone are insufficient, and that the decoder is crucial for enabling each region to retrieve complementary formation from its pair to learn distortion relationships and attributes. We conduct two more ab- lations on feature extractors and the number of Transformer blocks in the decoder. Our default backbone, DINOv2 (ViT-s), yields features of 384 dimensions, and we ablate DINOv2 (ViT-b) and SigLip (Zhai et al., 2023) with a dimension size of 768. By default, we adopt four Transformer blocks in the decoder of PANDA. We also ablate its sufficiency by considering two and six blocks as variations. Figure shows that design choices in PANDA are an optimal balance in network size and performance.
Comparison Accuracy
Distortion Accuracy
Severity Accuracy
Quality Score
0.60
0.8
0.6
0.8
0.55
0.7
0.50
0.6
0.5
0.6
0.45
0.5
0.4
0.4
0.40
0.35
0.2
0.3
0.3
Easy
Mediun
Hard
Easy
Medium
Hard
Easy
Medium
Hard
Easy
Mediun
Hard
SigLip
Panda
TBLock-2
TBlock-6
DINOV2-7684 768d
Figure 7: Design Choice Ablation. Accuracy comparison of different design choices: backbone feature extractors (solid line) and Transformer blocks (dotted line). PANDA maintains balance in size, performance, and efficiency.
Whole Image vs. Region-Wise. Our findings indicate that MLLMs performance is dependent on the granularity of decision-making If whole image, i.c., global view, is considered, the performance is non-trivial, but the same MLLMs suffer when reasoning over regions due to (i) lack of region-wise design considerations and (ii) rigidity induced by SFT on a fixed set of distortions/settings. PAND- ABENCH illustrates this struggle in tables 2, 3 and 4. Recall that in the easy split of PANDABENCH, a single distortion afflicts the whole image. While we reason over regions even in that split, it is pos- sible to query an MILM at the whole-image level for the distortion classification task. In table 5, we report results of an ablation study towards this end with DepiciQA (You et al., 2024a). Depic- iQA achieves low but non-trivial performance when asked to classify distortions at the whole-image level. In contrast, when queried region-wise, the performance sharply drops to chance.
Cost to Query MLLMs.
We analyze the cost to query MLLMs, both in terms of computation and monetary value. We also report the configuration of the MILM,, the LLM and vision tower,
17

---

Published as a conference paper at ICLR 2026
Type
Distortion Classification
Accuracy
Precision
Recall
F1
Region Wise
0.15
0,11 0.32
0.13
0,10
Whole Image
0.26
0.28
0.26
Table 5: Region-wise vs, Whole Image. Comparison of DepictQA (You et al., 2024a) across two
different evaluation setups: whole image and region-wise for distortion classification task.
Compute Cost
Monetary Value
Parameters (Billion)
Method
LLM
Vision Tower
Image Pair (secs)
Image Pair (USD)
2
Q-SiT (2025)
Qwen2 LLM
SigLIP
57. 74
N/A
3
Q-Insight (2025)
Qwen2.5 L.LM
Qwen2.5
274.74
N/A
DepictQA (2024a)
Vicuna v1.5
CLIP
245.42
N/A
Seagull (2024c)
Vicuna v1.1
CLIP
38.19
N/A
GPT-5 Nano (2025)
Pruprietary
Proprietary
N/A
0.002
Proprietary
GPT-5 Mini (2025)
Proprictary
Proprictary
N/A
0.007
Proprictary
GPT-4o (2024)
Proprietary
Proprietary
N/A
0.028
Proprictary
PANDA
N/A
DINOv2
3.53
N/A
0.028
Table 6: Cost Analysis. A summary of compute and monetary costs of different methods computed for single image pair with regions.
Clean
Compression
Darken
Oversharpen
Saturate Decrease
Blur
Contrast Decrease
Haze
Pixelate
Saturate Increase
Brightness
Contrast Increase
Noise
Rain
Snow
Figure 8: All Distortion Types. We visualize all 15 different distortion types on the same image taken from PANDASET. Each distortion degrades the image differently. Some distortions ruin the perceptual quality of the image more than others (e.g., haze, contrast decrease).
along with model size parameters in table For closed-source models, we do not compare the compute cost since they are exposed through an API. While for open-source models, we do not report monetary value given is hard to estimate. All costs are reported on single NVIDIA v100 GPU (32GB) with batch size and an image pair with 14 regions. Notably, PANDA is significantly cheaper terms of computational cost and parameters.
B. DISTORTION GRAPH FORMALISM AGGREGATES TO WHOOLE-IMAGE
We argue that region-first distortion analysis aggregates to and complements whole-image assess- ment naturally. We consider the instant rating task wherein given two images, the task is to rank which better image is perceptually superior. We adopt the KADIDIOK dataset (Lin et al., 2019). Due to lack of region information in KADID1OK, 2023) to generate region masks. We do zero-shot inference with PANDA trained on PANDASET, and report performance the ranking task using the predicted quality
query Segment Anything (SAM) (Kirillov et al.,
the comparative rela-
18

---

Published as a conference paper at ICLR 2026
Ranking
Inference
Method
Accuracy
Time +
Q-Insight (Li et al., 2025) GPT-5 Mini (OpenAI, 2025)
0.6970
8 hours N/A
0.8472
PANDA (ZS) Score Based
0. 0878
4 mins
PANDA (ZS) Predicate Based
0.7690
mins
Table 7: Whole-Image Instant Ranking. DG (generated by PANdA) aggregates naturally to whole- image assessment.
Anchor
Target
Region Focus
Region Crop
Systen Prompt: You careful vision inspector. Respond
only. include text
bounding 1423, 295, 473, 357] xx,,y,,x,,,)). inages: 1) A region crop of bench 2) The full image with everything outside the region of interest (in red) dimed. classify the region into one of the following distortion types: ['blur', "brightness', "clean', 'compression', 'contrast-decreaee" "carken', "haze", "noise", "oversharpen', "pixelate", "rain", "saturate- ["ninon", 'noderate', 'none", "severe'). examole: ("distortion":
Content: Analyze only specified region, bench,
gäven
contrast-increnn
vaturate-increase
the following "severity": ...) replace ... with appropriate distortion severity type.
'snow"
Figure 9: Prompt Type (a). template of prompt for open-source MLLMs. The tags for keywords like image, input, user, assistant, output, etc. that each method requires added as necessary.
Anchor
Target
System: are image quality assessor, Respond with **100 only* natching:
("regtons": [("Indox*) intoel, "a": ("distortion": of [dist-list],
"severity": e..3, "scone":
e..1), "6": ("distortion" same set, "severity"= a..3, "score"s o..1), "conparison": [conp-set] # n_regions]] Regions image sano as regions in inage B.
regions image image nanes. Identify distortion present each region, pick each region frou this 1ist ("blur', "brighhness", "clean", "conpression' , "contrast-decreaae decrease', , "contrast--nnee increase", "darken', "haze', "noise', "oversharpen', 'pixelate', "rain", saturate-dezriaa fsaturate- increase', "snoe']
Provide quality
hounding
Identify the severity of distortion for each ragion from: [a, 2, 3], whare means clean, and means highest severity. in Compane rankings
then
'significantl--arrse,, 'slighely-hetter', Give score botwoon and for each region, based on the dogradation and their soverity, where noans severoly bad inago, and 1 means clean inage.
(e:
"slightly-worrs
MlgniFicantly-betten
Regions:
Bogion: field
[x1+0.0, y2-250.0, x2-1023.0, y2-675.0]
Ragion:
[xl...]
Figure 10: Prompt Type (b). A template of prompt for closed-source MLLMs. Frontier methods have superior instruction following ability, and can reason about the regions from the prompt.
tionships (predicates). PANDA was not originally trained to provide whole-image ranking, and we directly use predicted relationship predicates region- wise scores with naive control logic, e.g., if more regions in image A are better (or score higher), then image A is better, to compute the ranking accuracy in table 7. Distortion graph (DG) naturally extends to whole-image assessment.
19

---

Published as a conference paper at ICLR 2026
Anchor
Target
Proupt (b): Provide an trage quality assessment for eoch of the regions in image A (first) and image (second) Bounding baxes and nares...
Corpressoon Nolse
Answer:
The overall clarity this rain subject, the trees, have lost most of their texture details. there is some noise present, and the corposition is poor, with tilt. The background is alse blurry. Quality this inago is very poon.
very low,
weak lighting.
Comparative Assessment, Score, Incomplete Degradation Identification, Insensitive to Inage
Proupt (b): Provide an image quality assesseent for each of the regions in image a (first) and irage (second) ... Bounding boses and nere....
Noise, Corpression
The overall clarity very low, weak Highting. subject, the treas, have lost most of their texture details. There is sone noise poor, with
irage rain
and the corpesition slight tilt. background is also blurry. Quality this image is very poon.
Figure 11: Q-Instruct Behavior. An illustration of output from Q-Instruct (Wu et al., 2023) when prompted for multiple instructions. It is insensitive the order of image, even when explicitly specified, misses degradation, struggles to follow instruction, and repeats irrelevant information.
B.2 GENERALITY OF DISTORTION GRAPH REPRESENTATION
Although PANDABENCH specifically designed for region-first, pairwise comparison, we find that the Dis- tortion Graph formalism generalizes beyond our proto- col and naturally aggregates to whole-image ranking on standard numan-annotate MOS benchmarks. In table 7,
Methods
Accuracy 1
mPLUG-Ow12 (2024)
48.5
LLaVA-1.6 (2024b)
57.0
evaluate PANDA on KADIDIOK (Lin et al., 2019) by using the predicted DG to rank image pairs and compar- ing against ground-trute truth Mean Opinion Scores. Using the DG's s scalar score attribute as the ranking signal yields an accuracy of 78.83%, while using the comparative predi- cate over regions yields 76.90%, indicating that a model trained on TOPIQ-derived region labels can still produce image-level rankings that align well with human judgments. the same protocol on TID2013 (Ponomarenko et al., 2015), another widely used IQA dataset with diverse distortion types and human-annotate MOS, see table 8. Here, PANDA again achieves strong performance, 77. 77.8% accuracy when ranking with DG scores and 78.4% with DG predicates, out- performing MLLM-bases based baselines such as mPLUG-OW12 (Ye et al., 2024), LLaVA-1.6 (Liu al., 2024b), and Q-Instruct (Wu et al., 2024b). These results support that DG is not merely a dataset- specific construct for PANDABENCH but generally useful representation that (i) can be collapsed into reliable whole-image rankings and (ii) transfers effectively to real-distortion, numan-labeled benchmarks.
Q-Instruct (2024b)
55.0
PANDA (Predicate)
78.4
PANDA (Score)
77.8
Table 8: Generality of DG Represen- tation TID2013.
To further test robustness, we repeat
C ANALYSIS OF MULTIMODAL LARGE LANGUAGE MODELS
In this section, we present analysis of the behavior of different Multimodal Large Language Models (MLLMs), both open-source and closed-source, considered in this manuscript on region-wise dis- tortion understanding. We divide the analysis into three settings based on the results in tables 2, and 4 (i) methods considered, but not reported, (ii) open-source methods considered, and reported, and (iii) closed-source methods considered, and reported. As discussed section 5, we divide the prompt templates into two categories: (a) open-source prompts and (b) closed-source prompts. This is because open- apen-source MLLMs fail to provide an answer since they can either consider the whole image just one region at a time. In contrast, closed-soured source models have superior instruction fol- lowing abilities and can predict distortion and severity type, along with comparative assessment and quality score for all the regions in the image pair. We present samples of both prompt types (a)
20

---

Published as a conference paper at ICLR 2026
in fig. 9 and (b) in fig. 10. Note that in prompt (a) fig. 9, three images are fed to the MLLM, namely the anchor or the target, region focus and crop of the region for the respective image.
C.I METHODS CONSIDERED, NOT REPORTED
Recall that among open-source methods, we also explored Q-Instruct (Wu et al., 2023) (LLaVA v1.5 (Liu et al., 2024a)), Co-Instruct (Wu al., 2024c), and lanus-Pro-7hs (Chen et al., 2025), but did not report in tables 2, and 4 due to unreliability in their respective outputs (see section 5). We detail their respective behavior on PANDABENCH, illustrate with sample outputs, and briefly discuss the reasons.
Q-Instruct. Q-Instruct (Wu et al., 2023), a distortion MLLM, is designed for single image distortion analysis, but not for comparative assessment. A common workaround for multi-image inputs is to stack two images before prompting, as Fu et al. (2024). However, we observe that Q-Instruct does not consistently adapt to the change in order of stacked images. For example, when the left image (anchor) is labeled as noise and the right image (target) as compression, flipping their order does not necessarily flip the predicted degradations. This order-dependence makes the outputs unreliable for comparative tasks, so we do not report Q-Instruct results, An illustrative example of such behavior is presented in fig. 11.
Co-Instruct (Wu et al., 2024c) is a distortion MLLM designed for multi-image com- parisons. Given two (or more) input images, it can describe the quality of the images and compare or rank them. Our experiments show that it struggles with the multi-instruction setting required by PANDABENCH. In practice, it often omits regions due to output length limits repeats the same region until tokens are exhausted. Moreover, it fails to generalize to tasks outside its supervised fine-tuning regime such as the four region-level tasks in PANDABENCH (distortion classification, severity classification, predicate identification, and quality score prediction). An example of this behavior shown in fig. 12.
Co-Instruct.
Region Quary
sayo deggt,, caego
negaone in each amagi anwer (2) dittrrtinn za presan. 1151) shat
No Comparative Assesseent, No Score, wrong Degradation Identification
(1v) and assign a quilityy scors...
Fersee: Queryll
noderate Lnat distertion, pioclatie sturtirg The cocars the persoe a cieeurrg quiliy energs. make dirriaat
uested oc, and there Jons
Co-Instruct
sigetficens distarsian, diacem catadia. The colore alse sonsout moadd The euility of but lmags poer.
4. Nanrial taage has aintmal dictortioe, with a clear avonate landscape is the torkgoondd The quality is good with noticcnble distertion.
Figure 12:
Co-Instruct Behavior. An illustration of output from Co-Instruct (Wu et al., 2024c) when prompted for multiple instructions. It fails to perform comparative assessment, frequently misses regions, and struggles with instruction following.
Janus-Pro-7B. Unlike Q-Instruct (Wu al., 2023) and Co-Instruct (Wu et al., 2024c), Janus-Pro- 7B (Chen et al., 2025) is a general-purpose open-source MLLM designed for various multimodal tasks. However, when applied to low-level vision, its effectiveness remains limited, reflecting broader trend among open-source general-purpose MILMs. We prompt Janus-Pro-7B with prompt type (a), fig. 9, PANDABENCH Medium split, and observe that it almost invariably predicts the same distortion and severity label for nearly every region. This indicates that the method lacks reliable distortion understanding ability and is insensitive degradation inherent in the inputs, mak- ing its performance uninformative. The output is similar to the one presented in fig. 13, except for each region predicted distortion and severity class are clean.
C.2 OPEN-SOURCE METHODS CONSIDERED, REPORTED
As discussed earlier, in open-source methods, we consider Q-Insight (Li al., 2025), Q-SiT (Zhang et al., 2025), Gemma 3-27B (Team et al., 2025), Seagull (Chen et al., 2024c), and DepictQA (You et al., 2024a). Other than Gemma 3-27B, all other methods are distortion-specine that images in PANDABENCH have variable number of regions, and these methods are limited in the number of new tokens they can generate, we adopt prompt type (a), see fig. 9. We modify the prompt with special tokens necessary for each method. Our findings indicate that their perfor-
MLLMs. Given
21

---

Published as a conference paper at ICLR 2026
Anchor
Region Focus
Target
Region Focus
Proept (o): Analyas only specified region, v. (4lyl (2-)))-
bench 2) The full
with everything outvide
the region of interest (ān rel) dimed. Classiry
the region into one of the rnlinging distortion tyses: and ore of the following severity syseas exaplee: ("distortion" distorttion severity
sprroptate
Region Crop
Region
("anchor": ("distortian"; "pixciate", "severity": "eoderate"), "target": ("distortion": "blur", "severity": "severe"))
Figure 13: Open-Source MILM Prompt/Output. A representative example of prompt type (a) along with output for all open-source MILLMs evaluated in this work.
Anchor
Target
Proupt (b): Provide for each of the riglons image (first) and Leage 8 (sccord) ... bounding beses and names...
quality asseccmett
Answen:
(dists blur, winer, a.72) underate, 0.85), tomparison: slightly
(dist: seturete-
16. At (dist: contrast-decreaas, sev: ninor, score: 0.88), B: (dist: contract-incren incresse, severity: eicor, scores a. 0.92), compart son: slightly-urrse)
Figure 14: Closed-Source MILMM Prompt/Output. representative example of prompt type (b) along with output for all closed-source MILMS evaluated in this work. Note that, in this example, the image has 16 regions.
mance is generally limited, especially on Hard split of PANDABENCE indicating a broader trend in lack of region-wise image understanding towards distortion analysis. While Seagull (Chen et al., 2024c) is a region-first method, it struggles with following instructions and can not do comparative assessment. Since the maximum new generated tokens are limited, we query all of these methods region-wisc for each image pair, i.e., separate forward pass for each region in each pair. Hence, we do not report performance on comparative task. For quality score prediction, we observe that the outputs are largely static, varying only in coarse steps of 0.25. This behavior leads low pearson linear correlation coefficient (PLCC) and spearman rank-order correlation coefficient (SRCC), and thus we consider the results unreliable and do not report them. A typical prompt/output example representative of open-source methods is presented in fig. 13.
C.3 CLOSED-SOURCE METHODS CONSIDERED, REPORTED
We evaluate four frontier closed-source LLMs on PANDABENCH, namely GPT-5 Nano (Ope- nAI, 2025), GPT-5 Mini 1ittt---min-2225----11
(OpenAI, 2025), GPT-4o (Hurst et al., 2024), and Gemini 2.5 Pro (Comanici et al., 2025). Compared open-source models, closed-source frontier MILMSs exhibit stronger instruction- following abilities and are not constrained by output length. In practice, we find that a limit of 8192 tokens is sufficient cover all regions in PANDABENCH images. Based on this observation, we employ prompt template type (b) (see fig. 10) for these models. For each split, we conduct three independent runs per method and report the mean performance across runs in tables 2, 3 and 4. We additionally report corresponding standard deviations in table 9 for accuracy metric on all four tasks of PANDA BENCIL. A typical prompt/output example representative of closed-source methods is presented in fig. 14.
(Gpt-44-2024-11-22
D ILLUSTRATION OF PANDABENCH
PANDABENCH comprises three splits, Easy, Medium, and Hard, taken from the test set of proposed dataset PANDASET. From Easy to Hard, the task difficulty progressively increases. Recall that in Easy split, both images in each pair degraded by the same distortion type applied uniformly
22

---

Published as a conference paper at ICLR 2026
Standard Deviation
GPT-5 Nano
GPT-5 Mini
GPT-4o
Gemini 2.5 Pro
Distortion
0,0077
0.0016
0.0075
0.0029
Severity
0.0086
0.0025
0.0083
0.0033
Comparison
0.0065
0.0027
0.0105
0.0060
Hard
SRCC
0.0099
0.0076
0.0067
0.0070
PLCC
0.0097
0.0050
0.0077
0.0081
Distortion
0.0025
0.0076
0.0115
0.0104
Severity
0.0084
0.0015
0.0040
0.0106
PANDABENCH Medium
Comparison
0.0059
0.0055
0.0138
0.0062
SRCC
0.0116
0.0054
0.0078
0.0079
PLCC
0.0103
0.0102
0.008
0.0077
Distortion
0.0061
0.0239
0.0271
0.0090
Severity
0.0065
0.0122
0.0183
0.0094
PANDABENCH Easy
Comparison
0.0088
0.0121
0.0118
0.0083
SRCC
0.0207
0.0100
0.0052
0.0063
PLCC
0.0309
0.0072
0.0064
0.0055
Table 9:
Standard Deviation on Accuracy Metric. The standard deviation for accuracy metric on
all four tasks of PANDABENCH computed over three independent runs of each method. We do not report standard deviation for precision, recall and Fl score for brevity, but they follow similar trends.
across all regions, but with differing levels of severity. In the Medium split, one image is consis- tently degraded by single distortion across all regions, while its paired image exhibits region-wise distortions sampled randomly from the full distortion set. We illustrate the three splits in fig. 15. No- tice how in PANDA BENCH Hard, each region has different degradation, e.g., the ground in middle image (last row) has noise, while it is free of noise in its pair on left (last row).
E HYPERPARAMETER SENSITIVITY ANALYSIS
The optimization objective of PANDA is C= L Ag AgL + Aa We search for the value of each À E (0.01, (obtained with each À set 1.0) serve 5.0} using cross-validation. A set of baseline values indication for early stopping, and we only train for full 30 epochs if particular combination is better than the baseline. We set the final objective as -= 0.1 Liel_ + 1.0 LCF 0.1 x L + 1.0 Lycare where Å₁ 0.1, = 1.0, À 0. = 1.0. Note that each X is common for both its respective heads, i.e., for both anchor and target the same used. We present results of different runs in fig. wherein each grey point denotes an experiment that performed significantly worse and we label top five settings with colored mark. PANDA trained for 30 epochs with batch size of on 8x NVIDIA v100 32GB GPUs, and it processes all regions for an image pair simultaneously. PANDA takes around 1.5 days to train for all 30 epochs, and in inference we employ one NVIDIA v 100 32GB GPU. For learning rate, swept through [le-2 2, le 1e-6,, and found that 1e best balanced speed of optimization and convergence of optimization procedure. As shown in fig. 16, performance remains largely consistent across most hyperparameter configurations, shown by the tight cluster of grey points around similar performance values, with only a few extreme combinations yielding noticeable differences. This suggests that PANDA overly sensitive hyperparameter selection, and that reasonable choices are sufficient for stable performance.
F DISTORTION GRAPHS
We discuss few limitations of this work, PANDA, detail directions for future work, and present reproducibility and data statement. We also present a sample of a dense distortion graph generated from an image pair with several regions in fig. 17.
23

---

Published as a conference paper at ICLR 2026
Clean
Saturate Decrease
Darken
PandaBench Easy
Clean
Mixed
Noise
PandaBench Mediun
Mixed
Mixed
PandaBench Hard
Figure 15: PANDABENCH Representative samples from Easy, Medium, and Hard splits of PAND- ABENCH. Easy split, only one distortion afflicts the entire image (and its regions, but with varied severity), while in Medium, mixed has region-wiss aegion-wise distortions (see the person in blue jacket). In Hard split, the distortion varies by region both images (see ground, bike, trees, etc.). Taken together, they represent spectrum of difficulty with subtle degradations inherent in the image to complicated region-wise distortions.
F.1 LIMITATIONS & FUTURE WORK
PANDA serves minimal yet necessary baseline for learning the proposed distortion graph task. While DG provides complete intra- (semantics) and inter-image (distortions) representation, PANDA remains intentionally simple and leaves room for improvement, particularly in handling complex regions. An interesting future direction would to employ DG as separate intermediate step where the graph is generated as part of the reasoning chains for region-wise distortion reasoning before final answer is generated. Furthermore, as DG provides general formalism for comparative reasoning, an interesting avenue for future work is to extend it beyond distortion analysis broader comparative tasks in vision and multimodal settings.
Another potential avenue for future work pertains the construction of PANDASET. While its scenes are natural, and we preserve the real-world ISP distortions, from Seagull- (Chen et al., 2024c), when they overlap with chosen distortion category, e.g., noise, blur, etc., the remaining distortions are synthetic following (You et al., 2024b;a). This design is intentional because control- lable distortions are what makes it possible to (i) assign deterministic region-level quality scores, (ii) match regions with comparative labels, and (iii) systematically vary difficulty from Easy to Hard in PANDABENCII. We show that our design choices are aligned with human preferences, see tables and 8, but it is possible that comparative relations inherit underlying IQA model's, TOPIQ (Chen et al., 2024a), perceptual biases. More broadly, the lack of large-scale, region- grrioneg real-world comparative dataset in the literature is key limiting factor. Building PANDASET with human- annotated region-level comparative relations at similar scale would require substantial annotation effort, which leave to the future work.
24

---

Published as a conference paper at ICLR 2026
0.59
Comparison Acc.
0.58
0.57
0.56
Dist. Acc.
0.72
0.70
0.68
0.66
Distt. Acc.
0.74
0.72
0.70
0.58
A Sev. AcC.
0.56
0.54
0.52
0.58
T Sev. Acc.
0.56
0.54
0.52
0.56
0.58
0.675 0.7000.725
0.700 0.725 0.750 T Dist. Acc.
0.5250.5500.575 A Sev. Acc.
0.5250.5500.57. T Sev. Acc.
Comparison Acc.
A Dist. Acc.
10.5011000
31.01006
03,103311
30,5000111
Figure 16: Hyperparmeter Sweep. Plot of optimization objective hyperparameter sweep with cross-validation on validation set of PANDASET. Each grey point denotes an experiment that per- formed noticeably worse, and we label top five settings with colored mark. A denotes anchor, T denotes Target, Dist. denotes distortion, Sev. is short for severity, and Acc, denotes accuracy.
We, therefore, view PANDASET the first dataset that enables large-scale, region-wise distortion understanding, and PANDABENCE as the first benchmark that supports systematic evaluation of such region-wise comparative reasoning. We hope that this work catalyzes scientific research on region-grounded comparative quality assessment, and our proposed distortion graph task serves as foundation towards that end.
F.2 REPRODUCIBILITY STATEMENT
We provide all the necessary details to reproduce our work, along with architecture details in sec- tion 3, experimental setup in section 5, hyperparameter details, and compute requirements. in ap-
25

---

Published as a conference paper at ICLR 2026
9els
#istertion
Dense Distortion Graph Sample. An example of image pair with several regions resulting in a dense distortion graph. Left image is Anchor (purple nodes in graph), Right image is Target (green nodes in graph). Legend is presented, and 'rels' is short for relations.
Figure 17:
pendix We will publicly release our code, trained models, and proposed dataset and benchmark to help further scientific research on comparative assessment and region-level understanding.
F.3 DATA STATEMENT
As we discuss in section 4, PANDASET is built with two open-source datasets (i) PSG (Yang et al., 2022) and (ii) Seagull- 100w tion maps, and scene information. In PSG, since it is an intersection of COCO Lin et al. (2014) and Visual Genome (Krishna al., 2017), scene level relationships (or predicates) are provided. While Seagull- 100w provides a short description of each region, we use scene parser (Wu et al., 2019) to parse region relations from these descriptions. Further, images PANDASET vary in resolution and orientation, e.g., portrait in fig. 13 and landscape in fig. 14, with a minimum spatial resolution of 640 x 480. We will release PANDASET with the same license the original datasets.
100w (Chen et al., 2024c). Both of these datasets have region-level segmenta-
26


Reviews:
[{"review_id": "F3aUwUsOGM", "reviewer": "Reviewer_ZyHe", "summary": "This paper introduces a novel task called \"distortion graph\", which represents regional-level quality comparison between image pairs as a structured graph. In this graph, nodes encode regional attributes such as distortion type, severity, and quality score, while edges represent comparative relationships. To support this task, the authors construct the PandaSet dataset and the PandaBench benchmark, and design an efficient model named Panda. Experiments show that existing multimodal large models perform poorly on regional distortion understanding, while Panda significantly outperforms them.", "questions": "1. If the visual encoder were replaced by a fine-grained, structure-preserving model like DeepSeek-OCR, would it be capable of capturing the complex scene relationships in the images? We look forward to the authors’ insightful answer.\n\n2. Does the stability of SAM-based segmentation affect the precision of region-level alignment?\n\n3. Is the proposed dataset organized in a manner analogous to the Tree-of-Thoughts (ToT) approach? The authors are requested to provide a comparative analysis, along with an evaluation of the associated training and inference time costs.\n\n4. Is this dataset inherently more suitable for video understanding scenarios by nature?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. It creatively extends the concept of scene graphs to the field of image quality comparison, defining a structured and interpretable representation method.\n2. The paper's figures are highly expressive, clearly illustrating both the motivation and performance of the work.\n3. The meticulously constructed PandaSet dataset contains over 500,000 image pairs with 15 distortion types and region-level annotations, demonstrating substantial scale.", "weaknesses": "1. Although the paper introduces the \"Distortion Graph\" structure, it essentially extends Scene Graph to image-pair comparison tasks. Meanwhile, region-level understanding has already been explored in recent IQA studies such as grounding IQA.\n\n2. PandaSet builds upon PSG and Seagull-100w without providing genuinely new real-distortion data, relying solely on synthetic augmentations.\n\n3. Comparing the specially-designed Panda model with MLLMs not fine-tuned for regional distortion tasks makes the conclusion \"MLLMs perform poorly\" appear unfair due to task mismatch.\n\n4. Ablation studies remain insufficient, lacking evidence of how existing open-source vision models (Qwen, LLava, Kimi-vl) would perform after fine-tuning on the authors' dataset.\n\n5. The paper provides limited analysis of the token-pool module, region-alignment mechanism, and loss-weight balancing strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0TZlQ3ainV", "reviewer": "Reviewer_w8pd", "summary": "This paper introduces Distortion Graphs (DG) that enable regional distortion analysis through topological representations of image regions. The authors propose a region-level training dataset, an evaluation benchmark, and an efficient architecture for generating distortion maps. Extensive experiments demonstrate the effectiveness of Panda in regional distortion analysis.", "questions": "1. More analysis and discussion of IQA methods that incorporate regional understanding.\n2. Broader evaluation to verify generalization capability.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Regional distortion evaluation through DG is reasonable and effective, and this is the first explicit pairwise regional distortion assessment.\n2. The proposed dataset PandaSet and benchmark PandaBench can promote development in the related research community.\n3. Comparisons with existing MLLM approaches, including both open-source and closed-source models, support the effectiveness of the proposed regional distortion analysis.\n4. The paper contains rich content, including numerous figures and tables that improve clarity.\n5. Supplementary materials provide additional examples and code, which further strengthen the solid.", "weaknesses": "1. The discussion on region-aware understanding in IQA is insufficient. Prior works such as [1] and [2] should be included to improve motivation and task background.\n2. The evaluation is mainly conducted on PandaBench, which limits the ability to fully assess performance. It is recommended to expand the evaluation to more general datasets or conduct user studies to validate generalization further.\n\n\n\n[1] Q-Ground: Image Quality Grounding with Large Multi-modality Models\n\n[2] Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aUm7ve0K4L", "reviewer": "Reviewer_uHjF", "summary": "This paper presents a novel paradigm for comparing the quality of image pairs, particularly those with structured region compositions. The core contribution is the distortion graph, a structure designed to represent diverse distortion information, including type, severity, comparison, and quality score. To support this, the authors introduce a new dataset, PANDASET, used to train a graph prediction network. The network is designed in a DETR fashion, with strong backbones (DINOv2 and SAM) and separate decoders to predict the distinct components of the distortion graph. This graph is then used as a structured input to LMMs to generate region-specific distortion summaries for the input image pairs. Experiments demonstrate that this approach significantly improves the region-wise distortion understanding of current LMMs.", "questions": "Please see weakness points above.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "* The paper is well-written, clearly organized, and easy to follow. The concepts are introduced logically.\n* The proposed distortion graph is a novel and intuitive method for structuring the complex task of regional quality comparison, moving beyond simple scalar scores or text descriptions.\n* The proposed model architecture is well-motivated and thoughtfully designed, intelligently combining powerful pre-trained backbones (DINOv2, SAM) with a DETR-style architecture suited for object-centric prediction. The use of separate decoders for different graph components is a logical design choice.\n* The ability to feed the DG outputs directly into an LMM to generate detailed, region-specific summaries is a practical and valuable application, bridging the gap between quantitative metrics and human-understandable feedback.\n* The reported experimental results are impressive, showing a significant improvement in detailed distortion understanding compared to baseline LMMs.", "weaknesses": "* **Insufficient Discussion of Related Work:**\n    The paper omits a discussion of closely related work in image quality grounding [a, b]. These prior works also aim to localize and describe quality issues, sharing a similar motivation. The paper would be strengthened by citing these works and clearly articulating how the proposed distortion graph paradigm differs from and improves upon existing grounding-based IQA methods. This context is crucial for accurately positioning the paper's contribution within the field.\n\n* **Limitations of the PANDASET Dataset:**\n    The newly collected PANDASET dataset presents potential limitations.\n    1.  **Scale:** At a reported 2,000 images, the dataset is relatively small, which raises concerns about model overfitting both to the semantics and distortion types.\n    2.  **Synthetic Distortions:** The dataset exclusively contains synthetic distortions. This reliance on synthetic data creates significant concerns: (1) The model may not generalize well to the diverse, subtle, and complex artifacts found in real-world images (e.g., camera noise, motion blur, compression artifacts from unknown codecs). (2) The model might be overfitting to the specific synthetic distortion types included in the dataset, potentially failing to identify or compare unseen distortion types.  \n    The authors should discuss these limitations and provide experiments (even on a small scale) to test the model's robustness to real-world or unseen distortions.\n\n* **Fairness of Experimental Comparison:**\n    The experimental comparison with general-purpose LMMs may be unfair, potentially exaggerating the proposed method's superiority. The proposed network is specifically fine-tuned on the PANDASET dataset, while the baseline LMMs are evaluated in a zero-shot inference setting without task-specific training. The observed significant improvements might primarily stem from this specialized training on the target data distribution.\n    It might be better to include this discrepancy in training protocols and discuss it as a confounding factor in the results analysis.\n\n[a] Q-Ground: Image Quality Grounding with Large Multi-modality Models. Chen et al., ACM MM 2024.  \n[b] Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment. Chen et al., arxiv 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JxCCrFQdhQ", "reviewer": "Reviewer_3yab", "summary": "This paper introduces the new task of Distortion Graph (DG), aimed at region-based pairwise image comparison and assessment. The authors argue that existing methods emphasize global analysis while neglecting region-level understanding, and therefore contribute: (1) a region-level dataset, PANDASET, comprising 528K image pairs and 15 distortion types; (2) a three-difficulty benchmark, PANDABENCH; and (3) the PANDA architecture, which employs a DETR-style decoder to predict inter-region comparative relations, distortion types, severities, and quality scores. Experiments show that PANDA substantially outperforms existing MLLMs on PANDABENCH (Tables 2-4), and the generated DGs can serve as chain-of-thought prompts to improve GPT-5 Mini’s accuracy by about 15% (Fig. 4).", "questions": "1. Given that the ground-truth labels for comparative relations and quality scores are entirely dependent on the TOPIQ model, did the authors conduct any human studies to verify that TOPIQ’s region-level scores and their thresholding indeed align with human fine-grained perceptual preferences? If PANDA is merely imitating TOPIQ, how can we ensure it does not inherit TOPIQ’s perceptual biases?\n\n2. The core comparison mechanism of the PANDA decoder allows region features from one image to query features of the other entire image (Eq. 6). Why choose this “region-to-global” attention rather than a more direct “region-to-region” comparison?\n\n3. Fig. 4 shows that using DG as a CoT prompt improves GPT-5 Mini’s performance, yet its original zero-shot accuracy is very low (e.g., comparative accuracy of only 0.31 under the Easy setting, Table 2). Is this improvement due to emergent reasoning capabilities of the MLLM, or is it simply copying answers from PANDA-generated prompts?\n\n4. All models experience a sharp performance drop on PANDABENCH-Hard (PANDA’s distortion accuracy falls from 0.78 to 0.27, Tables 2-3). Is the main cause of this drop that SAM fails to align regions in images with mixed distortions, or that the PANDA decoder cannot effectively compare two inputs both containing complex mixed distortions?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 1, "presentation": 3, "contribution": 3, "strengths": "The core strengths of this work lie in its clear motivation and the comprehensive construction of an ecosystem for the task. Shifting image quality assessment from a global to a structured, region-level representation is a valuable contribution; Fig. 2 compellingly demonstrates failure cases of current MLLMs in understanding region-level distortions. The paper provides a formal definition of DG, a large-scale dataset (2,200 images), a multi-difficulty benchmark (Easy/Medium/Hard), and an efficient baseline model (0.028B parameters). The experimental design is thorough, including systematic comparisons of open- and closed-source MLLMs, ablation studies, and validation of DG as an effective CoT prompt. PANDA’s SOTA performance across all settings (e.g., distortion classification accuracy of 0.78 under the Easy setting) substantiates the method’s effectiveness.", "weaknesses": "The most critical limitation of the paper is a circularity issue arising from the construction of the “ground-truth” labels. While the paper aims to propose a fine-grained assessment paradigm superior to existing evaluators, the two most central label types in PANDASET - region-level quality scores and comparative relations (e.g., “slightly better”) - are entirely derived from the existing TOPIQ model: quality scores directly adopt TOPIQ’s full-reference scores, and comparative relations are obtained by thresholding the differences between TOPIQ scores (p. 7; thresholds ±[0.1, 0.3), etc.). This implies that PANDA is essentially learning to imitate TOPIQ’s outputs rather than human perceptual decisions. The paper offers no human annotations to validate the perceptual validity of these automatically generated labels, which weakens the claim that DG can serve as a new evaluation standard. Moreover, the conclusion that “MLLMs cannot understand region-level degradations” (p. 2) may be overstated: the experiments more clearly show that MLLMs, in zero-shot settings, struggle with this novel out-of-domain task (Tables 2-4 indicate near-random performance across all MLLMs), whereas PANDA is trained specifically on PANDASET. Such a comparison is not entirely fair for assessing the fundamental understanding capabilities of MLLMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)