Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluators
Authors: Jongwoo Ko
Sungnyun Kim
Sungwoo Cho
Se-Young Yun
Venue: NeurIPS 2025 poster ()
Keywords: mllm-as-a-judge
automatic evaluation
multimodal reasoning
multimodal large language models
Primary area: deep_learning
TL;DR: We propose Flex-Judge, a multimodal judge model trained solely on a small corpus of high-quality text reasoning data.

Abstract: Human-generated reward signals are critical for aligning generative models with human preferences, guiding both training and inference-time evaluations. While large language models (LLMs) employed as proxy evaluators, i.e., LLM-as-a-Judge, significantly reduce the costs associated with manual annotations, they typically require extensive modality-specific training data and fail to generalize well across diverse multimodal tasks. In this paper, we propose **Flex-Judge**, a reasoning-guided multimodal judge model that leverages minimal textual reasoning data to robustly generalize across multiple modalities and evaluation formats. Our core intuition is that structured textual reasoning explanations inherently encode generalizable decision-making patterns, enabling an effective transfer to multimodal judgments, e.g., with images or videos. Empirical results demonstrate that Flex-Judge, despite being trained on significantly fewer text data, achieves competitive or superior performance compared to state-of-the-art commercial APIs and extensively trained multimodal evaluators. Notably, Flex-Judge presents broad impact in modalities like molecule, where comprehensive evaluation benchmarks are scarce, underscoring its practical value in resource-constrained domains. Our framework highlights reasoning-based text supervision as a powerful, cost-effective alternative to traditional annotation-intensive approaches, substantially advancing scalable multimodal model-as-a-judge.

FLEX-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluators
Jongwoo Ko Sungnyun Kim
Sungwoo Cho²
Se-Young Yun?
Microsoft
KAIST AI
j jongsooko@miccooooft com, (ksn4397, peter8526, yunseyoung/0kaist. ac.kr
https://flex judge, github.io
Abstract
Human- -ennanngeeertedd reward signals are critical for aligning generative models with human preferences, guiding both training and inference- timee evaluations. While large language models (LLMs) employed as proxy evaluators, i.e., LLM-as-a- Judge, significantly reduce the costs associated with manual annotations, they typically require extensive modality-specific training data and fail to generalize well across diverse multimodal tasks. this paper, we propose FLEX-Judge, a reasoning guidedd multimodal judge model that leverages minimal textual reasoning data to robustly generalize across multiple modalities and evaluation formats. Our core intuition that structured textual reasoning explanations inherently encode generalizable decision decision-making patterns, enabling an effective transfer to multi- modal judgments, e.g., with images or videos. Empirical results demonstrate that FLEX-Judge, despite being trained on significantly fewer text data, achieves com- petitive superior performance compared to state-of-the-art commercial APIs and extensively trained multimodal evaluators. Notably, FI.EX-Judge presents broad impact in modalities like molecule, where comprehensive evaluation benchmarks are scarce, underscoring its practical value in resource-constrained domains. Our framework highlights reasoning- baaed text supervision as powerful, cost-effective alternative to traditional annotation-intensive approaches, substantially advancing scalable multimodal model-ass saseelass--jaaa audge..
Introduction
Human- gennan-geeerated reward signals play a crucial role both training and deploying generative models. They are commonly used to fine-tune models toward human-aligned behavior through preference optimization [48, 56] or reinforcement learning [61, 91]. inference time, they also guide inference- time decisions, e.g., best-of-N selection [24], output reranking [43], filtering based quality or safety criteria, making them essential tools for test-time control. As models become more capable and are applied across diverse modalities and tasks, the need for high-quality, consistent human feedback continues to grow. However, scaling human feedback process is highly resource-intensin and challenging to generalize across different domains, highlighting critical demand for more scalable and cost-effective alternatives that can reliably approximate human judgments [71, 89].
A promising alternative to manual feedback collection is to use large language models (LLMs) as proxy evaluators an approach known as LLM-as-a-Judge [89]. These models emulate human preferences via costruction-followee across tasks such as summarization [60, 61] and dialogue [29]. In addition to reducing annotation costs, they can serve as reusable, modular evaluators and are often comparable to human judges in consistency. However, existing approaches are largely restricted to text-only scenarios [36] and often
prompts and have shown strong agreement with human ratings
'Two authors equally contributed
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Training Multimodal Judge
Multimodal Inference
MILMM backbone
Score Judge
Reasoning
Score
FLEX-Judge (LLM)
Jucian
Think Once,
Image/Video
Audio Encoder
Malecule
Judge Anywhere!
Model APB
Encoder
Encoder
Train
① Reasoning Dataset Curation
Eval. Task
Image
Video
Audio
Malonvee
Text Only Reasoning (Seed)
<ite start|psyster
You are helpfut assiatant.
enclesed within
Desurib Cheniial Structure
<thinko </thinkk and canswer> </anseer>> [...]
TIS
Reasoning- Focused Judge Model
We leverago structured
kunta with
Pytmisier
Text-Only Rationale behind judgments
Witch with
description!
Figure 1: Conceptual overview of FLEX-Judge. We train multimodal judge model using small amount of text-only reasoning data. Unlike previous approaches that require modality-speciffe supervision, FLEX-Judge leverages structured text-only rationale behind judgments enable supeeralice
generalization across modalities. Once trained, FLEX-Judge can be applied to various evaluation tasks, including vision-language tasks, audio quality scoring, and molecular structure, without the need for additional task-specific or modality- annotations.
require substantial amounts of paired preference data [29] to generalize across evaluation types, such as single-score grading, pairwise comparison, or batch-level assessments.
Extending this paradigm to multimodal domains (e.g.. vision-languee generation or image cap- tion ranking) presents unique challenges. A Although some efforts have adapted I.LM-as-a-Judge models into vision-langual
evaluators [11], they typically require extensive modality-specific anno- tations [76] and frequently fail to generalize across diverse formats without re-training or fine-tuning on each new task [32]. Moreover, the lack of publicly available multimodal preference datasets makes difficult to systematically evaluate and train such models, often resulting heavy reliance on proprietary models or benchmarks.
Motivated by the these challenges, we ask key question: Can just small amount of textual reasoning data serve to train cost-effcient, modality-agnoste judge model? Our core intuition that textual reasoning chains, such evaluative explanations or comparisons behind preference judgments, encode structured and interpretable rationale that can be transferred across modalities and evaluation formats. That is, models trained make judgments by reasoning about why one answer preferred over another may learn more generalizable decision rules. In addition, recent advances in multimodal large language models (MLLMs) suggest that their impressive generalization capabilities predominantly arise from their pretrained textual reasoning abilities.
Contribution & Organization. Inspired by this, we propose FLEX--Jude, multimodal judge model trained solely on a small corpus of high-quality text reasoning data. Our central finding that: do not need large-scale multimodal annotations to train effective MLLM judge-just small amount of good reasoning data enough. This text-only supervision not only cheaper but also avoids the need for complex annotation tools multimodal data curation, while achieving strong generalization across diverse modalities and evaluation settings. Refer to Figure for the preview of this work. Our key contributions are:
We propose FLEX-Judge, simple and cost-effective method that uses 1K-sized text-only reasoning data to generalize across modalities without modality-speciff training. This facilitates zero-shot evaluation unseen modalities with minimal annotation and compute overhead.
Efficient Approach (Section 2):
Comparison with State-of-the-arr (Section 3): We evaluate FLEX-Judge on image, video, and audio reward benchmarks against commercial APIs [1, 62], large vanilla MLLMs, and open-source judges trained on costly multimodal datasets [32, 76]. Despite its simplicity and efficiency, FLEX- Judge (7B model) outperforms open-source judges, even exceeding Gemini and GPT-4o on several MJ-Bench and GenAl-Ben Bennh-Benchh subtasks. In-depth analyses are presented in Section 5.

---

Broader Impact (Section 4): We demonstrate real-world applications of FLEX-Judge in the molecular domain by introducing FLEX-Mon MEEE--Mol-LLMM,, the first judge model designed for molecular modality [27]. We showcase its utility in two key scenarios: (1) serving as a best-of-N selector for inference-time scaling, and (2) constructing training data for direct preference optimization (DPO) [56]. In both cases, reward-guided molecular MLLM achieves significant improvements, highlighting the practical solution in domains where modality-speciffc reward models are infeasible.
2 Approach
2.1 Motivation
Problem Statement. Evaluating outputs across multiple modalities using foundation models is increasingly important, especially as generative models expand beyond language to include image, video, or audio [4, 16, 79]. While both proprietary LMs and open-source evaluators are widely used to assess the generative models' response quality, two main challenges remain:
Concern of Commercial API: Issues regarding transparency, controllability, and affordability persist when utilizing proprietary LMs for evaluation tasks [29]. API model changes can silently degrade evaluation quality, raising concerns about the reliability of LLM-aasaa-Jude with closed- source models [9]. For instance, Xiong et al. [76] re-evaluated GPT-4V on the MI.L benchmark [11 |] and found a significant drop evaluation performance.
Limited Support for Diverse Modalities: While judge models for language [29] and vision- language tasks [32, 76] have advanced significantly thanks the availability of assessment data, evaluating other modalities, e.g., audio [15, 16], thermal heatmaps [84]. 3D point clouds [25]. and molecular structures [41], remains underexplored, with few effective judge models publicly available training resources. For instance, evaluating a state-of- the-art molecular LLM [27] often relies GPT-4o to assess the soundness and relevance of its responses, which is not only hard to reproduce but also unreliable, as GPT-4o cannot handle molecular modalities.
Hypothesis. In multilingual LLMs [20, 53, 74], it has been observed that fine-tuning on downstream tasks in one language can lead to performance improvements in other languages as well, demonstrating cross-lingual generalization. This suggests that when a shared representation space exists, task knowledge can effectively transfer across different languages.
We hypothesize that a similar phenomenon may occur in multimodal settings: if model learns a unified cross-modal representation, then fine-tuning on single enable generalization to other modalities. However, such investigations on cross-modal transfer are still rare in MLLMs. Motivated by findings from multilingual LLMs, we propose to build a practical multimodal judge model using small amount of text-based reasoning data and demonstrate that this model can be applied across a variety of modalities, including those with scarce data resources.
text-may
2.2 FLEX-Judge: Reasoning-Guider MILM as a Judge
Building on our hypothesis, we propose FLEx-Judge, multimodal judge model framework trained a priori through textual reasoning annotations. Unlike existing judge models that heavily depend on extensive poddllit-sppecfff preference data, our framework strategically leverages small but care- fully curated corpus of reasoning data (THINK why certain outputs are preferred over others- to foster robust evaluation capabilities across diverse modalities (JUDGE ANYWHERE).
textual annotations indicating
Data Curation. We begin by generating high-quality "seed" textual dataset leveraging JudgeLRM [12], a reasoning-focusee judge LM explicitly trained to evaluate Al responses with struc- tured explanations. JudgeLRM uses crafted prompt templates assess single or paired Al-generated outputs, producing detailed rationales enclosed within specialized tags <think></think>> These reasoning annotations are detailed and comprehensive, explicitly addressing criteria such as correct- ness, completeness, consistency, relevance, and coherence.
A critical advantage of our framework is the minimal data requirement. Specifically, we rely on only K- corpus of high-quality textual reasoning annotations on text-only evaluation samples, making our approach highly cost-efficient, compared to MILM judges such as Prometheus- Prometheus-Vision [32] (150K image-text evaluation pairs for training) and VLLVV--Ciiie [76] (113K pairs). This corpus carefully curated to exhibit the following properties:
3

---

Panda
Panda
(Goneral)
LM
(Doneral)
(Oonerall
72.2
71.2
72
[Reasen'
325
LH
[Raseon
|Raseoni
LM
52.9
(Align.)
[ArtifacO)
(Align.)
(Arthoct)
(Algg..)
(Artifoct)
(Sherd)
[Tanc.)
0.5K
2K
4<
0.1
0.5
0.9
1.3
(a) Quality Length
(b) Number Samples
(c) Sampling Temperature
Comparisons different perspectives of the seed dataset curation. All evaluations are done with our FLEX-Judge, where its backbone model is Qwen2.5-VL-7B (refer to Section 3.1 for model details) and has been trained on the JudgeLRRM-7B response data.
Figure 2:
Quality & Difficulty: Following Muennighoff et [50], we prioritize high-quality and high- difficulty samples when curating training dataset for better sample efficiency. Specifically, we utilize JudgeLRM-7B to generate evaluation responses for prompts in the JudgeLM- 100K dataset [90], and filter out responses whose predicted ratings mismatch with those annotated selection process significantly improves the performance of the trained judge compared to random selection (iee. blue; low-quality in Figure 2a). Consistent with the findings of [50], we also observe that samples with longer reasoning chains (i.e., brown) improve judge performance across both language-only and multimodal settings. This observation supports the hypothesis that longer reasoning indicates higher difficulty, thereby enhancing the effectiveness of limited training data.
by GPT-4o in the original dataset. This quality-
Number of Data Samples On-policy: We observe that training with large number of samples can cause catastrophic forgetring [81], where the LLM backbone exhibits diminished capacity encode visual or audio features. As shown in Figure 2b, while JudgeLM [90] and PandaLM [70] performances improve with more training data, performances on multimodal benchmarks (MLLM-as---Judee and MJ- MJ-Bench) degrade, indicating modality shift detrimental to multimodal understanding. We also find in Figure 2c that lower-temperature decoding yields more effective training data, with lower initial losses. Since JudgeI.RM-BB (the data generator) shares its L.M backbone with FL,EX Fu.ge-Jugee, using these lower-loss, on-policy samples helps prevent catastrophic forgetting while preserving the language-side judge performance.
Compared to naïvely using unprocessed outputs from JudgeLRMM-75 which only supports pairwise scoring
Format Diversity:
1-10 scale shown in Figure 7, we post-proces, the model's outputs to support both single-score and pairwise grading, with scores mapped to either 1-10 or 1-5 scales depending on the instruction. FLEX-Judge trained these post-processed outputs demonstrates improved generalization to diverse evaluation formats, including a single-score grading and a batch-level ranking (i.e., where the number of response options exceeds three) as used in [11]. Furthermore, we find that the post- protesso variant is more robust to varied instruction styles, particularly when prompts emphasize different evaluation criteria across input pairs. The detailed results are provided in Section 3.2 (Table 1).
Training Multimodal Judge. Next, we use the reasoning seed dataset to fine-tune an MLLM, such as Qwen2.5-VL. [5] and Qwen2.5-Omni [79]. Despite being originally trained to handle both text and other modalities such as visual or audio inputs, the MLLM is one-tuned exclusively on our textual reasoning annotations. We call this fine-tuned evaluator FLEX-Judge. Consequently, the structured and explicit reasoning provided by JudgeLRM enables FLex-Judge to learn how to systematically evaluate and justify preferences, significantly improving zero-shot transfer capabilities.
Multimodal Inference: Reasoning-Guidet Preference Judgments. At inference time, our model performs multimodal evaluation without additional fine- tinettnnig or moodality-speciff annotations (i.e., training-free We visualize an example output from FLEX-Judge in Figure 3, demonstrating that the textual reasoning behind judgments has transferred across modalities. Furthermore, unlike most other evaluators, our FLEX--udge takes advantage of inference scaling to improve judgment performance by leveraging multiple reasoning paths, such as majority voting [69] or budget forcing [50], as addressed in Section 5.

---

User: Generate an inage "a cute dog is playing a ball".
<think
Assistant inage quite accurate
depiction dog playing
[...]
Aliganent with Prorpt: The dog is indeed playing with ball, which is the core action described. The dog's
the ball, and the autdoor setting (grass and sunlight) all
contribute clear depiction
scene described in the prongt.
Naturalness: The lighting and the the dog is interacting with the ball are also quito realistic, naking inage natural and believable.
Assistant
aligns with the pronpt, but it might slightly less accurate in terns of the context provided [...]
contrast, Assistant
inage
Naturalness: lighting the background (Indoors with furniture) night be
less realistic or natural in the context of a dog playing autside.
Visual Appealing: The
also visually appealing, but the indoor setting night be first image.
Assistant B
seen as visually engaging compared
outdoor setting
Judge: The first image night be considerad more accurate, as it aligns morg clasely, with an outdoor, sotting, which night be more typical for playing with ball ball.g/thinko /tthinko.cansserr
cansser>> [A [
Figure 3: Reasoning process of FLEX-Judge on the text-image alignment task (GenAl-Bench [35]). Additional qualitative examples are found in Appendix D.
3 Experimental Evaluation of FLEX-Judge
We comprehensively evaluate FLEX-Judge across diverse modalities, including images, videos, and audio, demonstrating its generalization capability and competitive performance against state-of-the-ae judge models. Notably, it matches closed-source commercial APIs on vision tasks and outperforms all training-free evaluators in audio understanding. These strong results suggest that FLEX-Judge can be used with confidence in modalities even when expert judge models not applicable, which further explore in Section 4.
3.1 Experimental Setup
Implementation. Based on KK-sized training dataset introduced in Section 2.2, we develop FLEX- Omni-7B (image, video, and audio) and FLEX-VL-7B (image and video) from Qwen2.5-Ommt 7B [79] and Qwen2.5-VL-7BB [5], respectively. We compare them against both commercial models with costly API usage [1, 62] and open-source models that require either extensive training data [32, 76] or significantly more parameters [40, 67]. For more implementation details, refer to Appendix
Evaluation Protocol for Judge Models. To evaluate the quality of judge models, we measure how closely their assessments align with human annotations (or human-verified model evaluations [14, 38]). The specific metric depends on the evaluation format: we measure (1) Pearson correlation [33] for single-score grading tasks, (2) accuracy for pairwise (A/B) comparisons, and (3) normalized Levenshtein distance [34] for batch-level rankings (e.g., ABCD). Detailed descriptions for each benchmark and the evaluation prompts are provided in Appendíx B.3.
Evaluation Benchmarks. We evaluate image understanding capabilities of FLEX-Judge using the MLLM-as-a-Judge benchmark [11], which comprises 14 diverse langua-anguuae tasks including captioning and website browsing, and the VL-RewardBench benchmark [38], which focuses complex reasoning tasks like visual hallucination detection. For image generation assessment, we use MJ-Bench [14] assess image quality and alignment. We use GenAl-Bench [35] for evaluating video generation and image editing. For audio understanding, following the prior work [68], we conduct speech quality assessment task, specifically performing mean opinion score (MOS) prediction by using the NISQA [49], BVCC [18], and SOMOS [46] datasets, and speaker similarity score (SS) prediction with VoxSim [2] dataset for assessing speaker similarity score (SS). For additional results of Multimodal RewardBench [82] and JuyteAnything [54] benchmarks, refer to Appendix C.I. We also provide the language-on) assessment results Appendix C.4.
3.2 Comparison with State-of-the-arts
Image Understanding. In Table 1, model judgments are compared with human ratings in MLLM- as-a-Judge benchmark. We compare FLex-Judge with the following model groups: commercial
Disclaimer: We completed all experiments on May 9h, but both VL- VL-RewwrdBench and MJ-Bendl were later modified on May 13 and t6 respectively.
5

---

Table 1: The overall performance of different MLLMs in judging, compared with human annotations on different datasets. We sample all judgments three times and average them to mitigate the bias.
w. and W.O. tie represents tie and non-tie situations, respectively, by following [11]. O reported results from LLaVA-Critic [76]. + results from the original paper of a-a---as-a [1]]. t: Prometheus- Siteethhes--iiioo---r 13B [32] only trained under the Score setting, incapable of following Pair/Batch instructions. Training-foi free (TF) models have not been trained on multimodal evaluation data.
Model
coco
Diff.
Graphics
Math
WIT
Chart
ViSTT
CC.M
M2W
SriQA
MM-Vei
Arc.
GPT 4yo
0.410
0.414
0,361
0.419
0
0.500
0.457
0.585
0.554
0.266
0.267
0,315
0.472
0.367
0.424
Semini-1,-Pro-iiiion
0.400
0.228
0.343
0.073
0.360
0,207
0,400
0.426
0.467
0.590
0.471
0.553
0.254
0.636
0.563
0.254
0.075
0.600
0.150
0.053
0.390
8.366
0.42"
0.106
"0112""
0.1E
0.514
0,205
0.234
6.256
0.228
"0000
".114
"0.365"
0.137
0.513
E1aVA-Crie:768
0.382
0.450
0.163
0.316
0.356
0.378
0.179
0.421
0.346
0.301
0.269
0.395
0.314
6-3VA-16-34B*
0.265
0.251
-0.012
0.262
0.238
0.258
0.351
0.318
0.19%
0,109
0.002
0.206
0.025
0.265
0.184
Qwen2.5 Ornii
0.150
0.017
0.045
0.087
0.003
0.049
0.060
0.010
0.136
0,073
0.136
0.097
0.145
0.072
Qwen2.5-VL--2
0.294
0.247
40,000
-1104]
0.005
0.170
0.056
0.011
0.328
0.178
0.255
0.311
0.327
0.103
0.165
FLEX- Qmai-7B
0.324
0.126
0.371
0.316
0.429
0.118
0.479
0.275
0.375
0.351
0.309
0.252
0.306
FLEX VL. 7B
0,363
0.215
0.114
0.338
0.448
0.423
0.125
0.471
0.452
0.189
0.357
0,350
0.407
0.343
0.333
GPT-4V
0.539
0.634
0.668
0.652
0,459
0.495
0.536
0,309
0.591
0.548
0,546
0,589
0.620
0.517
0.538
Semiia-1.0-Pr-Viiiioe
0.616
0.757
0.650
0.466
0.664
0.605
0.500
0.660
0.500
0.370
0.262
0.190
0.312
0.509
Pair (1)
Genini 2,5 Pro
0.540
0.606
0.753
0.618
0,455
0.532
0.508
0,570
0.604
0.565
0,660
0,365
0.694
0.527
0.556
(LaVA-Cme-77
6.591
0.007
0.387
0,432
0.564
0.33
0.561
0,350
0.586
L.aVVA-I 6-348*
0.499
0.600
0.570
0.300
0.374
0.551
0.545
0224
0. 0.508
0.902
0.513
0.634
0.524
0.499
0.460
Qwen2. 5-0mm-777 7D
0,482
0.479
0,733
0.422
0.385
0.433
0,411
0,5M4
0.650
0,501
0.508
0,395
0:317
0.462
0,471
Qwen2. V7
0.446
0.474
0.507
0.326
0.397
0,383
0.366
0.364
0,661
0,483
0.358
0.442
0.494
0.430
0.423
FLEX OB
0.496
0.647
0.713
0.490
11429
0.485
0.445
0.432
0.512
0.579
0.593
0.584
0.656
0.524
0.532
FLEX VB
0,538
0.685
0.653
0.532
0.446
0.534
0.458
0,356
0.586
0.586
0.5M5
0.391
0.636
0.500
GPP--49
0.729
0.722
0.884
0.853
0.665
0.661
0.760
0.405
0.785
0.707
0.607
0.619
0.741
0.654
0.717
Pair (i+(t)
Sxnini-l.&-Po-Viiiee
0,717
0.770
0.67%
0.654
0,622
0.35%
0.265
G.5.ial2.55.Po Pro
0.609
0.677
0.783
0.768
0.518
0.611
0,604
0.513
0,724
0.649
0.740
0.645
0.709
0.705
0.668
ElaVAACiio-78
0,771
0.714
0.755
0.758
0.996
0.688
0.680
01463
0.727
0.742
0.002
0.658
0.715
0.655
0.689
LLaVA-16-34B
0.607
0.324
0,855
0.412
0.547
0.758
0
0,503
0.564
0.713
0.679
0.694
0.648
Qwen2.5 Omni 7B
0.559
0.518
0.755
0.478
0,407
0.456
0.461
0.443
0.557
0.557
0.545
0.617
0.525
0.459
0.526
Qwen2. 7B
0,510
0.288
0:368
0.334
0,511
0.425
FLEX Omni 7B
0.648
0.720
0.748
0.621
0.560
0.566
0.534
0.609
0.711
0.679
0
0.706
0.655
0.674
0.650
PLEX VB
0.689
0.763
0,685
0.670
0,580
0.613
0.542
0.548
0,002
0.686
0,066
0.684
0.655
0.683
GPT-4V
0,318
0.353
0.070
0.385
0.348
0.319
0.290
0.347
0,300
0,402
0.597
0,452
0:453
0:411
0.361
Qemini-1_--P--viiiie Visinn
0.267
0.239
01773
0.462
0.430
0.344
0.530
0.426
0.357
0.613
0.412
0:467
0.529
0.422
Gemini-
0.250
0.595
0.599
0,488
0.557
0,501
0,512
Butch
E1aVA-Crre=78
0.541
0.455
0,525
0:62
0.576
0.599
0.603
0.580
0.481
0.592
0.588
0.627
0.618
0.515
0.565
LLaVA-1 6-34g*
0.500
0.561
0.575
0.552
0.437
0.500
0.501
Qnen2.--Ommii77
0.545
0.518
0.835
0.591
0.589
0.602
0.586
0.545
0.582
0.538
0.594
0.576
0.574
0.581
0.576
Qwen2.5. V-L-78
0.562
0.450
0,913
0.630
0.407
0.582
0.631
0.570
0.569
0,519
0.639
0,703
0.558
0.572
0.585
FLEX- -xnni-8
6.392
0.328
0.404
0.452
0,439
0.462
0.342
0.450
0,442
0.484
0.515
0.362
FLEX VL 7B
0.419
0.325
0.414
0.462
0.437
0.412
0.477
0.445
0.398
0.342
0.420
0.687
0.471
0.405
0.426
Table 2: Comparison of MLLM evaluator performances on VL-RewardBench (Left) and MJ-Bench (Right). 9 results from the original works [14, 38]. Best and second best results.
Overall
Macro
Model
TF!
Alignment w. Tie
Safety Tie
Artifact
GPI-460
49.1
67.6
70.5
65.5
62.4
w.o. Tic
W.o. Tie
W: Tie w.o. Tic
Gemmn-1 P
Grmim-25--P 2.5-ho
44.5
12.5
49.1
64..
53.1
62.55
45.4
48.31
18-4
GPT-40
61.5
35,3
100.0
97.6
98.7
35.6
32.2
41.1
20.1
59.0
57.1
44.5
20,6
36.5
45.2
Clande 3 Opus
Gemini Ulin
67.2
57.1
55.9
69.0
13.1
13.4
95.
78.9
55.7
11.9
96.7
70.4
Quesd 5.Qmni-7B
28.7
PacScoena19
x
59.8
64.6
37.2
42.2
89.6
Piatnd-138
59.9
HPS-v2.19
X
47.3
70.1
18.8
41.3
67.3
93.5
430
ImageRewand
x
50.9
64.7
25,9
38.7
63.5
81.8
Mihns-718*
LLaVA-1 6-13
27.9
45.6
62.5
NVLM-D-222
44.3
Prometheus- Vision- 1389
x
11.8
64.3
28.6
71.4
67.9
47.4
38.5
$3.8
43.7
46.6
Fuxx Orne 78
FLEX VL
361T
47.01
42.73
61.08
62.87
48.10
50.27
51.79
FLEX-VL-78
FLEX-Ommi-TB
58.16
60.84
59.13
62:46
47
57,51
65.3 21
66.88
62.32
75,80
99.05
91.66
models including Gemini and GPT-4V (state-of-the-ee dlosed-source models), and latest open-source baselines including Prometheus-Vision [32] and LLLaVA-Critie-77 [76], trained on large-scale (> (>100K) curated MILLM-as---jul datasets.
We highlight that prior open-source judges are limited to specific evaluation formats, e.g., LLa Critic-7B faltering in batch-level ranking, making them less utilitarian. In contrast, our models are capable of handling diverse evaluation criteria, matching or outperforming much larger models like Gemini, GPT-4V, and ALLVVA-1.6-44P Notably, LLa VA-VVi-ii trained on 1 113K lision-language understanding data, underperforms both FLEX-Ommi-7B and FLEX- VL-7B on VL-RewardBench (see Table 2; left). These results mark the simplicity and efficiency of our approach, learning from IK text reasoning annotations without any codallty-speciffe supervision training-free; TF).
6

---

Image Generation. Table 2 (right) presents the judge performance of generated images in MJ- Bench, whether they are well-aligned with a prompt, safe, or have artifacts. FLEX-Judge models achieve higher scores than some commercial models (e.g., Claude 3 Opus) and all training-required judge models (PickScore-vl [30], HPS-v2.1 [75], and ImageReward [78]) by a large margin. While the baselines have high variance according to tasks, especially poor in safety check, our models perform highly consistent. The superiority of FLEX-Judge in image generation assessment is further demonstrated in Table 3, where FLEX-VL-7B with majority voting evaluation [69] outperforms GPT-4o and Gemini- 1.5-Pr.
Image Editing & Video Generation. We fur- ther report judge performance on GenAl-Bench, including image generation, image editing, and video generation tasks, in Table 3. Notably, our FLEX-VL-7B achieves the highest over- all performance, as well as strong results in both image editing and video generation tasks, when inference-time scaling is applied [69]. A detailed discussion of inference-time scal- ing is provided in Section 5. While FLEX- Omni-7B shows lower performance initially, it also benefits significantly from inference- time scaling-unlike its non-reasoning baseline, Qwen2.5-VL-7B.
Table 3: Comparison of MLLM evaluator perfor- mance on GenAI-Bench. c; results from the origi- nal work [35].
Model
Imuge Gen.
Image Edit.
Video Gen.
Overall
GPT-46
45.59
53.54
49.20
Gemini-1,5-e P
44.67
55.93
46.21
48.94
Gemini-2.5.PP
47.55
65.51
50.33
54.46
LLaVA
37.00
26. 12.12
30.40
31.17
LLaAA-NeXT
22.65
25.35
21.70
23.23
Owen2.5-Ommi-7B
34.87
31.88
38:45
35.07
Qwen2.5. VI,
31.93
38.63
37.61
36.06
FLFX-Omai-78
38.15
46.73
37.10
40.66
Majarity Voting [69]
41.67
FLEX-VL VL-X-VL-7B
43.32
47.41
44.78
45.17
Mujarity Voting [69]
46.34
54.19
47.34
49.29
Audio Understanding. Table 4 demonstrates the performance of our approach in speech quality evaluation, where the linear correlation coefficient (LCC) and Spearman's 's rank correlation coefficient (SRCC) are calculated to assess the agreement between the model's 's predicted scores and the human- annotated MOS and SS values. As observed in [10, 68, 86], existing open-source source audio LLMs like Qwen2-Audio struggle in quality assessment without specific fine-tuning, often exhibiting widespread hallucinations. While audio LLMs are primarily pretrained on semantics-relates tasks like audio question answering (AQA) or captioning (AAC), they are less familiar with such qualitative evaluations, which are challenging due to different MOS standards depending on the dataset [68]. Still, -Onx--Oni outperforms all training- free judges and even Gemini-2.0-Fii 0lasshhh
Table 4: Audio MOS/SS prediction results on the test sets of the NISQA [49], BVCC [18], SO- MOS [46], and VoxSim [2] datasets. System-level results are computed by averaging the utterance- level results within each text-to- text-to-speech system (not provided for NISQA due to the absence of system labels). 9 task-specific fine-tuning results from [68].
NISQA (MOS)
BVCC(MOS)
SOMOS (MOS)
VoxSim (SS)
Model
TF?
utterance-level
utterance-level
system-level
utterance-level
system-level
utterance level
LCC
SRCC
LCC
SRCC
LCC
SRCC
1.CC
SRCC
L.CC
SRCC
LCC
SRCC
Gemini-2 2.d..200.11 2.0.-lashh Gemini-2.5-Pro
0.408
0.415
0.044
0.038
0.092
0.096
0,119
0.129
0.256
0.317
0.451
0.457
0.567
0,586
0.261
0,266
0.495
0.504
0.195
0.436
0.661
0.667
Single-task S
0.894
0.887
0,899
0.896
0.939
0.936
0.687
0.681
0.911
0.917
0.835
0.836
Qwen2-Audion A
0.768
0.780
0.678
0.800
0.797
0,583
0.572
0.850
0.873
0.415
0.505
Qwen2. 7B
0.210
0.243
0.056
0.055
.0.075
.0.079
0,074
0.097
0.136
0.159
0.211
0.204
Qwen2-Audio
0.004
-0.002
-0.067
-0.062
0.093
0.120
-0.016
-0.013
0.058
0.077
0.042
0.044
PLeX-Ommi-7H
0.545
0.590
0.081
0.067
0.144
0.128
0.150
0.138
0.323
0.325
0.271
0.268
4 Broader Impact: Case Study on Molecule Evaluator
Trained only textual reasoning data without any -specffc- supervision, FLEX-Judge demonstrates strong generalization across image, video, and audio tasks. This suggests that FLEX- Judge can serve as a practical solution in domains where constructing modality speciicc reasoning datasets is infeasible or prohibitively expensive.
A particularly compelling use case arises in scientific domains such as molecular modeling [3,58 58, 83], where no existing reward model or judge is available due to limited data and domain complexity. To test our hypothesis, we build molecular judge using MoI-LLaMA [27], which is based on LLaMA3.1-88 [23] and understands both 2D and 3D molecule representations. We fine-tune its LLM backbone a reasoning-awan judge, while keeping its modality-aware modules (LoRA
7

---

Model
Default
CoI
w/ Task Info.
80
Llama3, I-Stuttttucctt
56.51
46.19
1A400 Aoc (N1
po0ed (H4)
78
Mol-Instractions [22]
55.91
33.50
70.47
60-
3D-MoLM [39]
46.93
50.00
64.86
LLaMo (52)
Mol-LLaMA (27)
63.55
64.37
72.48
20
66-
Gready Becading
Best-of-N (N=16)
PLEX-Mol-TT MoM.---MMMMA judge sonrtng
68. 08.85
69.83
153
64
Tork
Preference Optimization
76.41
75.92
80.10
6.0
6.5
1.0
85
9.0
judge Soore
Figure 4: (Left) Accuracy (%) trends on the parallel artificial membrane permeability assay (PAMPA: [64]) task with different judgment scores. (Middle) Accuracy trends the number of sampled responses in best-of-N sampling. (Right) Performance comparison with reward-guider Mol-LLaMA. We report accuracy with prompt styles of default, CoT, and task information, described in [27].
Reason.
(tore
ML.LM-as-a-Judh Judge
(i)
VI-Reward Overall
MJ-Bench
Safety (f)
7.5
5.0
NSW
VI.7B
0.290
0.538
0,521
48.60
39.28
57.51
28.42
2.5
SeiAA, COCO
0.0
Omni-7B
0.224
0.486
0.532
39.92
27.01
-2.5
Grophics
MMM Vet
VtiT
CC-SM
400
450
500
$50
600
650
Reasoning Length
Figure 5: (Left) Performance comparison of FLEX-Judge with and without reasoning. (Right) Relationship between the average reasoning length of FLEX-VL-77 and the accuracy gain from reasoning over non-reasoning evaluation across subcategories in MLLM-as-a-Judge (Pair w. Tie).
adapters [26], molecular encoders, and Q-Formers [37]) unchanged. We refer to this model as FLEX-MoI-LLaMA.
We evaluate LLEX-MMo-LLLMMA via two approaches. (1) Best-of-N selector: FLEX- MLEX-Mol-LLaMA guides best-of-N N sampling from Mol-LLaMA outputs, a method correlated with judge model performance [38]. (2) Direct preference optimization (DPO): Given prompt Mol-LLaMA generates two responses Yı and Y2, and LoLX--MM judge selects the preferred (y..) over the less preferred (yr) by score evaluation, forming triplets (x, Yw: Yi) [51]. These are then used to fine- tune Mol-LLaMA via DPO [56]. Strong downstream performance indicates that FLEX-MOI-LLaMA effectively curates high-quality preference data.
to score the Mol-LLaMA's understanding of chemical property (permeability)) following Kim et al. [27]. Our key findings include:
Best-of-N Selector. We use
Score-Accuracy Correlation: The judge scores strongly correlate with actual task performance (Figure 4; left), demonstrating that MEEx-Mol-LLMMM captures meaningful evaluation signals from molecular content and effectively detects high-quality responses.
Best-of-N Sampling: Selecting the highest-scoring ones among sampled responses yields significant accuracy improvement (Figure 4; middle, up to 77.49% when N 16), indicating that our judge model provides reliable preference signals.
Reward-Guided Training via DPO. In this sense, we further explore using FLEX-Mol-LLLMM judge reward source for DPO. We collect 4K samples FLEX-MOl-LLaMa preferences Yw. of the training set set-covering chemical structures, properties, and biological features-ant and further fine-tune Mol-LLaMA. As shown in Figure 4 (right), DPO reaches up 80.10% surpassing the previous state-of-the-art. This result confirms that our judge model can also act as an effective reward model in underexplored modalities, enabling scalable preference optimization. For the LLEX-Mol-LLaMAN reasoning and judgment results, refer to Appendix D.3.
5 Analysis
Effect of Reasoning. To investigate the impact of reasoning in FLEX--Juge, compare it against a non-reasoning variant where the training segments <think> and <answer> are reversed that is. the model is trained to generate the final answer first, followed by the reasoning. This answer-

---

Majerity Voting MV RewardBonch)
Budget Forcing RewardBench)
Majerity Veting (MLLU judge)
Bunget Forring (ML (Mjjudge)
62-
1r (6)
1op Omni-78 78 190-1-70
04
11
15
# SSnnpled Restoring Parh
Samoled Reaswring Fech
+Smpp ed Reaconng Pac
Sumpped Reweninng Path
(a) Majority Voting (Pair) (b) Budget Forcing (Pair) (c) Majority Voting (Scorc) (d) Budget Forcing (Scorc)
Figure 6: Inference-tinn tenrence-timeeeeee scaling of FLEX-(VL/Omni) our model supports parallel inference-tine scaling via majority voting, showing consistent gains. Surprisingly, budget forcing also offers minor but consistent improvements in score-based evaluations.
Unlike prior MLLM evaluators [38],
then-reason format is commonly used open-source datasets [76, 90]. As shown in Figure 5 (lef), our proposed FLEX-Judge consistently outperforms this variant, demonstrating that reasoning-first evaluation leads to more robust and generalizable performance. Furthermore, in Figure 5 (right), we find that accuracy gains on MLLM-as-a-Judde correlate with the average reasoning length produced by FLEX-VL VL- This suggests that more difficult tasks elicit deeper reasoning, highlighting the importance of reasoning capabilities in accurate evaluation. Notably, the M2W dataset [19], which involves complex website browsing scenarios and thus requires fnnegrained reasoning paths, stands out with larger performance gain than the general trend.
Inference-time Scaling Works in FLEX-Judge. We examine inference- scaling techniques, including majority voting [69] and self-refinement through budget forcing [44, 50], to enhance the reasoning ability of our FLEX-Judge. In the VL-RewardBench (Reasoning) pairwise comparison, applying majority voting steadily improves performance, which is in stark contrast to prior work [38] that reported performance drops under inference-time scaling (Figure 6a). For budget foreing, we inject the keyword "Wait", shown to be effective in Muennighoff et al. [50]. Although performance drops after the first trial, it consistently improves in subsequent trials and nearly recovers to its original level (Figure 6b). In score-based evaluation, both methods show consistent gains (6c and 6d). These results highlight that, unlike existing MLLM-based judges, our evaluator benefits from increased inference- time computation, especially in reasoning- training. our FLEx-Judge produces more diverse reasoning paths than prior non-reasoning evaluators, allowing inference-time scaling methods to be more effective.
tasks. Thanks to its reasoning based
Data Quality vs. Modality Alignment. To further assess the effectiveness of our cost- efficient approach, we compare FLEX-VL-7B with variant trained on image-text evaluation pairs. Using the RLHF-V dataset [85]. we curated IK image-text pairs with reasoning- guided evaluation where FL.EX-VL-7BB judged the chosen response as getter-withour rely- ing on explicit EopT-to-annottted scores, po-
Table 5: Comparison of variants trained on poten- tially lower-quality image-text data. VL and HQ de- note vision- language training and high-quality data.
MILLMMassaaJudee Tie
VL-Reward Overall
GenAl
VL HQ
Score (1)
Video (t)
0.274
0.198
43.84
43.41
VI--BB
x
0,332
0.538
48.60
44.78
tentially resulting in weaker quality than our original dataset (A). As shown in Table 5, these variants consistently underperform FLEX-Judge, emphasizing that dataset quality outweighs modality- awareness.
6 Related Work
LLMs have increasingly been used as proxy evaluators in place of costly human annotators, a direction formalized under the LLM-as-a-Judge paradigm [8, 89], which has demonstrated strong alignment with human preferences [28, 60, 61, 63]. These model-based judgments can be leveraged in various downstream techniques such as best-of-N selection [24] and sifference-drivenr optimization using DPO [56, 65] or listwise reranking [43, 55]. However, most LLM-as-a-Judee approaches remain limited to text-only domains [36], while their multimodal variants (e.g.. ImageReward [78]) require large-scale, modality-spedd
annotations [6, 32, 76]. Recent work has highlighted the advantages of reasoning guided supervision, training models with explanations or chain-of-thought rationales [7, 71, 72], to improve judgment quality and generalization [12]. Yet, collecting high- quality multimodal rationales is costly and especially difficult for underexplored modalities [11, 30,

---

88]. limiting applicability to new domains. Our work addresses this gap by showing that textual reasoning alone can effectively train multimodal judge models generalize across modalities, including tasks like molecular evaluation where no modality-peecifi preference data exists [27]. Additional related works are discussed in Appendix A.
7 Conclusion
In this work, we introduce FLEX--udde, reasoning-gui guided multimodal evaluator trained solely on textual preference explanations. By leveraging structured reasoning from pretrained model, we have shown that FLEX-Judge generalizes to diverse modalities including image, video, audio, and molecules, without modality-specifff supervision. Despite using far fewer annotations, it matches or outperforms state-of-the-ard commercial APIs and open-source evaluators. These results highlight reasoning supervision scalable, cost-effective alternative for training general-purpos judge models in complex multimodal settings.
Acknowledgements
SK, SC and SY were supported by Institute of Information & communications Technology Planning & Evaluation (IIT)) grant funded by the Korea government (MSIT) (No. RS-2019-11190075
Artificial Intelligence Graduate School Program (KAIST); 10%) and the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2022-0-00871, Development of AI Autonomy and Knowledge Enhancement for AI Agent Collaboration; 90%). We would like to thank Yeongjun Kim from the Georgia Institute of Technology for helpful discussions on the molecule evaluator.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0874 2023.
[2] Junseok Ahn, Youkyum Kim, Yeunju Choi, Doyeop Kwak, Ji-Hoon Kim, Seongkyu Mun, and Joon Son Chung. Voxsim: A perceptual voice similarity dataset. arXiv preprint arXiv:2407.18500 2024.
[3] Microsoft Research Al4Science and Microsoft Azure Quantum. The impact of large lan- guage models scientific discovery: preliminary study using gpt-4. arXiv preprint arXiv:2311.07361, 2023.
[4] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
(5) Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang. Jun Tang, al. Qwen2.5-vl technical report. arXiv preprint 2322233502..3923 2025.
[6] Tianyi Bai, Hao Liang, Binwang Wan, Yanran Xu, Xi Li, Shiyu Li, Ling Yang, Bozhou Li, Yifan Wang, Bin Cui, et al. A survey of multimodal large language model from data-centric perspective. arXiv preprint arXiv:2405. 16640, 2024.
[7] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05882 2022.
[8] Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, et al. Llms instead of human judges? large scale empirical study across 20 nlp evaluation tasks. arXiv preprint arXiv:2406. 18403, 2024.
10

---

[9] Will Cai, Tianneng Shi, Xuandong Zhao, and Dawn Song. Are you getting what you pay for? auditing model substitution in Ilm apis. arXiv preprint 2XXv:2554.04715, 2025.
[10] Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, Chao-Han Huck Yang, and Eng Siong Chng. Audio large language models can be descriptive speech quality evaluators. arXiv preprint arXiv:2501.1720 2025.
[11] Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Milm-as-a-judge: Assessing multimodal Ilm-as- a-judge with rision-languann benchmark. In Forty-first International Conference on Machine Learning. 2024.
[12] Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. Judgeirm: Large reasoning models as a judge. arXiv preprint arXiv:2504.00050, 2025.
[13] Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. Rm-rl: Reward modeling as reasoning. arXiv preprint arXiv:2505.0m 02387, 2025.
[14] Zhaorun Chen, Yichao Du, Zichen Wen, Yiyang Zhou, Chenhang Cui, Zhenzhen Weng, Haoqin Tu, Chaoqi Wang, Zhengwei Tong, Qinglan Huang, et al. Mj-bench: Is your multimodal reward model really a good judge for text-to-image generation? arXiv preprint EXXiv:440700488 2024.
[15] Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang. Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:231 1.07919, 2023.
[16] Yunfei Chu, Jin Xu, Qian Yang. Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen2-audio technical report. arXiv preprint arXiv:2407. 10759, 2024.
[17] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph. Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instructionfiinttu language models. Journal of Machine Learning Research, 25(70): 2024.
[18] Erica Cooper and Junichi Yamagishi. How do voices from past speech synthesis challenges compare today? arXiv preprint wtt.::2015.02373:: 2021.
[19] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards generalist agent for the web. In Thirly-eeventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://opeareever.. wwtp:eeew..eeeooomm/ddtddoaaooo
[20] Ameet Deshpande, Partha Talukdar, and Karthik Narasimhan. When is BERT multilingual? isolating crucial ingredients for cross-lingual transfer. In Marine Carpuat, Marie-Catherin de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3610-3623, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/1 acl aellanohlogy.oo wwwthhoogg..org222222 naacl- 264/.
URL https://
[21] Daniel Deutsch, George Foster, and Markus Freitag. Ties matter: Meta-evaluating modern metrics with pairwise accuracy and tie calibration. arXiv preprint 23X%: 2023.
[22] Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. In The Twelfth International Conference on Learning Representations, 2024.
[23] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.

---

[24] Lin Gui, Cristina Garbacea, and Victor Veitch. BoNBon alignment for large language models and the sweetness of best-of-n sampling. In The Thirty-eighth Annual Conference on Neural Informa- lion Processing Systems, 2024. URL. htt.:ttts//pppereiieeeee httprriwwwwnnteeeeeeeeeeeeeeeeee bet.fformm? wwt.ee/orrumm.ddtaSSSSWW r-x5
[25] Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-LLM: Injecting the 3d world into large language models. In Thirty-sevent Conference on Neural Information Processing Systems, 2023. URL https:/ https://openrevaa http///ppenre....e/ d
[26] Edward Hu, yelong shen, Phillip Wallis. Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL pettps://opereevieee sett/orrumid=nZZeaWle id=nZeVKeeFYf9.
[27] Dongki Kim, Wonbin Lee, and Sung Ju Hwang. Mol-Ilama: Towards general understanding of molecules in large molecular language model. arXiv preprint arXiv:2502. 13449, 2025.
[28] Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, et al. Prometheus: Inducing fine-grained evaluation capability in language models. In The Twelfth International Conference Learning Representations, 2023.
[29] Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Prometheus 2: An open source language model specialized in evaluating other language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4334-4353, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.1
URL https: / /t////aaaaaaaltoooo//aoo org/2024.
248/
[30] Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for exxt-toimmae generation. Advances in Neural Information Processing Systems, 36:36652-36663. 2023.
[31] Andrew K Lampinen, Nicholas Roy, Ishita Dasgupta, Stephanic CY Chan, Allison Tam, James Mcclelland, Chen Yan, Adam Santoro, Neil Rabinowitz, Jane Wang, et al. Tell me why! explanations support learning relational and causal structure. In International Conference Machine Learning, pages 1168-111 1188-. PMLR, 2022.
[32] Seongyun Lee, Seungone Kim, Sue Park, Geewook Kim, and Minjoon Seo. Prometheus-vision Vision-language model judge for fine-grained evaluation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 11286-113 1315, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/v11200 Feee4.fiiinngaaoe acl.672. URL https: http:////aaaaallaa////// org/ 2024. feeffffiniingss.. acl. 672/.
[33] Joseph Lee Rodgers and W Alan Nicewander. Thirteen ways to look at the correlation coefficient. The American Statistician, 42(1):59 5956666, 1988.
[34] Vladimir I Levenshtein al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707-710. Soviet Union, 1966.
[35] Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengehuan Zhang, Graham Neubig, al. Genai-bench: Evaluating and improving compositional text-to-visual generation. arXiv preprint 13374:2406.13743, 2024.
[36] Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Llms-as-judeee a comprehensive survey on llmebased evaluation methods. arXiv preprint arXiv:2412.05579 2024.
[37] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett,
12

---

editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730-19742. PMLR, 23-29 Jul 2023. URL. https::t/////tt htttt::://rroeeedngg mlr. www.ppress//ee. http///22/111/// html.
[38] Lei Li, Yuancheng Wei, Zhihui Xie, Xuqing Yang, Yifan Song, Peiyi Wang. Chenxin An, Tianyu Liu, Sujian Li, Bill Yuchen Lin, et al. Vlrewardbench: A challenging benchmark for vision-language generative reward models. arXiv preprini arXiv:2411.1745/ 2024.
[39] Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. Towards 3d molecule-text interpretation in language models. In The Twelfth International Conference on Learning Representations, 2024.
[40] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296-26306, 2024.
[41] Yuyan Liu, Sirui Ding, Sheng Zhou, Wenqi Fan, and Qiaoyu Tan. Moleculargpt: Open large lan- guage model (IIm) for few-shot molecular property prediction. arXiv preprint arXiv:2406. 12950, 2024.
[42] Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint PrXiv:2504.02495, 2025.
[43] Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. Zero-shot listwise document reranking with a large language model. arXiv preprint arXiv:2305.0211 2023.
[44] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 2023.
[45] Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410.12832 2024.
[46] Georgia Maniati, Alexandra Vioni, Nikolaos Ellinas, Karolos Nikitaras, Konstamtinos Klapsas, June Sig Sung, Gunu Jho, Aimilios Chalamandaris, and Pirros Tsiakoulis. Somos: The samsung open mos dataset for the evaluation of neural text-to-speech synthesis. arXiv preprint arXiv:2204.03440. 2022.
[47] Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok- A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195-3204, 2019.
[48] Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-frec reward. Advances in Neural Information Processing Systems, 37:124198-124235. 2024.
[49] Gabriel Mittag, Babak Naderi, Assmaa Chehadi, and Sebastian Möller. Nisqa: A deep enn-self- attention model for multidimensional speech quality prediction with crowdsourced datasets. arXiv preprint arXiv:2 2021.
[50] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501. 1...29931..99,,, 2025.
[51] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730--77444, 2022.
[52] Jinyoung Park, Minseong Bae, Dohwan Ko, and Hyunwoo J Kim. Llamo: Large language model-based molecular graph assistant. In The Tiirth- Annual Conference on Neural Information Processing Systems, 2024.
13

---

[53] Telmo Pires, Eva Schlinger, and Dan Garrette. How multilingual is multilingual BERT? In Anna Korhonen, David Traum, and Lluís Märquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996-5001, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 18653/ URL. https:/ / ac aclanthology org/P19-140 1493/.
[54] Shu Pu, Yaochen Wang, Dongping Chen. Yuhang Chen, Guohao Wang, Qi Qin, Zhongyi Zhang. Zhiyuan Zhang, Zetong Zhou, Shuang Gong, et al. Judge anything: Mllm as a judge across any modality. arXiv preprint arXiv:2503. /7489, 2025.
[55] Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu. Jialu Liu, Donald Metzler, et al. Large language models are effective text rankers with pairwise ranking prompting, In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1504-1588, 2024.
[56] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36:53728-53741, 2023.
[57] Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv: / 1999000061,, 2019.
[58] Shaghayegh Sadeghi, Alan Bui, Ali Forooghi, Jianguo Lu, and Alioune Ngom. Can large language models understand molecules? BMC bioinformatics, 25(1):225, 2024.
[59] Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317-8326, 2019.
[60] Hwanjun Song, Taewon Yun, Yuho Lee, Jihwan Oh, Gihun Lee, Jason Cai, and Hang Su. Learning to summarize from LLM -eM-reeeretted feedback. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 255-857, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189-5 URL https: hatpa////alaaaaol////// org/ 2025. naacl-long..... 38/.
[61] Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances neural information processing systems, 33:3008-3022 2020.
[62] Gemini Team, Rohan Anil, Sebastian Borgeaud, lean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 2023.
[63] Weixi Tong and Tianyi Zhang. Codejudge: Evaluating code generation with large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20032-20051, 2024.
[64] Alejandro Velez-Arce, Michelle M Li, Wenhao Gao, Xiang Lin, Kexin Huang, Tianfan Fu, Bradley L Pentelute, Manolis Kellis, and Marinka Zitnik. Signals in the cells: multimodal and contextualized machine learning foundations for therapeutics. bioRxiv, 2024.
[65] Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusson model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 8228888888888 883888228,0888 2024.
[66] Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17900 17926, 2023.
14

---

[67] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, al. Qwen2-vl: vl: Enhancing rision-langual model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[68] Siyin Wang, Wenyi Yu, Yudong Yang, Changli Tang, Yixuan Li, Jimin Zhuang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, et al. Enabling auditory large language models for automatic speech quality evaluation. In ICASSP 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE, 2025.
[69] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL https:// http::///ppeeevieeeeee www/ffounnii fioummiid=iiii 1ediidd1-11pMM.M..
[70] Yidong Wang, Zhuohao Yu, Wenjin Yao, Zhengran Zeng, Linyi Yang. Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. PandaLM: An automatic evaluation benchmark for LLM instruction tuning optimization. In The Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.o tttet//ooummidddnnnn...
[71] Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. Aligning large language models with human: A survey. arXiv preprint 2rXiv:2307.12666 2023.
[72] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-though prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
[73] Chenxi Whitehouse, Tianlu Wang. Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swar- nadeep Saha. J1: Incentivizing thinking in ilm-as-a-judge via reinforcement learning. arXiv preprint X5X: 10320, 2025.
[74] Shijie Wu and Mark Dredze. Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing LEMNLP-IJCNLA pages 833-844, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19, 1077. URL https: / /ttp//aaaaannnnoooooee org/D19-1n 1077/.
[75] Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306. 09341, 2023.
[76] Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. Llava-critic: Learning to evaluate multimodal models. arXiv preprint arXiv:2410.02712, 2024.
[77] Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-ol: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10444 2024.
[78] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 15903- 2023.
[79] Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang. Yang Fan, Kai Dang, et al. Qwen2.5-omm technical report. arXiv preprint 253.22250.. 2025.
[80] Senqiao Yang, Jiaming Liu, Renrui Zhang, Mingjie Pan, Ziyu Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Hongsheng Li, Yandong Guo, et al. Lidar-llm: Exploring the potential of large language models for 3d lidar understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 9247-9255, 2025.
15

---

[81] Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. Self-distillation bridges distribution gap in language model fine-tuning. In Lei-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Pupers), pages 1028-1043, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/ v1/2024.acl-long.56 URL https: httt::::///cllaahhooogyyy hwwpchhoooogyoog//2222222 acl acl-long..1..
[82] Michihiro Yasunaga, Luke Zettlemoyer, and Marjan Ghazvininejad. Multimodal reward- bench: Holistic evaluation of reward models for vision language models. arXiv preprint arXiv:2502. 14/91, 2025.
[83] Botao Yu, Frazier N Baker, Ziqi Chen, Xia Ning, and Huan Sun. Llasmol: Advancing large language models for chemistry with large-scale, comprehensive, high-quality instruction tuning dataset. arXiv preprint arXiv:2402.09390 2024.
[84] Shoubin Yu, Jachong Yoon, and Mohit Bansal. CREMA: Generalizable and efficient video- language reasoning via multimodal modular fusion. The Thirteenth International Con- ference Learning Representutions, 2025. URL https://openreview net/ fest/foormmidd 3Ua01zDEt2.
[85] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu. Hai-Tao Zheng, Maosong Sun. et al. Rlhf-v: Towards trustworthy milms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 13807-13816, 2024.
[86] Ryandhimas E Zezario, Sabato M Siniscalchi, Hsin-Min Wang, and Yu Tsao. A study on zero-shot non-intrusive speech assessment using large language models. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2025.
[87] Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. The 4th Workshop on Mathematical Reasoning and Al at NeurlPS'24, 2024. URL /ttp///wwp/eeeeeee//////e net/ forum? Brrrmmid=CCHHoTLLni
[88] Yi-Fan Zhang, Tao Yu, Haochen Tian, Chaoyou Fu, Peiyan Li, Jianshu Zeng, Wulin Xie, Yang Shi, Huanyu Zhang, Junkang Wu, et al. Mm-rlhf: The next step forward in multimodal Ilm alignment. arXiv preprint arXiv:2502. 10391, 2025.
[89] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595-46623 2023.
[90] Lianghui Zhu, Xinggang Wang, and Xinlong Wang. JudgeLM: Fine-tuned large language mod- els are scalable judges. In The Thirteenth International Conference on Learning Representations, 2025. URL https: /ppprrvieeeeeeeeee/eeeeeeeeeeeee net/ forum? 7id aidd/id==sss piw.xxE.ExEE............
[91] Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford. Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint 1xXiv:1909008590 2019.
16

---

FLEX-Judge: Text-Only Reasoning Unleashes
Zero-Shot Multimodal Evaluators
Supplementary Material
A Additional Related Work
A.1
(M)LLMs as Judges
Large language models (LLMs) have recently been explored as a cost-effective alternative to human raters, paradigm referred to as LLM-as-a-Judge [89]. By encoding human-like reasoning and leveraging instruction prompts, these models can approximate human preference judgments in tasks such summarization, dialogue, code generation [29, 60, 61, 63]. Because human annotations are expensive and tinecunnuinngg to scale, the LLM-based evaluation promises a reusable, modular approach that can reduce reliance on direct human supervision. For instance, BoNBoN Alignment [24] explores using best-of-N selection guided by LLMM-enerrted scores, while LRL [43] relies on L.LM- based ranking signals to optimize model outputs inference time.
However, most M-a-----euudee approaches have focused on text-only use cases [36], and extending these models to multimodal content (e.g., vision-language or speech tasks) remains challenging [10, 11, 68]. While multimodal variants do exist [76, 78], they often rely on large-scale, manually crafted, and modality-specife annotated datasets for training or fine-tuning [6. 77, 88), which becomes overly expensive at scale. For example, LLaVA-Critte [76] curates 46K images and 113K evaluation data by using GPT-4/4V on 8 multimodal datasets, well manually crafted prompt templates. Prometheus- Vision [32] involves a number of data processing steps, with 5K images and 15K customized score rubrics, and GPT-4V generating 150K response-feedback pairs.
Furthermore, the lack of high-quality, publicly available multimodal preference benehmarks makes difficult to develop MLLM judge models [1 1, 30]. Our work addresses this gap by demonstrating that an MLLM can effectively judge multimodal outputs without requiring extensive preference supervision. This approach suggests that LLMs judge models can be extended to wider range of modalities (e.g., 3D point clouds [25], LiDAR [80], molecular [27, 83)) that have not yet been explored due to limited resources.
A.2 Reasoning-Guider Reward Models
A growing body of research suggests that equipping reward models with the ability to reason-often captured through chain-of-thouge prompts or explicit textual rationales-can significantly improve alignment with human preferences [7, 61, 72]. Rather than learning only from binary scalar labels, these models benefit from learning how humans arrive at a decision, providing a more interpretable and generalizable internal mechanism for preference modeling [31, 57].
Recent advances in this area have shown that training on textual explanations can improve zero- shot and few-shot performance in downstream tasks requiring nuanced judgments [29, 60]. For example, Zheng et al. [89] find that when LLMs articulate the rationale behind their preference for one sample over another, they achieve higher consistency with human annotators. Building on the success of reasoning-ban based models in other domains, Zhang et al. [87] and Mahan et al. [45] introduced generative reward models, which significantly outperform non-reasoning judge models. More recently. JudgeLRM [12] demonstrates that reinforcement learning with reasoning-annotated annotated data leads to substantial gains reasoning- intensive evaluation tasks, significantly outperforming standard SFT-trained models. Concurrent with JudgeLRM, models such as JI [73], DeepSeek- GRM [42], and RM-RI [13] also adopt reinforcement learning frameworks to enhance the reasoning capabilities of reward models.
ceasoning-nteensvve
Despite these benefits, most reasoning- based reward modeling remains constrained to text-only applications, due in part to the higher cost and complexity of collecting multimodal rationales [11, 76]. In contrast, our work shows that texiual reasoning supervision alone can be leveraged to enable robust multimodal evaluation. By training models on a small set of high-quality textual explanation data, demonstrate effective cross-modal generalization without the need for domain-speciffe
17

---

<I You are helpful assistant. The assistant first performs detailed, step-by-step reasoning process in its mind and then provides the with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, 1.e., <think> detailed reasoning process here, explaining each step of your evaluation for both assistants </think><answer> answer here </answer>. Now the asks you judge the performance of Score assistants 1-10 (highersbetter).. Criteria includes helpfulness, relevance, accuracy, and level of detail. Avoid order, length, style or other bias. After thinking, when you finally reach conclusion, clearly provide your evaluation scores within <answer> </answer> tags, i.e., for example, (aaeer333/anseer>>asseeeee <lim_end|>
assistants in reaponse the question.
< <lim_stare >user [Question] (question)
[Assistant 1's Answer]
[answer_1} -1}
[Assistant 2's Answer]
(answer_2)
< <lim_end|>
< liimssaart/aassiiee
<think>
Figure System prompt for JudgeLRM [12].
preference labels. This approach reduces annotation overhead and paves the way for more scalable, modality-agnoste judge models.
B Experimental Details
B.1 Dataset Description
We first describe training dataset (seed): To generate reasoning-based judgments, we JudgeLRM-7B [12] to sample responses based on the given contexts from JudgeL.M- M-0K [90], using the prompts shown in Figure 7 with temperature of 0.1. Among the 100K samples, filter out those with rating mismatches compared to the GPT-4o evaluation results provided Zhu et al. [90], resulting in approximately 20K samples remaining after this process.
To construct the single-score format, we post-process 500 samples by truncating the reasoning and answer of Assistant 2, selecting cases where the reasoning length of Assistant exceeds 375 tokens. For the pairwise format, we use the 500 longest reasoning samples where the combined length of both assistants' responses exceeds 750 tokens. For both pairwise and single-score formats, we randomly transform the grading scale from 1-10 to 1-5 by dividing the score by 2. We provide final training dataset in the supplementary material.
We also describe all the benchmarks used in our experiments:
The MLLM-as-a-Judee benchmark [11] is introduced to specifically assess the judgment ability of MLLMs in the domain of image understanding across diverse scenarios. The benchmark consists of 14 datasets covering tasks such as image captioning, mathematical reasoning, text recognition, and infographic understanding. In total, it includes 4,414 image- instruction pairs, curated evaluate whether MILMMs can generalize their evaluative abilities across different modalities.
MLLM-as-a-Juude:
Ben:ch:ardBench: VI.-RewardBench [38] diagnostic benchmark for evaluating vision- language models multimodal understanding, hallucination detection, and complex reasoning.
18

---

Table 6: Description of the evaluation format, metrics used, and bias handling approach for evaluation benchmarks in Section 3.
Benchmark
Eval. format
Metric
Bias handling
Single-score grading
Pearson correlation
Average on 3 samples
aLLM-aa-a-judge
Pairwise comparison
Accuracy
Average on 3 samples, tie option [21]
Batch-level ranking
Norm. Levenshtein distance
Average 3 samples
VL-RewardBench
Pairwise comparison
Accuracy
Average on 5 samples, random order [66]
MJ-Bench
Pairwise comparison
Accuracy
Order reverse [66]. tie option [21]
GenAl-Bench
Pairwise comparison
Accuracy
Audio MOS/SS
Single-score grading
ICC & SRCC
System-level evaluation [68]
It comprises 1,250 high-quality examples curated through an Al-assisted pipeline with human verification.
MJ-Bench: MJ-Bench [14] is a benchmark designed to evaluate multimodal foundation models in the role of judge for image generation tasks. It includes preference data across four key perspectives perspetives-ttxxt-iaaee alignment, safety, image quality, and generation bias each further divided into detailed subcategories. Each example consists of an instruction paired with a chosen and rejected image, enabling fine-grained assessment of model judgment.
GenA --nnnhh: GenAL-Bench [35] is human-curated benchmark for evaluating image and video generation models across diverse composition skills. It includes 1,600 prompts from professional designers, avoiding subjective inappropriate content, and covers over 5,000 human-vent aunin-verified skill tags. Unlike prior work, each prompt is annotated with multiple fine-grained tags. Specifically, GenAl-Bench supports image generation, image editing, and video generation tasks.
Audio MOS/SS Benchmark:
There is no unified, structured benchmark for audio evaluation tasks. Instead, Wang et al. [68] assessed speech quality and speaker similarity using four datasets: NISQA [49], BVCC [18], and SOMOS [46] for speech quality (712, 742, and 3,000 test samples, respectively), and VoxSim [2] for speaker similarity (2,776 test pairs). All datasets include human- annotated scores. For the MOS prediction task, auditory LLMs are asked to assign the MOS score scale from 1.0 to 5.0 for given speech input. While Wang et al. [68] designed datasct-specific prompts to help models account for each dataset's unique standards, we evaluate models using a unified prompt format without dataset-specific tuning. For the SS prediction task, models rate the similarity between two speech samples on a scale from 1.0 to 6.0, where higher scores indicate greater speaker similarity. Since the human annotations of MOS and SS averaged over multiple individuals, they are predicted in the form of floats.
B.2 Training Details
Here, we describe the hyperparameters and implementation details for training FLEX-VL-7B and FLEX-Omni-7h Using a IK-sized training dataset, we fine-tune Qwen2.5-VL-7B and Qwen2.5- Omni-7B with learning rates of 10 and 10-6 respectively. For both models, we use batch size of 2 and maximum sequence length of 4096 for single epoch. Training is conducted on NVIDIA A6000 GPUs, taking approximately 1.5 hours per run, which highlights cost-efficiency of our Fudx- For FLEX-MoI-LLaa LLaMA, we use the same hyperparameters as for FLEX-VI VL-7B.
B.3 Details of Evaluation Protocol
All experiments in this work are designed evaluate the quality of judge models, focusing on how closely their judgments align with human preferences when scoring comparing Al-generated responses. Since our goal is to assess models that act as evaluators, we compare their decisions directly to human annotations across several evaluation formats.
Table summarizes the evaluation formats, metrics, and bias-hand strategies used across all benchmarks used in our paper. Below, we elaborate on the evaluation settings:
Single-score Grading: The judge assigns scalar score to single response. We evaluate
performance using Pearson correlation (for vision-language tasks) or Spearman and LCC (for
19

---

<I Simmstart/>syee
You helpful assistant. The assistant first performs detailed, step-by-step reasoning process in its mind and then provides the with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, 1.e., <think> detailed reasoning process here, explaining each step of your evaluation for an assistant </think><answer> answer here </answer>. Now the user asks you to judge the performance of to the question. Score assistant 1-10 (higher-better). Criteria includes helpfulness, relevance, accuracy, and level of detail. DO NOT assign the same score to multiple assistants. Avoid order, length, style other bias. Åfter thinking, when you finally reach conclusion, clearly provide your evaluation scores within <answer> </answer> tags, i.e., for example, hattua/r/3///mmeeeaaaaeee////// haapa///e.a.aaaaeaaaaaaaaaaa/aaa
asaistant (multiple assistants) in reaponse
<lim_end|>
<lim_start|>uee >user
[Question] {question}
[Assistant 1's Answer] (answer_1)
[Assistant 2's Answer] [anawer_2}
[Assistant 3'8 Answer] (answer_3)
[Assistant 4's Answer] (answer_4) _4
< <li__nd|>
< im__tarrt Dassistan
<think>
Figure 8: System prompt for single-score and batch-level ranking evaluations. The part colorized in red denotes the additional instruction used only for batch-level ranking evaluation.
audio), measuring the alignment between the judge's scores and human-annotann ratings, For audio benchmarks, evaluation is also conducted the system level, averaging across all utterances from the same text-to-speech system.
The judge selects the preferred response between two candidates. Accuracy is computed by counting the agreement between the judge and human preferences. We handle position bias by using randomized response orders [66] or incorporating the tie option [21], depending on the benchmark.
Pairwise Comparison:
The judge ranks multiple candidate responses (more than two) based on quality. Human-annotated rankings are treated as ground-truth, and we consolidate the ranking results into sequences (e.g., ABCD ++ CDAB) and measure their similarity using Normalized Levenshtein Distance [34].
Batch-level Ranking:
List of Prompts. We provide our applied system prompts for diverse evaluations setup. For pairwise evaluation which is the most common setup in our experiments, we utilize the system prompt as in Figure and post- post-process the results by comparing the scores provided from the FLEX-Judge. For single-score and batch-level ranking evaluations, our prompts are introduced in Figure 8, thanks to its applicability via post-processing. Despite minor variations among evaluation setups, models trained on datasets that do not incorporate the format diversity we introduce in Section 2.2 often fail to follow the given instructions. In contrast, our judge model-trainee on data that reflects this adheres to the prompts across all setups.
format
20

---

<I You helpful assistant. The assistant first performs detailed, step-by-step reasoning process in its mind and then provides the with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, 1.e., <think> detailed reasoning process here, explaining each step of your evaluation for an assistant </think>>answer> answer here </answer>. Now the user asks you to judge the performance of
assistants. You have only FOUR Option:
Option 1. Model A is better: [[A>B]]
Option 2. Model B is better: [[B>A]]
Option 3. Tie, relatively the acceptable quality: [[A-B=Good]]
Option 4. Both are bad: [[A=B-Bad]]
Assess the quality of generated videos. Consider inappropriateness the following sub-dimensions: Alignment with editing prompt, Overedited, Naturalness, Artifact,
Visual Appealing, are correctly represented. Avoid order, length, style or other bias. After thinking, when you finally reach a conclusion, clearly provide your evaluation scores within <anaver> </answer> tags, i.e., for example, [mnwer>[[B>A]]]]] </anseer>>
<lim_end|>
<I iim_start/zuser
[Question] {question)
[Assistant A's Video] (video_1}
[Assistant B's Video] (video_2)
< <lim_end|>
< lliisttrr//aatte Passistant
<think>
Figure 9: System prompt for GenAl-Bench (edition) evaluation. We colorized the different parts in red compared to the training samples and observed that FLEX-JUdge closely follows the given instructions. shown in Figure 15.
For GenAl-Bench [35] and audio evaluations, which use different label formats (e.g., [[A=B=Good]] or [[A=B=Bad]] for GenAl-Bench, and the first decimal point for audio evaluation) compared to our training samples, we use different prompts, as shown in Figure 9 and Figure 10. Thanks to our format-diversity-awar training dataset, FLEX-Judge can reliably follow instructions with high variation, as demonstrated in Figure 15 and https: htt:://flxxxjjeeeee betigg..gthub.... io/. Further detailed prompts can be found our provided code implementation.
B.4 FLEX-MOI-LLaMA. Judge
In this section, we present the training details of FLEX-MoI-LLLaMA reasoning-augmentte molec- ular judge model built on top of Mol-LLaMA [27]. Mol-LLaMA a molccule-focused LLM, pretrained and fine-tuned LLaMA3.1 I-BB, with LoRA adapters, molecule encoders, and Q-Formers attached to enable effective encoding of 2D and 3D molecular structures.
molecular understanding datasets. Its structure is based on the frozen
To construct PLEX-MoI-LLAMA. we reuse the molecular encoders and adapter modules of Mol- LLaMA and fine-tune only the LLaMA3.1-2 1-BB backbone using the same text-only reasoning dataset employed for training FLEX-Omni-7B and FLEX-VL- This results in model that retains full molecular understanding while acquiring generalizable reasoning capabilities. LoRA modules are re-attached after fine-tuning, allowing us to preserve the domain-specific functionality of MoM-LLaMA while transforming into judge model.
21

---

<I diimssart/>ayee
You are helpful assistant. The assistant first performs detailed, step-by-step reasoning process in its mind and then provides the with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, 1.e., <think> detailed reasoning process here, explaining each step of your evaluation for an assistant </think><answer> answer here </answer>. Now the user asks you to judge the performance of audio generative AI assistant in response question. Listen to the generated speech audio, and score this speech on scale from 1.0 to 5.0 in FIRST DECINAL. Consider the following criteria when scoring:
the
Very Bad: The speech is very unnatural, has poor audio quality, nearly impossible to understand.
2 Poor: The speech sounds unnatural and/or noisy. Only a few words are understandable.
3 Fair: The speech somewhat unnatural contains noticeable noise, but
the overall meaning is understandable.
4 Good: The speech is generally natural and clear, with most of the content easy to understand.
5 Excellent: The speech is very natural, high in audio quality, and fully intelligible.
Do NOT consider the content of the speech. After thinking, when you fimally reach conclusion, clearly provide your evaluation scores within <answer> </answer> taga, i.e., for example, <answer>3 &<ansuer>> <lim_end|>
< lim_start/>use
[Question]
Generate clear, natural, and understandable high-quality speech audio.
[Assistant's Answer]
Here is the speech I generated:
{audio}
< <mim_ndd/>
<mmmstarrt Dassistant
<think>
Figure 10: System prompt for speech quality assessment. We colorized the different parts in red compared to the training samples.
B.4.1 Best-of-N Sampling
We first evaluate FLEX-MoI- MoM-LLLMA as a reward model for inference-time scaling. Specifically, we apply the best-of-N sampling setup, where N number of responses are sampled from the base Mol-LLaMA, and the best one is selected using FLEX-Mol-LLaMAN predicted scores. Importantly, each response includes only the Mol-LLaMA's final prediction label (i.e., "high permeability" or "low-to-moderate permeability") but also its accompanying analysis and explanation for the prediction. Scores assigned by FLEX-MoI-LLaaee strong correlation with downstream task accuracy (see Figure 4; left), suggesting that the model effectively distinguishes between higher- and lower-quality outputs.
generally range between 6.0-9.5 and show
Since there exist responses that receive identically best scores, we repeat the sampling and selection process over 10 random trials and report the average performance. As shown in Figure (middle), increasing N consistently improves accuracy, validating that FLEX-MOI-LLaMA provides reliable, fine-grained reward signals.
B.4.2 DPO Training
Beyond inference-time selection, we also use FLEX-MoI-LLLMM as reward model for DPO to further fine-tune Mol-LLaMA. For this, we curate DPO training dataset from Mol-LLaMA's instruc-
22

---

tion tuning corpus, which consists of two main types: (1) detailed chemical structural descriptions, and (2) structure-to-featee relationship explanations covering both chemical and biological attributes. We excluded the multi-turn conversation type.
For each query x, sample two responses from Mol-LLLMMA using different decoding temperatures, 0.8 and 1.2. We use FLEX-MoI-LLawe
to compare the two and include the example in the DPO training only if the response from temperature 0.8 receives higher score than the one from 1.2. Also, since there is a position bias [66] in pairwise comparisons, we flip the order of the two responses in prompt and evaluate again, retaining only those pairs where the winning response remains consistent. Consequently, we construct 4,253 high-qualij preference triplets (x.
Fine-tuning Mol-LLaMA with this DPO dataset results in substantial accuracy boost the downstream permeability prediction task. As shown in Figure 4 (right), the final model achieves up to 80.10% accuracy, surpassing prior models by large margin. This result confirms that FLEX- Mol-LLaMA provides not only reliable evaluation at inference but also effective supervision for preference-based training in specialized, underexplored domains.
C Additional Results
C.1 Additional Benchmarks
We extend our evaluation to include results on two recent and comprehensive MILM evaluation benchmarks: Multimodal RewardBench [82], which focuses on image understanding, and JudgeAny- thing [54], which covers wide range of any-to-any tasks. JudgeAnything includes not only im- age/video/audio understanding (I - T, V - T, A + T) but also inage/video/audioo generation from text (T + T. V,T T + A) and more complex tasks like video-to-audio (V + A) or audio-visual- V, to-text (V+A T).
The results are summarized Table and Table 8, respectively. We note that FLEX-VL-7B is not capable of evaluating audio-related tasks. Across both benchmarks, FLEX-Judge achieves comparable performance proprictary models while consistently outperforming existing open-source baselines.
Table 7: Comparison of MILM evaluator performance on Multimodal RewardBench. from the original work [82].
results
General
Knowledge
Reasoning Coding
Safety
VQA
Model
Overall
Correctness
Preference
Math
GPT-4o
0.626
0.690
0.720
0.676
0.621
0.748
0.872
0.715
3LaMA-332-900-VII &
0.600
0.684
0.612
0.563
0.531
0.771
0.624
115VA-1.5-1388
0.533
0.552
0.505
0.535
0.493
0.201
0.518
0.489
PLEX-Omni-7B
0.616
0.612
0.657
0.625
0.582
0.362
0.777
0.631
FLEX-VL-7B
0.620
0.618
0.630
0.677
0.550
0.732
0.837
0.686
Table 8: Comparison of MLLM evaluator performance on JudgeAnything within the Overall setting and pair comparison with tie. o results from the original work [54].
Multimodal Understanding
Multimodal
Generation
Modell
T-yT I-IT
V--T A-+T V+A-T
T-I T-VV
T-A
I-I I-VV I-tA
V-VV V-+A
A-V
A-
Overall
52.5
73.0
58.0
69.5
53.0
52.5
56.0
26.0
49.0
42.0
68.0
78.0
38.0
58.5
$7.5
55.4
Gemini- Pro
49.0
79.0
57.5
68.5
48.5
52.0
56.0
38.0
$7.0
39.0
69.0
88.5
33.0
47.0
67.5
56.6
Dcmini-20--Flstt 0
43.5
51.5
46.5
510
43.5
57.5
Twea2.5-0mmii-
35.5
50,5
42.9
51.5
44,5
36.0
31,5
39,5
29.5
40.0
32,5
34,5
35.5
53.0
40.5
39.8
Qwea2.5-VL-7B
36.5
36.5
34.0
FLEX-Omai-7B
46.5
70.0
53.5
67.0
$2.0
49.0
56.0
40.5
47.0
45.0
72,5
78.0
44.0
70.0
38.0
55.3
FLEX-VI--7B
45.0
53.5
43.5
54.5
47.5
42.0
70.5
C.2 Scaling Law for FLEX-Judge
To examine how model scale affects judge performance, we conduct an additional experiment using smaller LLM backbone, where results are found in Table 9. Specifically, train FLEX-V -VLBB
23

---

by fine-tuning Qwen2.5-VL-3B on our curated seed dataset. Despite its significantly smaller size, FLEX-VL-33B also demonstrates reasoning capabilities and generalizes across modalities. However, consistently underperforms compared to our base 7B model, indicating that larger-scale LLMM-based models are better equipped to internalize reasoning patterns and serve as more reliable judges. These results suggest that while reasoning sesooninggguieed guided supervision is effective even at smaller scales, model capacity remains an important factor in achieving high-quality, generalizable judgments. Meanwhile, despite its slightly lower performance compared to the 7B model, FLEX-VL-31 shows comparable performance to LLaVA-1.66 on MLLM-as-a-Judde and both Qwen2.5-VL-7B and LLaVA-NeXT on Gennl-Bench.
Table 9: Comparison of FLEX-Judge performance across different MLLM sizes. results from Table 1. 9 results from Table 2. 9 results from Table 3. t: 32B model was trained with LoRA [26].
GenAl-Bench Score (*) (1) w.o. (f) Batch (t) General (1) Hallu.(t) Reason. (t) Image Edition (1) Vid
MLLM-as-a-Judge
VL-Reward
Model
Size
LLaVA-1.6*
34B
0.184
0.460
0.648
0.501
Qwen2-VL
72B
38.1
32.0
61.0
LLVA--NXTT
Unk
22.65
25.35
21.70
Qwen2.5.VT.
7B
38.63
37.61
3B
0.176
0.493
0.650
0.478
46.39
36.85
58.08
36.71
33,62
42.93
FLEX-VL
7B
0,332
0.538
0.655
0.426
46.11
43.39
62.87
43.32
47.41
44.78
32B
0.361
0.587
0.716
0.432
54.52
41.92
64.46
44.32
53.54
47.33
C.3 Reliability of FLEX-Judge
Length Bias. In Chen et al. [11], models such as GPT-4V [1] and Gemini [62] tend to favor longer answers over concise yet correct ones, exhibiting a phenomenon known as verbosity bias [89]. In contrast, our FLEx-Omni-7B and FLEX- VL.-7B demonstrate length preferences that are more consistent with human evaluators, unlike previous judge models reported in Chen et al. [11]. As illustrated Figure 11, our models do not systematically prefer lengthy answers, but instead align well with human judgments regardless of response lengths. This suggests that FLEX-Judge can serve as a reliable judge.
Length Distribution by judge Preference
Length Distribution by jud ge Preference
500
500
Flex Omni-7B
Human
VL-78
Human
Ostribution
400
Length Distrbution
400
300
200
Winner
Laser
Winner
Loser
Preference
Preference
(a) FLEX--mmi-7B vs. Human
(b) FL-E-VV--7B vs. Human
Examination length bias of FLEX-Omni77B and FLEX-VLL VL-7B compared to human evaluators in pairwise comparisons (excluding ties) on the MLLM-as LLLLM-as-aJJdeee Judge benchmark [11].
Figure 11:
Position bias. Models as judges consistently favor answers in specific positions, often influenced by training data that typically place correct responses at the beginning or end of prompts [89]. We observed similar behavior in our FLEX-Judge during pairwise comparison evaluations. In Figure 12, we examine the behavior of our judge models against human preferences. While human evaluators show less bias towards the first or second response and frequently opt for the Tie option when appropriate, our judge models, particularly FLEX-Ommi-7B, tend favor the first response more often. Additionally, our models are less likely to output Tie judgments, instead preferring one over another. As detailed in Table 6, we address the positional bias randomly changing the response orders.
24

---

MILMMas-e--uudee
GerWi-Bench (Image
Ornr
Gnei
Deni
VL
Dendte
Denuty
0.3
0.2
0.3
6.1
AWhn
Wii
Yic
Wiin
Tie (0oocl
1e (60d)
A
Tie (Good)
Th: (budi
(a) Image Understanding
(b) Image Edition
(c) Video Generation
Figure 12: Examination on position bias of FLEX- FrxX-Omi-77B and FLEX-VL-778 compared human evaluators in pairwise comparisons (including ties). "A Win" indicates that the model preferred the first response, and "B Win" indicates preference for the second response.
C.4 Text Judgment Performance
We also evaluate FLEX-Judgs text-only assessment tasks, with results shown in Table 10. Interest- ingly, both FLEE--Omii-7 and FLEE-VLL-7B outperform the base judge model, JudgeI in terms of judgment accuracy-desplee being trained the BudgeLRM'ss response data. This suggests that our high-quality dataset curation (as discussed in Section 2.2) may lead to stronger textual reasoning performance.
While increasing the size of the training set could further improve test-time judgment accuracy, our primary focus on multimodal generalization. Moreover, discussed in Section 2.2, we observe catastrophic forgetting on the non-textual modalities when training with excessive text data. To balance effectiveness and modality retention, limit the training set to IK-sized text corpus throughout our experiments.
Table 10: Comparison of (M)LLM evaluator performance JudgeLM [90] and PandaLM [70]. results from Chen et al. [12]. I; re-implemented results.
JudgeLM (GPT-4o as Ground-Truth)
PandaLM (Human as Ground-Truth)
Model
Agreement
Precision
Recall
F1
Agreement
Precision
Recall
F1
GPT-3,5
73.83
70.70
52.80
52.85
62.96
61.95
63.59
58.20
GPT-4
66.47
66.20
68.15
61.80
Pandal.M-P7
68.61
40.75
38.82 88.82
39.41
$9.26
$7.28
$9.23
$4.56
4
74.86
57.53
58.14
JudgeLM-3318
89.03
84.76
82.64
75.18
74.93
69.73
Qwen2..5-7BIInstruce
76.85
78.71
77.85
78.28
63.96
61.95
59.81
JudgeLRM-7B
83.74
85.84
83.65
84.73
78.28
74.90
75.74
75.05
JudgeLRM-7B
82.26
84.86
82.47
83.64
76.41
71.41
71.30
71.16
FLEX-Ommi-7B
84.12
80.95
93.51
86,78
76.24
73.38
71.15
71.74
-L-7-
84.21
82.65
90.06
86.65
76.84
72.15
70.06
70.74
C.5
Non-judgment Visual-Language Task Performance
To further examine potential catastrophic forgetting degradation of multimodal abilities, also evaluate our model on standard visual question answering (VQA) tasks that are unrelated to judgment, specifically TextVQA [59] and OK-VQA [47]. As shown in Table 11, FL.E-VI--7 exhibits only a slight decrease on TexiVQA and even outperforms the base model on OK-VQA. These results suggest that well-structured methodology preserves the multimodal capabilitiess of the base model and does not cause forgetting.
Table 11: Evaluation standard visual question answering tasks.
Model
OK-VQA Dwen2.5-VL-77B
TextVQa
80.80
67.46
VL-
80.31
72.
25

---

D Qualitative Examples
We provide qualitative examples of FLEX-Judge's S reasoning and judgments across different modalities. These examples illustrate how the model understands, compares, and scores Al-generated outputs as well as multimodal imputs using structured textual reasoning.
D.1 Image Understanding and Generation Tasks
We showcase examples of the image understanding and generation tasks with vision-language benchmarks. All judgments are made by FLEE--VL-7B.
Figure 13 illustrates an image understanding task, involving OCR of Hindi phrase. Both assistants output similar-looking responses, but FLEX-Judge correctly recognizes the image content by referring to an "unexpected election" rather than "financial crisis". assigns a significantly higher score to Assistant 2, whose response accurately reflects the OCR content.
Figure 14 presents a quality assessment task of generated images. Both assistants generate images, and FLEX-judge evaluates them based on clarity, focus, and presence of artifacts. Assistant receives
User: <OCR> (4) pepefre Reber Retver ofieh ereetor
firz Rier 3
(Hindi -anndi--ingii Engllshh "By- elyletectoon" refors to the type of election that is conductted to #111 sudden or unexpected vacancy-)
Assistant 1: (83 "anftfar Prity à ag firte uuer at anftfr feufa aht faez аев čατ B.
Assistant 2: (5) "Mireieee लिपीयs" & पह सितtय अоडीरt t अी अnmrecu Ritत मो अररर # सिर सिता अात
Judge:
<thinks
Assistant 1's answer inaccurate and irrelevant question asked. question spectfically asked about "Mmmiem हरीापा, which translates to "sudden election" or "unexpected election" in English. This held fill sudden vacancy position, such legislative body position in governentt.
refers
The enswer provided by Assistant 1, "anfér firshh गे aæ सियोद पाςा डी mhīee सिात सो aiı fिmn याा टता 8, translates to "The financial crisis leads to situation that nakes the financial situstion difficult." This answer does not accurately describe the concept of an "unexpected election, which is the question.
financial crisis, which different topic and does not align with the context of an unexpected election.
be discussing
effects
In contrast, Assistant 2's answer accurate and relevant. It correctly defines "अााीी काीययत" as vacancy, which is the
"the kind of election that held fill well with the question asked, This answer is accurate because it correctly identifies the nature of an unexpected election, which seat in parliament or position in governmen that has becane vacant unexpectadly.
definition and aligns
specific type of election that
fill sudden varancy, such
Given the accuracy and relevance the definitions provided, it is clear that Assistant 2's answer is more accurate and relevant, the question asked, while Assistant 1's answer 1s inaccurate and irrelevant. </telvv (enssee32</ansseee Kenswerriee//asserr
Figure 13: Reasoning process of FLEX-Judge on the OCR task (VL-RewardBench))
User: Generate image "A tolevision is turned on, displaying cartoon featuring cats".
<thinks
Assistant provided clear, focused
of living with television nounted [...]
Clarity and Focus: image is clear and well-focused, naking easy identify objects and the scene, The television is the central focus, and the cartoon heing [...] inage,
are visible artifacts illogiall representations. The image is straightforward and does identify spacific details.
blurriness, distortion,
Assistant
In contrast, the inage provided by Assistant is quite blurry and lacks clear focus, [...] Clarity and Focus: The Image is significantly blurred, which makes It hard to discern the exact details
scene. This of clarity could due novenent camera, which night nislead the viewer about the content being displayed the television. on, or
lou-quality
the being displayed is cartoon, as the details are not cleor.
night
even
content
Assistant
of inage provided by Assistant it more accurate re- Judge: presenting the scenarto described in the question. saiiiinn///tink///./ " "aasserrnnneaee Bataerrrr////weenne canswer 2eaeers//////eeeere
clarity
Figure 14: Reasoning process of FLEX-Judge on the image quality assessment task
26

---

User: Edit
image
large plephant that ie standing up ooting +> standing up and enjoying waternelon".
Inage
Assistant
Assistant
Judge:
Assistant A's edited inage is more accurate in aligning with the editing proupt, which states that the eating watermelon. [...] By adding watermelon slices the elephant's Assistant A has correctly interpreted the requirenent to have the elephent eating waternelon. This inage
elephant
change
and appropriate
task, naking
aligned
desired outcome.
Assistant B's edited inage is significantly nore overedited and less natural. The watermelon slices are placed in way that they appear be growing from the elephant's does not fit the natural behavior or appearance of an elephant. This kind of editing can be seen as a form of overediting,
which highly unrealistic and
it introduces elements that do not naturally belong to the scene, such the large, floating waternelon slices, which might seen artificial or unrealistic. editing detract from the naturalness and realisn of the image, which 1s significant drawback in terms of quality.
level of
Given the clarity and relevance of the inage provided by Assistent 1, it is pore accurate in re- presenting the scenario described in the question. (eation.//ttink) "canswer>[ (</asseers
Figure 15: Reasoning process of FLEX-Judge on the image editing task (GenAl-Bench)
higher score due its "clear and well-focused" image with "no visible artifacts", whereas Assistant 2's image shows blurs, which could be "due to movement or a low-quality camera".
Figure 15 shows an image editing task, where the judge assesses whether the edited image by each assistant aligns with the revised prompt. FLEX-Judge identifies that Assistant B's output exhibits "overediting", detracting from the "naturalness and realism". and thus assigns it lower score.
These examples highlight the FLEX-Judge's ability explain its preferences based semantic accuracy, relevance, and consistency with visual content, even in fine-grained evaluation scenarios.
D.2
Video and Audio Tasks
Since we cannot include video and audio content directly into the paper, we provide full qualitative examples-including input prompts, AI responses, and FLEX-Judge's reasoning- video and audio evaluation tasks at the following anonymized project page."
D.3 Molecular Tasks
Figure 16 and Figure 17 illustrate the reasoning and judgment outputs of FLEX-MOI-LLaMA, our molecular judge model. Figure 16 presents two examples from the PAMPA prediction task, the primary evaluation task for Mol-LLaMA. These samples were used in the best-of-N sampling experiment (Section 4). We display both low-scoring and a high-scoring response examples, along with FLEX-Mol-LLaMA's reasoning, to highlight its ability to distinguish response quality based on molecular analysis. Figure 17 shows a pairwise judgment example from Mol-LLaMA's instruction tuning set, which includes prompts about biological property prediction. Here, we sample two responses from Mol-L_LaMA and use FLEX-MoI-L..aMA to evaluate and score both. These pairwise preferences are then used to construct training triplets for DPO, guiding further alignment of Mol-LLaMA with high-quality response patterns.
hnttps:/fflex: judge. wit.ggtthhiii
27

---

Predict permability given tolecule the artificial Paralle1 Artificial Membrane Perreability Assay (PAMPA), high tow-ro-odeerat.
O Input
Vol-LLaMA: Low-to-noderate pereability
bround-truth: High permeabllity
Flex-Rol-LLana Judge:
<thinks
The given nolecule, 1,2,3, d-ttetralylroo---guiiootzz
derivative of quinolizine, which is bicyclic compound. tatranydro fors,
[..-]
Liposhtiicity: The presence the quinoline core, which is typically aroratic, is reduced
which can lead to decrease in Hipopiiiiiity [...] potantially leading to moderate pernsbility.
2. Moleculan Size and Weight: The tetrahydro forn of the noletule is likely to be snaller and less bulky than its fully
aronatic counterparts, which enhance its ability to rigid the membrane. [...]
which night affect its ability tetrahydro
Rigidity and
compared
fully aronstic quinaline,
into Hipid bilayer. does not form strong R-n interactions with the lipid bilayen, which can be factor in its permebiliiyy
aronaticity right also nean that molecule
6. Hydration Energy: the reduced aroraticity night also nean that the volecule has lower hydration energy, which can be beneficial in porseating the resbrane, it night require less [..-]
to evercore the hydnation harrier.
Overall, the combination these has halanced set propecties that can facilitate passage through ifiiiia meshrane,
suggests
given molecule would likely
moderate permeabllity
the PAPPA assay, but not to the extent that it would be highly permeable. </thhits cansuer>s. Saauerrs.5c//nser.
9. User: Predict the permeability of given rolecule to the artificial membrane in the Parallel Artificial Nembrane Perreability Assay (PMPA), high low-to-voderate,
Hol-LLaMA: High permeability
Ground-truth: permssility
Input Molecule
Flex-Pol-LLaMA Judge:
<think>
The given molecule is a complex organic compound that includes several structural features which can influence its perreability in the PAMA
[...]
rings, such as methoxypheny! groups, significantly enhance lipophilicitty the nolecule, Aronatic rings are known to increase the hydrophobic character, which can Improve the nolecule's ability to perseate through the Hipdd bilayer of the PAMPA sechrane. [...]
Lipophilitity;
presence
6. Aronsticity: The arovatic rings not only contribute to Hipophilicitty but can also play role in the nolecule's
Interaction with the mexbrane, Aroratic rings can participate in n-n Interactions [...] which can erhance perseability. the exadiazole ring lower hydration ability to perseate nembrane.
Hydration Energy: The
help
rolecule [...]
bound water volecules, thus improving
Overalll, the combination of these structural features suggests that the given salecule would Hikey have a high pernesbility in the PAMPA assay, it is likely to have favorable balance between hydrophobic and hydrophilic the membrane effectively. 4/thinks
interactions, and suitable size and shape (ooevess..o//sserr
Figure 16: Reasoning process of FLEx-MOI-LLaMA judge on the PAMPA task.
E
Limitations
We have demonstrated that textual reasoning alone can effectively train multimodal judge models that generalize across modalities. While training only the LLM part is an innovative idea, consists of pitfall: the underlying LLM must possess (or at least can learn) sufficient reasoning capability. Our method efficient and broadly applicable, but it implicitly requires that the backbone model be capable of generating coherent, structured reasoning.
We observed this limitation when attempting extend approach to 3D-LLM [25], which can process and encode 3D point clouds using Flan-T5-XL [17] as its language backbone. Despite the model's 3D understanding, pretraining made it unsuitable for generating high-quality reasoning data. As result, we were unable to successfully train a FLEX-3D-LLM judge using our framework. This highlights a key constraint of our approach: it is less effective when applied to MLLMs built on weak or constrained LLMs that lack the capacity for textual reasoning.
limited context window (512 tokens) and lack of strong reasoning
While FLEX-Judge naturally inherits the capabilities of its MLLM backbone, this is an inherent assumption shared by all LL-M-as-a-Judge paradigms. Our key contribution lies not in developing better backbone but in demonstrating that minimal textual supervision can yield strong cross-modal evaluation with fixed MLLM, eliminating the need for expensive modality-specific training.
28

---

Input Molecule (IUPAC nane): 1-----rinnn22[((25-22------ htt.-aaloo----peellpppp-p-ee pto-phyopyppppnylll -aattyylllllttllllinonn | J-N-naphthallen-22 --pett-naaapt User: tlaborate in detail the biological attributes the provided cospound, explaining how its peinary linked properties.
Vol-itaMA (response 1):
The given nolecule, Henastyl-entany,, is a synthetic optoid analogue closaly related to fentanyl. It is characterized prinarily by the presence of several key substructures that contribute to its biological properties, particularly its opioid-like effects.
The substructure active peptides and proteins. This linkage the rolecule involved in forsing stable bond that allous ecule to resist estabolic degradation, thereby enhancing its duration of action in the hndy. [...]
given
includes
linkage,
in nany biologically
Hol-LLaMA (response 2):
the given nolecule is a derivative of L-lysine connected to two naps napsigegen molecules. Let's delive into the biological properties analyzing the care substructures:
side in protein synthesis, serving as an acid, base, and a playen in enzynatic binding sites. With its charged groups, L-lysine can interact and stabilize protein structures through ionic bonds and hydrogen bonds. In the given molecule, this structure likely provides scaffold stability and contributes its binchesical interactions.
arina acid bearing another anino group. Its configuration lends essential group, carboxylic acid group,
L-lysine Bacibone: L-lysine
petnany
irplies incorporation naphehalene, aronatic hydrocarbon. naphthyl groups are nentioned, each acting as large, planar, aronstic solety that can engage in pi-stacking interactions. [..-]
kaphthyl Groups:
Being naphthyl
Flex-Pol-LLaNA Judge: <thinks
Assistant
very
explanation biological properties of N-Acety substructures nolecule, such as 1inkage,
synthatic optold analogue. and the phenyl naphthyl groups, are cructal in defining biological activity.
asino group,
1. Anide Linkage:
This is a key feature that contributes to the molecule's stability and resistance to netabolic they relatively stable and presence enzymes right atherwise important for the prolonged action nolecule analogues. [...]
degradition. Amide bonds often found biologically active conponnds
the
structure
stability body, is typical characteristic opioid
contrast, the description provided Assistant is less accurate and relevant to the biological properties of N- keety3fentanyl. The substructures
L-lysine backbore acid, naphthyl graups, typical characteristica opioid analogues. L-lysine an typical feature in apioid structures, and the presence of naphthyl groups night be more relevant to different type of rolecule or drug, such as those involved in protein interactions or specific enzyre inhibitors. This night lead to different set of biological Interactions and offects, which are not typical for oplold analogues 1ike I-ktatyl
answer,
not align
Overall, explanation provided assistant k-Acetyyffentany. which consistent Kannsser (e S37anwerr cansier an7anss
aligned with potent opioid analogue.s/thinkk s/thinks
properties nechanisss of actioon of
Reasoning process of FLEX- LLEXMMM judging the preference of two Mol-LLaMA responses on biological attributes prediction data. Here, the response I is preferred (score 9) against the response 2 (score 4).
Figure 17:
29

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract and Introduction sufficiently contain our main contributions: utilizing small amount of text reasoning corpus to train FLEX-Judge, multimodal reasoning judge model.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Our method is cost-efficient and generalizable, but implicitly requires that the backbone LLM should be capable of reasoning. We have included the detailed limitations in Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
30

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
We do not include any theoretical results in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
have elaborated the experimental details to enable the reader to reproduce our approach and results. We also include the seed dataset (IK-sized text reasoning) and the source code in the supplementary material for better reproducibility.
Justification: Section 3.1 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
31

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have added our source code and dataset in the supplementary material. We also plan to make all the assets available upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
public/ public/guides. weggggg//odeuuuaiigge
oniolicy) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //nips wwtpcccpppccppppttcccccccccctttt wttpc//gg/es/eeeeeeeeeeeeeeeeeee Chtpe///CoSSSSsisseeeee semiissiinni ssionnolicyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Section 3.1 and Appendix B contain all the experimental settings and details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In our main results (Table 1-Table 4), we have reported average performances over few random samples (see Appendix B.3 for details). Also, Figure and Figure show standard deviations across random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
32

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Appendix B.2, we elaborate the details on compute resource we have used for the experiments and report the elapsed time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //tner/isesssseee cc/public//aeeccccccccccccccctct wtwwiieiiieleeeeeeeeeeennn line??
Answer: [Yes]
Justification:
We have reviewed the NeurlPS Code of Ethics and conform to the instruction.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This study does not invoke considerable societal concerns.
33

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not contain such risks for data/model misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: We follow the conventional evaluation protocols for every experiment and dataset, which we assume are properly cited and detailed in Appendix B.3.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
34

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We detailed the dataset curation process in Section 2.2, as well as providing the training dataset and instruction file along with the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not deal with crowdsoureing nor human subjects in this research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not deal with crowdsourcing nor human subjects in this research.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
35

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We do not involve LLMs for the core method development in this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /tttpa:/ncee wttp///een...peeeeeee cc/ 2 for what should or should not be described.
36


Reviews:
[{"review_id": "rMJ9vY2uG2", "reviewer": "Reviewer_rdTF", "summary": "The paper presents the pipeline and the model for serving as a task evaluator that is trained using only textual reasoning data that can generalize its judgment capabilities to various modalities (vision, audio, video, even molecular inputs) without additional modality-specific training. The core idea of the proposed approach is that structured explanatory rationales (specifically generated for the responses evaluation and comparison) capture general decision-making patterns that can transfer across modalities. In the experiments, the authors fine-tune a base multimodal language model (specifically, Qwen-2.5 VL/Omni, 7B) on a small curated corpus of high-quality text-only evaluation examples with rich reasoning. After this training, the KIRBY-Judge (the fine-tuned model) can judge any input, i.e. perform zero-shot evaluation on image-text tasks, audio transcripts, video descriptions, and biological data, by leveraging the base model’s multimodal inputs. Empirically, KIRBY-Judge achieves competitive or superior performance to state-of-the-art evaluators despite using orders of magnitude less training data.", "questions": "I have the following questions for the authors:\n1. Did you evaluate the performance of the specifically prompted, but instruct-tuned multimodal models on your judgement tasks?\n2. Did you experiment with the size of the dataset for finetuning, can you assess the number of textual data necessary for training without sacrificing the multimodal abilities of the initial model?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The main strengths of the paper are as follows:\n\n1. The paper analyzes and empirically validates an important idea: that it is possible to effectively train models using small-scale datasets composed solely of high-quality, text-only reasoning examples. The main results suggest that when collecting modality-specific annotations is challenging, such datasets can still yield performance comparable to that of models trained on large-scale, modality-specific data.\n2. The specific molecular case study strengthens the contribution by demonstrating the method’s applicability to specific domain data. This shows that the proposed approach can generalize beyond conventional multimodal settings.\n3. The paper provides extensive experimental evaluation across a diverse set of benchmarks and modalities, which reinforces the robustness and reliability of the proposed method.\n\nThe main weaknesses of the paper are as follows:\n\n1. Despite the thorough comparisons with existing models, it would be valuable to include additional ablation studies to better isolate the effects of the proposed fine-tuning approach. First, it would be useful to evaluate the performance of the base models (e.g., Qwen-VL and Qwen-Omni) without any fine-tuning on the text-only dataset. This could clarify whether the instruction-following capabilities of the base models alone are sufficient for judgment tasks, or if the fine-tuning step is indeed necessary.\nSecond, assessing the performance gap on standard vision-language benchmarks before and after text-only fine-tuning would help determine whether this fine-tuning affects the model’s inherent multimodal understanding capabilities (either positively or negatively).\n2. An additional ablation study examining the effect of dataset size would also be valuable. Specifically, it would be helpful to understand how much text-only data is actually needed to achieve strong generalization performance without sacrificing the multimodal abilities of the initial checkpoint.", "was_revised": true, "final_justification": "I have read the rebuttals from the authors for my review and other reviewers, and I am quite satisfied with the responses. One of my initial concerns was that including more textual data for training can drop the model's performance significantly; the authors provided additional evaluation, showing that this is not the case. So, I would like to increase my score and suggest accepting the paper because the topic of evaluating multimodal inputs is essential in the current domain, where benchmarks are concentrated on generative answers that require oracles to judge the responses.\n\nRevised Score: 5 (Accept)", "rebuttal": "Thank you for your insightful feedback. We have rephrased your comments for simpler reference and have included our respective responses. We look forward to any further discussions.\n\n>***Q1. [Comparison with base model performances]*** Did authors compare the performance of your base models and fine-tuned models on judgment tasks, and did you assess how text-only fine-tuning affects their multimodal understanding on standard vision-language benchmarks?\n\n***A1.*** Thank you for your comment. As shown in the table below, we directly compared our Kirby-Judge models (Kirby-VL-7B and Kirby-Omni-7B) to the base instruction-tuned models (Qwen2.5-VL-7B and Qwen2.5-Omni-7B) across multiple benchmarks. With only a small amount of text-only data, Kirby-Judge achieves substantial performance improvements over the base models—for example, up to +32.38 on VL-RewardBench (Reasoning) and +14.85 on GenAI-Bench (Edition). This demonstrates that our fine-tuning approach is highly effective compared to simply prompting the instruction-tuned base models.\n\n| Model | MLLM-as-a-judge (Score, Ave.) | MLLM-as-a-judge (Pair w Tie, Ave.) | MLLM-as-a-judge (Pair w.o Tie, Ave.) | MLLM-as-a-judge (Batch, Ave.) $\\downarrow$ | VL-RewardBench (General) | VL-RewardBench (Hallu.) | VL-RewardBench (Reason.) | GenAI-Bench (Image) | GenAI-Bench (Edition) | GenAI-Bench (Video) |\n|-|-|-|-|-|-|-|-|-|-|-|\n| Qwen2.5-VL-7B | 0.165 | 0.423 | 0.425 | 0.585 | 37.65 | 33.11 | 48.19 | 31.93 | 38.63 | 37.61 |\n| Kirby-VL-7B | 0.332 | 0.538 | 0.655 | 0.426 | 46.11 | 43.39 | 62.87 | 43.32 | 47.41 | 44.78 |\n| Δ (Kirby - Qwen) | +0.167 | +0.115 | +0.230 | -0.159 | +8.46 | +10.28 | +14.68 | +11.39 | +8.78 | +7.17 |\n| | | | | | | | | | | |\n| Qwen2.5-Omni-7B | 0.072 | 0.471 | 0.526 | 0.576 | 32.60 | 18.30 | 28.70 | 34.87 | 31.88 | 38.45 |\n| Kirby-Omni-7B | 0.306 | 0.532 | 0.650 | 0.425 | 47.01 | 42.72 | 61.08 | 38.15 | 46.73 | 37.10 |\n| Δ (Kirby - Qwen) | +0.234 | +0.061 | +0.124 | -0.151 | +14.41 | +24.42 | +32.38 | +3.28 | +14.85 | -1.35 |\n\n\nTo further address concerns about potential catastrophic forgetting or degradation of multimodal abilities, we also evaluated our models on standard visual question answering (VQA) tasks that are unrelated to judgment, specifically TextVQA (Signh et al., 2019) and OK-VQA (Marino et al., 2019). As shown below, Kirby-VL-7B exhibits only a slight decrease on TextVQA and even outperforms the base model on OK-VQA. These results suggest that our well-structured methodology preserves the multimodal capabilities of the base model and does not cause forgetting.\n\n| Model | TextVQA | OK-VQA |\n|-|-|-|\n| Qwen2.5-VL-7B   | 80.80   | 67.46  |\n| Kirby-VL-7B     | 80.31   | 72.10  |\n\nIn summary, our experiments demonstrate that Kirby-Judge achieves significant improvements over the base models on judgment tasks, while largely maintaining (and in some cases improving) performance on non-judgment multimodal benchmarks.\n\n&nbsp;\n\n>***Q2. [Effect of dataset size]*** How does the amount of text-only data used for fine-tuning affect generalization and multimodal performance? Did you perform ablation studies on this?\n\n***A2.*** Thank you for the comment. As shown in ***Section 2.2 and Figure 2(b)***, we have provided an ablation study on the effect of dataset size for both text-only (PandaLM, JudgeLM) and vision-language (VL-RewardBench, MJ-Bench) judge evaluations. We find that text-only tasks benefit from larger datasets, showing continuous improvement as the number of samples increases. In contrast, for vision-language tasks, performance improves up to a certain threshold (1K), but adding more data beyond that point results in a slight and gradual performance drop, likely due to catastrophic forgetting. These results highlight the importance of careful dataset size selection, and we have designed our methodology with these findings in mind."}, {"review_id": "3XthlfGQRH", "reviewer": "Reviewer_1xgQ", "summary": "This work proposes Kirby-Judge, a multimodal judge model for automatic evaluation of image, videos, audio, and even molecules. The model is fine-tuned from Qwen2.5-VL/Omni-7B on a small scale (1K) of text-only data for LLM-as-judge data with high-quality reasoning steps. The hypothesis is that by learning from structured, reasoning-rich text data, the model can generalize to other modalities that the model originally can understand. Despite the minimal training cost, experiments show that Kirby-Judge can produce effective judging results aligned with human preferences.", "questions": "Please check the Weaknesses described above.", "limitations": "The authors have discussed the limitations in the supplementary material. Another minor limitation is that this work does not show whether this method can further scaled to larger models (e.g., 32B).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. The training (fine-tuning) cost is very affordable.\n\n2. The idea of transferring knowledge from text-only data to other modalities for LLM-as-judge is interesting.\n\n3. The comprehensive experiment results show promising performance compared with the best open-source and closed-source models.\n\n## Weaknesses\n\n1. In the main Tables 1 and 2, the base models' (Qwen2.5-VL-7B, Qwen2.5-Omni-7B) performance should be included to clearly understand the relative improvement brought by Kirby-Judge.\n\n2. In Figure 6, the standard deviation of multiple runs seems large. How does it compare with baseline models? Some more analysis on the model prediction stability/robustness could be helpful.\n\n3. As acknowledged in the supplementary material, the success of Kirby-Judge relies on the capabilities of the underlying multimodal LLM. If the base model does not have strong reasoning capabilities already, it would be hard to develop a judge model with limited fine-tuning data. In other words, the performance of Kirby-Judge may be heavily dependent on the base models (Qwen2.5-VL/Omni-7B), especially their base reasoning capabilities. Considering reasoning LLMs are being actively developed in the open-source community, this limitation is rather minor, but worth some investigation (e.g., training weaker models with the same data as Kirby-Judge).", "was_revised": true, "final_justification": "The previous concerns are addressed in the response. I will keep the positive rating.", "rebuttal": "We appreciate your constructive comments. We have rephrased the question for ease of reference and provide our corresponding responses below. We look forward to any further discussions.\n\n> ***Q1. [Comparison with base model performances]*** Could the authors include the results of the base models (Qwen2.5-VL-7B, Qwen2.5-Omni-7B) in the main tables to clearly show how much improvement is achieved by Kirby-Judge?\n\n***A1.*** Thank you for your comment. As shown in the table below, we directly compared our Kirby-Judge models (Kirby-VL-7B and Kirby-Omni-7B) to the base instruction-tuned models (Qwen2.5-VL-7B and Qwen2.5-Omni-7B) across multiple benchmarks. With only a small amount of text-only data, Kirby-Judge achieves substantial performance improvements over the base models—for example, up to +32.38 on VL-RewardBench (Reasoning) and +14.85 on GenAI-Bench (Edition). This demonstrates that our fine-tuning approach is highly effective compared to simply prompting the instruction-tuned base models.\n\n| Model | MLLM-as-a-judge (Score, Ave.) | MLLM-as-a-judge (Pair w Tie, Ave.) | MLLM-as-a-judge (Pair w.o Tie, Ave.) | MLLM-as-a-judge (Batch, Ave.) $\\downarrow$ | VL-RewardBench (General) | VL-RewardBench (Hallu.) | VL-RewardBench (Reason.) | GenAI-Bench (Image) | GenAI-Bench (Edition) | GenAI-Bench (Video) |\n|-|-|-|-|-|-|-|-|-|-|-|\n| Qwen2.5-VL-7B | 0.165 | 0.423 | 0.425 | 0.585 | 37.65 | 33.11 | 48.19 | 31.93 | 38.63 | 37.61 |\n| Kirby-VL-7B | 0.332 | 0.538 | 0.655 | 0.426 | 46.11 | 43.39 | 62.87 | 43.32 | 47.41 | 44.78 |\n| Δ (Kirby - Qwen) | +0.167 | +0.115 | +0.230 | -0.159 | +8.46 | +10.28 | +14.68 | +11.39 | +8.78 | +7.17 |\n| | | | | | | | | | | |\n| Qwen2.5-Omni-7B | 0.072 | 0.471 | 0.526 | 0.576 | 32.60 | 18.30 | 28.70 | 34.87 | 31.88 | 38.45 |\n| Kirby-Omni-7B | 0.306 | 0.532 | 0.650 | 0.425 | 47.01 | 42.72 | 61.08 | 38.15 | 46.73 | 37.10 |\n| Δ (Kirby - Qwen) | +0.234 | +0.061 | +0.124 | -0.151 | +14.41 | +24.42 | +32.38 | +3.28 | +14.85 | -1.35 |\n\n&nbsp;\n\n> ***Q2. [Large standard deviation of Kirby-Omni-7B]*** In Figure 6, the standard deviation of multiple runs (especially for Kirby-Omni-7B) appears to be large. How does this compare to the baseline models? Some further analysis on model prediction stability or robustness would be helpful.\n\n***A2.*** Thank you for your question. The table below reports the average (standard deviation) of model accuracy across multiple runs, where $k$ is the number of reasoning paths used for majority voting. We observe that the baseline instruction-tuned Omni model (Qwen2.5-Omni-7B) already exhibits high variance, and this characteristic is inherited by Kirby-Omni-7B. In contrast, both the baseline VL model and Kirby-VL-7B show much lower standard deviation, indicating greater stability. Importantly, for both VL and Omni variants, the Kirby-Judge model either matches or reduces the standard deviation relative to its base model at each value of $k$. These findings suggest that prediction stability (as measured by standard deviation) is mainly a property of the base model. Our method does not introduce additional instability and, in some cases, even improves evaluation consistency compared to the baseline.\n\n| Model | $k=1$ | $k=3$ | $k=5$ | $k=7$ | $k=9$ |\n|-|-|-|-|-|-|\n| Qwen2.5-VL-7B      | 22.53 (4.95)  | 42.73 (2.91)  | 47.79 (1.99)  | 51.20 (1.92)  | 53.29 (1.75)  |\n| Kirby-VL-7B        | 55.39 (1.36)  | 57.31 (1.22)  | 58.32 (0.71)  | 59.52 (0.80)  | 60.12 (0.61)  |\n| Qwen2.5-Omni-7B    |  9.48 (4.79)  | 22.45 (4.63)  | 28.58 (4.22)  | 32.89 (3.33)  | 37.37 (2.21)  |\n| Kirby-Omni-7B      | 55.01 (1.79)  | 56.44 (2.14)  | 57.68 (3.14)  | 57.38 (2.91)  | 57.64 (2.03)  |\n\n&nbsp;\n\n> ***Q3. [Dependence on base model capabilities]*** As acknowledged in the supplementary material, the effectiveness of Kirby-Judge is influenced by the reasoning capacity of the backbone model. Could the authors comment on how your approach performs with weaker or smaller backbone models, and whether you conducted related experiments?\n\n***A3.*** Thank you for your insightful question. We agree that the performance of Kirby-Judge is closely tied to the multimodal understanding and reasoning ability of its backbone MLLM. To directly assess this, we conducted additional experiments with a weaker backbone: we trained Kirby-VL-3B by fine-tuning Qwen2.5-VL-3B on our curated seed dataset and compared its performance to Kirby-VL-7B and other larger models (see ***Table 7 and Appendix C.1*** in the supplementary material). Notably, Kirby-VL-3B demonstrates strong generalization and reasoning ability, achieving better results than LLaVA-NeXT and Qwen2.5-VL-7B, but underperforms compared to our Kirby-VL-7B model.\n\nAdditionally, we further experimented with another weaker backbone, Qwen2-VL-7B, in place of Qwen2.5-VL-7B. This resulted in a slight further performance drop: 0.538 → 0.524 (MLLM-as-a-Judge, pair w tie) and 46.1 → 42.5 (VL-RewardBench, General), but still highly competitive to the open-source baselines. These results confirm that while the choice of backbone affects the performance, our Kirby-Judge consistently brings significant gains over the base models, even with smaller or weaker MLLMs.\n\nMoreover, as the reviewer pointed out, reasoning-capable MLLMs are rapidly evolving. Recent models like DeepSeek-R1-Distill show promising reasoning ability even at smaller scales, and our framework is designed to benefit from these ongoing advancements. Once a stronger or more efficient backbone becomes available, our method can be seamlessly applied using only a small amount of high-quality text-based supervision.\n\n&nbsp;\n\n> ***Q4. [Larger Kirby-Judge]*** Another minor limitation is that this work does not show whether this method can be further scaled to larger models (e.g., 32B).\n\n***A4.*** Thank you for pointing this out. We have trained Kirby-VL-32B, based on Qwen2.5-VL-32B-Instruct (see table below), and these results confirm the scalability of our framework. Although we trained the 32B model using parameter-efficient LoRA (Hu et al., 2021), which is generally less effective than full fine-tuning due to limited computational resources, it outperforms the fully fine-tuned 7B version on most evaluation benchmarks and is competitive with GPT-4o. This validates that judgment quality improves as the backbone’s reasoning capacity increases.\n\n| Model | MLLM-as-a-judge (Score, Ave.) | MLLM-as-a-judge (Pair w Tie, Ave.) | MLLM-as-a-judge (Pair w.o Tie, Ave.) | MLLM-as-a-judge (Batch, Ave.) $\\downarrow$ | VL-RewardBench (General) | VL-RewardBench (Hallu.) | VL-RewardBench (Reason.) | GenAI-Bench (Image) | GenAI-Bench (Edition) | GenAI-Bench (Video) |\n|-|-|-|-|-|-|-|-|-|-|-|\n| GPT-4V/4o | 0.424 | 0.538 | 0.717 | 0.361 | 49.10 | 67.60 | 70.50 | 45.59 | 53.54 | 48.46 |\n| | | | | | | | | | |\n| Kirby-VL-3B   | 0.176 | 0.493 | 0.650 | 0.478 | 46.39 | 36.85 | 58.08 | 36.71 | 33.62 | 42.93 |\n| Kirby-VL-7B   | 0.332 | 0.538 | 0.655 | 0.426 | 46.11 | 43.39 | 62.87 | 43.32 | 47.41 | 44.78 |\n| Kirby-VL-32B (LoRA) | 0.361 | 0.587 | 0.716 | 0.432 | 54.52 | 41.92 | 64.46 | 44.32 | 53.54 | 47.33 |"}, {"review_id": "L4QlEbCTOg", "reviewer": "Reviewer_zegN", "summary": "The paper introduces a new multimodal judge model, Kirby-Judge, designed to evaluate the performance of large multimodal models across a broad range of tasks. The proposed approach demonstrates strong robustness in handling multiple modalities, even when trained with minimal textual reasoning data. Experimental results show that Kirby-Judge outperforms state-of-the-art judge models that rely heavily on modality-specific training data, particularly in tasks such as image understanding, image editing, video generation and audio understanding. Additionally, the authors present an effective case study in which their approach is successfully adapted to evaluate multimodal models for molecular modeling.", "questions": "- Could the authors comment on how their model would compare against the recently proposed MLLM-as-a-Judge models such as InternLM-XComposer2.5-Reward, CAREVL, and JudgeAnything, or provide preliminary results if available?\n- Have the authors considered evaluating Kirby-Judge as a response selector as part of an inference-time scaling strategy?\n- Could video understanding assessments be carried out using the JudgeAnything benchmark?", "limitations": "The authors have adequately discussed the limitations of their approach.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- The paper presents Kirby-Judge, a novel multimodal judge model designed to evaluate large multimodal models across a diverse range of tasks, which addresses a critical need in the field.\n- The proposed approach is robust to different input modalities and requires only minimal textual reasoning data for training, making it broadly applicable and data-efficient.\n- Experimental evaluations demonstrate that Kirby-Judge, in general, outperforms existing judge models trained with extensive modality-specific data, across a diverse  set of tasks.\n- The method significantly reduces the reliance on large amounts of specialized training data, enhancing its practicality and scalability in real-world applications.\n- The paper includes a well-executed case study showing the adaptability of Kirby-Judge to the molecular modeling domain, further illustrating the model’s flexibility and broader applicability.\n\nWeaknesses\n\n- The image understanding capabilities of Kirby-Judge are evaluated using the MLLM-as-a-Judge and VL-RewardBench benchmarks. However, more recent and comprehensive benchmarks such as MM-RewardBench (Yasunaga et al., 2025), TaskAnything, and JudgeAnything (Pu et al., 2025) are not considered in the analysis.\n    - Yasunaga et al., Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models, arXiv preprint arXiv:2502.14191, 2025\n    - Pu et al., Judge Anything: MLLM as a Judge Across Any Modality, arXiv preprint arXiv:2503.17489, 2025\n- The video understanding capabilities of Kirby-Judge are not evaluated. For this, the authors could consider benchmarks like JudgeAnything (Pu et al., 2025), which support multi-modality assessment, including video.\n- The evaluation of video generation and image editing capabilities is limited to the GenAI-Bench benchmark. A more comprehensive evaluation could include the recently proposed Q-Eval-100K (Zhang et al., 2025) and Video-Bench (Han et al., 2025).\n    - Zhang et al., Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content, CVPR 2025\n    - Han et al., Video-Bench: Human-Aligned Video Generation Benchmark, CVPR 2025\n- Several recently proposed MLLM-as-a-Judge models such as InternLM-XComposer2.5-Reward (Zang et al., 2025), CAREVL (Dai et al., 2025), and JudgeAnything (Pu et al., 2025) are not included in the experimental comparison, which limits the assessment of Kirby-Judge's competitiveness.\n    - Zang et al., Internlm-xcomposer2.5-reward: A simple yet effective multi-modal reward model, arXiv preprint arXiv:2501.12368, 2025\n    - Dai et al., From captions to rewards (CAREVL): Leveraging large language model experts for enhanced reward modeling in large vision-language models, arXiv preprint arXiv:2503.06260, 2025\n- The paper demonstrates that classical inference-time scaling techniques, such as majority voting over multiple reasoning paths, can further improve Kirby-Judge's performance. However, it does not explore whether Kirby-Judge itself can be used as a tool for inference-time scaling, e.g., selecting the best response from a set of diverse outputs generated by an MLLM. Evaluating Kirby-Judge in this context would provide valuable insight into its practical utility as a response selector.", "was_revised": true, "final_justification": "The authors provided detailed responses to my earlier comments. I appreciate the clarifications and the additional results the authors provided, which highlight the contribution of the paper better.", "rebuttal": "We thank the reviewer for their constructive feedback. Below we address each concern in detail. We look forward to any further discussions.\n\n\n>***Q1. [More recent benchmarks on image/video understanding]*** Have authors evaluated Kirby-Judge on newer image and video understanding benchmarks like MM-RewardBench or JudgeAnything to more comprehensively validate its performance?\n\n***A1.*** Thank you for your suggestion. In response, we have extended our evaluation to include results on two recent and comprehensive MLLM evaluation benchmarks: Multimodal RewardBench (Yasunaga et al., 2025), which focuses on image understanding, and JudgeAnything (Pu et al., 2025), which covers wide range of any-to-any tasks. JudgeAnything includes not only image/video/audio understanding (I → T, V → T, A → T) but also image/video/audio generation (T → I, T → V, T → A), image editing (I → I), image-to-video (I → V), and image-to-audio (I → A) tasks. The results are summarized in the table below.\n\n\n**Multimodal RewardBench**:\n\n| Model | Overall | Correctness | Preference | Knowledge | Math  | Coding | Safety | VQA   |\n|-|-|-|-|-|-|-|-|-|\n| GPT-4o | 0.715 | 0.626 | 0.690 | 0.720 | 0.676 | 0.621 | 0.748 | 0.872 |\n| Llama-3.2-90B-Vision-Instruct | 0.624 | 0.600 | 0.684 | 0.612 | 0.563 | 0.531 | 0.520 | 0.771 |\n| Llava-1.5-13B | 0.489 | 0.533 | 0.552 | 0.505 | 0.535 | 0.493 | 0.201 | 0.518 |\n||||||||||\n| Kirby-VL-7B | 0.686 | 0.620 | 0.618 | 0.630 | 0.677 | 0.550 | 0.732 | 0.837 |\n| Kirby-Omni-7B | 0.631 | 0.616 | 0.612 | 0.657 | 0.625 | 0.582 | 0.362 | 0.777 |\n\n**JudgeAnything**:\n\n| Model | I → T | V → T | A → T | T → I | T → V | T → A | I → I | I → V | I → A |\n|-|-|-|-|-|-|-|-|-|-|\n| GPT-4o | 73.0  | 58.0  | 69.5  | 52.5  | 56.0  | 26.0  | 49.0  | 42.0  | 68.0 |\n| Gemini-1.5-Pro   | 79.0  | 57.5  | 68.5  | 52.0  | 56.0  | 38.0  | 57.0  | 39.0  | 69.0 |\n| Gemini-2.0-Flash | 67.5  | 59.0  | 69.0  | 46.5  | 58.0  | 50.5  | 51.0  | 43.5  | 57.5 |\n|||||||||||\n| Qwen2.5-Omni-7B  | 50.5  | 42.9  | 51.5  | 36.0  | 31.5  | 39.5  | 29.5  | 40.0  | 32.5 |\n| Kirby-Omni-7B | 70.0  | 53.5  | 67.0  | 49.0  | 56.0  | 40.5  | 47.0  | 45.0  | 72.5 |\n\nAcross both benchmarks, Kirby-Judge achieves comparable performance to proprietary models while consistently outperforming existing open-source baselines. However, we would like to emphasize that our ultimate goal is not to surpass proprietary models on conventional benchmarks, but to enable capabilities that they currently lack. Specifically, Kirby-Judge is designed to be adaptable to new, underexplored domains, such as molecular tasks, where existing models cannot be directly applied—which we detail further in the following response (***A4***).\n\n&nbsp;\n\n\n>***Q2. [More recent benchmarks on video generation]*** Have authors evaluated Kirby-Judge on newer benchmarks like Q-Eval-100K and Video-Bench for video generation and image editing tasks?\n\n***A2.*** Thank you for your suggestion. We carefully considered the recent benchmarks you mentioned. However, to the best of our knowledge, the official GitHub (or Hugging Face) repository for Q-Eval-100K does not provide access to their test split, which is necessary for a fair and standardized evaluation. For Video-Bench, their evaluation framework involves using both MLLMs and LLMs simultaneously, which is not directly compatible with our evaluation protocol. \n\nInstead, the JudgeAnything results (as shown in ***A1***) show promising performances of Kirby-Omni-7B in text-to-video (T → V) and image-to-video (I → V) generation tasks. We currently believe GenAI-Bench (Table 3 in the paper) is the most widely used and representative benchmark for video generation and image editing. Our results show that Kirby-Judge is effective across a wide range of visual tasks, including video and image generation, image understanding, image editing, audio generation, and molecule understanding.\n\n\n&nbsp;\n\n\n\n>***Q3. [More recent baselines]*** Did authors consider including recently proposed MLLM-as-a-Judge models, such as InternLM-XComposer2.5-Reward or CAREVL, in your experimental comparison to more thoroughly assess the competitiveness of Kirby-Judge?\n\n***A3.*** Thank you for the suggestion. Recent models like CaReVL and InternLM-XComposer2.5-Reward (IXC-2.5-Reward) are primarily designed only for pairwise comparison and trained on large-scale visual-language pairs, whereas Kirby-Judge is built to generalize across broader evaluation formats, including single-score grading and batchwise ranking, trained with ~1K text data.\n\nImportantly, Kirby-Judge demonstrates strong generalization not only in evaluation format, but also across domains. While many existing judge models are limited to vision-language tasks, our work uniquely extends to molecular generation (potentially to other modalities like 3D space), where curated preference data is scarce. Despite these differences, Kirby-Judge remains competitive even when evaluated against recent models, as presented in our results below.\n\n| Model | VL-RewardBench (General) | VL-RewardBench (Hallucination) | VL-RewardBench (Reasoning) | MLLM-as-a-Judge (Pair w Tie, Ave.) | MLLM-as-a-Judge (Pair wo Tie, Ave.) |\n|-|-|-|-|-|-|\n| Qwen2-VL-7B | 31.6 | 19.1 | 51.1 | 0.365 | 0.329 |\n| IXC-2.5-Reward | 84.7 | 62.5 | 62.9 | - | - |\n| CaReVL (Qwen2-VL-based) | 74.2 | 66.3 | 56.3 | 0.538 | 0.677 |\n| Kirby-VL (Qwen2-VL-based) | 42.5 | 31.8 | 57.2 | 0.524 | 0.625 |\n\n&nbsp;\n\n>***Q4. [Practical utility of Kirby-Judge for response selection]*** Have you evaluated the practical utility of Kirby-Judge as a response selector at inference time, for instance by selecting the best answer from diverse model outputs?\n\n***A4.*** As described in ***Section 4 (Broader Impact) and Figure 4***, we have already demonstrated the use of Kirby-Judge as a best-of-N selector for inference-time scaling, particularly in the molecular domain. This shows that Kirby-Judge can effectively select the best response from a set of diverse outputs in practical scenarios. Furthermore, we demonstrated that Kirby-Judge can also serve as an off-the-shelf reward model for DPO training, where Kirby-Mol-Llama effectively guides Mol-Llama's preference optimization, highlighting its applicability to downstream training."}, {"review_id": "3xBNLmFVIO", "reviewer": "Reviewer_VVyF", "summary": "This paper introduces KIRBY-Judge, a reasoning-guided multimodal evaluator that uses minimal textual reasoning data to asses model outputs across various modalities like text, images, and videos. The key insight is that structured textual explanations capture generalizable decision-making patterns, which enables effective transfer to multimodal evaluation without extensive modality-specific training. Empirical results show that KIRBY-Judge achieves performance competitive with or better than existing methods, even in low-resource domains.", "questions": "- Could the authors elaborate more on the intuition of generalizing from textual reasoning data to multimodal judge capability?\n- How exactly was the seed training data curated?  Could the authors provide some examples of their data?  Also, the data is mainly synthetic - could the authors elaborate on how to ensure the synthetic data is high quality and not collapsed?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The method is solid.  The approach is modality-agnostic, and once trained, the KIRBY-Judge can be applied to diverse evaluation tasks (vision-language tasks, audio quality scoring, molecular structure, etc.)\n- Overall, good experiments and execution.  Evaluation is conducted across diverse tasks, such as Image Understanding, Image Editing, Video Generation, Audio Understanding, and Molecule Evaluator.\n\nWeakness\n- It would be better if experimental results (Table 1, 2, 3) include stronger baselines like latest GPT4-o, o3 and Gemini 2.5 Pro.\n- Misses some related work. For example, there are various multimodal judge or reward benchmark works worth discussing or evaluating on: video LM as judge (https://arxiv.org/abs/2503.05977) and Multimodal RewardBench (https://arxiv.org/abs/2502.14191), in addition to MLLM-as-a-Judge and VL-RewardBench, etc. discussed in the paper.", "was_revised": true, "final_justification": "Issues resolved", "rebuttal": "We thank the reviewer for their constructive feedback. Below we address each concern in detail. We look forward to any further discussions.\n\n> ***Q1. [Inclusion of latest baseline models]*** It would be better if the experimental results included stronger baselines such as the latest Gemini 2.5 Pro.\n\n***A1.*** Thank you for the suggestion. We have presented Gemini-2.5-Pro results in the table below. The latest Gemini-2.5-Pro model shows mixed superiority over previous versions. \n\n| Model             | MLLM-as-a-judge (Score, Ave.) | MLLM-as-a-judge (Pair w Tie, Ave.) | MLLM-as-a-judge (Pair w.o Tie, Ave.) | MLLM-as-a-judge (Batch, Ave.) $\\downarrow$ | VL-RewardBench (General) | VL-RewardBench (Hallu.) | VL-RewardBench (Reason.) | GenAI-Bench (Image) | GenAI-Bench (Edition) | GenAI-Bench (Video) |\n|-|-|-|-|-|-|-|-|-|-|-|\n| Gemini-2.5-Pro    | 0.390 | 0.556 | 0.668 | 0.512 | 44.28 | 49.13 | 53.01 | 47.55 | 65.51 | 50.33 |\n| Gemini-1.5/1.0-Pro | 0.304 | 0.509 | 0.615 | 0.432 | 50.80 | 72.50 | 64.20 | 44.67 | 55.93 | 46.21 |\n| Kirby-Omni-7B     | 0.306 | 0.532 | 0.650 | 0.425 | 47.01 | 42.72 | 61.08 | 38.15 | 46.73 | 37.10 |\n| Kirby-VL-7B       | 0.332 | 0.538 | 0.655 | 0.426 | 46.11 | 43.39 | 62.87 | 43.32 | 47.41 | 44.78 |\n\nHowever, we would like to emphasize that our ultimate goal is not to surpass proprietary models on conventional benchmarks, but to enable capabilities that they currently lack. Kirby-Judge is designed to be adaptable to new, underexplored domains, such as molecular tasks, where existing models cannot be directly applied—which we detail further in ***Section 4 (Broader Impact)***.\n\n&nbsp;\n\n> ***Q2. [Additional related work]*** Misses some related work. For example, there are various multimodal judge or reward benchmark works worth discussing or evaluating on: Multimodal RewardBench, in addition to MLLM-as-a-Judge and VL-RewardBench, etc. discussed in the paper.\n\n***A2.*** Thank you for the suggestion. We will include the mentioned works and discussion in our related work section. Additionally, we have extended our evaluation by including results on a recent MLLM evaluation benchmark, Multimodal RewardBench (Yasunaga et al., 2025) and JudgeAnything (Pu et al., 2025).\n\nMultimodal RewardBench (Yasunaga et al., 2025) focuses on image understanding, while JudgeAnything (Pu et al., 2025) covers a wide range of any-to-any tasks. JudgeAnything includes not only image/video/audio understanding (I → T, V → T, A → T) but also image/video/audio generation (T → I, T → V, T → A), image editing (I → I), image-to-video (I → V), and image-to-audio (I → A) tasks.\n\n**Multimodal RewardBench**:\n\n| Model | Overall | Correctness | Preference | Knowledge | Math  | Coding | Safety | VQA   |\n|-|-|-|-|-|-|-|-|-|\n| GPT-4o | 0.715 | 0.626 | 0.690 | 0.720 | 0.676 | 0.621 | 0.748 | 0.872 |\n| Llama-3.2-90B-Vision-Instruct | 0.624 | 0.600 | 0.684 | 0.612 | 0.563 | 0.531 | 0.520 | 0.771 |\n| Llava-1.5-13B | 0.489 | 0.533 | 0.552 | 0.505 | 0.535 | 0.493 | 0.201 | 0.518 |\n||||||||||\n| Kirby-VL-7B | 0.686 | 0.620 | 0.618 | 0.630 | 0.677 | 0.550 | 0.732 | 0.837 |\n| Kirby-Omni-7B | 0.631 | 0.616 | 0.612 | 0.657 | 0.625 | 0.582 | 0.362 | 0.777 |\n\n**JudgeAnything**:\n\n| Model | I → T | V → T | A → T | T → I | T → V | T → A | I → I | I → V | I → A |\n|-|-|-|-|-|-|-|-|-|-|\n| GPT-4o | 73.0  | 58.0  | 69.5  | 52.5  | 56.0  | 26.0  | 49.0  | 42.0  | 68.0 |\n| Gemini-1.5-Pro   | 79.0  | 57.5  | 68.5  | 52.0  | 56.0  | 38.0  | 57.0  | 39.0  | 69.0 |\n| Gemini-2.0-Flash | 67.5  | 59.0  | 69.0  | 46.5  | 58.0  | 50.5  | 51.0  | 43.5  | 57.5 |\n|||||||||||\n| Qwen2.5-Omni-7B  | 50.5  | 42.9  | 51.5  | 36.0  | 31.5  | 39.5  | 29.5  | 40.0  | 32.5 |\n| Kirby-Omni-7B | 70.0  | 53.5  | 67.0  | 49.0  | 56.0  | 40.5  | 47.0  | 45.0  | 72.5 |\n\n\n&nbsp;\n \n> ***Q3. [Intuition on multimodal generalization]*** Could the authors elaborate more on the intuition of generalizing from textual reasoning data to multimodal judge capability?\n\n***A3.*** Thank you for the question. Our intuition is motivated by two key lines of recent research:\n\n1. It has been shown that when multilingual LLMs are pretrained to encode shared representations across languages, fine-tuning on a specific task in one language often leads to **improved performance on the same task in other languages**, even without additional data. This demonstrates the potential for knowledge transfer and skills to generalize across different domains when the underlying representations are unified.\n\n2. Works such as s1 (Meunnighoff et al., 2025) and LIMO (Ye et al., 2025) have shown that pretrained LLMs embed rich knowledge and reasoning capabilities, which can be **\"elicited\" with only a small amount of additional reasoning supervision**. This pattern has also been observed in preference alignment research, such as LIMA (Zhou et al., 2023), where even a small set of high-quality instructions can unlock strong task performance in pretrained models.\n\nBased on these findings, our core intuition is that a well-pretrained multimodal LLM already possesses unified cross-modal representations and general reasoning ability. Therefore, with only a small amount of high-quality textual reasoning data, we are able to evoke a model to elicit multimodal reasoning-based judgment capabilities, without the need for large-scale, modality-specific annotations. Our work empirically verifies this hypothesis.\n\n&nbsp; \n\n> ***Q4. [Question on curated training dataset]*** How was the seed training data curated? Could the authors provide some concrete examples of the data? Given that the data is mainly synthetic, how do you ensure its high quality and avoid issues such as model collapse?\n\n***A4.*** Thank you for your question. Our seed training data was curated by applying a set of quality criteria, which we systematically investigated through various ablation studies ***(see Figure 2 and Section 2.2)***. These criteria proved effective for producing high-quality synthetic data without suffering from mode collapse in our experiments. To further address concerns about possible collapse or forgetting, we additionally evaluated our models on visual question answering (VQA) tasks unrelated to judgment, specifically TextVQA and OK-VQA. As shown below, Kirby-VL-7B exhibits only a slight decrease on TextVQA and even outperforms the base model on OK-VQA. \n\n| Model | TextVQA | OK-VQA |\n|-|-|-|\n| Qwen2.5-VL-7B   | 80.80   | 67.46  |\n| Kirby-VL-7B     | 80.31   | 72.10  |\n\nThese results suggest that our well-structured methodology does not cause model collapse or catastrophic forgetting. Also, we want to highlight that we have included the curated seed data in the supplementary material for your reference."}]

Author rebuttal:

Meta-review: This paper was reviewed by four knowledgeable reviewers. The reviewers raised concerns about ablation studies w.r.t base models [rdTF, 1xgQ], the effect of dataset size [rdTF], the performance on understanding tasks [rdTF], the selected benchmarks [zegN, VVyF] and baselines considered [VVyF, zegN]. The rebuttal adequately addressed the reviewers' concerns and introduced the requested ablation studies, provided evaluations on VQA tasks, analyzed the prediction robustness of the models, and showed results on weaker backbones and larger models. The rebuttal also incorporated the more recent benchmarks and comparisons suggested by the reviewers. After rebuttal and discussion, there is a clear consensus to accept the paper. The AC appreciates the efforts put in the rebuttal, which substantially strengthen the contribution, and agrees with the reviewers' recommendation. Therefore, the AC recommends to accept.
Decision: Accept (poster)