Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning
Authors: Hulingxiao He
Zijun Geng
Yuxin Peng
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Model
Fine-Grained Visual Recognition
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs

Abstract: Any entity in the visual world can be hierarchically grouped based on shared characteristics and mapped to fine-grained sub-categories. While Multi-modal Large Language Models (MLLMs) achieve strong performance on coarse-grained visual tasks, they often struggle with Fine-Grained Visual Recognition (FGVR). Adapting general-purpose MLLMs to FGVR typically requires large amounts of annotated data, which is costly to obtain, leaving a substantial performance gap compared to contrastive CLIP models dedicated for discriminative tasks. Moreover, MLLMs tend to overfit to seen sub-categories and generalize poorly to unseen ones. To address these challenges, we propose Fine-R1, an MLLM tailored for FGVR through an R1-style training framework: (1) Chain-of-Thought Supervised Fine-tuning, where we construct a high-quality FGVR CoT dataset with rationales of "visual analysis, candidate sub-categories, comparison, and  prediction”, transition the model into a strong open-world classifier; and (2) Triplet Augmented Policy Optimization, where Intra-class Augmentation mixes trajectories from anchor and positive images within the same category to improve robustness to intra-class variance, while Inter-class Augmentation  maximizes the response distinction conditioned on images across sub-categories to enhance discriminative ability. With only 4-shot training, Fine-R1 outperforms existing general MLLMs, reasoning MLLMs, and even contrastive CLIP models in identifying both seen and unseen sub-categories, showing promise in working in knowledge-intensive domains where gathering expert annotations for all sub-categories is arduous. Code is available at [https://github.com/PKU-ICST-MIPL/FineR1_ICLR2026](https://github.com/PKU-ICST-MIPL/FineR1\_ICLR2026).

Published as a conference paper at ICLR 2026
FINE-R1: MAKE MULTI-MODAL LLMs EXCEL IN VISUAL RECOGNITION BY CHAIN-OF THOUGHT REASONING
Hulingxiao He, Zijun Geng, Yuxin Peng"
Wangxuan Institute of Computer Technology, Peking University
hehulingxian pku. edu. en, gengzijjn222240633 com, pengyuxin@pku.ee edu, en
Closed Broliatioon
Open-wourd Evaluation
Unseen Sub-Category
Many-shot Samples
Qurstion:
the bia theplou??
Degformptes-1
Training
4t (1--1))
Esteting
Catngoey
Calegory
7hinking Process:
Disuul Asulysis) The showe amad hird perthed wine with dusrectio that andyyud idiestityy to gpote:
repprio Sarppess
Few-shot Saw-les
(Candiitate Subcategarieg jesg Bxved
of "ixhhe" sew mewhreee
Camposs Gobdiidds Korzn tarto beatt fambbed paid, cala body and yelkor whn sutchee...
Training
[Comariion) Penopeax Goldttcck ws Aneeiion Gakditich The Acerricaa otfich
beight nod locehaad
Fine-81
(ours)
yalon (Prsdlitioo) Exer en the dicel bet viua foauie, the andt tetti la mert liedy Ka Karapean Golditech.
hight
dhe Burapas Galddbich...
Catagory
Gategory N
toddiedh
Figure 1: Fine-R1 generates Chain-of-Thouglh (CoT) before producing the final fine-grained visual recognition (FGVR) answer. It utilizes CoT supervised fine-tuning (SFT) and Triplet Augmented Policy Optimization (TAPO), learning the reasoning process with only few-shot samples per cate- gory. In comparison to general and reasoning MLLMs, and contrastive CLIP models, Fine-RI R1 excels identifying both seen and unseen categories.
ABSTRACT
Any entity in the visual world can be hierarchically grouped based on shared char- acteristics and mapped to fine-grained sub-categories. While Multi-modal Large Language Models (MLLMs) achieve strong performance on coarse-grained vi- sual tasks, they often struggle with Fine-Grained Visual Recognition (FGVR). Adapting general-purpose MLLMs to FGVR typically requires large amounts of annotated data, which is costly to obtain, leaving a substantial performance gap compared to contrastive CLIP models dedicated for discriminative tasks. More- over, MLLMs tend to overfit to seen sub-categories and generalize poorly to un- seen ones. To address these challenges, we propose Fine-R1, an MLLM tailored for FGVR through an R -sstyle training framework: (1) Chain-of-Thought Super- vised Fine-tuning, where construct high-quality FGVR CoT dataset with rationales of "visual analysis, candidate sub-categories, comparison, and predic- tion", transition the model into strong open-world classifier; and (2) Triplet Aug- mented Policy Optimization, where Intra-class Augmentation mixes trajectories from anchor and positive images within the same category improve robustness to intra-class variance, while Inter-class Augmentation maximizes the response distinction conditioned on images across sub-categories to enhance discriminative ability. With only 4-shot training, Fine-RI outperforms existing general MLLMs, reasoning MLLMs, and even contrastive CLIP models in identifying both seen and unseen sub-categories, showing promise in working in Enowledge-intensive do- mains where gathering expert annotations for all sub-categories is arduous. Code is available http:///gthhbbccm////CCCCCSAA///
ICLR2202
"Corresponding author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
The visual world exhibits inherently fine fine-grained characteristics, which pose significant challenges for visual understanding. Objects are organized hierarchically according to shared traits and span a vast number of fine-grained categories (Zhang et al., 2024c). For example, the coarse-grand -aaase-r super-category bird can be further subdivided into thousands of fine-grained sub-categories such as Acadian Flycatcher, Great Crested Flycatcher, and Least Flycatcher. Moreover, new categories can emerge in real-world applications, requiring to identify unseen concepts (Geng et al.) 2020). According to the latest statistics from the International Omithologists' Union (IOC), as of 2024, 11,276 bird species have been identified worldwide, and the number continues to grow with new species discovered.
Recent advances in Multi-modal Large Language Models (MLLMs) have achieved impressive re- sults on general vision-language tasks such as image captioning and visual question answering (Liu itt al. 2024a bb. However, prior studies (Zhang et al., 2024e; Liu et al. 2024c He et al. (0255) reveal substantial drop in performance when MLLMS are applied to knowledge-intensive
fine-grained visual recognition (FGVR) task. FGVR, long- songg-ttnding challenge in computer vision, requires dis- tinguishing subtle differences among visually similar categories, such as animal species (Wah al.,
2011}, plant varieties (Nilsback & Zisserman] 2008), or specific models of cars (Krause et al. 2013) and aircraft (Maji et al. 2013). These tasks are difficult even for humans, as they demand extensive domain knowledge to resolve minimal inter-class variance and large intra-class variations. Notably, even state-of-the-arr generative MLLMS such as GPT-4 (Achiam et al. 2023) and GeminiPro (Team)
et al. 2023) underperform compared to contrastive CLIP models (Radlord et al. 2021: Zhai et al. dedicated for discriminative tasks.
2023
To improve FGVR capability, early studies have explored fine-tuning MLLMs with classification data (Zhang et al.) 2024e, et al. 2025 Shi et al. 2025b). While this can yield gains, transforming general-purpose MLLMS into fine-grained classifiers requires extensive labeled data, which is costly to obtain. Moreover, these models tend to overfit to seen categories during training, limiting their utility in real-world scenarios where recognition of novel concepts is crucial (Geng et al. 2020
To address these challenges, we propose a framework that enables MLLMs to deploy intrinsic knowl- edge for FGVR while generalizing effectively unseen categories with limited data. It comprises two key components: (1) two-stage training framework while chain-of ohain-of-thought supervised fine- tuning (CoT SFT) establishes foundational FGVR capabilities through knowledge-integrann soning chains, followed by reinforcement learning that optimizes the capability to deploy knowledge for FGVR via reward signals; and (2) a policy gradient algorithm named Triplet Augmented Policy Optimization (TAPO) designed to tackle the problem of high intra-class variance and low inter-class variance for FGVR. The key idea behind TAPO is to introduce implicit contrastive signals by a posi- tive and negative sample for the anchor image. By mixing trajectories from both anchor and positive image sampled from the same sub-category. it demonstrates improved robustness. By maximize two versions of policy, conditioned on either the anchor negative image sampled from the most similar sub-category, it encourages the model to distinguish visually-simila objects. Through this framework, we develop Fine-R1, an MILM that enhances FGVR guided by strong CoT's
rea-
Extensive experiments on six FGVR datasets under the few-shot base-to-new generalization setting yield three main findings: (1) State-of-the-art performance: Fine-RI achieves superior results in both closed-world and open-world settings, surpassing general MLLMS (e.g.. closed: +8.51 1% and open:
+23.75% over Qwen2.5-VL-7BB) reasoning MLLMs (e.g.. closed: +5.59% and open: +30.98% over DeepPerception- 7B), and even contrastive CLIP models (e.g., closed: +4.27% over SigLIP- L) dedicated for discriminative tasks. (2) Stronger generalization. Fine-R1-33 exhibits superior cross-domain generalization on unseen categories (e.g., +15.59% over SFT, +10 +10.28% over CL.S- RI. (Li et al., 2025), and + +10005 over No-Thinking Reinforcement Learning (No-Thinking RI.) (L et al. (025)). confirming that its improvements stem from enhanced knowledge deployment rather than memorization. (3) Broader benefits. Fine-RI provides more accurate answers to non- classification questions where object recognition is a prerequisite (e.g., +3.60% over Qwen2.5-VL- while preserving or even surpassing on general VQA tasks.
3B on
In summary, our contributions threefold: (1) We propose Fine-R1, an MLLM with enhanced ability to deploy intrinsic knowledge for FGVR on seen categorics while simultaneously demon- strating generalization unseen categories, merely with few-shot data available. To achieve this
2

---

Published as a conference paper at ICLR 2026
develop two-stage framework integrating CoT SFT with TAPO, dedicated for the problem of high intra-class and low inter-class variances. (2) Extensive experiments demonstrate the strong FGVR capability of Fine-R1 in both closed-world and open-world evaluation, and superior base-to- new category generalization performance. Notably, Fine-R1 surpasses MILMS with larger param- eters and reasoning ability, and even CLIP models dedicated for discriminative tasks. (3) Through a series of analyses, we show that both visual features extracted and knowledge about fine-grained sub-categories do not change substantially through our training, but Fine-RI does improve the de- ployment of this knowledge in the context of FGVR task.
RELATED WORK
MLLMs for FGVR. Recent efforts to adapt MLLMs for FGVR either fine-tune them with classi- fication data or design training-free approaches (Peng et al. 2025). Fine-tuning studies show that explicit object mentions in training data are crucial (Geigle et al. 2024 Zhang et al. (002e)). and that integrating sufficient classification-focuus focused data enables MLLMS bridge the gap between state-of- the-art classifiers while enhancing object-centric reasoning. Some works attribute underperformance to misalignment between visual objects and category names, addressing through attribute-based alignment (He et al. 2025) or interpretable data synthesis (Shi et al. 2025a). In contrast, training- free methods such as Sparse Attention Vectors exploit sparse attention activations for discriminative tasks MMitra et al. 2024a), but performance remains limited without large annotated datasets.
Reinforcement Learning. Reinforcement learning (RL) has shown strong potential in enhancing reasoning abilities of both LLMs and MLLMs by reducing reliance on large annotated datasets
(uue et al. 2024). In LLMs, the GPT series (Hurst et al., 2024: Achiam et al.] [023] Jaech et al.
2024) leveraged RL with human feedback (Ouyang al., 2022
optimize reasoning, excelling Yang et al., 2024 Luong
in mathematics (Cai et al.) [024 YYng et al. 2024, Shao et al.
2024
al. 2024) and coding tasks (Hui et al., [202 Zhang et al. [224
IT. DeepSeek RI-Zero (Guo
iet al. 2025) further demonstrated RL-only training for reasoning improvement. In MLLMs, series
of work (Liu et al. 20256 et al., 2025 Huang et al.) 2025| Ma al. 2025) advanced visual perception and inference through RL with verifiable rewards. Together, these works highlight RL's effectiveness in interactive visual-linguise
reasoning. While policy optimization has shown great potential in image classification task (Liu et al.) 2025b Ma et al.] (025) for MLLMs, how it can be optimized for solving the key challenge of fine-grained image classification task hasn't been investigated. Instead, we equip policy optimization with contrastive paradigms to make the model more robust to the high intra-class variance and discriminative to the low inter-class variance.
CoT Reasoning with MLLMs. Mhain-of-thought (CoT) reasoning provides explicit intermediate steps that connect a problem to its solution (Wei et al. (022b)) and such rationales have been shown to substantially improve LLM reasoning (Cheng et al. 2024 Fu et al. 2023b] [Wang et al.) 2023
Diao et al., 2023). In multimodal contexts, CoT reasoning is enabled through two complementary strategies: CoT prompting (Gao al. [024]] Mitra al.) 2024b Lu et al. 2023) and CoT SFT (Luo) et al. 2025 Xu et al. TTawakkr et al. 2025). These approaches empower MLLMs to tackle challenging tasks including visual math reasoning (Zhng et al.] 2024b) and embodied decision- making (Mu et al. 2023). CoT prompting is typically applied in zero-shot (Kojima et al. 2002) or few-shot (Zhang et al. 2023) settings, guiding models such as GPT-4o (Hurst et al. (024) and Claude 3.5 Sonnet (Anthropic 2024) to articulate reasoning steps before answering. In contrast,
CoT SFT relies on multimodal instruction tuning with datasets containing explicit reasoning traces, and its success hinges the quality of these examples. In this work, we adopt CoT SFT with AI- generated and human-verified samples. Rather than using generic "visual analysis and prediction" rationales, utilize the structured reasoning procedure of "visual analysis, candidate subcategories, comparison, and final prediction". ", eliciting the model to first propose candidate subcategories (the most likely categories base model confuses it for) and then utilize text knowledge to resolve this confusion by detailed comparison between candidates.
PRELIMINARIES
FGVR with MLLMs. Let us define an MLLM as a function fMLLM generating a text output y in the space T given image x in the space X' and text query g E T, SMULM X x T T. To
3

---

Published as a conference paper at ICLR 2026
Stage 1: Chain-of Thought Thooght Supervised Fine-tuning
SFT UpTupa
Thinking Process
American Goldfinch, Eurupeun Goldfinch
American Goldfiich X Eurupeun Goidfinch
What
the
80002 of
bird photo?
the
Amchor Image
Question
Candidate Sub-categories
Visual Analysis
Comparison
Prediction
Stage 2: Triplet Augmented Policy Optimization
Pollcy Update
Klyey
Minimise
Reference * Model
A
What nome the MILM
Fly
Group Computation
An,
Image
bind
Obyer
Verilier "European Goldfinch" Eo?
Question
-ig
lng
Capy
Positive Image
Regalerion Entropy
Og
What the
Hisal
photo?
MLLM
Klieser
Masiiiie
Negative Image
x
H (P
Question
Regalarizz Eatropy
Figure 2: Overview of the proposed two-stage training framework integrating CoT SFT and TAPO.
perform FGVR with MLLMs in the open-world setting, the query g contains prompt of the type "What is the name of the bird in the photo?". We let MLLM predict naturally on its original output space T without any constraint, and expect the output y to be a sub-category E T. In the case of closed-worldd setting, we have predefined list C of classes and we modify by specifying the set C via a multi-choice question. As consequence, the model is required to pick from the set of all candidate subcategories, with C.
FGVR with CLIP models. Let us define CLIP model as two mapping functions: fiext gen-
erating text embedding étext the space given text input fiext T E; fimage generat- ing image embedding éimage in the space & given an image I, fimage X + E. CLIP models can only be used closed-world setting where the label set known. Following (Radford et al. (2021), we use the prompt "a photo of a <cllas>>" as the text input t. The sub-category with the highest cosine similarity the image embedding Cinage is selected the predicted answer: orgmaz;ec Sim(eeva Cinuge).
4 METHODOLOGGY
4.1 OVERVIEW
this section, we introduce our Fine-RI model and the associated progressive two-stage frame- work. As shown in Figure this method begins with chain-of-though supervised fine-tuning (CoT SFT), which teaches the model to perform open-world FGVR in a "human-like" manner by SFT. Then, apply our proposed Triplet Augmented Policy Optimization (TAPO) to guide the model to explore potentially better thinking process that is robust to intra-class variance and discriminative with respect to inter-class variance.
4

---

Published as a conference paper at ICLR 2026
4.2 CHAIN-OF-THOUGHT SUPERVISED FINE-TUNING
The primary objective of the first stage training is to enhance models' open-world FGVR capabil- ities by training them on synthesized CoT reasoning data, as illustrated in Figure 2 This process endows the model with structured reasoning procedure of "visual analysis, candidate subcate- gories, comparison, and final prediction" by explicitly imitating human-like reasoning pathways. Such structured training lays the foundation for forming aoooledge-sssociation patterns that sup- port subsequent reinforcement learning (RL) optimization.
wprn-world CoT Data. For data construction, we sample one image per sub-category and employ Qwen2.5-VL-333 VL-32B (Bai et al. 2025), to generate open-world FGVR CoT data in two key stages: (1) Image-level Visual Concept Selection (Shi et al.] 2025b): Given image and its ground- truth sub- category, we first extract image-specific concepts that capture the connection between visual content and the subcategory. Specifically, we leverage the MILM's captioning ability to produce multiple descriptions of the same image, each emphasizing different visual attributes. By aggregating this diverse set of descriptions, we approximate the distribution of discriminative features and mitigate the incompleteness of individual captions. To refine these features, we further apply an information bottleneck strategy, retaining only the most relevant ones. This process explicitly transforms critical visual cues into textual representations, providing a richer foundation for reasoning than raw image inputs alone. See qualitative examples of the extracted visual concepts associated with each image in Appendix image-question pair to guide the MLLM in focusing on discriminative details for FGVR. Inspired by human cognition,
A (2) Structured CoT Prompt: The extracted visual concepts are concatenated with the
design structured CoT prompt that decomposes the reasoning process into clear stages: visual analysis, candidate subcategories, comparison, and final prediction. An example of the CoT prompt template is shown in Appendix To ensure data reliability, we design some ded- icated strategies, ultimately yielding a high-quality open-world FGVR CoT dataset containing 404 samples: (1) Multiple responses are sampled until the CoT leads to exactly matched subcategory. (2) We detect CoT with mixed language and manually correct them to English. (3) We manually check the predicted subcategory in the CoT rationales, and maintain the samples whose predictions are both included in the candidate subcategories and consistent with the ground truth.
B
CoT SFT. We fine-tune the model on the curated CoT dataset, enabling it to develop strong open- world FGVR capabilities. Through this process, the model learns to integrate domain knowledge when generating candidate subcategories and to conduct meticulous comparisons that lead to more accurate predictions. The resulting model provides a robust foundation for further optimization through RL in the subsequent training stage.
4.3 TRIPLET AUGMENTED POLICY OPTIMIZATION
Following CoT SFT, we further explore the use of Decoupled Clip and Dynamic Sampling Pol- icy Optimization (DAPO) (Yu et al. 2025) to enhance the model's open- world FGVR capabilities, building upon the structured reasoning foundation. DAPO, a representative successor to GRPO (Shao et al. 2024), introduces several improvements such Clip-Higher, Dynamic Sampling, and Token-LLeve Policy Gradient Loss. To address the unique challenges of FGVR, we propose Triplet Augmented Policy Optimization (TAPO). The central idea is to encourage the policy to remain discriminative under low inter-class variance while maintaining robustness under high intra-class variance when predicting the final answer. Specifically, we augment each anchor image x with positive image Ipos from the same subcategory and a negative image Ineg from the most visually similar but distinct subcategory. This forms triplets T (z, Pops Fneg). supporting both intra-class and inter-class augmentation.
Intra-class Augmentation. To improve robustness against substantial intra-class variation, we introduce Intra-class Augmentation, a strategy designed to enrich sample diversity within target classes. Inspired by (Liu et al., 2025a), IntraClassAug employs hybrid sampling approach that leverages predicted answers from both anchor images and their positive counterparts 2pos This approach captures a broader range of intra-class variants. Concretely, for each input pair (z. (2.g). we randomly sample positive image Topos from the same category. As illustrated in Figure 2 the old policy Tesa generates two sets of rollouts: n, responses conditioned on the anchor (x, q) and na responses conditioned on the positive (I 9). All rollouts are then aggregated into single pool
5

---

Published as a conference paper at ICLR 2026
for reward computation:
frr f = (r(z, 9, 0,0)) U fr ok) (
(I)
Importantly, the policy update step remains conditioned only the anchor (2, q). promoting more effective exploration.
This design yields main advantages for FGVR: (I) Intra-class diversity modeling: Positive trajectories from different images within the same subcategory introduce varied visual perspectives, helping the policy better capture subtle within-class variations. (2) Discriminative guidance: When anchor and positive images yield different predictions for the same query, discrepancies in rewards provide informative signals that encourage the model to focus on fine- anne-griied, image-specific cues, thereby strengthening category-level distinctions.
Inter-class Augmentation. To address the challenge of low inter-class variance, we propose Inter- class Augmentation, which encourages the policy to generate distinct responses for visually similar images from different subcategories. To quantify whether a model can effectively distinguish be- tween matched and mismatched image-category pairs, we define the ratio:
To (o (....
mnter (0)
(2)
To ( q. Kncg)
where is generated sequence of tokens, q is the question and x. is the anchor image pos- itive image Tipos This ratio measures how much the model's output distribution changes when the input image replaced by a near-neighbor from another sub-category. higher ratio indicates that the model assigns significantly lower probability to the correct output under the negative image, suggesting that it has learned leverage category-specific diseriminative features. Conversely, low ratio suggests that predictions remain largely unchanged, even when informative features are removed and misleading ones introduced. implying difficulty in localizing fine-graine cues. Thus, for well-trained model 0, we expect g ( to be high.
Inspired by (Wang et al. 2025j, we introduce an additional loss term into the DAPO objective by maximizing the KL divergence between the output distribution conditioned on the anchor/positive image and that conditioned on the negative image:
[KL [Tel ا me (ollgx( Te( q, Fneg)]
(3)
Combining intra-class and inter-class augmentation, the resulting objective is:
Ty ti Jacl
(0) F. (. ((..))pp.(o)]]] ~ (ot) mo (1912p))))
21 +02 402 [0s1
1=1
min (r.,(() clip (rr,, q, +e) A DDLL(TTT with < [o: | is included(e o:)}| <m +
[To|
7/1 H [T]]
77 (T
(4)
where is the weight for the KL-divergence term [ (inler()) log g ( fol- lowing (Hershey & Olsen 2007). Here, indexes the i-th rollout response. Since the KL divergence is unbounded, we adopt the double entropy regularization strategy from (Wang et al., 2025) to con- strain both na and a thereby encouraging stable and low-entropy distributions. The entropies are log To Fneg). is is_included(a, o;) checks whether the ground truth is contained in the response. ?/i and 7/2 are hyperparameters used to weight the corresponding loss terms.
defined H[TT]] = log Na (olo, x.). H|TT Reg
EXPERIMENTS
5.1 EXPERIMENT SETTINGS
Datasets. We conduct experiments on several popular FGVR datasets that include CaltechUCSD
Bird-200 /Wah al. 2011), Stanford Car-196 (Krause al. 2013), Stanford Dog- 120 (Krause al.

---

Published as a conference paper at ICLR 2026
Table 1: Closed-world FGVR evaluations in terms of accuracy (%). The best results are bolded and the second best results are underlined in all following tables. All results are averaged with 3 trials.
Seen Categories
Unseen Categories
Models
Avg.
Air.
Bird
Car
Dog
Flower
Pet
Avg.
Air.
Bird
Car
Dog Flower
Pet
Avg.
CLIP Models
CLIP-L
47.95 73.96
80.50
77.12
87.59
95.45
77.10
45.68
62.35
79.81
73.58
57.24
89.22
67.98
72.54
EVA-G
40.96
81.37
90.06
76.02
81.70
94.21
77.39
45.83
63.39
87.06
76.66
54.83
89.82
69.60
73.49
SigL.P-LL
67.08
85.10
96.03
86.18
97.59
98.02
88.33
69.35
74.17
92.97
84.44
69.07
93.24
80.54
84.44
SigLIP2-LL
43.36
71.26
93.43
78.51
89.05
92.14
77.96
40.87
58.03
90.00
77.15
61.39
93.03
70.08
74.02
General MLLMs
Idefics2-8B
49.90
52.37
90.85
58.24
82.10
81.85
69.22
48.53
40.20
84.67
44.79
60.54
83.86
60.43
64.83
Idefess-LLLMMA-BBB
43.31
34.51
75.18
47.50
65.12
72.98
56.43
48.38
37.47
70.59
43,80
55,30
68.59
54.02
55.23
L.aVA-vI.6-mistra
7B
49.60
55.96
71.02
45.34
62,86
62.59
57.90
47.71
50.32
67.62
37.20
46.16
65.71
52.45
55.17
7B
32.07
54.81
71.38
70.72
73.83
76.52
63.22
30.43
52.92
67.21
53.78
48.94
66.31
53.27
58.24
V1..521.2.5.BB
36.71
65.23
63.40
53.70
65.39
75.46
59.98
40.57
62.74
63.53
42.62
46.35
60:21
52.67
56.33
Intem VL2.544
38.31 29.54
51.44
34.65
61.44
51.01
44.40
39.29
31.29
44.95
31.99
35.83
54.25
39.60
42.00
Intem
46,70 53.14
62.17
54,26
68.25
76.65
60.20
45.38
47.12
61.15
48.82
47.95
62.83
52.21
56,20
Qwen2-VL.-22
66.18 60.15
94.28
64.67
91.93
85.39
77.10
65.51
44,92
87.89
58.60
50.21
79.84
64.50
70.80
Qwen2-VL-7B
78.27 67.41 94.60
71.70
93.84
91.04
82.81
79.86
56.25
89.63
66.16
67.47
78.30
72.95
77.88
Qwen2.5-VL-33
64.24
65.40
86.70
70.51
94.24
83.46
77.43
68.29
58.98
80.65
67.10
68.74
87.88
71.94
74.68
Qwen2.5-VL-7B
74.28
70.54
90.75
80.19
96.20
91.91
83.98
71.60
66.29
84.02
77.54
65.63
93.44
76.42
80.20
Reasoning MLLMs
DeepPerception-75
83.52
94.89
80.40
97.05
91.91
86.99
89.72
77.85
72.80
87.41
79.24
83.12
Fine-R1-3B (ours)
76.87
86.79
92.14
87.85
96.25
93.89
88.97
75.73
79.10
87.40
80.93
73.93
91.36
81.41
85.19
Fine-R1-7B (ours)
90.50
94.03
97.22
96.05
91.71
87.54
87.99
96.92
85.70
2013). Flower-102 (Nilsback & Zisserman 2008), Oxford-IIIT Pet-37 (Parkhi et al., 2012). and FGVC-Aircraft (Maji al. 2013). To facilitate evaluation base-to-new category generalization, we randomly select 60% of the categories as seen categories and the remaining 40% as unseen categories for each dataset. We train a unified model for all six datasets with 4-shot data per seen category, and do evaluation on test sets of the seen and unseen categories, respectively.
Evaluation Metries. We define success on single example as whether the ground-truth choice is included in the MLLM generation. We report the success rate of all test examples as the accuracy in the doseddworld setting. Since evaluating models the open-world world setting presents additional challenges, as predictions may differ in granularity (e.g., Boeing 737 vs. Boeing 737-200), or ground truth may include redundancy for distinguishing from others (e.g.. "Coupe 2012" in Audi A5 Coupe 2012 and Audi S5 Coupe 2012), we use two complementary metrics: (1) text inclusion (Zhang) cet al., 2024c),, evaluating strict string matching. (2) relative semantic similarity between the text embeddings of predictions and ground truth calculated by the SigLIP (Zhai al. (023) text encoder. Instead of using the similarity as reward directly, we use the similarity between the super- cuper-category and the ground truth subcategory the standard to calculate the relative value. Formally, the relative semantic similarity S:S is expressed as:
S m0 Sim(e.c")- Sem((,.
(5)
1 I-Sim(E,c")
where é and c* denote the super-category. predicted and ground truth subcategory, respectively. We defer prompts for evaluations, implementation details, and compared models to Appendix B. and D respectively.
5.2 MAIN RESULTS
Closed-world Evaluation. As shown in Table I although trained solely on worn-world FGVR tasks, Fine-RI achieves substantial performance gains in the closed-word world setting with the guidance of COT. This validates our hypothesis that a human-inspired reasoning process enables Fine-RI I to better distinguish visually similar sub-categories. On seen categories, Fine-R1-7B reaches an
7

---

Published as a conference paper at ICLR 2026
Table Open-world FGVR evaluations in terms of relative semantic similarity (%). All results are averaged with 3 trials.
Seen Categories
Unseen Categories
Models
Avg.
Air.
Bird
Car
Dog
Flower
Pet
Avg.
Air.
Bird
Car
Dog
Flower
Pet
Avg.
General MLLMs
Idefics2-8B
3.64
19.68
19.54
10.03
14.94
2.50
11.72
3.69
15:35
20.81
10.19
5.84
2.47
9.73
10.72
idefics-LLLMMA3-BB
9.66
27.72
22.96
35.39
40.92
20.17
26.14
7.09
24.93
22.08
27.67
21.99
24.84
21.43
23.79
LLLVA-v1.6-mistra 7B
2.73
16.02
21.75
19.35
10.33
12.47
13.78
2.51
17.40
23.24
18.16
8.20
10.62
13.36
13.57
LLaVA- Onevi-oniiii
9.90
31.74
21.35
30.13
45.24
17.87
26.04
7.47
29.81
19.56
25.44
16.53
19.35
19.69
22.87
V.2.5VL22.52B
7.26
21.32
27.29
25.87
23.08
26.56
21.90
5.22
20.12
24.73
24.84
12.59
24.55
18.68
20.29
In..m
14.71
25.41
32.19
33.13
23.84
28.19
26.25
12.64
23,91
29.11
30.80
13.10
26.32
22.65
24.45
Inter VL2.582
23.76
28.44
30.08
27.11
21.73
29.03
26.69
20.34
24.38
27.55
24.73
13.56
26.23
22.80
24.75
Qwen2-VL-22 VL-2B
47.49
48.72
52.95
51.22
66.56
19.88
47,80
48.88
39.32
49.33
45.16
33.87
22.43
39.83
43.82
Dwwn2-VVL.70
56.47
56.46
55.31
67.03
75.02
36.97
57.88
52.75
41.17
52.46
61.01
32,57
30.74
45.12
51.50
Qwen2.5-V1.-3 VL.-BB
56.98
66.77
52.49
65.12
68.96
26.27
56.10
52.50
48.09
51.75
59.02
34.19
28.78
45.72
50.91
Owen2.5-V1-778
65.97
56.94
59.02
62.61
35.59
56.50
45.26
55.39
54.59
32.74
36.98
45.60
51.05
Reasoning MLLMs
DeepPenception-B8
44.24 47.63
54.14
49.16
47.30
40.90
47.23
40.03
37.10
52.27
49.05
28.38
35.57
40.40
43.82
Fine-R1-3B (ours)
54.36
78.90
82.46
78.21
64.55
81.60
73.35
46.43
58.00
74.60
70.08
39.54
79.11
61.29
67.32
Fine-R1-7B (ours)
73.53 86.12
90.73
80.71
81.46
83.14
82.62
65.21
60.69
82.19
70.97
40.74
82.04
66.97
74.80
curacy of 91.71%, outperforming all baselines of comparable scale (e.g.. +7.73% over Qwen2.5- VL-7B) and even surpassing strong contrastive CLIP models (e.g., +3.38% over SigLIP-L). For un- seen categories, achieves 85.70% accuracy, yielding even larger improvements (e.g..+9.288) +9.28% over Qwen2.5-VL-7B and 15.16% over SigLIP-L). These results further demonstrate that Fine-RI not only leverages knowledge effectively for FGVR but also generalizes well to novel categories. Per- formance comparison in terms of text inclusion is presented in Appendix E
worldworld Evaluation. As shown in Table Fine-RI-7B establishes new state-of-the-an per- formance with only 4-shot training samples per sub-category, achieving 74.80% relative semantic similarity on average. This represents a substantial improvement of 23.75% over Qwen2.5-VL. Wovn2.5-VLL7BB. Notably, Fine-R1 still demonstrates strong base-to-new category generalization in the open-world setting. The superior performance in both glosed-world and open-world FGVR scenarios demon- strates that CoT guidance provides two key advantages: (1) enhances the model's ability to discern subtle discriminative features among visually similar candidates, and (2) it enables more effective integration of inherent knowledge identify candidates that accurately capture the ground truth sub-category. A more detailed analysis of the performance gain is presented in Section 5.4
5.3 ABLATION STUDY
We conduct several ablation studies to verify the effectiveness of our design. For the ablation study, we use Qwen2.5-VL-3B and Fine-R1-3B by default.
Training Methods. Figure 3a compares different training methods in closed-world setting. SFT greatly improves accuracy on seen categories (+3.98%) but severely harms unseen categories (- 6. 6.12%, showing overfitting and poor generalization. CLS-RL (Li et al., 2025) alone reduces the unseen drop but still underperforms the zero-shot baseline (71.13% vs. 71.94%). Moreover, degrades the accuracy by 5.92% on seen categories models with limited capabilities struggle to generate high-quality CoT for RL. Though No-Thinking-RL (Li et al. 2025) achieves performance gains seen categories, it still lags behind SFT (80.86% 31.41%). Our two-stage framework combines the strengths of SFT and RL, significantly outperforming SFT on seen categories (+7.56%) while significantly surpassing No- No-Thinking- RL on unseen categories (+10.05%).
Inference Strategies. We investigate two inference strategies, including CoT prompting, and In- Context Learning (ICL) in the closed-workd evaluation. For CoT prompting, leverage the zero- shot CoT prompting technique by adding "let's think step by step" at the end the prompt (Wei) et al. 2022a: Kojima et al. 2022). For CLIP-like models, we additionally add prompt ensem-
8

---

Published as a conference paper at ICLR 2026
30.30%
50%
5uen
Unseen
71.300
Unseon
51.4550
31485
11.39%
13.62%
N4.00
Overall
75.12%
Dverall
Over
50
60
70
80
90
100
Reutive Semanii Simianiy
75
Accuracy (96)
CAT prengstig
IMAO
(ou))
(a) Training methods.
(b) Inference strategies.
(c) Key components of Fine-RI.
Figure 3: Ablation study training methods, inference strategies, and key components of Fine-R1.
bling results for SigLIP using 80 prompt templates from ImageNet dataset. As shown in Figure 3b compared to the baseline (74.68%), direct CoT prompting without training for CoT reasoning (74.7%%) has limited impact on FGVR, which is also affirmed in (Zhang et al.] 2022d). For ICL, we randomly sample one demonstration for each candidate once it belongs to seen categories (i.e., occurs in the training data). However, since we can only retrieve demonstrations for seen categories in the candidates, the context may introduce bias. The results show that Fine-R1-3B surpasses Qwen2.5-VI. VL-3BB with ICL by 17.90%, strengthen the effectiveness of Fine-R1. worth noting that Fine-R1 outperforms CLiP--ike models even with prompt ensembling optimization.
Key Components. As illustrated in Figure 3c we evaluate the effectiveness of each key compo- nent of the training framework. CoT SFT improves relative semantic similarity by 13.30%, laying the foundation for high-quality reasoning. Utilizing DAPO after CoT SFT brings a performance gain of 1.60%, demonstrating effective integration of domain knowledge. Adding Intra-class and Inter-class Augmentation individually both outperform CoT SFT + DAPO, and combining them to- gether achieves the best results of 67.32%, confirming that Fine-R1 benefits from complementary augmentations.
control n1 + n2 10, and change the anchor-to- totoo-tt-positive rollout ratio n1 M2.. Since n1 : 72 = 1 achieves the best performance, confirm- ing that the performance gain of intra-class augmentation is from increasing the diversity of rollouts instead of generating more rollouts (i.e., using x merely to generate more rollouts).
Anchor-to- Possiiive Ratio. As illustrated in Table 3a
CoT Generalization. We construct more CoT data from the same model Qwen2.5-VL-32BB, and evaluate SSyelative on unseen categories to test scaling behavior when using larger CoT data. As shown in Table 3b] the model performance increases with the number of CoT data, confirming that the model does not overfit to synthetic patterns in the limited synthetic set. Additionally, we can observe that quality out-weights quantity of CoT data, alleviating the cost to construct a large scale of data for COT SFT, showing the high data efficiency.
Cross-model Evaluation. We conduct experiments on the Qwen2-VL-2B -n-stt-tt (Wang et al.) 2024) model to further enhance the evidence of the effectiveness. Architecturally, Qwen2.5-VL. difters from Qwen2-VL. through the use of an updated vision encoder and language model and a new iision-languse fusion module. As shown in Tablc3c| the consistent gains compared to CoT SFT+DAPO again shows the generality of TAPO to different model architectures.
We defer the general capability analysis and qualitative examples to Appendix F and G
5.4 PERFORMANCE GAIN ANALYSIS
To better understand why Fine-RI outperforms baselines FGVR, we propose three hypotheses inspired by the essential capabilities of MILMs for fine- fine-graines -reiine recognition (He et al.) 2025). HI: Fine-RI improves the extraction of visual cues needed to distinguish objects; H2: Fine-RI funda- mentally enhances knowledge of subcategories; H3: Fine-RI improves the ability to deploy existing subcategory knowledge in FGVR tasks. We analyze each hypothesis below.

---

Published as a conference paper at ICLR 2026
Table Ablation study on n2; #CoTs in stage 1, and cross-model evaluation.
Table 4: Left: Linear probing of visual features and differences. Right: Differences in cosine similarities between species pairs belonging to the same and different genus.
(a) 122:
(b) #CoTs.
(c) Cross-model evalua- tion on Qwen2-VLLe VL-2B.
na
Avg.
#CoTs
Avg.
Embedding Similarity
10
59.47
Method
Avg.
Models
Probing (%)
8
59.21
404
58.66
A
Zero-shot
2 59.33
804
62.06
CoT SFT DAPO
62.32
Qwen2.5-VI..c VL.3
84.26
0.0088
-0.3872 0.6993
59.39
1199
62.64
COT TAPO
64.65
Fine-RI (ours)
85.00
0.0531
HI: Fine-R1 extracts better visual cues. We perform linear probing on image features. Specifi- cally, we retrieve image token embeddings from the residual stream of the final LLM layer, apply mean pooling, and train a linear classifier on CUB-200 training set with batch size 512, learning rate le-4, Adam optimizer, and 500 training epochs. The best test performance during training is reported. Results in Table + show negligible differences between Fine-R1 I and the base model, indicating that Fine-RI does not produce more effective visual embeddings for FGVR.
H2: Fine-RI R1 encodes more subcategory knowledge. We evaluate whether Fine-RI reserves more knowledge about sub-categories, like taxonomy-aware relationships among bird species. For each target species, we compute similarities with one species from the same genus and with four from different genus, then take the difference between intra-genus and inter-genus similarities. If this difference is larger for Fine-R1, 1 it would suggest stronger taxonomy-awan encoding. However, Table[sshows little difference between Fine-R1 and @0we2..5-VI.33. suggesting that Fine-RI does not fundamentally alter subcategory knowledge.
H3: Fine-R1 better deploys subcate- gory knowledge. We study the distin- guishability of positive image-category pairs from negative ones, and exam- ine whether this distinction is reflected in the holistic representation of the in- put context (e.g., "<image> Is the bird species fcorrectiincorreet name] Specifically, we use the last hidden state of the final L.M layer as the sum- mary representation of the full context, encompassing both the image and ques- tion. We then test whether inputs con- taining positive pairs can be separated from those containing negative pairs through PCA. Figure(4present the first two principal components of input representations from Qwen2.5-VL-3B and Fine-R1, with positive and negative pairs color-coded. The results show that positive and negative pairs are more linearly separable in Fine-RI representations, suggesting that Fine-RI are better at deploying fine-grained subcategory knowledge, achieving genuinely different representational states compared to Qwen2.5- VL-3B when the task context requires utilizing knowledge for FGVR
40
20
20
20
-10
-10 10-30020-11
20-10
10 20 30
-40 -20
66
PCI
PC
Poettive
Negative
Posttive
Negetive
Figure 4: PCA projections of the last hidden state rep- resentations of inputs containing positive and negative image-category pairs, extracted from Qwen2.5-VL-3B and Fine-R1.
CONCLUSION
In this work, we tackle the challenges of data inefficiency and base-to-new generalization in FGVR tasks by proposing a framework that strengthens the ability to leverage intrinsic knowledge through COT SIT and TAPO. By augmenting policy optimization with triplets consisting of an anchor image, positive image, and negative image drawn from the same different subcategories, our method effectively addresses the issues of high intra-class variance and low inter-class variance. By guiding MLLMs to generate CoTs in "human "iuman-likee manner, Fine-RI R1 achieves ofte-of-theear results in both closed-world and open-world evaluations, outperforming contrastive CLIP models dedicated for discriminative tasks, thereby paving the way for more fine-grained visual applications.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTE
This work was supported by the grants from the National Natural Science Foundation of China (62525201, 62132001, 62432001) and Beijing Natural Science Foundation (L.247006, L.257005). This work was partially supported by PKU Kunpeng&Ascend Center of Excellence.
REPRODUCIBILIIY STATEMENT
The main implementations of our proposed models are Section 4.2tion/422an and 4.3 The evaluation metrics
presented in Section 5.1 The prompts for evaluation and implementation details are in Appendix and C respectively.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint drXiv:2303.0877, 2023.
Anthropic. Grok-1.5 vision preview. httpp::/ww..tttieeeeeeeemm
2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Siho Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprint 2rXiv:2403.172 2024.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024.
Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. ChainLM: Empowering large lan- guage models with improved chain-of of-thought prompting. In COLING, pp. 2969-29999999 2999, 2024.
Shizhe Diao, Pengeheng Wang, Yong Lin, and Tong Zhang. Active prompting with thain-of-thhuggee for large language models. arXiv:2302.122 122466 2023.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang. Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.1 13394, 2023a.
Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, and Tushar Khot. Chain-of-thought hub: A continuous effort to measure large language models' reasoning performance. arXiv:2305.17306 2023b.
Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yunhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, et al. Cantor: Inspiring multimodal chain-of- thought of mllm. 4rXiv:2404.16033, 2024.
Gregor Geigle, Radu Timofte, and Goran Glavaš. African european swallow? benchmarking large vision-language models for fine- greingraiddd object classification. arXiv preprint arXiv:2406. 14496, 2024.
Chuanxing Geng. Sheng-jun jun Huang, and Songcan Chen. Recent advances in open recognition: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(10))3614-331) 2020.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501. 25..2.2419.... 12948, 2025.

---

Published as a conference paper at ICLR 2026
Hulingxiao He, Geng Li, Zijun Geng, Jinglin Xu, and Yuxin Peng. Analyzing and boosting the power of fine fine-graing grained visual recognition for multi-modal large language models. arXiv preprint arXiv:2501.15140, 2025.
John Hershey and Peder A Olsen. Approximating the kullback leibler divergence between gaus- sian mixture models. In 2007 IEEE International Conference Acoustics, Speech and Signal Processing-ICASSPP volume 4, pp. IV-317. IEEE, 2007.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.067453 2025.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint urXiv:2412.16722 2024.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. pp. 22199-22213, 2022.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCVW, pp. 554 561, 2013.
Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and bet- ter understanding vision-language models: insights and future directions. arXiv preprint arXiv:2408. /2637, 2024a.
Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? arXiv preprint arXiv:2405.02246, 2024b.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03:2 03326, 2024.
Bohao Li, Rui Wang, Guangzhi Wang. Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Bench- marking multimodal Ilms with generative comprehension. arXiv preprint arXiv:2307.66125, 2023a.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICMI., pp. 19730-19742. PMLR, 2023b.
Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, Haoquan Zhang, Wang Bill Zhu, and Kaipeng Zhang. Think think: A study of explicit thinking in rule-based visual reinforcement fine- tuning. arXiv preprint arXiv:2503. 16188, 2025.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 26296-26306, 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
Huan Liu, Lingyu Xiao, Jiangjiang Liu, Xiaofan Li, Ze Feng, Sen Yang, and Jingdong Wang. Revisiting milms: An in-depth analysis of image classification abilities. arXiv preprint 1rXiv:2412116411 2024c.
12

---

Published as a conference paper at ICLR 2026
Xiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shich. Noisyrollout: Reinforcing visual reasoning with data augmentation. arXiv preprint arXhv:2504.13055 2025a.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp. 216-233. Springer, 2024d.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint MxXiv2553.007883 2025b.
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language mod- els. pp. 43447-43478 2023.
Ruilin Luo, Zhuofan Zheng, Yifan Wang, Yiyao Yu, Xinzhe Ni, Zicheng Lin, Jin Zeng, and Yujiu Yang. Ursa: Understanding and verifying chain-of-thought reasoning in multimodal mathematics. arXiv:2501.04686 2025.
Trung Quoe Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Rea- soning with reinforced fine-tuning, arXiv preprint TrXiv:2401.08967, 2024.
Xinyu Ma. Ziyang Ding, Zhicong Luo, Chi Chen, Zonghao Guo, Derek F Wong, Xiaoyi Feng, and Maosong Sun. Deepperception: Advancing rl-like cognitive visual perception in milms for knowledge-intensive visual grounding. arXiv preprint arXiv:2503.12799 2025.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint 1rxivv1306.5151, 2013.
Chancharik Mitra, Brandon Huang, Tianning Chai, Zhiqiu Lin, Assaf Arbelle, Rogerio Feris, Leonid Karlinsky, Trevor Darrell, Deva Ramanan, and Roei Herzig. Sparse attention vectors: Gener- ative multimodal model features are discriminative vision-language classifiers. arXiv preprint 1XXi:24422.00142. 2024a.
Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig, Compositional chain-of- thought prompting for large multimodal models. pp. 14420-14431, 2024b.
Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. pp. 25081-25094, 2023.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In ICVGIP, pp. 722-729. IEEE, 2008
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 17730-27744 2022.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, pp. 3498 -508. IEEE, 2012.
Yuxin Peng, Zishuo Wang, Geng Li, Xiangtian Zheng, Sibo Yin, and Hulingxiao He. A survey on fine-grad fine-grained multimodal large language models. Authorea Preprints, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pp.
PMLR, 2021.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseckmath: Pushing the limits mathematical reasoning in open language models. arXiv preprint arXiv:2402.03330 03300, 2024.
13

---

Published as a conference paper at ICLR 2026
Yucheng Shi, Quanzheng Li, Jin Sun, Xiang Li, and Ninghao Liu. Enhancing cognition and explainability of multimodal foundation models with self-syythesized data. arXiv preprint 1rXiv:2502.14044,, 2025a.
Yucheng Shi, Quanzheng Li. Jin Sun, Xiang Li. and Ninghao Liu. Enhancing cognition and explainability of multimodal foundation models with sytthesnteee data. arXiv preprint arXiv:2502. 14044, 2025b.
Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang. and Yue Cao. Eva-clip: Improved training techniques for clip scale. arXiv preprint arXiv: 2023.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. 2023.
Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethink- ing step-by-step visual reasoning in llms. arXiv:2501.06/88 2025.
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786 2025.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint (rXix:2409.12199 2024.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery. and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. 2023.
Zhenhailong Wang, Xuehang Guo, Sofia Stoica, Haiyang Xu, Hongru Wang, Hyeonjeong Ha, Xiusi Chen, Yangyi Chen, Ming Yan, Fei Huang, et al. Perception-aware policy optimization for mul- timodal reasoning. arXiv preprint arXiv:2507.06444 00300000444, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et Chain-of-thought prompting elicits reasoning in large language models. Advances in 2022a.
neural information processing systems,
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, al. Chain-of-thought prompting elicits reasoning in large language models. pp. 24824 24837, 2022b.
Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-ol: Let vision language models reason step-by-step. 1. 2024.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvemment arXiv preprint arXiv:2409. 12122, 2024.
Huaiyuan Ying. Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong. Kuikun Liu, Ziyi Wang, et al. Intemll-math: Open math large language models toward verifiable reasoning. arXiv preprint arXiv:2402.06332 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu. Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. /4476, 2025.
14

---

Published as a conference paper at ICLR 2026
Sheng Yue, Yongheng Deng, Guanbo Wang, Ju Ren, and Yaoxue Zhang. Federated offline reinforce- ment learning with proximal policy evaluation. Chinese Journal of Electronics, 33(6):1360-1377 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
Kechi Zhang, Ge Li, Yihong Dong, Jingjing Xu, Jun Zhang, Jing Su, Yongfei Liu, and Zhi Jin. Codedpo: Aligning code models with self generated and verified source code. arXiv preprint arXiv:2410.05605, 2024a.
Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Yichi Zhang, Ziyu Guo, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, al. Mavis: Mathematical visual instruction tuning. arXiv:2407.08733 2024b.
Ruru Zhang, E Haihong, Lifei Yuan, Yanhui Wang, Lifei Wang, and Meina Song. Fgm-spel: Open- set reconnition network for medical images based on fine-grained data mixture and spatial position constraint loss. Chinese Journal of Electronics,
2024c
Yuhui Zhang, Alyssa Unell, Xiaohan Wang. Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Serena Yeung-Levy. Why are visually-grounded language models bad at image classification? In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024d. URL
ht httpss://ppenevile hetr
net/ forum? ddwWwdmmgwWwWWVV
Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Screna Yeung-Levy. Why are visually-groundee language models bad at image classification? arXiv preprint arXiv:2405..0.. 18415, 2024e.
Yuxiang Zhang, Shangxi Wu, Yuqi Yang, Jiangming Shu, Jinlin Xiao, Chao Kong, and Jitao Sang. ol-coder: an ol replication for coding. arXiv preprint arXiv:2412.00154, 2024f.
Zhuosheng Zhang. Aston Zhang. Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. 2023.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint wxXiv2243..3377 2024.
15

---

Published as a conference paper at ICLR 2026
A QUALITATIVE RESULTS OF VISUAL CONCEPTS
Plesag Oveadl Invemibe ano
vona appeppte
Fastleras aa dily Santeiie clost belly lees
seoae seaid singe
Skape: Seall with
Fastar Oifie sus fire aue apwe
digence
Car Patere whie bas out incatie prrtos
of
Labes Lut deancal
Rakas Nich voikal
and thage
Can Fatura Didistre wide has aud sags ingpllar packis
Telt Long, fucheso hartucet sclendd appnrct
hovss outh friurnyg spocls mutted
Exponies
Swaker
Englis
Figure 5: Different image-level visual concepts for objects with the same subcategory.
B PROMPT DESIGN
Table 5: Prompt template for FGVR CoT data construction.
This is a picture of a [label} with the following visual features: formation provided, please answer the following question. Question: {question]. Note that you MUST first analyze the visual features that help you provide at most four candidate sub- categories of the same super-category. then pay attention the differences between candidate subcategories and make detailed comparison between them to find evidence that help you make prediction. The visual analysis process, candidate subcategories, comparison process, and fi- nal predicted subcategory are enclosed with <analysis> <amalysis></anallsssss... </analysis>> <options> </oponss>>/op/o>>>>>>> respectively. i.e., <anal-
{concepts). Based on the in-
<comparison>>< c/tosssassn///moooososooooo..o// and <prediction> <prediition=<<ppccciitttoot ysis>visual analysis process here </analysis>> coptissssscsiiisssssssi subcategories here </op- tions> Moons>>Compaiiooncccomaaar process here <rttompariion>>ppreettttiinnnnni predictedd subcat- egory here </prediction>>.
For CLIP models, only closed-worke evaluation conducted. In concrete, CLIP models select the subcategory with the highest cosine similarity the image feature from the four candidates. We do not include prompt ensembling to fairly compare with MLLMs. For MLLMs, we evaluate FGVR in both cloled-world and open-world settings. Example prompts for Fine-RI are:
(1) Closed world: "Given the question: {Question). This is a fine-grained question, so you need to output titeigr categories, such specific animal species car, airplane model. Output the thinking process in <think></think formal should be as follows: <think> </hh/kkk <answer>> your answer</answer>> Please follow this format exactly."
final answer in <aaower></nnswr>>teee The response
(2) Open-world: "Given the question: (Question), based the options provided in {Options), output the thinking process in <think><<thinn>>> aeathiin>andd final choice in <<nswer><</nswwr>>tte The response format should be as follows: < follow this format exactly."
Please Please
C IMPLEMENTATION DETAILS
For the CoT SFT data preparation, we utilize the advanced MILMM Owen2.5-VIL-322 (Bai et al. and
(Bai et al., 2025) as the base models, and full fine-tune them for 10 epochs using Llama-Factory framework (Zheng et al.) 2024). After CoT SFT, we subsequently train 3B and 7B models for 10 and 5 epochs on the separate subset of 4-shot training data, using the proposed TAPO instantiated from DAPO baseline (Yu et al. 2025) with clipping factors set E 0.2, Ej 0.28, reference KL removed, token-level loss averaging enabled, and dynamic sampling with a maximum of 20 retries. For other RL-related hyperparameters, adopt the default settings from PAPO (Wang et al. 2025): global batch size
2025). We adopt Qwven2.5-VL-3B-Instee
16

---

Published as a conference paper at ICLR 2026
of 128, rollout batch size of 384, learning rate of lc-6 and weight decay of 1e-2. We and generate n 5 response per prompt. All training is conducted on A6000 GPUs.
D EVALUATED MODELS
Several models are evaluated for comparison, including:
CLIP Models:
(shortened as CLIP-L, same below) (Radford al.
2021 EVA-ViT-G/14 (EVA-G) (Sun et al. 2023). SigLIP (SigLIP-L) (Zhai et al. 2023
and SigLIP2 (SigLIP2-L) Tschannen et al. 2025
General MLLMs: Idefics2-8B (Laurençon et al.
2024b). Idefies-LLaMA33-88
(Laurençon et al. 2024a). LLaVA-v1. 6-- 7B (Liu et al. (222b). LLaVA-Onevisionn
7B (Li et al.) 2024), InterVi.2.522B/4BB88 (Chen al. 2024). Qwen2-V1-2BB7B (Wang)
et al. 2024). and Qwe32.5-VI-33B/B (Bai et al. 2025).
Reasoning MLLMs: DeepPerception-7B8 (Ma et al. 2025
Notably, CLIP-L, EVA-G and SigLIP-L are utilized by the LLa VA series Liu et al. (2024b), the BLIP series [i al. (2023b), and Idefics series Laurençon et al. (2024b) as vision encoders, respectively. Therefore, the MLLMs should theoretically have the competitive even better FGVR capacity as these vision models.
E OPEN-WORLD EVALUATION RESULTS WITH TEXT INCLUSION
Table 6: Open-world FGVR evaluations in terms of text inclusion (%). All results are averaged with 3 trials.
Seen Categories
Unseen Categories
Models
Avg.
Air.
Bird
Car
Dog
Flower
Pet
Avg.
Air.
Bird
Car
Dog
Flower
Pet
Avg.
General MILMS
Idefics-88B
7.49
11.02
17.94
18.01
54.27
11.12
19.98
6.76
6.01
13.37
12.14
0.38
11.45
8.35
14.16
IdefickT-L
3.30
4.79
5.57
17.04
32.37
3.72
11.13
3.16
3.50
2.97
9.59
2.12
7.57
4.82
7.98
ALaVA-v1..-mistrall7
2.00
3.27
9.85
16.05
24.75
13.97
11.65
2.03
3.25
8.39
9.59
0.09
7.97
5.22
8.44
LL
21.33
46.50
3.26
18.23
3.46
5.54
15.36
13.01
0.14
12.46
Intem
3.90
6.03
10.37
17.13
25.79
20.50
13.95
2.33
4.20
7.24
9.41
1.27
13.06
10.10
IntemVLL2.544B
8.59
7.67
16.98
19.92
25.12
16.91
15.87
7.06
7.10
10.96
11.20
1.84
10.11
8.05
11.96
VL2.58V
10.04
10.97
12.78
18.76
26.34
23.12
17.00
8.64
9.00
8.42
11.53
1.79
13.66
8.84
12.92
Qwen2-VL-2B
34.37
25.04
60.42
41.32
59.06
4.50
37.45
42.90
8.91
42.17
30.42
3.49
4.89
22.13
29.79
Qwen2-VL-HH
45.50
37.93 66.16
53.32
69.81
27.48
50.03
51.16
17.52
45.60
41.71
2.26
15.41
28
39.49
Qwen2.5-VL..
37.96
48.78
58.08
51.19
61.15
13.92
45.18
40.80
22.33
43.78
36.62
5.61
12.12
26.88
36.03
Qwen2.5 V-L-7B
46.85
58.28
67.14
68.90
73.09
33.55
57.97
44.78
28.60
45.88
49.03
10.80
27.19
34.38
46.17
Reasoning MLLMs
DeepPerception-T7
40.66
47.63
66.62
64.78
75.47
67.37
60.42
|42.37
21.33
47.28
48.21
5.04
40.12
34.06
47.24
Vine- R1-38 (ours)
71.15
73.45
77.16
37.27
30.68
16.50
Fine- R1-B (ours)
63.44
75.22
78.88
78.41
86.92
86.76
78.27
44.85
29.77
51.52
51.24
10.37
69.86
42.94
60.61
GENERAL CAPABILITY
To comprehensively assess the model's general capabilities endowed with FGVR capability, we conduct evaluations on two set of datasets: (1) classification-basel VQA benchmark: ImageWikiQA caattionnnnswrinng dataset collected by feeding the Wikipedia pages of ImageNet classes to GPT-4. (2) General VQA benchmarks: MME (Fu jet al., 2023a). MMBench (Liu et al. 2024d). and SEED-Bert (Li et al.) 2023a). As shown in find that current MIL.MS perform poorly in answering these questions, suggesting that
(Zhang et al. 2024e), which is a multiple-choice
TTble
17

---

Published as a conference paper at ICLR 2026
their poor FGVR performance is fundamental limitation for more advanced capabilities. However, Fine-R1 raises the performance from 54.85% to 58.45%. demonstrating that FGVR is indeed a foundation for MLLMs' advanced capabilities. Moreover, Fine-RI demonstrates competitive in peneral-purpose performance and even achieves improvements on MMBench and SEED-BBecch worth noting that Fine-R1 is post-trained solety the FGVR task without incorporating general instruction tuning data, proving that RL drives performance gains not by mere answer memorization. These results suggest that Fine-RI can serve both as a specialized assistant for users interested in FGVR and as general-purpose MLLM for broader applications.
Table 7: Performance comparison on three general MLLM benchmarks.
Model
ImageWikiQA MME MMBench SEED-Bench
Qwen2.5 VL--BB
54.85
1591
79.30
73.74
1-ne-R1338 (ours)
58.45
1587
79.55
73.99
G QUALITATIVE RESULTS
We provide a qualitative analysis to better demonstrate the effectiveness of our approach. As shown in Figure 6, we can easily observe the model's capability generate accurate answers through structured "visual analysis-candidar cally integrates domain-specific knowledge with visual observations, in contrast the tendency of the baseline model (i.e., Qwcn2.5-VL-3B) to produce incorrect responses directly from superficial pattern recognition.
pnocaatgoriesccmppaiisoonpo prreiictionriiiiionn" process that systemati-
QwenZ.5-V1-30 What liind of car this? Optione
aireraft
[Meecedes Bera SL Class Coups 2009,
Options:
QwenZ. (- 5-VB-3B
Aston Martin Vinage Coupe 2012,
[EAE 146:300. 300,
Chevroles Corvettee 2R1 2012.
Astinn Martin 98 Vanttage Coups 2012]
Martin
2012
X
LEE-200000 200,
Folder
Answer:
X
Thinking Precess:
(Visual Analysta)
dlekk, low-stung with several distinctive Bootline: anatyzed ideatity
Fine-R1-30
(Visual Analysis) shows atrenitt with the following visual
Thinking Precess:
Fine-R1
(Candidate Subcategories)
aerodgnami: profile...
flowing rootine that towards
visual features the follouting
observed, the (Candidate Subcutegortes) Based the "regional candicure subcategories
Pusstage The faselage appears relattrety compared
regional jets. indieating dedggee for suorte
considered:
candidate
(all within super catogeyy "spocts
146-300 146 different Foldeer
Mercedes Class Coupe
fustbadk
sleck
roofline.
confiurations (Comparison) BAE Foldier 100/Folder 70: The his and Foblicr 70,
(Compurison) Morcedes SL Class 2009 festback strcamlinci clegent uppasmmnas, while mundel uginceivve (Prediction) Busrd
Martin coupe profile Santbeck design but typically
and
the Mercedes
Martin
compared
appearance. The T-ail the BAE 146's
100
Coupe 2012:
the Hencdds spiicaly
Martin Virage tinds
(Prediction) Based analyiis image engans,
the conligurationn nadkel sugaests uperuaad by regional
side profile, detailed of Mercades Sla-Clas Coupe Answer:
including design, the image likely
Additionlly presence
regonal aligns
Mercedes Benz SL- Class 2009
BAE 346-300
Annwer:
Figure 6: Case study comparing Fine-R1-3B and Qwen2.5-VL-3B on Stanford Car-196 (Left) and FGVC-aireraft (Right).
H THE USE OF LARGE LANGUAGE MODELS
LLMs were used solely for polishing writing and error correction the preparation of this paper, and all suggestions generated by the models were carefully reviewed and verified by the authors.
18


Reviews:
[{"review_id": "aWmC2ahVyz", "reviewer": "Reviewer_Ace4", "summary": "This paper targets FGVR by proposing a two-stage approach to improve MLLMs. The first stage, CoT SFT, uses supervised fine-tuning with a structured Chain-of-Thought to teach the model an interpretable, fine-grained reasoning process. The second stage, TAPO, employs a triplet-augmented policy optimization to sharpen the model's ability to distinguish between highly similar classes. In a 4-shot, base-to-new setting across six FGVR datasets, the method reports significant gains over both general-purpose MLLMs and strong contrastive models like SigLIP.", "questions": "1、Novelty of CoT Application Needs Clarification. The core idea is to combine structured CoT with policy optimization. However, using CoT to enhance reasoning is already a well-explored area in LLMs. The paper needs to more clearly articulate the fundamental difference between its \"CoT SFT\" and existing work on few-shot CoT prompting or standard CoT-based supervised fine-tuning.\n\n2、Details on Semantic Similarity Metric are Lacking. The paper relies on a SigLIP text encoder to calculate semantic similarity, which is a key metric. However, it needs to provide more details on the threshold selection, the metric's sensitivity to different class granularities, and the potential impact of text normalization or synonyms.\n\n3、Clarity on Acronyms. Several new acronyms (\"CoT SFT,\" \"No-Thinking-RL,\" \"TAPO,\" etc.) should be defined with their full names upon first use to improve readability.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1、t's widely acknowledged that general MLLMs underperform contrastive models like CLIP/SigLIP on fine-grained tasks. The paper's attempt to close this gap using structured CoT and reinforcement learning, rather than relying solely on massive labeled datasets, is a practical and appealing direction.\n\n2、The two-stage approach is well-motivated. The CoT SFT stage provides an interpretable reasoning framework (\"visual analysis → candidate subclasses → comparison → prediction\"), while the triplet-based policy optimization (TAPO) directly targets the core challenge of FGVR: maximizing inter-class variance while minimizing intra-class variance.\n\n3、The experiments are extensive, covering both closed-set and open-set scenarios. The use of multiple evaluation metrics (including semantic similarity) and thorough ablations helps to clearly identify the sources of performance improvement.\nWeaknesses & Suggestions", "weaknesses": "1、The method feels like an application of existing CoT and RL techniques, not a new paradigm for FGVR. A direct comparison against a generic CoT prompt is needed to prove the proposed reasoning structure is truly beneficial. \n\n2、Using a SigLIP encoder to calculate a key metric while also comparing against SigLIP is a potential conflict. The results should be cross-verified with another encoder (like CLIP's) to ensure fairness. \n\n3、The reported gains are marginal and lack error bars, making them unconvincing given the high variance of RL and CoT methods. \n\n4、It's unclear if the gains come from the reasoning structure or just from generating longer text, a known confounder. The paper needs length-controlled experiments to prove its central claim. Weak Baselines. \n\n5、The CLIP/SigLIP baselines seem under-tuned, as they lack standard optimizations like prompt ensembling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k4H690ZBvU", "reviewer": "Reviewer_8sko", "summary": "The paper proposes Fine-R1, a multimodal large language model (MLLM) framework for fine-grained visual recognition (FGVR). It introduces a two-stage R1-style training pipeline: (1) Chain-of-Thought Supervised Fine-Tuning (CoT-SFT), where the model learns structured reasoning steps from synthesized CoT data; and (2) Triplet Augmented Policy Optimization (TAPO), a reinforcement-learning method that augments intra- and inter-class examples (anchor, positive, negative) to improve robustness and discrimination. Fine-R1 achieves superior accuracy on six FGVR benchmarks, outperforming both general and reasoning MLLMs (e.g., Qwen2.5-VL, DeepPerception) and even contrastive CLIP models, especially in few-shot and unseen-category settings.", "questions": "**1. Data efficiency and generalization:**\nThe CoT-SFT dataset contains only 404 samples. Could the authors elaborate on how they ensure generalization beyond this limited synthetic set? For instance, were any experiments conducted to test scaling behavior when using larger or more diverse CoT data?\n\n**2. On cross-model validation:**\nSince all base models are Qwen-VL variants, have the authors attempted to reproduce the results on alternative architectures (e.g., LLaVA or InternVL) to confirm that the proposed TAPO framework generalizes across backbones?\n\n**3. On training efficiency and computational cost:**\nTAPO introduces triplet sampling and additional rollouts. Could the authors quantify the training-time or GPU-hour overhead compared to DAPO or standard GRPO?\n\n**4. On incremental novelty and relation to prior work:**\nThe paper positions Fine-R1 as an R1-style framework. Could the authors clarify the specific conceptual or algorithmic distinctions from prior RL-based reasoning methods such as Visual-RFT, Vision-R1, or VLM-R1? What unique design choices make TAPO fundamentally different rather than a variant?\n\n**5. On CoT faithfulness and quality control:**\nThe CoT rationales are generated automatically by Qwen2.5-VL-32B. Did the authors evaluate their correctness or consistency? How sensitive is model performance to potential noise in these synthesized CoTs?\n\nI will adjust my score based on the authors’ response.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Clear motivation**: The paper clearly articulates the limitations of existing MLLMs in fine-grained visual recognition and motivates the need for improved reasoning and generalization capabilities.\n\n**Methodological novelty**: The proposed TAPO combines reinforcement learning with triplet-based augmentation, conceptually bridging contrastive and policy-optimization paradigms.\n\n**Strong empirical results**: Fine-R1 consistently surpasses prior MLLMs and CLIP baselines across multiple FGVR datasets and evaluation settings (closed/open-world, seen/unseen).\n\n**Well-structured analyses**: The ablation studies and hypothesis testing (H1–H3) are thoughtful, showing that Fine-R1’s gains stem from better knowledge deployment rather than merely improved visual features or memorization.", "weaknesses": "**Limited data scale for CoT-SFT**: The CoT dataset reportedly contains only 404 samples, raising doubts about generalization and potential overfitting to synthetic patterns.\n\n**Evaluation bias toward Qwen-based baselines**: All base models are Qwen-VL variants; cross-model validation (e.g., on LLaVA or InternVL foundations) is missing, which might limit claims of generality.\n\n**Complexity vs. gain**: TAPO adds considerable training and sampling overhead (triplet construction, multi-rollout reward computation), but the improvement over DAPO (+1.6%) is relatively modest.\n\n**Conceptual overlap**: While well-positioned as an R1-style method, the framework’s connection to previous RL-based reasoning systems (e.g., Visual-RFT, VLM-R1) could be made more precise to clarify incremental novelty.\n\n**Interpretability of CoT generation**: The reasoning chains are auto-synthesized by another MLLM (Qwen2.5-VL-32B), but no human verification or quality metrics are provided, leaving uncertainty about rationale faithfulness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mptHnFhkoT", "reviewer": "Reviewer_ENRW", "summary": "This paper introduces Fine-R1, a multi-modal large model (MLLM) designed to excel at fine-grained visual recognition (FGVR). The authors propose a two-stage framework, starting with Chain-of-Thought Supervised Fine-tuning (CoT SFT) to teach the model structured reasoning, followed by Triplet Augmented Policy Optimization (TAPO). TAPO is an RL algorithm based on DAPO that uses anchor, positive, and negative image triplets to handle the FGVR challenges of high intra-class and low inter-class variance.", "questions": "If the authors can clearly address the following points during the rebuttal, I am open to reconsidering my score:\n1. Can you further justify the novelty of TAPO as an RL algorithm, distinguishing it from simply applying a known FGC regularizer ($x^{neg}$) and a data augmentation strategy ($x^{pos}$) to a DAPO baseline?\n2. Can you provide a decoupled ablation study that shows the individual performance contributions of $x^{pos}$ (Intra-class Augmentation) and $x^{neg}$ (Inter-class Augmentation)? I am particularly interested in the (Baseline + $x^{pos}$ only) result.\n3. Please provide an ablation study on the ratio of anchor-to-positive rollouts ($n_1$ vs $n_2$).", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper's primary strength lies in its impressive empirical results. Fine-R1 achieves state-of-the-art performance on six FGVR datasets, outperforming general MLLMs, reasoning-focused MLLMs, and even strong contrastive CLIP models. The model shows particularly strong generalization to unseen categories, which is a key challenge in FGVR. The analysis (hypotheses H1-H3) provides an insightful conclusion that the gains come from an improved ability to deploy existing knowledge, rather than learning new features or knowledge.", "weaknesses": "Weakness\n* Limited Novelty of TAPO: The core algorithmic contribution, TAPO, does not appear to be a novel RL algorithm. It feels like a forced \"splicing\" of positive ($x^{pos}$) and negative ($x^{neg}$) sampling techniques onto an existing baseline (DAPO/GRPO). The use of a $D_{\\text{KL}}$ loss for the $x^{neg}$ sample is a common regularization technique in standard (non-RL) fine-grained classification, which calls into question its novelty as a policy optimization method.\n* Unclear Ablation: The paper fails to clearly disentangle the individual contributions of the $x^{pos}$ and $x^{neg}$ components. It is unclear if the $x^{pos}$ (hybrid rollouts) provides any significant benefit on its own. The ablation study is missing crucial comparisons (e.g., Baseline + $x^{pos}$ only, Baseline + $x^{neg}$ only) and also lacks an analysis of the $n_1:n_2$ ratio (anchor vs. positive rollouts).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)