Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data
Authors: Yucheng Shi
Quanzheng Li
Jin Sun
Xiang Li
Ninghao Liu
Venue: ICLR 2025 Poster (2025)
Keywords: Multimodal Foundation Models
Synthetic Data
Explainability
Visual Reasoning
Fine-grained Visual Categorization
Rejection Sampling
Primary area: foundation or frontier models, including LLMs

Abstract: Large Multimodal Models (LMMs), or Vision-Language Models (VLMs), have shown impressive capabilities in a wide range of visual tasks. However, they often struggle with fine-grained visual reasoning, failing to identify domain-specific objectives and provide justifiable explanations for their predictions. To address the above challenge, we propose a novel visual rejection sampling framework to improve the cognition and explainability of LMMs using self-synthesized data. Specifically, visual fine-tuning requires images, queries, and target answers. Our approach begins by synthesizing interpretable answers that include human-verifiable visual features. These features are based on expert-defined concepts, and carefully selected based on their alignment with the image content. After each round of fine-tuning, we apply a reward model-free filtering mechanism to select the highest-quality interpretable answers for the next round of tuning. This iterative process of synthetic data generation and fine-tuning progressively improves the model's ability to generate accurate and reasonable explanations. Experimental results demonstrate the effectiveness of our method in improving both the accuracy and explainability of specialized visual classification tasks.

Published as a conference paper at ICLR 2025
ENHANCING COGNITION AND EXPLAINABILITY OF MULTIMODAL FOUNDATION MODELS WITH SELF-SYNTHESIZE DATA
Yucheng Shi Quanzheng Li Jin Sun' Xiang Li Ninghao Liu-
School of Computing, University of Georgia
Department of Radiology, Massachusetts General Hospital and Harvard Medical School
ABSTRACT
Large Multimodal Models (LMMs), or Vision-Language Models (VLMs), have shown impressive capabilities in a wide range of visual tasks. However, they often struggle with fine-grained visual reasoning, failing to identify domain- specific objectives and provide justifiable explanations for their predictions. To address the above challenge. we propose a novel visual visualeeecction sampling frame- work to improve the cognition and explainability of LMMs using self-syythesized data. Specifically, visual fine-tuning requires images, queries, and target answers. Our approach begins by synthesizing interpretable answers that include human- verifiable visual features. These features are based on expert-defindd concepts, and carefully selected based on their alignment with the image content. After each round of fine-tuning, we apply a reward model-free filtering mechanism to select the highest-quality interpretable answers for the next round of tuning. This iterative process of synthetic data generation and fine-tuning progressively im- proves the model's ability to generate accurate and reasonable explanations. Ex- perimental results demonstrate the effectiveness of our method in improving both the accuracy and explainability of specialized visual classification tasks.
INTRODUCTION
Large Multimodal Models (LMMs) have recently demonstrated remarkable performance in visual descriptive tasks, excelling in applications such as image captioning and visual question answer- ing (Liu al., 2024a; Alayrac al., 2022; Li et al., 2022; Liu al., 2024b). However, recent research (Kim & Ji, 2024; Geigle et al., 2024) reveals significant drop in their effectiveness when applied to cnowledde-intensiing
gomain-speciffe visual classification tasks. We observe that even advanced models like LLaVA-1.5 (Liu et al., 2024a) can only achieve mere 12.2% classification accuracy in the Stanford Dogs dataset (Khosla et al., 2011). Further investigation reveals that cur- rent LMMs struggle to utilize key visual features for reasoning in classification and fail to provide justifiable explanations for their predictions, as illustrated by the examples in Figure 1.
User: Identify the hird species and expluin your reasoning.
The core issue stems from insufficient Nomain-specift aligmment, as the model struggles to recognize key vi- sual features and link them to the correct labels. To ad- dress this problem, we propose enhancing the LMM's domain-specific cognition and explainability through fine-tuning (Touvron et al., 2023; Gu et al., 2021). However, this approach hindered by a lack of data, as creating high-quality, feature- -eeell image annotations is both complex and resource- inttuneieeeeteivveeeeeee (L.iu et al., 2024c). While labeling images by category and identifying key features for each class independent of the image is man- ageable, annotating the specific visual characteristics per image requires an extensive workload. Moreover, this level of detailed annotation goes beyond the pacity of standard annotators and current LMMs (Chen al., 2024a), making it impractical scale.
LMM:
The bird species shown in this image a seaguil. The reason for that
IMM:: The bird species is blaok crow. for this identification is that
this idcutification the bird is flying the accan, which is typical seagulls. Label: Stack footed Albutross
perched on tree brunch. which comnoo behavior for crows.
Label: Covome billed And
LLaVA-1.5 struggles utilize key visual features in images reasoning and ex- plaining predictions in classification tasks.
Figure 1:

---

Published as a conference paper at ICLR 2025
The biggest challenge now is synthesizing high-quality training data, specifically interpretable target answers. Given a dataset with images and labels, a naive approach would be to use only labels as target answers. However, training on such data may result in shorteut learning, where models pick up spurious correlations instead of truly understanding key visual features (Geirhos et al., 2020). While including general lasel-associated features as target answers might seem beneficial, it often results in overly generic explanations that lack the image-specific details necessary for accurate interpretation. We illustrate these shortcomings with examples in Figure 2.
To tackle this challenge, we propose framework that allows LMMs to seff-synthesize interpretable answers without rely- ing on explicit image-specific annotations. For a given image, we first leverage the LMM's captioning ability to generate de- scriptions, which are then used to identify visual features rel- evant to that specific image. Each description may only cover part of the key features, but by collecting a large set of descrip- tions from the LMM, we can approximate the true distribution of the image's features, reducing the incompleteness in indi- vidual descriptions. We provide a formal justification for this approach in Section 3.2. Moreover, to ensure precise identifi- cation, we apply an information bottleneck technique to select the most relevant features. Once the image-specific concepts are identified, they are rewritten into interpretable answers.
Q: What this bird's species? Explain your reasoning.
Synthesized
Answer
Ronly label
This is Black Footed Albatross.
#label and general label-level features
Black Footed Albatross, because .. Black fest; long,
wingor booked hill dork
plabel and specific image- Teaturest
Black Footed Albatross
based on its large, hooked bill, dark foathers, small, dark eyes either side of its head
For the training procedure, also design an iterative fine- tuning approach to further improve performance over a one- shot training scheme. We begin by extracting image-level fea- tures and transforming them into interpretable answers, which, together with the corresponding images and queries, form the initial training dataset. Fine-tuning on this data results in an updated model that can generate more accurate answers. The updated model is then used to repeatedly generate answers, with the best one selected for the next round of fineituning. This self- boosting process progressively improves the LMM's ability to deliver reliable explanations.
Examples of synthetic answers for query Q. Training with the first two types leads to shortcut learning or overgeneralization.
Figure 2:
In summary, our contributions are threefold: (1) We propose a novel framework that improves LMMs' interpretable visual classification abilities without requiring extensive manual labeling, (2) We introduce an information-theoreii approach to select interpretable visual concepts for each im- age and reward model-free filtering method to ensure high-quality data selection from synthetic outputs, and (3) We develop an iterative process of data synthesis and model fine-tuning to progres- sively enhance LMMs' cognitive abilities and explainability.
2 PRELIMINARY
Problem Statement. We aim developing Large Multimodal Model (LMM) for explainable visual classification. Let fo denote the LMM model, X be the input image, and q be the query prompt. The model's answer is denoted as ÿ fo(X, 9). where ý is expected to correctly predict the label and explain its prediction by using the visual features observed in the image. To build such model, a straightforward strategy is fine-tune the LMM with a training dataset that contains the ground-truth answer for each input image. However, most available datasets D = {(X,, )) only consist of images X; and class labels c; (Khosla et al., 2011). To solve this, we propose data synthesis approach that transforms the raw dataset into an augmented dataset D {(X,, di y/ M with queries and explainable answers. While generating queries gi is straightforward, the challenge lies in synthesizing cxplainable answers yıs which must include detailed visual features 2; that humans can identify and use for explanations. Therefore, the key problem is developing a method to automatically annotate visual features Z for each image X,, given their labels c.
Visual Tine-Tuning. Visual fine-tuning adapts a pre-trained LMM to understand specific visual information by training on image-text pairs. Typically, an LMM fe consists of vision encoder that extracts visual embeddings from an input image X, a projector that maps these embeddings into the language embedding space, and language model that processes the combined visual and
2

---

Published as a conference paper at ICLR 2025
X
Base LMM (LLaVA)
Initial Synthesis
Tuned LMM fround
Rejection Sampling
Tuned (round N)
Final LMM
Black Trohed
(((x.2))) AI(Z);
Query Prompt
Tuned LMM
P2. Hal
fround Ni
Label level Concepts 2
X
The bind spacies shownn in this image a dlock footed Albutrons. The image festures, such the sniib large, hooked brtl, dark and this identiicctooo
Z
Prrddcminnnt dank plumage?
Image: Concepts
"Long, adapted searing".
Whhite-ttppdd teathers",
"Large, hocked rypically gray'. "Small, cyes "Predomirantyy plumoge".
hookeed tysicaly dark "Small, apes
Concepts Selection
3"
a5k maas (p): 2")
Figure Our framework: An iterative approach of data synthesis and model fine-tuning.
textual information (Liu et al., 2024b;a; Lin et al., 2024). Formally, given round of conversation containing an image X, a question and an answer y, the model is trained to maximize the likeli- hood of generating the target answer: L(0) Σm po X, g, Here, y is the i-th token of the answer y. and ly/ the length of the answer. The fine-tuning process typically optimizes performance by freezing the pre-trained visual encoder to preserve learned visual representations while updating the projector and language model parameters to improve language understanding for visual inputs (Liu et al., 2024b; Lin et al., 2024). Recognizing the critical role of fine-tuning data quality in model performance, our research proposes synthesizing high-quality conversation data to improve performance.
3
METHODOLOGY
3.1
OVERVIEW
Our approach leverages synthetic data for visual fine-tuning enhance both the cognitive ability and explainability of LMMs, as illustrated in Figure 3. There are two major steps:
Image-level Visual Concept Selection: Given an image and its label, our first task is extract concepts that explain the connection between the image and label. We propose
set of a selection method that identifies subset of concepts most relevant to the image content while ensuring the subset is concise. Using these concepts, we prompt the LMM to rewrite them into textual answers that explain the label. These answers are used for the first round of fine-tuning.
After the initial fine-tuning, the model can generate synthetic answers, which can used for next-round fine-tuning, but their quality still varies. To filter out low-quality explanations and only select the best quality one training data, we use previously selected concepts as filtering criteria, quantifying alignment between explanations and the concepts. The synthetic answer with the best-aligned explanation is then selected and combined with the corresponding image and query, forming a new data pair for subsequent rounds of tuning.
2. Reward Model-Free Rejection Sampling:
3.2 STEP IMAGE-LEVEL VISUAL CONCEPT SELECTION
In this step. our goal is to identify the visual concepts present in a given image. Let X represent the true content of the image and its class label. Each label class e is associated with set of expert-defined visual concepts Z, which can be obtained by consulting domain experts or using a large language model. However, not all concepts in Z will necessarily be present in the image X. Therefore, we aim to select a subset of concepts, 2 C Z. that are observable in the image X.
To achieve this, we propose concept selection by leveraging the Information Bottleneck (IB) prin- ciple, which seeks a compressed representation that preserves maximal information about another variable (Tishby et al.. 2000). In context, need find Z" that maximizes its mutual infor- mation with the image content X, i..., [[;; Z*), while minimizing the redundancy by penalizing the mutual information between the selected concepts Z and the full concept set Z, i.e T(Z"; Z).
3

---

Published as a conference paper at ICLR 2025
Formally, define the optimization problem:
z = arg max (T(X:Z) -(( Z'CZ
()),
where 3 is a Lagrange multiplier that balances relevance and redundancy. However, directly comput- ing T(X; 7') is intractable due the high dimensionality and complexity of the image space. To ad- dress this, we introduce an intermediate variable: a set of image descriptions D = {dj, d2..... dn J. generated by prompting an LMM (i.e., the base LLaVA-1.5 (Liu al., 2024a)), with instructions like "Please describe the image." Each description d; attempts to capture some aspects of the image content X. By increasing the number of collected descriptions n with different prompts, we aim to approximate the true distribution of X.
This approach is analogous to assembling pieces of a puzzle: Each description provides partial in- formation about the image, and collectively, they form more complete representation. Similarly, multiple potentially partial descriptions generated from different prompts can collectively approx- imate the true image content. Under this intuition, we formalize the approximation in Theorem 1, with proof provided in the appendix.
Theorem 1. Let X be the true image content with label e and D = {d1, d2...... and identically distributed (f..d.) samples from P.D Let Z be an expert-defined concept list about label Under the assumptions of conditional independence and convergence (Assumptions I and 2), as n - oC, the mutual information I IDD Z) converges to Z):
........d, dn be independent
lim ID: IX;
In practice, we cannot sample an infinite number of descriptions, and LMMs may generate inconsis- tent or contradictory descriptions due to ballucinations or uncertainties. To mitigate this, we design high-quality prompts and encourage diverse responses to improve the reliability of the generated descriptions. By doing so, we assume that most of the descriptions will accurately reflect the image content. This assumption further validated through experiments in Section 4.4. From another per- spective, if feature is not consistently covered by image descriptions, it means that the model is not certain about its presence, which will naturally result in lower MI scores for the associated concepts, reducing their likelihood of being selected. Using the set of descriptions D, reformulate the IB objective in Equation. as:
Z = arg max [(D; Z') - 3I(Z':
(2)
Z'ZZ
However, computing mutual information in high-dimensional space directly remains challenging. Therefore, we employ the InfoNCE loss (Oord et al., 2018) as a lower-bound estimator of mutual information. For each concept 2j E Z, we calculate an InfoNCE score sy:
exp (sim (ea, ez,) /T)
8j = log
(3)
exp
(sim (ea. 02,) /T) + EdEe exp (sim (edz (2,)/T]
diED
where sim(..) denotes the cosine similarity between embeddings, ed, and es, are the embeddings of description d, and concept 2j. respectively, and 7 is a temperature parameter. are the descriptions for other images as negative samples. We can easily obtain the above embeddings through an off- the-shelf language embedding model (e.g.. BERT (Devlin, 2018)). Next, we approximate I I(D; 2*) Σx,ez- Sj. To minimize I[Z"; Z), we selectively include concepts to reduce redundancy. Given that Z is a subset of Z, have: (Z*)) = 7 Z) -E. z- p(2;) log p(2;). where H H(Z)) is the entropy of Z A selected subset Z* with a smaller size and higher probabilities for its z, elements will result in lower entropy. Combining these approximations, our selection criterion for Z' becomes:
as the sum of InfoNCE scores for the selected concepts: I(D;
2, E 2 31
a Bo
(4)
where y and a are the mean and standard deviation of the InfoNCE scores {s,) |2] parameter / controls the trade-off between including relevant concepts and avoiding redundancy. The selected concepts Z are not only relevant to the image but also capture the most informative features unique to the class label, providing strong evidence for the classification result and serving as reasonable explanations. Once we obtain Z*, we can generate an explainable answer for a classification query the image X. Specifically, we prompt the base LMM with these concepts to produce a coherent explanation. The prompts used are detailed in the appendix. After gathering the image and query-answer pairs, use them to fine-tune LMMs.
respectively. The
4

---

Published as a conference paper at ICLR 2025
3.3 STEP 2: REWARD MODEL-FRREE REJECTION SAMPLING
After the initial round of fine-tuning with explainable visual query-answer pairs, the LMMs have significantly improved their ability to generate reasonable explanations. This improvement allows us to leverage the current fine-tuned model to generate new data for subsequent training rounds. However, the quality of newly generated data can vary considerably terms of label accuracy and explanation quality. Training on low-quality data could lead to performance degradation. To address this issue, we propose a rejection sampling technique that filters out low-quality outputs.
Rejection sampling, also known as Bfst-of-N, is an inference- time strategy that generates multiple candidates and selects the best one for further tuning (Touvron et al., 2023; Stiennon et al., 2020). In our work, we adapt this idea for visual fine-tuning to iteratively improve LMM performance. Our proposed rejection sampling process begins by generating series of answer candidates, using the fine-tuned model f/ from the last round T, and then identify the best answer from these candidates. For language-only conversations, this selection is typically performed by a reward model, which assigns higher rewards to answers aligning with desired criteria (Touvron et al., 2023). The answer with the highest reward is then selected and used for the next round (T + 1) of training. However, in the visual domain, finding a reliable reward model remains challenging, noted by (Chen al., 2024a). To address this, we propose a reward model-free data filtering method to select the highest quality interpretable answers.
Specifically, we leverage the selected concept set 7* from Section 3.2 as a reference to evaluate explanation quality. Our aim is to select the answer that best aligns with these relevant concepts. Formally, let Y = {uI, .2...... Jm J represent the set of answers generated by the fine-tuned model for a given image. These answers are obtained by prompting the model with questions like "Whar is the {item) in this image? Please provide your reasoning." " The "item" " here is set to be an coarse- level label, like bird, airplane. Our goal to select the answer y" Y that maximizes the mutual information with z*:
y arg max Ty6E 2*). W
(5)
We approximate (21;; Z*) using the InfoNCE score:
exp (sim (ey,, Cz2)) /T)
5/ = Σ log cxp (sim (Cy,, Cz2) /T) + E= exp (sim (Cy,, Cz..) /TT)
(6)
Z,EZ*
where ey. is the embedding of answer Vi- We select the answer with the highest score: y² arg m s/. The InfoNCE score provides a quantitative measure of how well the generated an- swer aligns with the relevant concepts. Therefore, our method eliminates the need for a separate reward model, which is particularly beneficial given the lack of reliable reward models for multi- modal data (Chen et al., 2024a). Additionally, we add another empirical constraint: the selected answer should contain the correct label c; otherwise, it will be discarded.
In summary, our framework, outlined in Algorithm 1, enhances the model's fine-grained classifi- cation capabilities by iteratively fine-tuning on diverse, high-quality, synthetic visual classification query-answer pairs. Step 1 identifies the most informative concepts in the image, while Step 2 se- lects the explanations that best align with these concepts. This two-step approach improves both the accuracy and interpretability of the model's predictions, enabling it perform more effectively in complex visual classification tasks.
3.4 THEORETICAL JUSTIFICATION
Our approach is grounded in information theory, specifically the maximization of mutual informa- tion among the image X, the expert-defined concepts Z for the whole class, and the generated answer Y with explanations. Theoretically, the problem of finding the best Y is equivalent to maximizing the three-way way mutual information:
max (X:;Y Y
where y captures the maximum amount of information from both the image and the expert-define concepts. However, directly optimizing this objective is intractable due the high dimensionality of X and Y (Poole et al., 2019). To make the problem manageable, we decompose it into two subproblems corresponding to our two-step method.
5

---

Published as a conference paper at ICLR 2025
Algorithm Iterative synthesis and fine-tuming tne-tuning.
In Step 1, we select subset of concepts z S Z that maximizes I(X; 2²). capturing the most relevant concepts for the image. This aligns with the Information Bottleneck princi- ple, where Z" serves as distilled subset of Z that retains maximal information about X. In Step 2, we select the generated explanation Y* that maximizes I(Y*; Z"), ensuring that the generated explanation closely aligns with the selected concepts. With this decomposition, we establish lower bound on the whree-way mu- tual information (proof provided in appendix).
Require: Dataset 2 {(X,,, ) Concept Sets Z = [e) of every label, Pre-
trained LMM fg. synthetic query set Q
Ensure: { Descriptive query qa
Fine-tuned LMM fo for accurate, in- terpretable explanations
1: Initialize training set D +-
2: for each (X,,cz) ) do
3:
Z + 2 [c
4:
D, - (Xx;
5:
for 2j E Z do
Theorem 2. Let X, Y, and Z be discrete ran- dom variables. Define:
6:
Sty -- InfoNCE(D;,
7:
end for
Z arg arg max -K3zz'--1(Z'' 7/7%
8:
H4 < mean( (siz H). 0; t -tdd( {ssst)
9:
2 + {zj E Z Sij > Mi + Bor
10:
y: f- assemble Z
Y arg max mag IYY' Y'CY
11: D < D U {X,, q:: v1)
12: end for
Then, the following inequality holds:
13: FT: Fine-tune ʃ D to obtain f
14: repeat
I(X;Y;Z) 2 T(Y";"") + -
15: for each (X -) E do
16: Yi - f (X., qi)
for Mi E Y, do
18:
s f InfoNCE( Vig)
end for
This theoretical foundation justifies our two-
step optimization process, ensuring that our
20:
y < arg max s/ D + D U ((Xi, 46, v()} end for
method effectively captures the essential infor-
21:
mation shared among the image, concepts, and explanations.
22:
23: Iterative FT: Fine-tune f D
24:
until Max iterations T
25:
return fm
4 EXPERIMENTS
We conduct experiments address the following questions. Q1: How effectively does our method improve performance on fine-grained visual classification tasks? Q2: How does our method enhance the explainability of the fine-tuned model? Q3: How does our concept selection strategy compare to baseline methods? Q4: Does our training scheme mitigate shortcut learning? Q5: How usable is our trained model?
4.1 EXPERIMENTS SETTINGS
We evaluated our approach on variety of datasets, including fine-grained classification datasets (CUB-200 (Wah et al., 20,1), Stanford Dogs (Khosla et al., 2011), FGVC- Aircraft (Maji et al., 2013)), medical datasets (HAM10000 (Tschandl al., 2018), Chest X-Ray for Pneumonia (Kermany et al., 2018)), and the Plant Disease Dataset (PLD) (Vipooool, 2020). By testing datasets from diverse domains, demonstrate the versatility of proposed frame- work with self-synthesized data. Our experiments used LLLvv--155-BB (Liu et al., 2024a) as the base LMM, and for the Chest X-Ray datasets, we employed its medical version (Li et al., 2024b). Ad- ditionally, ES (Wang et al., 2022) served as the embedding model. We fine-tuned the LMMs using LoRA, focusing on all linear layers. Training was conducted on 8 H100 GPUs, utilizing Hugging- Face (Wolf, 2019) and DeepSpeed frameworks for efficient distributed training and optimization. Further details are provided in the appendix. We also provide our code here!.
Implementation Details:
Baselines: Given the novel problem of generating interpretable answers without image-specific annotations, we design the following baselines for comparisons: (1) Base LMM: Assesses the base multimodal model's performance in the zero-shot setting. (2) Naive Label Fine-tuning (NL)):
hhtpps:// thu.. com/ Syeny/Selnllellee fäynthX

---

Published as a conference paper at ICLR 2025
Table 1: Our method achieves superior accuracy and explanation quality across diverse datasets.
Accuracy t Per Iteration
Explanation Quality
General Ability
Dataset
Method
3
EE f
CS
FS 4
MMMU 1
Base
2.69
0.92
0.67
4.28
35.56
NL
73.42
78.25
79.94
82.21
0.00
35.67
CUB-200
L+GE
61.48
72.23
73.23
73.06
1.00
0.70
6.84
34.89
Ours
80.24
83.76
84.69
85.02
1.00
0.82
6.53
35.00
Base
12.2
0.94
0.69
5.47
35.56
Stanford
NL
82.73
82.34
84.03
84.27
0.00
34.67
dogs
L+GE
73.45
77.89
78.15
76.55
1.00
0.77
7.50
34.56
Ours
85.29
86.75
86,86
86.91
1.00
0.86
7.41
34.56
Base
3.00
0.97
0.42
5.39
35.56
NL
83.47
87.28
87.82
87.73
0.00
35.56
FGVC-A
L+GE
72.13
79.87
82,45
82.69
1.0
0.76
8.59
35.56
Ours
88.78
90.91
91.42
91.99
1.0
0.79
7.00
37.33
Base
0.00
0.95
35.56
NL
89.38
94.52
94,29
93.95
0.00
34.78
PLD
L+GE
24.03
25.27
24.56
24.90
1.00
0.76
10.45
35.44
Ours
75.96
92.80
96.59
97.16
1.00
0.86
9.01
35.22
Base
1.62
0.98
0.63
3.93
35.56
77.28
80.75
82.49
81.71
0.00
35.33
HAM10000
L+GE
7.47
8.83
9.35
8.45
1.00
0.94
9.68
35.22
Ours
79.37
82.29
83.69
85.06
1.00
0.87
7.43
35.89
Base
62,50
1.00
0.24
3.49
Chest X-ray Pneumonia
NL
85,58
89.10
85.90
89.58
0.00
L+GE
62.50
62.50
62.98
62.66
1.00
0.79
7.19
(LLaVA-Med)
Ours
97.60
96.31
99.04
98.72
1.00
0.87
8.25
Base:
original model: NL: only train with labels; L+ GE: train labels and general explanations
Fine-tunes the base model using only class labels and a simple template (e.g.. "This a picture of (label)") "). (3) Label with General Explanations (L+GE): We adopt the data synthesis approach introduced the LLaVA paper (Liu et al., 2024b) and (Kim & Ji, 2024), where training data is generated using language-only model (e.g., GPT-4), based on class labels and their corresponding label-level knowledge. More details, including the prompts, are in the appendix.
4.2 RQI: : TRAINING ON OUR SYNTHETIC DATA IMPROVES CLASSIFICATION
We evaluated our model's classification capabilities using a multi-round, progressive rejection sam- pling training process comprising four iterations, each with two epochs. Classification accuracy was measured after each iteration, with success defined as the presence of the ground truth label in the model's 's response (Kim & Ji, 2024). All trainable baselines were trained and evaluated under identi- cal settings to ours for a fair comparison. More details on training are provided in the appendix.
Our proposed method achieves higher accuracy than both NL and L+GE baselines, shown in Ta- ble 1. While the baselines, particularly NL, exhibit overfitting, our method improves accuracy with each iteration, demonstrating its effectiveness in fine-grained classification tasks. Our approach's ' resilience to overfitting stems from the use of rejection sampling, which generates a more diverse training dataset. This expanded data pool enhances the model's generalization capabilities. More- over, training with our dataset, which includes specific visual features, helps the model learn more detailed visual knowledge, contributing to further gains in classification performance. In contrast, the L+GE baseline struggles with certain datasets, notably HAM10000, likely due to irrelevant information in general explanations impeding effective learning. Moreover, we assessed the gen- eral ability of the models using the MMMU metric (Yue et al., 2024) evaluated by the LMMs-eval tool (Zhang et al., 2024). The results indicate that all trained models maintain comparable general abilities to the base model, with negligible degradation. Note that the LMMs-eval tool does not sup- port the LLaVA-M M-M model; therefore. the general ability metric is not reported. More experiment results can be found in the appendix.
7

---

Published as a conference paper at ICLR 2025
ckent the speato name the bad
Ruan Mlldkidd Bary Blacbidd andiassited Madibl
"he sperific
Oena The aiyles cans dai longt Conna
slenke Mack hel it brons
muy
naved
Uie like aosall cumpur dog with angarodie frewe ahet lops and seve with disteetec undera bake"
cf dacals und de dusk juskes bil dasmcii in dight daniuee conto and
baitiinng seues the stien lang tidds communty trand rittame
head
Thasy
Eonn tail petforance candil lad bdiiilig.
Weaten Hendewberk The Wosken
Tibetta Terverr The lecad kanva lir zn detreese upparess whick indako baad and recutacks and prpertiond boly The dog the lmage blckk and stiee out whick
New the ary oe is caggo a
cocialcy
Naudectii au
btip yollie
dee non lakken whic demoeenen Aidabiii aidess, diodd 120 not
and ling, powni hell in counurty fooe Lvand Stics.
Andas wien coserne tus
Figure 4: Our generated answers contain detailed visual explanations.
4.3 RQ2: OUR METHOD PROVIDES HIGH-QUALITY EXPLANATION
Assessing the quality of generated explanations is challenging. especially without case-by-case ground truth annotations (Ding et al., 2022; Schuff et al., 2022). Following Mohseni et al. (2021), we evaluate our explanation from three aspects: explanation existence, cognition level, and fluency.
Explanation Existence (EE). This metric assesses model's capability to generate explanations (Xu et al., 2023). Following Bills et al. (2023); Bricken et al. (2023), we employ an advanced proprictary LLM (GPT-40) to determine whether each model-generated answer includes an explanation. For- mally, for a set of generated answers Y = {yr, 1/2-- 1/22.................. Un queried by prompt like "What is the bird name? Provide your reason." ", EE is defined as: EE P. >i=1 Ci where e; = 1 if the hth answer y; includes an explanation, and e;- O otherwise. We provide the evaluation prompt in the appendix.
Results; As shown in Table our method achieves an EE of 1.00 across all datasets, indicating that it consistently produces explanations for its predictions. The base model can generate explanations most times but fails do so in some cases. In contrast, the NL baseline records an EE of 0.00, reflecting its inability to generate explanations due to training solely on class labels. The L+GE method also attains EE of 1.00 but falls short in other quality metries compared to our approach.
Cognition Score (CS). This metric evaluates the coherence and logical flow of generated expla- nations (Nourani et al., 2019; Fan al., 2020). Following Liu et al. (2024e): Bills al. (2023); Lieberum et al. (2024), we employ an advanced proprietary LLM (GPT-40) to assess the ratio- nal integrity of explanations by analyzing their alignment with expert knowledge. For an an- swer Vi containing label and explanation, we first extract label-level concepts Z correspond- ing to the label. We then use an evaluation prompt to obtain a cognition score from LLM: cs; LLM(), Z, Promptt. The Evampt is detailed in the appendix. Scores range from knowledge. We only CSt, where Ye is the set of assess correct answers. The total CS is calculated as: CS = [.. Eiril Yol correct answers.
0 to 1, with higher scores indicating better alignment with expert-predefine
Results: Our method generally achieves higher CS values than the baselines, demonstrating superior coherence and logical reasoning in the explanations. This improvement reflects the effectiveness of our approach in generating explanations that are more useful and trustworthy to human users. The reason behind our better cognition is that we select image-level concepts to build the training dataset, rather than using all label-level features. This ensures that our model focuses relevant features present in the image and effectively connects the image content with expert-defined con- cepts, avoiding distractions from irrelevant features not shown in the image.
Fluency Score (FS). This metric evaluates the grammatical correctness and naturalness of the ex- planations (Liu et al., 2021). We use perplexity, standard metric that measures how well language model predicts the text, with lower values indicating better fluency.
Results; Our method demonstrates good fluency in the generated explanations. The base model receives the best fluency. This is because training on expert-level knowledge inevitably increases the complexity of explanations, which can lead higher perplexity scores. However, our method generally achieves lower perplexity than the L+GE baseline, indicating that our explanations are more natural, highlighting the robustness of our approach.
In summary, our proposed method not only enhances classification accuracy but also consistently generates high-quality explanations that are coherent, logical, and fluent. We also show some quan- titative results in our explanation in Figure 4. More results can be found in the appendix.
8

---

Published as a conference paper at ICLR 2025
4.4 RQ3: EVALUATION ON INTERPRETABLE CONCEPTS SELECTION
To evaluate the efficacy of our proposed model in select- ing expert-defined features from images, we conducted comprehensive study with human experts. Experts were invited to identify and annotate the top-4 relevant con- cepts for each image from a dataset of six bird species, providing ground truth annotations for our evaluation. Our investigation examined two primary aspects: the ef- feet varying the number of descriptions on concept se- lection performance, and a comparative analysis against several baseline methods. These baselines include GPT- 4o for concept extraction, which relied crafted prompt to guide the model identifying and re- turning the four most probable concepts for each image. We applied the same approach to the LL model for concept extraction. In contrast, for the CLIP model, con- cept selection was performed by identifying top-4 con- cepts with the highest CLIPScore (Hessel et al., 2021) relative to the target image.
Concept Selection Precision
Precision (%)
Ours
GPT.4o select
carefully
CLIP select
10 15
30
Number of Descriptions per Image
Figure Our method demonstrates superior precision in concept selection compared to applying GPT-4o.
We evaluated performance using precision, comparing model-selected concepts with human- annotated ground truth. Results demonstrated that our model outperforms strong baselines in the concept selection task. Its precision improves proportionally with the increased number of descrip- tions, peaking at 72.89% with 25 descriptions. In contrast, GPT-4o maintained relatively high precision 63.95% while both LLaVA and CLIP showed weaker performances at approximately 55%. These findings highlight our model's superior ability to leverage repeated sampled descrip- tions for more accurate concept selection. We show some qualitative results Figure 6, with more in the appendix.
Derp-or febil Bidd
+ White chespent-shaped markings around the eyes
Undertiie, oftnn rocienbble when the mosth clead
Yellowing of load margits and intervciaal prces
> Dark sooty-brown plumage
Winkle led shott mrzzeld fice Distective ficial espresiion othen dheastrrbadd is "aomoot or "houggtffl"" Rocnd, promieer oyo.
Ovenil recunal compuraal hesillly liaves Cuiling distuction l'le cdges
> Dusky, almost black hend
> slender wings Streamlined body
gleay
Curling and distortion leaf edges
Incressed prominence
* Often seen gliding with stiff?
Short erright limbt
sernctiess
wing posture
Underbizz ofizn netiomblic when the
Upaani downontd cupping larva
> Subtic light streaks on upper wings
nouth is close
Wirkked deatt-raceeed fize Squere, sturly buil?
Leaves may appear more rigid and less flexible
Seaty Altutrom
Puz
Temato Taler Leal
Figure 6: Different image-level visual concepts for objects with the same label.
4.5 RQ4: VISUALIZATION OF VISUAL FOCUS OF LMMS
To interpret model's predictions, visualize the outputs of LMMs using gradient-based ex- planation method (Wu et al., 2023). This approach identifies the image regions most influential in generating the model's answer by computing gradients of answer token probabilities with respect to image pixels. Figure 7 presents case examples demonstrating that training on our synthetic an- swers with cognitive explanations effectively prevent the model from learning spurious features, whereas directly linking images to naive labels may result in shortcut learning.
4.6
RQ5: EVALUATION ON USABILITY
Previous L.Ms often struggle with specialized questions due to difficulty recognizing domain- specific objects. Our method enhances LMMs' ability to identify specialized objects and provide verifiable visual details, as demonstrated in Table 10. In Case 1, the LMM accurately identifies a bell pepper leaf disease, describing specific visual cues such as brown necrotic spots and chlorosis. It then recommends an appropriate treatment. In Case 2, the LMM correctly identifies male Northern Cardinal, noting distinctive features like its bright red plumage and black mask. These improvements enable LMMs to serve as effective visual assistants in specialized domains, offering users accurate and interpretable responses that can be verified against the provided images.

---

Published as a conference paper at ICLR 2025
Stanford dogs
FGVC-A
Chest-Xray
Figure 7: Visualization of heatmaps for different datasets. From left to right in each triplet: Original Image, Heatmap of NL model, Heatmap of Ours.
Table 2: LMMs fine-tuned with our method provide verifiable details, enhancing user trust.
User: LMM:
What the name of this leaf disease? Can you give me your reason? The name of this leaf disease is Bell Pepper Leaf with Bacterial Spot Dis- ease. The reason for this name is that the leaf has brown or black necrotic spots, which characteristic of bacterial infections. These spots coalesce, forming larger necrotic areas, and the leaf may exhibit chlorosis or yellowing of surrounding leaf tissue. The spots are often located on the underside of the leaf. and the leaf may become crinkled or deformed near affected areas. OK then, what chemical should use to treat this disease?
User:
LMM:
It is recommended to use copper- based fungicide to treat this disease.
User:
Identify this bird. What features led to your conclusion?
LMM:
The bird is male Northern Cardinal with a bright red plumage on its head, throat, and chest. The presence of crest on head and black "mask" " around its eyes further supports the identification of this bird. Where can I find this bird?
User:
LMM:
You find this bird in the United States, specifically in the eastern and central parts the country. The Northern Cardinal is common resident bird areas with dense woodlands, shrubs, and gardens.
5 RELATED WORK: SELF-IMPROVEMENT AND DATA SYNTHESIS
In the field of LLMs, self-generated data has become a powerful tool for enhancing model per- formance (Liu et al., 2024d). Researchers have explored various techniques, such as rejection sampling (Touvron et al., 2023; Dubey et al., 2024), self-rewarding (Yuan et al., 2024), and self- play (Chen et al., 2024c), enabling models improve using synthetic data. Recent studies (Snell et al., 2024; Brown et al., 2024) have proposed the "inference-time scaling law", suggesting that in- creasing inference samples size boosts the likelihood of generating high-quality data. Data synthesis techniques have also been applied to LMMs, improving general vision tasks like visual question an- swering (Zhou et al., 2024a; Deng et al., 2024; Gao et al., 2023; Zhou et al., 2024b) and enhancing instruction-folloiin capabilities (Luo et al., 2024). Our work extends this line of research by fo- cusing on domain -omain-specific visual classification to enable effective visual assistance in professional tasks. In contrast to existing methods, address the unique challenges of data synthesis in special- ized domains, thereby extending these techniques to support expert-driven applications.
6 CONCLUSION
In this work, we addressed LMMs' limitations in domain-specific visual classification tasks by in- troducing novel framework that enhances their cognition and explainability through iterative fine- tuning with self-synthesized interpretable answers. By applying the IB principle to select relevant visual concepts without extensive image-specific annotations, our approach significantly improved classification accuracy and explanation quality across various datasets. By enabling LMMs gener- ate accurate, interpretable explanations grounded in domain -ppeifff visual features, our framework advances their applicability in specialized domains, paving the way for more reliable multimodal models in -eteeee-intesiive applications. Future work may explore more complex tasks than classification tasks, and refinements enhance scalability and generalizability
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The work is, in part, supported by NSF (#I1S-2223768.. Dr. Xiang Li is supported by the Google Research Scholar Program. The views and conclusions in this paper are those of the authors and should not be interpreted as representing any funding agencies.
REFERENCES
Jean-Bape Mann-Bppistte Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716- 23736, 2022.
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in lan- guage models. URL htps://pppnniiibblic blob. core. windows. net/neuron hwwreeuonnexlllieeppppprrneeee html.(Date accessed: 14.05. 2023), 2, 2023.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jenmyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, 7ac Hatfiel-DDods, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https//transformer- htt.0////mmoommaantfeettttreeeee
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprini arXiv::407.0000 1787, 2024.
Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Pan Zhou, Yao Wan, and Lichao Sun. MIlm-ass as-a-jude:: Assessing multimodal llm-as-a-judge with vision-language benchmark. arXiv preprint vrXiv:2402.04788. 2024a.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.203300 2024b.
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:2401.01335 2024c.
Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai-Wei Chang, and Wei Wang. En- hancing large vision language models with self-training image comprehension. arXiv preprint arXiv:240S.19716 2024.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arviv: 1810004805 2018.
Weiping Ding, Mohamed Abdel-Basset, Hossam Hawash, and Ahmed Ali. Explainability of artificial intelligence methods, applications and challenges: A comprehensive survey. Information Sciences, 615:238-292. 2022.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint 47712:2407.21783, 2024.
Ming Fan, Wenying Wei, Xiaofei Xie. Yang Liu, Xiaohong Guan, and Ting Liu. Can trust your explanations? sanity checks for interpreters in android malware analysis. IEEE Transactions on Information Forensics and Security, 01888-8853 2020.

---

Published as a conference paper at ICLR 2025
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. Mme: a comprehensive evaluation benchmark for multi- modal large language models. corr abs/2306. 13394 (2023), 2023.
Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, al. G-llava: Solving geometric problem with multi-modal large language model. arXiv preprint arXiv:2312.. 22.. 13700, 2023.
Gregor Geigle, Radu Timofte, and Goran Glavaš. African or european swallow? benchmarking large vision-language models for fine-graen fine-grained object classification. arXiv preprint arXiv:2406. 14496, 2024.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 16:6)6666666666 673, 2020.
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural language processing, ACM Transactions Computing for Healthcare (HEALTH), 3(1): 1-23, 2021.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-fre evaluation metric for image captioning. arXiv preprint arXiv:1104.0778,, 2021.
Daniel S Kermany, Michael Goldbaum, Wenjia Cai, Carolina CS Valentim, Huiying Liang, Sally Baxter, Alex McKeown, Ge Yang. Xiaokang Wu, Fangbing Yan, et al. Identifying medical diag- noses and treatable diseases by image-based deep learning. cell, 172(5):1122-111))) 2018.
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine- grained image categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, June 2011.
Jeonghwan Kim and Heng Ji. Finer: Investigating and enhancing fine-grained visual concept recog- nition in large vision language models. arXiv preprint arXiv:2402. 16315, 2024.
Bohao Li, Yuying Ge, Yi Chen, Yixiao Ge, Ruimao Zhang, and Ying Shan. Seed-bench-2-plus Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint ar arXiv:2404. 16790, 2024a.
Chunyuan Li, Cliff Wong. Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang. Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assis- tant for biomedicine in one day. Advances in Neural Information Processing Systems, 36, 2024b.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. In International conference on machine learning, pp. 12888-12900, PMLR, 2022.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anea Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma arXiv preprint 20X1 2024.
Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre- training for visual language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition, pp. 26689-26699, 2024.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recogni- fion, pp. 26296-26306. 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
12

---

Published as a conference paper at ICLR 2025
Jiacheng Liu, Alisa Liu, Ximing Lu. Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, and Han- naneh Hajishirzi. Generated knowledge prompting for commonsense reasoning. arXiv preprint 1..iv:2110.088877
Mingxuan Liu, Subhankar Roy, Wenjing Li, Zhun Zhong, Nicu Sebe, and Elisa Ricci. Democratiz- ing fine-grained visual recognition with large language models. arXiv preprint arXiv:2401.13887 2024c.
Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, et al. Best practices and lessons learned on synthetic data for language models. arXiv preprint arXiv:2404.077 400400000,,,. 2024d.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp. 216-233. Springer, 2024e.
Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang. Minzheng Wang, Pengpeng Zeng, Lianli Gao, et al. Mmevol: Empowering multimodal large language models with evol-instruct. arXiv preprint (XX1:24409.058400 2024.
S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013.
Sina Mohseni, Niloofar Zarei, and Eric D Ragan. A multidisciplinary survey and framework for design and evaluation of explainable ai systems. ACM Transactions on Inteructive Intelligent Systems (TiiS), (1(3-4): 2021.
Mahsan Nourani, Samia Kabir, Sina Mohseni, and Eric Ragan. The effects of meaningful and meaningless explanations on trust and perceived system accuracy in intelligent systems. In Pro- ceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 7, pp- 97-105, 2019.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint 1rXiv:1807.03748, 2018
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pp. 5171- 5180. PMLR, 2019.
Hendrik Schuff, Heike Adel, Peng Qi, and Nguc Thang Vu. Challenges in explanation quality evaluation. arXiv preprint arXiv:2210.07126. 2022.
Yucheng Shi. Yushun Dong, Qiaoyu Tan, Jundong Li, and Ninghao Liu. Gigamae: Generalizable graph masked autoencoder via collaborative latent space reconstruction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 2259- 2269, 2023.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can more effective than scaling model parameters. arXiv preprint arXiv:2408.0333 03314, 2024.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008 30211, 2020.
Naftali Tishby, Fernando Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/00040057. 2000.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
13

---

Published as a conference paper at ICLR 2025
Vipooool.
New plant diseases dataset.
https: /tt///wwwww wwww kaggle. com/datasets/ 2020. Accessed: 2024-09-25.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Cub-200. Technical Report CNS-TR- 2011-001, California Institute of Technology. 2011.
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Ma- jumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint 1rXXi:2212.0353:: 2022.
Wolf. Huggingface's ' transformers: Statc-of-the-art natural language processing, arXiv preprint arXiv:1910.03377, 2019.
Xuansheng Wu, Wenlin Yao, Jianshu Chen, Xiaoman Pan, Xiaoyang Wang, Ninghao Liu, and Dong Yu. From language modeling to instruction following: Understanding the behavior shift in Ilms after instruction tuning. arXiv preprint arXiv:2310.00492 2023.
Lin Xu, Zhiyuan Hu, Daquan Zhou, Hongyu Ren, Zhen Dong, Kurt Keutzer, See-Kiong Ng. and Jiashi Feng. Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration. ICIR 2024 Workshop on Large Language Model (LLM) Agents, 2023.
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho. Sainbayar Sukhbuatar, Jing Xu, and Jason Weston. Self-rewarding language models. arXiv preprint 2.Ki::2401.10020 2024.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9556-9567, 2024.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models, 2024. URL https::///// htttpp::/arxxi.oot www.iiivorrr/..ooooooooo orgpab/////02.. 12772.
Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large language models via preference fine-tuning. arXiv preprint arXiv:2402. 11411, 2024a.
Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405. (40::22,5.444,,,, 2024b.
14

---

Published as a conference paper at ICLR 2025
A THEORETICAL PROOF ON THEOREM 1
Theorem 3. Let X be the true image content with label с and D {d1, d2.d2....dd, be independent and identically distributed (i.id.)) samples from P.D Let Z be expert-defined concept list about label c. Under the assumptions of conditional independence and convergence (Assumptions and 2), JC, the mutual information I ID; Z) converges I(X; Z): lim 1); (X;
Proof. We proceed with the following steps:
A.1 ASSUMPTIONS
- dn conditionally independent Z given Assumption 1. Conditional Independence: d1, d2,-..,
X.
Assumption 2. Convergence of Mutual Information: As co, the mutual information between D and X converges to the entropy of X:
lim I(D:X) H())
1-000
A.2 PROOF STEPS
1. Mutual Information Expression: We start with the definition of mutual information:
I(D;)) = H(D) - H(D]Z)
2. Expanding HDDZZ) Using the Chain Rule: Apply the chain rule for entropy:
H (D|Z) HDD X) + (x H( 2)
Substituting this back into the mutual information expression:
T(D; 7) H(D) [H] + H(X (X Z))
3. Applying Conditional Independence: By Assumption 1, we have:
H(D|Z, = H(D[X)
Therefore,
((;2)==(()) - H)D H(XIZ) H Recognizing (D; = H (((())(((j()))) we have:
[(D;2)=1((>)
= -((I)))H(XXIZ + H(X 2)
4. Taking the Limit as n oc: Apply the limit both sides:
lim I(D lim [TD;;X) H(XIZ) H(X 2)]
By Assumption 2, we have:
lim I(D;X) H(X)
For the term H(XD, Z), we argue that:
lim H(X D. =0 n
This follows from Assumption 2, as it implics that becomes sufficient statistic for X + Therefore, conditioning on Z does not add any information about X once have D.
5. Conclusion: Substituting these limits into our equation:
lim I) = (()) HXX + 1(X;
This demonstrates that the number of sampled descriptions increases indefinitely, the mutual information between the aggregated descriptions D and the human concepts converges to the mutual information between the true image content X and Z.
15

---

Published as a conference paper at ICLR 2025
A.3
ADDITIONAL NOTES
A.3.1 CONVERGENCE OF I(D;X) X) TO H()
The statement:
lim I(D; H()
n-000
This is directly from Assumption 2. To understand its implications:
Recall that mutual information defined as: [(D;] - ((IDD)
For this equality to hold 0o, it must be true that:
lim H =0 120000
This means that we gather more samples (D), we eliminate all uncertainty about X.
In other words, with infinite samples, D contains all information about X.
A.3.2 CONVERGENCE OF Z, TO
The statement:
lim H(X =0 120000
This follows from the previous point. Here's the reasoning:
We've ve established that as n + co, contains all information about X.
This means D becomes a sufficient statistic for X.
A sufficient statistic contains all the information that the sample provides about the param- eter (in this case, X).
Therefore, once we know D, knowing Z doesn't provide any additional information about X.
Mathematically. this means: H( 2) = H(
But we know from the first part that lim,-+00 H(X = 0
Thus, limn- H H(XD, D, Z) =0
B THEORETICAL PROOF ON THEOREM
Theorem 4. Let X, Y, and Z be discrete random variables. Define:
Z* max
max (Xx; (1(Z';
2'C%
Y' arg max I(Y':
Y'CY
Then, the following inequality holds:
I(X;Y; I(Y" ((Y) + I(X; ()
(8)
where ((:-) denotes mutual information and H()) denotes entropy.
Proof. We prove this theorem using fundamental principles information theory:
1. Recall the definition of multivariate mutual information:
T(X; Y: 2) = T(X;Y) -1(X-
(9)
2. By chain rule mutual information (Shi al., 2023), we can rewrite this as:
T(X;Y; 2) T(X; T(Y T(X,Y;
(10)
16

---

Published as a conference paper at ICLR 2025
3. Consider subsets Z and Y*. By definition of mutual information and the data process- ing inequality:
S $ T(
(11)
T(X; & Z(
(12)
4. Substituting these into our equation:
T(:;Y: 2 T(X; + 2²) T(X,Y:
(13)
5. For any random variables A and B:
T(A; B) - (Hin(H(()) H(B)) ((B)
(14)
Therefore:
(X,Y;2)) & mi,) H)) s ((Z:2)
(15)
6. Applying this to our inequality:
1x; > ((X; ((x))
(16)
7. Therefore, we can conclude:
(17)
17

---

Published as a conference paper at ICLR 2025
C
PROMPTS FOR DIFFERENT TASKS
EXAMPLE PROMPT FOR BIRD IMAGE DESCRIPTION IN CUB-200
Prompt
Focus solely the bird shown in the image. Describe the bird's appearance in detail, emphasizing its most prominent physical features. Avoid mentioning the background other elements not related to the bird.
Prompt
Provide a focused analysis the bird this image, detailing distinctive physical features. Concentrate exclusively on the bird and describe its appearance without referencing the surtoundings any extrancous details.
Prompt
Directly observe the bird depicted and offer precise description its visual attributes. Ensure your deseription is limited to the bird itself, detailing its primary features and omitting any unrelated background elements.
Table 3: Prompts for bird image analysis in CUB-200 dataset
EXAMPLE REwRITE PROMPTS FOR DIFFERENT DATASETS
Dataset
Prompt
This is a picture of {label} with the following visual features: {concepts.str sttr.. Based on the information provided, please answer the following question.
cub-200/ / stanford dogs
Question: '(query)'
This is a dermatoscopic image of [label] disease with the following visual features: {concepts str}. Based on the information provided, please answer the following question.
HAM10000
Question: '(query)'
This is a picture of (label} with the following visual features:
(concepts.sti) Based on the information provided, please answer the following question.
PLD/fgve
Question: '(query)'
This is chest-xray of (label) with the following visual features:
{concepts str). Based on the information provided, please answer the following question.
chest-xray
Question: "[query]'
Table 4: Answer rewrite prompts for different datasets.
L+GE SYNTHESIZE PROMPT EXAMPLE
Dataset
Prompt
There is picture of a [label), which is known for the following
characteristics: (concepts_stre
L+GE synthesize prompt
Act as if you can see the picture. Please answer the following question based on the above information. Make your answer concise.
Question: (qquery]'
Answer:
Table 5: Prompt for answering questions based on image characteristics for general image datasets.
18

---

Published as a conference paper at ICLR 2025
EVALUATION PROMPTS FOR EXPLANATION EXISTENCE (EE) AND COGNITION SCORE (CS)
Prompt Type
Prompt
Determine whether the following answer contains valid explanation supporting its conclusion. Respond with only 'true' or 'false'.
EE Prompt
Answer: {answer}
Contains an explanation?
Evaluate the coherence and logical alignment the following explanation with the provided concepts. Please note: the explanation does not need to fully encompass all concepts.
Assign a consistency score between and 1, where indicates the explanation contains irrelevant information to the listed concepts, and 0 indicates complete misalignment with entirely irrelevant information. Only give the score.
CS Prompt
Concepts: (concepts.formattel) formatted] Explanation: (explanation)
Consistency Score:
Table 6: Prompts for Explanation Existence (EE) and Cognition Score (CS).
CONCEPT EXTRACTION PROMPTS
To obtain label-level concepts, we used GPT-4o with prompts designed to elicit detailed visual features associated with each class label. An example prompt is:
Prompt Type Concept
Prompt
Please provide list of visual characteristics that are commonly associated with the bird species (Class Name). Include features such as color patterns, shapes, and distinctive markings.
Extraction
Prompt
Table 7: Prompts for Concept Extraction.
19

---

Published as a conference paper at ICLR 2025
D EXPERIMENTS DETAILS
D.1 TRAINING FRAMEWORK IMPLEMENTATION
Our experimental setup leverages state-of-the-art techniques for fine-tuning the LLaVA-I.5-7B model. We initialize the model using pre-trained weights obtained from the Hugging Face model hub?. Our training process uses the SFTTrainer for supervised fine-tuning and implement Low- Rank Adaptation (LoRA) to achieve efficient parameter fine-tuning. In our configuration, LoRA rank (r) to 128, the scaling factor (ar) to 256, and the dropout rate to 0.1. We fine tune model mixure of domain-specinn synthetic data with LLaVA-1.55 original general instruction- following training data to stabilize training without introducing additional data. We enable gradient checkpointing to reduce memory usage, allowing all intermediate activations. In addition, we employ DeepSpeed ZeRO Stage to fully utilize our multi-GPU configuration. We also utilize mixed precision training with the BF16 format, which further reduces memory requirements while maintaining computational performance. Gradient ac- cumulation is implemented with steps and per-device train batch size of 16. Moreover, slight data augmentation are applied to the image data to create diversity and enhance training robustness.
set the
process larger batch sizes by not storing
D.2 DATASET OVERVIEW AND EXPERIMENTAL RUNNING TIMES
Table 8: Summary of Selected Datasets
Dataset Name
Test Training Cases
Label Categories
Running Time Per Iter
CUB-200-2011 200-2011
5,794 / 5,994
200
143 minutes
FGVC-Aireraft
3,333 / 6,667
100
171 minutes
Stanford Dogs
8,580 12,000
120
310 minutes
Chest X-ray images
624 / 5,216
2
113 minutes
HAM1000
5,008 / 5,007
7
107 minutes
PLD
47.523 5,279
37
109 minutes
We use the original labels and train/test split from the original datasets. Detailed train/test data staties reported in Table 8. Regarding the experiments running time, each iteration of rejection sampling and fine-tuning took approximately 2.5 hours on single NVIDIA H100 GPU for datasets like CUB-200 and FGVC-Aireraf. A more detailed report be found in Table 8.
E VISUALIZATION RESULTS
hhttp::/huugiingffcc..ooo 1ava- lava-1 76
20

---

Published as a conference paper at ICLR 2025
Table 9: Generated Answer Examples.
User:
Based on the visual content, what the specifie name of this bird? Provide an explanation.
LMM:
The specific name of this bird is the Yellow breasted Chat. This bird is known for its bright yellow throat and chest, olive-green back, wings, and tail, and short, rounded wings, is a small bird that is commonly found the Americas, particularly in the United States and Canada.
User:
Based on the visual content, what the specific name of this bird? Provide an explanation.
LMM:
The specific name of this bird is the Blue Grosbeak. The Blue Grosbeak is small, colorful bird with deep blue overall plumage in males and a brown over- all coloration with subtle hlue tinges and faint streaking in females. The hird a slightly glossy sheen to blue feathers and known for its melodious songs.
User:
Based on the visual content, what the specifie name of this bird? Provide an explanation.
LMM:
The specific name of this bird is the Orchard Oriole. The bird has black head, back, throat, and upper chest, with rich chestnul- underparts. It has black and black throat patch developing. white wing bars, and it has a black bill. The Orchard Oriole is small bird with a distinctive appearance, and it is known for its melodious songs.
bird's wings are black with
User:
Based on the visual content, what is the specific name of this bird? Provide an explanation.
LMM:
The specific known for distinctive black throat patch and white stripe below its eyes. is small bird with brown and white coloration, and is often seen perched on branches or low vegetation.
of this bird is the Black throated Sparrow. This bird is
User:
Based on the visual content, what the specific name of this bird? Provide an explanation.
LMM:
The specific name of this bird is the Henslow Sparrow. This bird is known for its olive-colored head, short, pointed bill, and pale eye-ring. It is small bird with brown and white coloration, and it is often seen flttiigg low in dense grassy habitats.
F ADDITIONAL EXPERIMENTS
F.1 IMPACT ON GENERAL ABILITIES
To assess the effect of our proposed training method on the general abilities of the models, conducted evaluations on four widely used multimodal benchmarks: MMStar, SEED-Bench-2 Plus, MMBench, and MME (Cognition). Table 11 summarizes the results.
MMStar (Chen al., 2024b): MMStar is benchmark designed evaluate vision- iision-indispensaalle multimodal capabilities of MLLMs. It comprises 1,500 high-quality samples selected through a meticulous process from initial pool of 22,401 samples. The benchmark assesses six core ca- pabilities, each containing 250 samples, distributed across 18 detailed axes. This structure ensures a comprehensive evaluation of models' performance tasks that require visual understanding.
SEED-Bench-2 Plus (Li et al., 2024a): SEED-Bench-2 Plus is specifically designed to evalu- ate text-rich visual comprehension in MILMs. It features 2,300 multiple-choice questions with precise human annotations, spanning three broad categories: Charts, Maps, and Webs. These categories encompass wide spectrum of real-world text-rich scenarios, effectively simulating environments where visual and textual information are intertwined. The benchmark aims to as- models' proficiency in interpreting complex visual data embedded with textual content.
21

---

Published as a conference paper at ICLR 2025
Table 10: Selected Concepts Examples.
Concepts: "Powerful, thick neck" "Broad and strong back" "Large and muscular build"
"Well-sprung ribs"
Concepts: "Pure white cream-colored fur" tail that blends with the body fur"
"Strong, straight legs" "Broad head with slightly rounded skull"
Concepts:
"Straight, arched tail that rests on the back" "Broad, flat skull" "Thick double coat, either rough smooth"
Concepts:
"Deep-set, almond-shaped eyes" "Erect, triangular ears"
"Thick ruff of fur around the neck"
"Thick double coat, either rough smooth"
Concepts: "Ears: Large, bat-like ears that stand erect and move attentively." "Distinctive facial fcatures: Elongated muzzle with moderate stop and expres- sive eyes."
"Whiskers:
Short or absent due the lack of hair follicles around the muzzle area."
MMBench (Liu et al., 2024e): MMBench is comprehensive benchmark that evaluates MLLMs across 20 ability dimensions. includes approximately 3,000 multiple-choice questions, each with single correct answer. MMBBnch addresses limitations of traditional benchmarks by focus- ing on fine-grained abilities and introducing robust evaluation strategies. The benchmark employs ChatGPT match a model's prediction with the choices of question, providing more reliable assessment of model performance.
MME (Cognition) (Fu et al., 2023): MME (Cognition) benchmark evaluates the cognitive abili- ties of MILMSs through tasks requiring reasoning and understanding of visual and textual inputs. The cognition part includes four subtasks: Commonsense Reasoning, Numerical Calculation, Text Translation, and Code Reasoning. These tasks test the model's ability to integrate multimodal in- formation, such interpreting visual scenes, performing arithmetic based images, translating text in images, and reasoning about code snippets. The dataset consists of carefully curated images and instruction-answer pairs ensure robust and fair assessment.
As shown in Table 11, the fine-tum tne-t models exhibit improved performance across multiple bench- marks. Notably, the model fine-tuned on HAM10000 achieves significant 12.9%, indicating that our training method enhances sppeiff cognition without compromis- ing and, in cases, improves the models' general abilities.
overall improvement of
22

---

Published as a conference paper at ICLR 2025
Table 11: General ability evaluation across additional benchmarks. The results demonstrate that our fine-tuned models not only retain their general abilities but also achieve overall improvements compared the base LLa model.
Model
MMStar
BEED-Beach-2 Plus
MMBench
MME (Cugnition)
Overall Improvement
LLaVA-1.5 (Base)
34,46
41.81
63.05
334.28
Trained CUB-200
33.40
41.78
63.14
355.00
3.2%
Trained on Stamford Dogs
34.93
40.97
63.06
365.71
8.3%
Trained PGVC-Aircraf
63.23
Trained on PLD
35.30
40.89
63,14
337,14
1.1% f
Trained on HAM10000
34.46
41.11
64.08
378.21
12.9%
F.2 EFFECTIVENESS OF FILTERING STRATEGIES
To evaluate the importance of our reward model-free rejection sampling method described in Sec- tion 3.3, we conducted an ablation study comparing our approach with baseline that does not employ the filtering mechanism. In this baseline, the model generates the most probable responses during each iteration, which are used directly for training without any filtering. Table 12 presents the classification accuracy and cognition score (CS) across four iterations for both baseline without filtering and our proposed method.
Table 12:
Comparison of accuracy and cognition scores (CS) for the baseline without filtering and proposed method across iterations.
Dataset
Method
Accuracy (lter I)
Accuracy (Iter 2)
Accuracy (Iter 3)
Accuracy (Tter 4)
CS Score
woo Fillering
68.90
70.11
70.85
70.45
0.71
CUB 200
Ours
FGVC-Aireratt
w/o Fillering
76.36
76.60
77.11
76.78
0.72
Ours
88.78
90.91
0.79
w/o Filtering
Stanford Dogs
Ours
85.29
86.75
86.86
86.91
0.86
w/o Filtering
61. 19
63.74
62.32
62.59
0.66
Ours
75.96
92.80
96.59
97.16
0.86
w/o Filtering
71.79
72.10
72.34
72.29
0.75
HAM10000
Ours
79.37
82.29
0.87
From Table 12, it is evident that filtering strategy significantly enhances both classification accuracy and explanation quality, as measured by the CS score. The baseline without filtering shows marginal improvements initially but fails to achieve comparable performance our method. This demonstrates the critical role of our filtering mechanism in refining synthetic data and improving the model's performance iteratively.
F.3
IMPACT OF TEXT ENCODER
Our framework relies on a text embedding model for estimating mutual information during con- cept selection (Section 3.2). To assess the impact of different text encoders on concept selection accuracy, we compared three models: E5 (Wang et al., 2022), BERT-Large (Devlin, 2018), and BERT-Base (Devlin, 2018). The results are presented in Table 13.
Table 13: Concept selection accuracy using different text encoders.
Text Encoder
Concept Selection Accuracy (%)
E5
72.9
BERT-Large BERT-Base
71.4
69.7
As shown in Table 13, the E5 model achieves the highest concept selection accuracy. This indicates that using a more powerful text embedding model improves the quality of concept selection, which in turn enhances the effectiveness of our overall framework.
23

---

Published as a conference paper at ICLR 2025
F.4 CROSS-DATASER TRANSFER PERFORMANCE
We conducted experiments to evaluate the cross-dataset transferability of our method. Specifically, trained the model the CUB-200 dataset and evaluated its performance the Stanford Dogs dataset. The results are shown in Table 14.
Table 14: Cross-dataset transfer performance when training CUB-200 and evaluating on Stanford Dogs.
Training Dataset
Evaluation Dataset
Accuracy (%)
None
Stanford Dogs
12.20
CUB-200
Stanford Dogs
Stanford Dogs
Stanford Dogs
86.91
The results in Table 14 indicate that training on CUB-200 provides marginal improvement when evaluated on Stanford Dogs. However, the performance remains significantly lower than when the model is trained directly on Stanford Dogs. This suggests that while our method may improve general cognition to some extent, domain-specific fine-tuning is crucial for achieving high accuracy in specialized tasks.
F.5 EXPLANATION QUALITY IMPROVEMENT ACROSS ITERATIONS
To assess the benefits of our iterative fine-tuning approach, we evaluated the Cognition Score (CS) of the models across four iterations for various datasets. The results are summarized in Table 15.
Table 15: Cognition Scores (CS) across iterations for various datasets. The CS Improvement repre- sents the percentage increase from Iteration 1 to Iteration 4.
Datasel
CS Value (Iter I)
CS Value (lter 2)
CS Value (Iter 3)
CS Value (Iter 4)
CS Improvement
CUB-200
0.77
0.76
0.78
0.82
6.5%
Stanford Dogs
RGVC-Arrerat
0.78
0.78
0.78
0.79
1.3%
PLD
0.85
2.4%
HAM10000
0.77
0.84
0.83
0.87
13.0%
Chesi may
0.67
0.80
0.81
0.87
29.9%
As observed in Table 15, the CS scores generally improve over iterations, indicating that our iterative fine-tuning process enhances the explanation quality of the models. The most significant improve- ments are seen the HAM10000 and Chest X-ray datasets, with 13.0% and 29.9 % reported.
24


Reviews:
[{"review_id": "e7xD55KSVS", "reviewer": "Reviewer_mX83", "summary": "This paper propose a new framework to improve LMMs in domain-specific visual classification by enhancing cognition and explainability through iterative finetuning and the Information Bottleneck principle. This approach boosts accuracy and interpretability without extensive annotations, making LMMs more applicable in specialized fields. Future work may explore more complex tasks and scalability improvements.", "questions": "* is the choice of text encoder significantly impact the results of data selection?\n* is there any qualitative example that demonstrate the effectiveness of the data selection strategy?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The writing of the paper is logically clear and easy to follow.\n* An interesting and effective self-improving approach, which enhances the model's performance in fine-grained classification tasks.", "weaknesses": "* need evaluate on more general benchmarks, e.g. MMVet, MMStar, and SEED-Bench2...\n* it is better to report experimental runtime.\n* missing important ablation study: to validate the effectiveness of the two data selection strategies, results using unfiltered data for training should be provided as a comparison baseline.\n* minor: In Table 1, highlighting the best results would improve readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DrYEQzeFVX", "reviewer": "Reviewer_xodD", "summary": "In this paper, the authors argue that the limited domain-specific knowledge in the current the LMMs. The main challenge for such limited parametric knowledge is on lack of data to further fine-tune the models. Accordingly the authors propose a new synthetic data generation pipelines from the model themselves and fine-tune the models with the generated data to enhance domain-specific cognitive capabilities.", "questions": "1. What are the trainable parameters during the fine-tuning stage? The text-aligned vision encoder such as CLIP lacks detailed visual perception due to the training nature (focusing on contrastive learning with paired text supervision). The reviewer wonders where the performance gain is obtained from. If fine-tuning is only applied to the parametric knowledge of the LLM side, does this adequately address the encoder’s deficiencies in processing complex visual data?\n\n2. Continually, the reviewer wonders whether there are any negative effects of using synthesized data for fine-tuning. Could this potentially harm the model's general QA capabilities? Can the authors compare the general benchmark results before and after the finetuing?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The main challenge for data generation synthesis is on how to assure the self-generated data quality and its reliability. The authors cleverly employ information bottleneck theory to extract intersect knowledge from exterior knowledge (such as from expert or gpt-like models) and the visual contents. In addition, by introducing a doubly robust rejection sampling strategy improves both the accuracy and interpretability of the model prediction.", "weaknesses": "Evaluation is somewhat limited due to the main results are mainly conducted on classification task. Therefore, during the fine-tuning stage with the synthesized data, it can potentially negatively impact the general model QA capabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hCsU2Bden1", "reviewer": "Reviewer_Hq7v", "summary": "This paper proposes a novel framework to improve the explainability of large multimodal models (LMMs) in visual classification tasks. The authors address the challenge of LMMs struggling to identify domain-specific objectives and explain their predictions. Their approach involves fine-tuning LMMs using self-synthesized data, which includes interpretable answers containing human-verifiable visual features. The framework iteratively generates high-quality training data by leveraging the model's captioning abilities and filtering synthesized outputs using a reward model-free mechanism. This method enhances the model's ability to generate accurate and justifiable explanations for its classifications, without relying on extensive manual annotations. The paper presents experimental results demonstrating the effectiveness of the proposed framework across various datasets and discusses the theoretical foundations behind it.", "questions": "If the authors can well address the questions in the weakness section, I will consider increase the rating.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "*   **Clear Problem Identification and Justification:** The authors successfully identify the challenge of LMMs struggling with domain-specific visual classification and explainability. They pinpoint insufficient domain-specific alignment as the root cause, highlighting that LMMs often fail to link key visual features with correct labels and provide justifiable explanations for predictions. \n*   **Significant Accuracy Improvement:** The paper demonstrates a substantial increase in classification accuracy through its proposed method compared to training with labels alone. The authors showcase this improvement across multiple iterations, highlighting the efficacy of their approach in achieving more robust classification.\n*   **Novel Framework and Theoretical Grounding:** The authors introduce a novel framework for enhancing LMMs' domain-specific cognition and explainability using self-synthesized interpretable answers. The framework's two key steps, image-level visual concept selection and reward model-free rejection sampling, are thoroughly described and grounded in information theory. \n*   **Multifaceted Evaluation of Explanation Quality:** The authors address the difficulty of assessing explanation quality without ground truth annotations by employing a multi-faceted evaluation approach. This involves metrics like Explanation Existence (EE), Cognition Score (CS), and Fluency Score (FS), each offering insights into different aspects of the generated explanations. \n*   **Clear Writing and Presentation:** The paper is commended for its clear writing, making it easy to understand the presented concepts and methods. The use of illustrative figures and tables further enhances the clarity and impact of the research.", "weaknesses": "*   **Generalization to Unseen Categories:** The paper doesn't explicitly address the model's ability to generalize to unseen categories. While the method shows promise in fine-grained classification within specific datasets, it remains unclear whether the model can effectively transfer its learned knowledge to classify entirely new object categories. Further investigation is needed to determine if the performance gains are limited to the fine-tuned categories or if the model truly unlocks a broader understanding of visual concepts that can extend to novel classes.\n\n*   **Dependence on Base LMM and Lack of Ablation Studies:** The proposed method relies heavily on the capabilities of the chosen base LMM (LLaVA-1.5). The paper doesn't provide ablation studies exploring the impact of different base LMMs or their pre-training data on the final performance. It is possible that the performance is still bounded by the limitations of LLaVA-1.5. To strengthen the paper's claims, investigations into the sensitivity of the method to different base LMMs and their training data would be beneficial.\n\n*   **Label Generation and Testing Set Validity:** The paper doesn't fully explain how to create a valid testing set when the labels used for training are generated by the model itself. This raises concerns about potential biases and circularity in the evaluation process. Further clarification is needed on how to ensure the independence and representativeness of the testing set to provide a more reliable assessment of the model's true performance.\n\n*   **Acquisition of Label-Level Concepts:** The process of obtaining the Label-level concepts (Z) is not explicitly outlined in the paper. While the paper mentions expert-defined concepts, it is unclear whether this implies manual annotation by human experts or if these concepts can be acquired automatically. If human expert annotation is still required, it would introduce a significant bottleneck in the scalability of the approach. Further details on the origin and acquisition of these concepts would be helpful.\n\n*   **Potential Overfitting to Explanation Format:** The paper focuses on generating explanations in a specific format. It is possible that the model overfits to this format, limiting its ability to generalize to other explanation styles or answer formats. Exploring the model's performance with different explanation formats or encouraging the model to generate more diverse responses would strengthen the paper's conclusions regarding the general explainability of the fine-tuned model.\n\n*   **Limited Comparison with External Works:** The experimental evaluation in Table 1 primarily compares the proposed method against self-created baselines (NL and L+GE). While these baselines offer a useful reference point, including comparisons with other external works in explainable visual classification would provide a more comprehensive assessment of the method's strengths and limitations. Additionally, comparing the performance with methods that use manually annotated labels as an upper bound would be valuable for contextualizing the achieved results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E01EGUNI2C", "reviewer": "Reviewer_hKB4", "summary": "This paper proposes to improve LLMs’ capabilities in fine-grained image classification. In particular, it focuses on an iterative approach that derives synthetic explanations by identifying discriminative properties of the stimuli, and leveraging them for fine-tuning. The key properties are selected by measuring their cosine similarity with a pool of expert annotations, where the properties for each stimulus are determined by prompting a pretrained vision-language model. Experimental results on multiple datasets demonstrate the usefulness of the approach in enhancing the model performance and generating detailed explanations for the answers.", "questions": "(1) Please justify the effects of domain shift over improvement in reasoning capabilities and explanability. \n\n(2) After fine-tuning on the specific tasks, do the LLMs still perform well in general reasoning? \n\n(3) Does learning on one dataset benefit fine-grained classification in general? For instance, does training on CUB-200 lead to improvement in Stanford dogs?\n\n(4) Please consider including additional baselines for comparison, as noted in the Weaknesses section.\n\n(5) How does the iterative process benefit the derivation of discriminative concepts? A quantitative analysis would be reasonable.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "(1) Improving the interpretability of LLMs is an important problem, and the focus on challenging classification problems is a reasonable direction.\n\n(2) The paper proposes a general framework for automatically constructing explanations for fine-grained classification, which shows promise in simultaneously enhancing the performance and interpretability of LLMs.\n\n(3) The paper takes into account various automatic evaluation metrics and also conducts user studies for model validation, providing a comprehensive view of the effectiveness of the proposed method.", "weaknesses": "(1) It is not surprising that off-the-shelf LLMs can fail to provide reasonable explanations for fine-grained classification, as their aim for general applications does not align with the requirement for expert-level knowledge for specific tasks. My general concern for the paper is that, how much of the improvement comes from the shift of domain caused by fine-tuning, and whether or not such changes will harm the generalizability of LLMs. As shown in Table 1, despite the advantage of the proposed method,  a general model actually has equivalent performance among many metrics. \n\n(2) Related to the previous comment. I believe the comparison in Table 1 is missing some important baselines. In particular, the compared methods either are not trained on fine-grained classification (Base), or do not take into account explanations with reasonable quality (i.e., NL does not use any explanation and thus is expected to fail at interpretation, while L+GE only has access to explanations derived from the class labels alone). On the other hand, the proposed method has full access to both visual and language data, enabling it to perform a more thorough domain shift. For a fair comparison, it would be reasonable to include another baseline that is trained with explanations derived by prompting a vision-language model on each image. This experiment can also be used to validate the effectiveness of the proposed sampling method, i.e., how well its filtered properties outperform unfiltered ones. \n\n(3) Furthermore, I would encourage the authors to additionally consider evaluations on both the fine-grained and general classification (e.g., via zero-shot setting). This will enable us to explore additional research questions, e.g., whether fine-tuning on expert-level explanations leads to stronger general reasoning capabilities, or whether the challenge behind LLM explainability is rooted in the conflict between general and specific domains.\n\n(4) The paper emphasizes an iterative approach for selecting key features for explanations. Nevertheless, other than reporting the model accuracy for each iteration, there is no analysis regarding how it affects the quality of explanations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)