Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum

## Abstract

Abstract The effectiveness of Multimodal Chain-of-Thought (MCoT) prompting is often limited by the use of randomly or manually selected examples. These examples fail to account for both model-specific knowledge distributions and the intrinsic complexity of the tasks, resulting in suboptimal and unstable model performance. To address this, we propose a novel framework inspired by the pedagogical principle of “tailored teaching with balanced difficulty”. We reframe prompt selection as a prompt curriculum design problem: constructing a well ordered set of training examples that align with the model’s current capabilities. Our approach integrates two complementary signals: (1) model-perceived difficulty, quantified through prediction disagreement in an active learning setup, capturing what the model itself finds challenging; and (2) intrinsic sample complexity, which measures the inherent difficulty of each question–image pair independently of any model. By jointly analyzing these signals, we develop a difficulty-balanced sampling strategy that ensures the selected prompt examples are diverse across both dimensions. Extensive experiments conducted on five challenging benchmarks and multiple popular Multimodal Large Language Models (MLLMs) demonstrate that our method yields substantial and consistent improvements and greatly reduces performance discrepancies caused by random sampling, providing a principled and robust approach for enhancing multimodal reasoning.

## 1 Introduction

Multimodal Large Language Models (MLLMs)  Yin et al. (2024); Liu et al. (2023a) have emerged prominent research focus alongside the rapid advancement of artificial intelligence. Built upon powerful foundation language models  Touvron et al. (2023); Bai et al. (2023), MLLMs leverage cross-modal alignment mechanisms to achieve understanding and processing of information across multiple modalities, including text, images, videos, and audio. A typical approach to deploying MLLMs is the in-context learning paradigm  Brown et al. (2020); Xie & Min (2022), which drives models to perform predictions by providing a large number of instructions and input-output pair examples. Chain-of-Thought  Wei et al. (2022); Zhou et al. (2022) enhances the logical reasoning capability of models by constructing examples that decompose complex problems into step-by-step subproblems and solve them sequentially. Multimodal Chain-of-Thought (MCoT)  Zhang et al. (2023), on the other hand, further extends this core idea to the application scenarios of multimodal large language models, achieving an improvement in cross-modal reasoning capabilities.

Figure: Figure 1: Illustration of the motivation and key highlights of our proposed framework. (a) CoT uses random/manual prompts without analyzing model knowledge distribution or dataset features; (b) Auto-CoT uses clustering for representative prompts but ignores inter-model knowledge differences; (c) CAMS (Ours) screens optimal prompts via active learning uncertainty and complexity analysis, balancing difficulty to enhance effectiveness. We adopt the same multimodal input consisting of images and questions as in (a) and (b) for CAMS.
Refer to caption: x1.png

MCoT encourages large language models to perform multi-step reasoning by providing explicit problem-solving rationales, rather than mapping questions directly to answers. This approach has been effectively applied to scientific question answering across domains such as natural sciences, linguistic sciences, and social sciences. However, as illustrated in Figure [1](https://arxiv.org/html/2508.18673v2#S1.F1)(a), MCoT prompt examples are typically randomly selected or manually crafted without considering the model’s internal knowledge distribution or the characteristics of the dataset. This leads to prompts that are unstable and insufficiently tailored to the model or task. As a result, MCoT prompting often suffers from poor generalization, hallucinated outputs, and inconsistent performance. To enhance the specificity and effectiveness of prompt examples, Auto-CoT  Zhang et al. (2022) constructs prompts by selecting representative examples via clustering, as shown in Figure [1](https://arxiv.org/html/2508.18673v2#S1.F1)(b). However, due to the inherent differences between modalities, the effectiveness of Auto-CoT in multimodal scenarios is limited. It fails to adequately account for the distributional differences in internal knowledge across different models.

These challenges raise a critical question: how to selectively identify the most effective multimodal prompt examples? In human learning, individuals often compile personalized sets of challenging problems ranging from basic misunderstandings due to knowledge gaps to complex tasks that require multi-step reasoning. Inspired by the idiom “Tailor teaching to individual needs”, we treat each multimodal large model as a unique learner and select prompt examples through two dimensions: uncertainty analysis and complexity evaluation.

We propose CAMS (Complexity-Guided Active Multimodal CoT Sampling), a novel joint selection framework that integrates active learning and data complexity assessment. Rather than relying on randomly selected or manually crafted MCoT examples, CAMS constructs a tailored “error sets” for each model based on the training sets of various reasoning tasks, aiming to enhance the models’ performance on test sets. As shown in Figure [1](https://arxiv.org/html/2508.18673v2#S1.F1)(c), CAMS identifies highly targeted, effective, and difficulty-balanced prompt examples by jointly analyzing sample uncertainty and complexity. We conduct experiments on five benchmark datasets and three multimodal large models. The results show that CAMS both improves model performance and greatly reduces accuracy variability caused by random prompt selection. Our key contributions are as follows:

- •
We introduce CAMS,the first framework to select prompt examples based on both model-internal knowledge and dataset characteristics.
- •
We demonstrate that effective prompting requires a balance of easy and hard examples; neither extreme is sufficient on its own.
- •
CAMS greatly reduces the instability of traditional prompt selection methods and enables MLLMs to achieve stable, high performance on complex reasoning tasks.

## 2 Related Work

### 2.1 Prompt Selection based on Active Learning

Active learning is a machine learning paradigm focused on maximizing model performance using the fewest possible labeled samples. It aims to identify the most informative unlabeled data points for annotation, thereby reducing labeling costs while maintaining high accuracy. Active learning methods are typically grouped into the following three categories based on how unlabeled data is queried: membership query synthesis Angluin (1988); King et al. (2004), where the model generates new instances for labeling; stream-based selective sampling Dagan & Engelson (1995); Krishnamurthy (2002), where data points are evaluated one at a time for potential labeling; and pool-based sampling Lewis (1995), where the model selects the most informative samples from a large pool of unlabeled data.

Active Prompt Diao et al. (2023) applies active learning principles to prompt selection by quantifying uncertainty through prediction disagreement metrics (e.g., variance, entropy, disagreement) and choosing high-uncertainty samples as prompt examples. However, Active Prompt focuses solely on model’s prediction disagreement with samples (i.e., distributional differences in model-internal knowledge) without considering sample complexity. Our approach introduces a data complexity evaluator that assesses the inherent difficulty of samples. This allows for more customized and effective prompt selection, combining insights from both model knowledge and dataset characteristics.

### 2.2 Chain-of-Thought in Visual Question Answering

The Multimodal Chain of Thought (CoT) technique is widely adopted to enhance the multi-step reasoning abilities of large language models (LLMs). Its core idea is to guide models to generate intermediate reasoning steps that help them solve complex problems more effectively. Benchmarks such as VQA Antol et al. (2015), VQAv2 Goyal et al. (2017), OK-VQA Marino et al. (2019), A-OKVQA Schwenk et al. (2022), and ScienceQA Lu et al. (2022) provide structured visual question answering (VQA) tasks across various domains, including natural science, social science, semantics, and everyday reasoning. For complex reasoning tasks, recent approaches like MCoT Zhang et al. (2023), Auto-CoT Zhang et al. (2022), Self-Consistency Wang et al. (2022), and Active Prompt Diao et al. (2023) leverage carefully designed prompt examples to improve model performance.

Despite these advancements, many methods Wei et al. (2022); Wang et al. (2022); Zhou et al. (2022) rely on either randomly selected or manually crafted prompt examples. These examples often fail to align with the specific demands of individual VQA tasks, limiting model performance. In particular, they tend to overlook key factors such as the distributional characteristics of the model’s internal knowledge and the multimodal nature of VQA tasks, focusing primarily on unimodal scenarios. To address these limitations, our framework jointly considers the model’s uncertainty about the dataset and the intrinsic complexity of each example. By integrating both model-centric and data centric perspectives, we dynamically construct prompt examples that are better tailored to the model’s current capabilities and the reasoning requirements of the task, leading to more effective and adaptive prompting in visual question answering.

Figure: Figure 2: The illustration of our CAMS framework. Dataset consists of multimodal inputs of images and text. Complexity-Based Dataset Feature Estimation calculates complexity by integrating question text and image captions to evaluate dataset characteristics. Analysis of Multimodal Model Internal Knowledge reveals the distribution of the model’s internal knowledge through the uncertainty of the model’s multiple predictions.
Refer to caption: x2.png

## 3 Methodology

Figure [2](https://arxiv.org/html/2508.18673v2#S2.F2) illustrates the three core modules of CAMS: (i) Characterization of Multimodal Model Internal Knowledge, which quantifies the model’s predictive uncertainty through multiple independent samplings of the model, revealing the distribution characteristics of the model’s internal knowledge; (ii) Complexity-Based Dataset Feature Estimation, which is used to quantitatively evaluate dataset characteristics; (iii) Examples Sampling Strategy, which incorporates model uncertainty indicators, dataset complexity scores, and the “easy-hard” selection principle to dynamically identify the most representative and effective prompt examples for the target model.

### 3.1 Problem Definition

We define a visual question answering dataset as $D=\{(x_{i},v_{i},y_{i})\}^{N}_{i=1}$, where $x_{i}$ represents the linguistic text query, $v_{i}$ represents the corresponding visual image input, $y_{i}$ denotes the ground truth, and N is the total number of test samples. The goal of the prompt optimization task is to search for the optimal prompt $p^{*}$ that maximizes the performance $\mathcal{A}(\cdot)$ of large language models (LLMs) on a given task. This task can be formally defined as:

$$ $p^{*}=\mathop{argmax}\limits_{p\in\mathcal{P}_{space}}\sum^{N}_{i=1}\mathcal{S}(\mathcal{A}(x_{i},v_{i};p),y_{i})$ (1) $$

where $\mathcal{P}_{space}$ denotes the set of all possible prompts (automatically selected or manually crafted), and $\mathcal{S}(\cdot)$ represents the corresponding evaluation metric.

### 3.2 Characterization of Multimodal Model Internal Knowledge

#### 3.2.1 Disagreement.

We define the training sample set as $\mathcal{D}_{train}=\{q_{1},q_{2},...,q_{n}\}$, where $q_{i}$ denotes an unlabeled sample and $n$ is the total number of training samples. We instruct the MLLM $\mathcal{F}_{\theta}(\cdot)$ to sample from the training set $\mathcal{D}_{train}$, generating the sample results:

$$ $\mathcal{F}^{i}_{\theta}=\{\mathcal{F}^{i}_{\theta}(q_{1}),\mathcal{F}^{i}_{\theta}(q_{2}),...,\mathcal{F}^{i}_{\theta}(q_{n})\}$ (2) $$

where $\mathcal{F}_{\theta}(q_{i})$ represents the model’s response to sample $q_{i}$.
After performing k sampling iterations, the final results are denoted as :

$$ $\mathcal{A}=\{a_{1},a_{2},...,a_{k}\}$ (3) $$

where $a_{i}=\{\mathcal{F}^{1}_{\theta}(q_{i}),\mathcal{F}^{2}_{\theta}(q_{i}),...,\mathcal{F}^{k}_{\theta}(q_{i})\}$ represents the k sampling outcomes for sample $q_{i}$, $\mathcal{F}^{j}_{\theta}(q_{i})$ denotes the j-th sample generated by the model for input, and $\mathcal{A}$ represents the collection of $k$ sampling results for all samples. We then compute unique answer via set operations to remove duplicates, yielding $k^{\prime}$ unique items $a_{i}=\{\mathcal{F}^{1}_{\theta}(q_{i}),\mathcal{F}^{2}_{\theta}(q_{i}),...,\mathcal{F}^{k^{\prime}}_{\theta}(q_{i})\}$, where $\frac{k^{\prime}}{k}\in(0,1.0]$ denotes the disagreement metric $u$. A larger value of the disagreement metric $u$ indicates a higher difficulty level for the sample.

#### 3.2.2 Clustering.

After assigning each training sample a corresponding uncertainty metric $u$, we perform clustering on all training samples based on the numerical values of $u$, yielding a new training sample set. Specifically, we classify samples with
$u\in(0.5,1.0]$ as difficult questions; conversely, samples with $u\in(0,0.5]$ are classified as easy questions.

$$ $\displaystyle\mathcal{D}^{diff}_{train}$ $\displaystyle=\{\mathcal{D}_{difficulty},\mathcal{D}_{easy}\},$ (4) $\displaystyle\mathcal{D}_{difficulty}$ $\displaystyle=\{\mathcal{D}_{u_{i}}\}_{0.5<u_{i}\leq 1.0},$ $\displaystyle\mathcal{D}_{easy}$ $\displaystyle=\{\mathcal{D}_{u_{j}}\}_{0<u_{j}\leq 0.5}.$ $$

where $u_{i}$ represents the specific disagreement value, and $\mathcal{D}_{u_{i}}$ represents the set of samples whose disagreement metric equals $u_{i}$.

### 3.3 Complexity-Based Dataset Feature Estimation

##### Training of the Complexity Scorer.

We introduce an evolution-based metric, Evol Complexity. We define a small-scale seed dataset
$\mathcal{D}=\{(I^{(0)}_{1},R^{(0)}_{1}),(I^{(0)}_{2},R^{(0)}_{2}),...,(I^{(0)}_{N},R^{(0)}_{N})\}$, where $(I^{(0)}_{i},R^{(0)}_{i})$ denotes instruction-response pairs and $N$ denotes the number of instruction-response pairs. For each instruction sample $I$ , we enhance its complexity through Technique $\mathcal{F}_{\alpha}(\cdot)$, which involves adding constraints, specification, and increasing reasoning steps. After $N$ iterations, a set of instructions with varying complexities is obtained:

$$ $\{(I^{(0)}_{i},R^{(0)}_{i}),(I^{(1)}_{i},R^{(1)}_{i}),...,(I^{(M)}_{i},R^{(M)}_{i})\}$ (5) $$

where $I^{(m)}_{i}=\mathcal{F}_{\alpha}(I^{(m-1)}_{i})$ , $M$ denotes the number of sample evolution iterations and is set to 5. Further, we utilize the scoring function $\mathcal{S}(\cdot)$ (i.e. ChatGPT) to rate and rank these 6 samples, generating a set of instructions with scoring labels:

$$ $\{(I^{(j)}_{i},\mathcal{S}(I^{(j)}_{i}),R^{(j)}_{i})\}^{M}_{j=0}$ (6) $$

Composed of these labeled instruction groups, a dataset is constructed to train LLaMA as the complexity scorer. We use the Llama2-7B model trained on the “6k SFT + 10k DPO” dataset as the final complexity scorer. Experiments demonstrate that this model can effectively classify sample complexity, with an AlpacaEval score of 90.06% Liu et al. (2023b).

##### Computation of Complexity.

This scheme aims to convert images in multimodal data into text form (image captions) and integrate them with question texts to form unimodal inputs, enabling the reuse of existing unimodal complexity scorers. Specifically, we observe that each image in every multimodal dataset is accompanied by a corresponding textual description (i.e., a caption), and we thus decide to make full use of these image captions. The image captions are then integrated with the corresponding questions and options to form complete unimodal inputs in pure text. We adopt the concatenation format of “question text + option text + image caption”, using the delimiter ${}^{\prime}\backslash n^{\prime}$ to ensure that the scorer can recognize the text representation of visual information. Finally, the integrated text data is fed into the complexity scorer, which outputs the corresponding sample complexity score.

### 3.4 Strategies for Selecting Examples

We employ a difficulty-balanced sample selection strategy. Specifically, for subsets of questions categorized as difficult and error-prone, we select an equal number of high-complexity and low-complexity questions; this balanced selection criterion is equally applied to subsets of simple and fundamental questions. This selection strategy fully incorporates two dimensions: uncertainty and complexity. !!!! It is important to note that all test samples within a given dataset share the prompt examples selected through this method, rather than each test sample using a distinct set of prompt examples.

## 4 Experiments

In this section, we conduct extensive experiments to validate the effectiveness of CAMS (Complexity-Guided Active Multimodal CoT Sampling). Our experiments aim to address the following core questions:

- Q1.
Compared with existing baseline methods, can CAMS improve the accuracy of final answers?
- Q2.
Can CAMS eliminate the instability of randomly selecting prompt examples?
- Q3.
Are the designs of our two modules (active learning and complexity scoring) both meaningful?
- Q4.
Can CAMS enhance the accuracy of multimodal large models in subdivided domains?
- Q5.
Why should we select examples with uniform uncertainty for multimodal large models?

### 4.1 Experimental Settings

#### 4.1.1 Datasets and Metrics.

Following the standard evaluation settings in MLLMs reasoning research,we conduct testing on five popular benchmarks: ScienceQA Lu et al. (2022), A-OKVQA Schwenk et al. (2022), OK-VQA Marino et al. (2019), VQAv2 Goyal et al. (2017) and TextVQA Singh et al. (2019). The problem designs in these datasets include both multiple-choice and open-ended answer formats, and we follow the practice Liu et al. (2024); Li et al. (2024); Bai et al. (2023) of using accuracy as the metric.

#### 4.1.2 Baselines and Model Variants.

In our experiment, the following five methods are used as the main baselines: Chain-of-thought (ZS-CoT) Wei et al. (2022), Few shot CoT (FS-CoT), Self-consistency (Self-Con) Wang et al. (2022), Auto-CoT Zhang et al. (2022) and Active Prompt (Active-Pro) Diao et al. (2023). FS-CoT uses the same annotation process as our method, the only difference being that it randomly selects questions from the training data for annotation. We select multiple cutting-edge MLLMs as experimental models, including llama3.2-vision:11b, llava:7b, and Qwen2.5-VL:7b.

### 4.2 Implementation Details

#### 4.2.1 Hyperparameters.

In the training set sampling phase, we follow the optimal value of 10 Diao et al. (2023) for the number of sampling iterations. During both the sampling and the inference phases of the experiments, the temperature is uniformly set at $0.5$. Unless otherwise specified, the multimodal large language models used in the experiments are llama3.2-vision:11b, llava:7b, and Qwen2.5-VL:7b.

#### 4.2.2 Uncertainty Assessment.

In the experimental process, we adopt a zero-shot sampling strategy, which does not rely on additional examples or guidance information. For ScienceQA, A-OKVQA, and VQA, we perform sampling on the complete training sample sets, while for the VQAv2 and TextVQA datasets, we only sample 10,000 samples(^1^11Because the training sets of these two datasets are too large, full sampling would require substantial computational and time costs.). For the sampling frequency parameter, We follow the optimal value Diao et al. (2023) to set $k=10$. We consistently use “Disagreement”(^2^22In preliminary experiments, variance and entropy exhibit suboptimal performance in multimodal scenarios.)—a more intuitive and accurate method—as the uncertainty metric.

#### 4.2.3 Constructing Examples.

We focus on the innovating and optimizating strategies for prompt example selection. To reduce human labor as much as possible, we eliminate the need for manual annotation of prompt examples. Instead, we construct prompt instances by directly concatenating the question, reasoning, and answer from the dataset. The solution processes in the dataset are carefully crafted and rigorously screened by the authors, considering domain knowledge or task-specific styles. This ensures the generalization ability of our method to a certain extent. Details are provided in Appendix [A.1](https://arxiv.org/html/2508.18673v2#A1.SS1).

**Table 1: Overall results ($\%$) on five benchmarks. In each setting, the best results are displayed in bold and italics, with gray shading indicating the degree of improvement compared to ZS-CoT.**
| METHOD | Datasets | Avg. |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| ScienceQA | A-OKVQA | OK-VQA | VQAv2 | TextVQA |  |  |
|  | Llama3.2-vision:11b |  |  |  |  |  |
| ZS-CoT | 38.08 $\uparrow 0.000$ | 47.42 $\uparrow 0.000$ | 28.74 $\uparrow 0.000$ | 57.45 $\uparrow 0.00$ | 37.52 $\uparrow 0.000$ | 41.842 $\uparrow 0.0000$ |
| FS-CoT | 60.42 $\uparrow 22.34$ | 55.88 $\uparrow 8.460$ | 50.92 $\uparrow 22.18$ | 59.95 $\uparrow 2.50$ | 65.00 $\uparrow 27.48$ | 58.434 $\uparrow 16.592$ |
| Auto-CoT | 39.07 $\uparrow 0.990$ | 59.82 $\uparrow 12.40$ | 48.13 $\uparrow 19.39$ | 63.57 $\uparrow 6.12$ | 61.14 $\uparrow 23.62$ | 54.346 $\uparrow 12.504$ |
| Active-Pro | 40.44 $\uparrow 2.360$ | 57.55 $\uparrow 10.13$ | 46.86 $\uparrow 18.12$ | 60.12 $\uparrow 2.67$ | 58.86 $\uparrow 21.34$ | 52.776 $\uparrow 10.924$ |
| Self-Con | 50.00 $\uparrow 11.92$ | 54.41 $\uparrow 6.990$ | 43.56 $\uparrow 14.82$ | 66.04 $\uparrow 8.59$ | 54.85 $\uparrow 17.33$ | 53.772 $\uparrow 11.930$ |
| Ours | 68.22 $\uparrow 30.14$ | 59.39 $\uparrow 11.97$ | 51.89 $\uparrow 23.15$ | 61.89 $\uparrow 4.44$ | 62.76 $\uparrow 25.24$ | 60.830 $\uparrow 18.988$ |
|  | Llava:7b |  |  |  |  |  |
| ZS-CoT | 41.10 $\uparrow 0.000$ | 59.39 $\uparrow 0.00$ | 3.910 $\uparrow 0.000$ | 7.820 $\uparrow 0.000$ | 0.300 $\uparrow 0.000$ | 22.504 $\uparrow 0.0000$ |
| FS-CoT | 59.79 $\uparrow 18.69$ | 62.31 $\uparrow 2.92$ | 31.82 $\uparrow 27.91$ | 29.36 $\uparrow 21.54$ | 20.43 $\uparrow 20.13$ | 40.742 $\uparrow 18.238$ |
| Auto-CoT | 56.12 $\uparrow 15.02$ | 62.18 $\uparrow 2.79$ | 34.34 $\uparrow 30.43$ | 30.80 $\uparrow 22.98$ | 20.48 $\uparrow 20.18$ | 40.784 $\uparrow 18.280$ |
| Active-Pro | 57.20 $\uparrow 16.10$ | 61.39 $\uparrow 2.00$ | 34.10 $\uparrow 30.19$ | 30.69 $\uparrow 22.87$ | 20.58 $\uparrow 22.87$ | 40.792 $\uparrow 18.288$ |
| Self-Con | 57.91 $\uparrow 16.81$ | 62.10 $\uparrow 2.71$ | 14.06 $\uparrow 10.15$ | 11.00 $\uparrow 3.180$ | 9.140 $\uparrow 8.840$ | 30.842 $\uparrow 8.3380$ |
| Ours | 62.53 $\uparrow 21.43$ | 63.41 $\uparrow 4.02$ | 34.46 $\uparrow 30.55$ | 33.07 $\uparrow 25.25$ | 20.78 $\uparrow 20.48$ | 42.850 $\uparrow 20.346$ |
|  | Qwen2.5-VL:7b |  |  |  |  |  |
| ZS-CoT | 40.16 $\uparrow 0.000$ | 39.39 $\uparrow 0.000$ | 6.910 $\uparrow 0.000$ | 5.180 $\uparrow 0.000$ | 1.900 $\uparrow 0.00$ | 18.708 $\uparrow 0.0000$ |
| FS-CoT | 83.44 $\uparrow 43.28$ | 71.35 $\uparrow 31.96$ | 20.86 $\uparrow 13.95$ | 29.98 $\uparrow 24.80$ | 4.330 $\uparrow 2.43$ | 41.992 $\uparrow 23.284$ |
| Auto-CoT | 74.09 $\uparrow 33.93$ | 69.78 $\uparrow 30.39$ | 19.90 $\uparrow 12.99$ | 30.49 $\uparrow 25.31$ | 4.020 $\uparrow 2.12$ | 39.656 $\uparrow 20.948$ |
| Active-Pro | 77.79 $\uparrow 37.63$ | 71.26 $\uparrow 31.87$ | 21.70 $\uparrow 14.79$ | 28.92 $\uparrow 23.74$ | 4.040 $\uparrow 2.14$ | 40.742 $\uparrow 22.034$ |
| Self-Con | 74.70 $\uparrow 34.54$ | 64.45 $\uparrow 25.06$ | 9.540 $\uparrow 2.430$ | 7.610 $\uparrow 2.430$ | 2.420 $\uparrow 2.12$ | 31.744 $\uparrow 13.036$ |
| Ours | 84.08 $\uparrow 43.92$ | 71.79 $\uparrow 32.40$ | 24.52 $\uparrow 17.61$ | 31.14 $\uparrow 25.96$ | 4.580 $\uparrow 2.68$ | 43.222 $\uparrow 24.514$ |

### 4.3 Main Results

#### 4.3.1 For Q1: CAMS consistently outperforms nearly all baseline methods.

Among the three models — Llama3.2-vision:11b, Llava:7b, and Qwen2.5-VL:7b — the average accuracy (Avg.) of the proposed method consistently outperforms nearly all baseline methods. As shown in Table [1](https://arxiv.org/html/2508.18673v2#S4.T1), taking Llama3.2-vision:11b as an example: our method achieves an average score approximately 45.38 points higher than that of ZS-CoT, and more than 5 points higher than those of Auto-CoT, Active Prompt, and self-consistency. The best-performing FS-CoT is around 2 points lower than CAMS. This indicates that CAMS can stably enhance the performance of multimodal large models on complex multimodal reasoning tasks, demonstrating robust effectiveness compared to existing baseline methods across diverse model configurations.

#### 4.3.2 For Q2: CAMS can greatly reduce accuracy instability caused by randomly selecting prompt examples.

To ensure the fairness and reliability of the experiments, five tests are conducted on the FS-CoT method using different random seeds on the test set, with the average accuracy across the five tests selected as the final result. Figure [3](https://arxiv.org/html/2508.18673v2#S4.F3) illustrates the specific performance of FS-CoT in these five tests. The line chart reveals that the random selection strategy exhibits significant instability, with substantial fluctuations in accuracy across tests — the difference between the highest and lowest accuracy exceeds 5. CAMS not only significantly reduces the impact of randomness on performance but also consistently achieves above-average accuracy, outperforming the random selection strategy across the board.

Figure: Figure 3: Accuracy fluctuations across five tests of FS-CoT and CAMS, where Test 1–5 denote the serial numbers of each test.
Refer to caption: x3.png

**Table 2: Results of ablation study ($\%$) on five benchmarks. Unc-Eva selects examples randomly only from different uncertainty categories. Com-Eva selects only an equal number of high-complexity and low-complexity samples.**
|  | Module | Datasets | Avg. |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Unc-Eva | Com-Eva | ScienceQA | A-OKVQA | OK-VQA | VQAv2 | TextVQA |  |  |
| Llama3.2-vision:11b |  |  |  |  |  |  |  |  |
| (a) | ZS-CoT | 38.08 $\downarrow 48.58\%$ | 47.42 $\downarrow 20.15\%$ | 28.74 $\downarrow 44.61\%$ | 57.45 $\downarrow 7.17\%$ | 37.52 $\downarrow 40.22\%$ | 41.84 $\downarrow 31.22\%$ |  |
| (b) | ✔ |  | 63.40 $\downarrow 7.07\%$ | 56.68 $\downarrow 4.56\%$ | 43.65 $\downarrow 15.88\%$ | 59.75 $\downarrow 3.46\%$ | 61.32 $\downarrow 2.29\%$ | 56.96 $\downarrow 6.36\%$ |
| (c) |  | ✔ | 61.92 $\downarrow 9.23\%$ | 55.92 $\downarrow 5.84\%$ | 48.10 $\downarrow 7.30\%$ | 60.66 $\downarrow 1.99\%$ | 60.40 $\downarrow 3.76\%$ | 57.40 $\downarrow 5.64\%$ |
| (d) | ✔ | ✔ | 68.22 | 59.39 | 51.89 | 61.89 | 62.76 | 60.83 |

#### 4.3.3 For Q3: Both the uncertainty analysis and complexity evaluation modules can effectively improve accuracy.

Table [2](https://arxiv.org/html/2508.18673v2#S4.T2) presents the findings of the ablation experiments. To clarify the design of each method compared in the table, key definitions are as follows: ZS-CoT solely employs the simple prompt “Let’s think step by step” to guide the model; and Ours refers to a complete multimodal thought-chain reasoning method enhanced by active learning. Compared with the baseline method, both the standalone Unc-Eva module and Com-Eva module enhance model accuracy in complex reasoning tasks to varying degrees. The Unc-Eva module demonstrates better performance across different disciplinary types and difficulty levels compared to the Com-Eva module. The Com-Eva module, while eliminating randomness, still improves model performance and ensures data stability and reliability. The experimental results indicate that the proposed method effectively combines the advantages of both modules, further enhancing model accuracy while mitigating randomness.

#### 4.3.4 For Q4: CAMS can improve the model’s accuracy in subdivided domains.

To further investigate the efficacy of CAMS in specific subfields, we conduct systematic experiments on the ScienceQA dataset. Specifically, based on disciplinary attributes, the ScienceQA dataset is categorized into three major groups: natural sciences (NAT), social sciences (SOC), and linguistic sciences (LAN). According to difficulty gradients, it is also divided into two levels: grades 1–6 (G1-6) and grades 7–12 (G7-12), thus constructing multidimensional subscenarios that capture both domain specificity and cognitive complexity. As shown in Figure [4](https://arxiv.org/html/2508.18673v2#S4.F4), in the five subfields mentioned above, CAMS almost universally outperforms the three baseline methods - Self-con, ZS-CoT, and FS-CoT - in terms of precision, especially in the NAT, SOC, and G1-6 domains. Whether in knowledge-centric scenarios emphasizing disciplinary dimensions or in learning settings focused on difficulty levels, CAMS demonstrates significant advantages, confirming its ability to substantially improve model accuracy within specialized domains. Meanwhile, this also demonstrates the generalization capability of the CAMS framework.

Figure: Figure 4: Comparison of accuracy between CAMS and three baseline methods in subdivided domains.
Refer to caption: x4.png

Figure: Figure 5: Comparison of different example selection strategies. “AP_high” denotes the selection of only difficult examples (i.e., those with high uncertainty); “AP_low” denotes the selection of only easy examples (i.e., those with low uncertainty).
Refer to caption: x5.png

#### 4.3.5 For Q5: A selection strategy that combines easy and difficult examples is more beneficial to the model.

To evaluate the effectiveness of combining easy and difficult examples, we conducte additional experiments using two additional selection strategies: one that includes only difficult examples (i.e., those with high uncertainty) and another that includes only easy examples (i.e., those with low uncertainty). As shown in Figure [5](https://arxiv.org/html/2508.18673v2#S4.F5), while the all-difficult example strategy performs relatively well across the five target datasets, it still falls short compared to the balanced strategy employed by CAMS. This finding highlights that relying exclusively on either high- or low-difficulty samples is insufficient to capture the full spectrum of knowledge complexity and scenario diversity required for complex reasoning tasks. In contrast, CAMS’s approach of integrating both easy and difficult examples enables the model to learn more diverse and representative knowledge patterns, ultimately leading to improved reasoning accuracy.

## 5 Conclusion and Future Work

In this work, we address key challenges in existing Chain-of-Thought (CoT) prompting methods for multimodal models, which often suffer from unstable and suboptimal performance due to their reliance on random or manually selected examples. To overcome these limitations, we propose CAMS, a novel framework inspired by the pedagogical principle of “tailored teaching with balanced difficulty”. CAMS integrates two key dimensions — model-perceived difficulty
and sample complexity — to construct a customized prompt curriculum that balances between easy and challenging examples. We demonstrate the effectiveness of CAMS through experiments on five benchmarks using multiple state-of-theart multimodal large language models. We also highlight promising directions for future work, including adapting CAMS to broader multimodal settings and further exploring dataset feature dimensions for deeper curriculum design.

## Reproducibility statement

We have submitted the relevant code in the supplementary materials.
The names of the experimental benchmarks, the prompt templates used, and the model’s hyperparameter settings can all be found in The Appendix [A.1](https://arxiv.org/html/2508.18673v2#A1.SS1),  [A.2](https://arxiv.org/html/2508.18673v2#A1.SS2),  [A.3](https://arxiv.org/html/2508.18673v2#A1.SS3) and Table [3](https://arxiv.org/html/2508.18673v2#A1.T3). Section [4](https://arxiv.org/html/2508.18673v2#S4) (Experiments) provides a detailed description of the experimental setup for the mechanism experiments.

## References

- Angluin (1988)
Dana Angluin.
Queries and concept learning.
*Machine learning*, 2:319–342, 1988.
- Antol et al. (2015)
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh.
Vqa: Visual question answering.
In *Proceedings of the IEEE international conference on computer vision*, pp. 2425–2433, 2015.
- Bai et al. (2023)
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al.
Qwen technical report.
*arXiv preprint arXiv:2309.16609*, 2023.
- Brown et al. (2020)
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.
Language models are few-shot learners.
*Advances in neural information processing systems*, 33:1877–1901, 2020.
- Dagan & Engelson (1995)
Ido Dagan and Sean P Engelson.
Committee-based sampling for training probabilistic classifiers.
In *Machine learning proceedings 1995*, pp. 150–157. Elsevier, 1995.
- Diao et al. (2023)
Shizhe Diao, Pengcheng Wang, Yong Lin, Rui Pan, Xiang Liu, and Tong Zhang.
Active prompting with chain-of-thought for large language models.
*arXiv preprint arXiv:2302.12246*, 2023.
- Goyal et al. (2017)
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh.
Making the v in vqa matter: Elevating the role of image understanding in visual question answering.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pp. 6904–6913, 2017.
- King et al. (2004)
Ross D King, Kenneth E Whelan, Ffion M Jones, Philip GK Reiser, Christopher H Bryant, Stephen H Muggleton, Douglas B Kell, and Stephen G Oliver.
Functional genomic hypothesis generation and experimentation by a robot scientist.
*Nature*, 427(6971):247–252, 2004.
- Krishnamurthy (2002)
Vikram Krishnamurthy.
Algorithms for optimal scheduling and management of hidden markov model sensors.
*IEEE Transactions on Signal Processing*, 50(6):1382–1397, 2002.
- Lewis (1995)
David D Lewis.
A sequential algorithm for training text classifiers: Corrigendum and additional data.
In *Acm Sigir Forum*, volume 29, pp. 13–19. ACM New York, NY, USA, 1995.
- Li et al. (2024)
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al.
Llava-onevision: Easy visual task transfer.
*arXiv preprint arXiv:2408.03326*, 2024.
- Liu et al. (2023a)
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.
Visual instruction tuning.
*Advances in neural information processing systems*, 36:34892–34916, 2023a.
- Liu et al. (2024)
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee.
Improved baselines with visual instruction tuning.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 26296–26306, 2024.
- Liu et al. (2023b)
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He.
What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning.
*arXiv preprint arXiv:2312.15685*, 2023b.
- Lu et al. (2022)
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan.
Learn to explain: Multimodal reasoning via thought chains for science question answering.
*Advances in Neural Information Processing Systems*, 35:2507–2521, 2022.
- Marino et al. (2019)
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi.
Ok-vqa: A visual question answering benchmark requiring external knowledge.
In *Proceedings of the IEEE/cvf conference on computer vision and pattern recognition*, pp. 3195–3204, 2019.
- Schwenk et al. (2022)
Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi.
A-okvqa: A benchmark for visual question answering using world knowledge.
In *European conference on computer vision*, pp. 146–162. Springer, 2022.
- Singh et al. (2019)
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach.
Towards vqa models that can read.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 8317–8326, 2019.
- Touvron et al. (2023)
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.
Llama: Open and efficient foundation language models.
*arXiv preprint arXiv:2302.13971*, 2023.
- Wang et al. (2022)
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.
Self-consistency improves chain of thought reasoning in language models.
*arXiv preprint arXiv:2203.11171*, 2022.
- Wei et al. (2022)
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.
Chain-of-thought prompting elicits reasoning in large language models.
*Advances in neural information processing systems*, 35:24824–24837, 2022.
- Xie & Min (2022)
Sang Michael Xie and Sewon Min.
How does in-context learning work? a framework for understanding the differences from traditional supervised learning.
*A framework for understanding the differences from traditional supervised learning*, 2022.
- Yin et al. (2024)
Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen.
A survey on multimodal large language models.
*National Science Review*, 11(12):nwae403, 2024.
- Zhang et al. (2022)
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola.
Automatic chain of thought prompting in large language models.
*arXiv preprint arXiv:2210.03493*, 2022.
- Zhang et al. (2023)
Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola.
Multimodal chain-of-thought reasoning in language models.
*arXiv preprint arXiv:2302.00923*, 2023.
- Zhou et al. (2022)
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al.
Least-to-most prompting enables complex reasoning in large language models.
*arXiv preprint arXiv:2205.10625*, 2022.

## Appendix A Appendix

### A.1 The construction of Prompt

We construct prompt examples by integrating questions, reasoning steps, and answers from the dataset:

$$ $E=\{(q_{1},c_{1},a_{1}),(q_{2},c_{2},a_{2}),...,(q_{n},c_{n},a_{n})\}$ (7) $$

where $q$ denotes questions, $c$ denotes reasoning steps, $a$ denotes answers, and $n$ denotes the number of examples.

### A.2 Comparison between CAMS and Auto-CoT

Figure [6](https://arxiv.org/html/2508.18673v2#A1.F6) demonstrates that the prompt examples selected by CAMS can, to a certain extent, enhance the strengths and mitigate the weaknesses of multimodal large models, thereby better facilitating their completion of reasoning tasks.

Figure: Figure 6: Comparative cases between CAMS and Auto-CoT
Refer to caption: x6.png

### A.3 Prompt Demonstration

Details are provided in Table [3](https://arxiv.org/html/2508.18673v2#A1.T3). It is important to note that the captions in the examples are replaced with actual images in real-world applications; text is used here solely for the convenience of demonstration.

### A.4 The Use of Large Language Models (LLM)

In order to enhance the language quality and clarity of this academic paper, the author utilized AI-powered tools for text refinement during the writing process. The specific details are as follows:

Purpose of Use: The primary purposes for using AI tools were to:

- •
Check grammar and spelling for certain sentences.
- •
Optimize vocabulary choices for more precise and academic expression.
- •
Adjust sentence structures to improve logical coherence and readability between paragraphs.

Method of Use: The author input original paragraphs written by themselves into the AI tools and then manually judged, filtered, and revised the text based on the refinement suggestions provided. All adopted changes were carefully considered by the author to ensure they fully align with the original intent and academic rigor of the paper.

Disclaimer of Responsibility: All academic content in this paper, including core arguments, research data, result analysis, argumentation process, and final conclusions, was independently created and is the sole responsibility of the author. The AI tools were used purely as an auxiliary aid and did not generate any critical academic viewpoints, research data, or conclusions. The author assumes full responsibility for the final content of the paper.

Tools Used: The AI tool used in this process is: GPT-4.

**Table 3: Prompt template of query formulation.**
| SYSTEM: |
| --- |
| You are a professional VQA task solver |
| USER: |
| Given the problem and its related images, you need to generate answers for the VQA task. |
| The generated answers must be derived from direct visual observation of the image—do not include speculative content, assumptions, or information not visible in the image. |
| When answering questions: if the image clearly shows the required information, output a specific, concise answer; if the image does not contain enough information to answer (e.g., object not visible, detail unclear), output “insufficient information”; if the image contradicts the question (e.g., question asks about a “red car” but image shows a blue car), output a negative answer with correct visual details. |
| Do not answer subjective judgment questions (e.g., “Is the image interesting?”)—visual facts only. |
| You MUST only respond in the format as described below. |
| DO NOT RESPOND WITH ANYTHING ELSE. |
| Response Format:The answer is …, because … . |
| Here are an example: |
| “question”:“Which of these organisms contains matter that was once part of the phytoplankton?” |
| “choices”:[“black rockfish”, “sea otter”] |
| “hint”:“Below is a food web from an ocean ecosystem in Monterey Bay, off the coast of California. A food web models how the matter eaten by organisms moves through an ecosystem. The arrows in a food web represent how matter moves between organisms in an ecosystem.” |
| “caption”:“A painting of a penguin on a wall.” |
| “lecture”:“A food web is a model. A food web shows where organisms in an ecosystem get their food. Models can make things in nature easier to understand because models can represent complex things in a simpler way. If a food web showed every organism in an ecosystem, the food web would be hard to understand. So, each food web shows how some organisms in an ecosystem can get their food. Arrows show how matter moves. A food web has arrows that point from one organism to another. Each arrow shows the direction that matter moves when one organism eats another organism. An arrow starts from the organism that is eaten. The arrow points to the organism that is doing the eating. An organism in a food web can have more than one arrow pointing from it. This shows that the organism is eaten by more than one other organism in the food web. An organism in a food web can also have more than one arrow pointing to it. This shows that the organism eats more than one other organism in the food web.” |
| “output”:“The answer is A, because Use the arrows to follow how matter moves through this food web. For each answer choice, try to find a path of arrows that starts from the phytoplankton. The only arrow pointing to the sea otter starts from the sea urchin. The only arrow pointing to the sea urchin starts from the kelp. No arrow points to the kelp. So, in this food web, matter does not move from the phytoplankton to the sea otter.There are two paths matter can take from the phytoplankton to the plainfin midshipman: phytoplankton$->$plainfin midshipman. phytoplankton$->$zooplankton$->$plainfin midshipman. There is one path matter can take from the phytoplankton to the black rockfish: phytoplankton$->$zooplankton$->$black rockfish. There is one path matter can take from the phytoplankton to the zooplankton: phytoplankton$->$zooplankton.” |
| Now complete your output with following the above rules. |
| Input: |
| Output: |

Figure: Algorithm 1 CAMS

## Peer Reviews

confidence: 4
contribution: 1
initial_rating: 2
presentation: 2
questions: Please see the weakness.
soundness: 1
strengths: 1. This paper targets a practical issue that how to construct the prompt demonstration for multimodal CoT reasoning.
2. The experiments cover multiple VQA benchmarks and several multimodal models, showing the effectiveness of  the proposed CAMS framework.
3. The overall pipeline is relatively easy to follow and easy to implement.
summary: This paper proposes CAMS, a prompt demonstration selection framework for multimodal CoT reasoning. CAMS uses a disagreement-based uncertainty score and a complexity scorer to build the example pool. Experiments on several VQA benchmarks show gains over baseline CoT methods. However, key design choices are ad hoc, and the complexity signal is only weakly justified, and there is no serious analysis. The contribution mainly refines existing ideas in LLMs.
weaknesses: 1. The core contribution feels like a combination of existing ideas in LLMs rather than a new concept for multimodality. The improvement over simple baselines is often modest. The proposed method also introduces additional computation cost in prompt selection.
2. The proposed complexity scorer is trained on text-only data using an outdated LLM (llama-2) and applied to caption-plus-question inputs for multimodal tasks, but this paper does not offer convincing evidence that this score meaningfully reflects sample difficulty for these VQA benchmarks.
3. Key choices such as the disagreement threshold for splitting easy and hard examples and the fixed mixing of high- and low-complexity samples are not thoroughly studied. There is no sensitivity analysis that shows the method is robust to these hyperparameters.
4. It would be better to see quantitative discussion of computational cost because CAMS requires multiple passes of a large MLLM over a training subset and additional passes of a complexity scorer.  For realistic scenario, processing the whole training dataset may be costly or infeasible.
5. I would like to know why CAMS constructs a single global prompt pool per dataset and does not adapt exemplars to individual test questions, which contradicts the "tailored".

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 2
questions: Plz answer my concerns in the weakness section
soundness: 2
strengths: - The paper uniquely reframes MCoT prompt selection as a curriculum design problem. It innovatively combines two complementary signals—model-specific uncertainty and model-agnostic sample complexity—to create a more principled and effective selection strategy.

- The empirical validation is robust, featuring extensive experiments on five benchmarks with three different MLLMs. The ablation studies are a key strength, clearly demonstrating the synergistic benefit of the two signals, and the work provides strong quantitative evidence of improved performance stability.
summary: This paper addresses the suboptimal and unstable performance of Multimodal Chain-of-Thought (MCoT) prompting, which it attributes to the reliance on randomly or manually selected examples. The authors propose CAMS, a framework to automate prompt selection by constructing a "prompt curriculum" based on two signals: model-perceived difficulty, measured via prediction uncertainty, and intrinsic sample complexity, determined by a separate scorer. By jointly analyzing these dimensions, the method selects a balanced set of examples intended to be more effective. The approach is evaluated on five multimodal reasoning benchmarks, with the authors reporting improved accuracy and reduced performance variance compared to baseline prompting strategies.
weaknesses: - A core component of the proposed framework is the "Complexity Scorer," yet its development and impact are not fully interrogated. The scorer is trained on a dataset created via an "evolution-based metric" that relies on ChatGPT for ranking. This introduces several issues: (1) It creates a dependency on a powerful, proprietary model, which complicates reproducibility. (2) The performance of CAMS becomes implicitly dependent on the quality of this external LLM's judgments. The paper lacks a sensitivity analysis exploring how the final MCoT performance is affected by the accuracy of this scorer.

- The paper's key claim is that a "balanced difficulty" curriculum is superior. However, the implementation of this balance—selecting an equal number of samples from four discrete quadrants (easy/hard uncertainty vs. easy/hard complexity)—is a rigid heuristic. It is not obvious that a 25/25/25/25 split is optimal across different models, which have unique knowledge gaps, or across different datasets with varying intrinsic difficulty distributions.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: See above
soundness: 3
strengths: 1. Clear framing of prompt selection as a tailored curriculum with two orthogonal signals (model-uncertainty & data-complexity) and a balanced difficulty principle; simple and general recipe usable across MLLMs. 

2. Multi-benchmark, multi-model evaluation; ablations showing both modules matter; analysis on sub-domains (ScienceQA NAT/SOC/LAN; grade bands). Reported average gains over strong baselines plus reduced seed-variance. 

3. Method flow (Fig. 2) and pseudocode (Algorithm 1) make the approach reproducible; implementation details (temperature, k, sampling limits) are provided; prompt templates are in the appendix; code promised. 

4. Addresses a practical pain point—random exemplars cause instability—and offers a principled fix with measurable benefits for MCoT on common VQA-style tasks.
summary: The paper proposes CAMS (Complexity-Guided Active Multimodal CoT Sampling), a prompt–curriculum framework for multimodal chain-of-thought (MCoT) reasoning. Instead of randomly or manually picking few-shot exemplars, CAMS selects a balanced set of examples by combining: (i) model-perceived difficulty via prediction disagreement across k stochastic forward passes (unique answers divided by k), and (ii) intrinsic sample complexity estimated by a learned complexity scorer (trained on “evolved” instruction variants and applied to text formed by concatenating question, options, and image caption). Samples are partitioned into easy/hard by the disagreement score and into low/high complexity by the scorer; CAMS then samples equally from the four buckets to build the prompt exemplars (shared across all test items). Experiments on ScienceQA, A-OKVQA, OK-VQA, VQAv2, TextVQA with LLaMA3.2-Vision-11B, LLaVA-7B, Qwen2.5-VL-7B report consistent gains over ZS-CoT, FS-CoT, Auto-CoT, Active Prompt, and Self-Consistency, plus reduced variance vs random few-shot selection. Ablations indicate both uncertainty and complexity contribute, and analyses on ScienceQA sub-domains suggest broad benefits.
weaknesses: 1. The scorer depends on caption quality and a text-only transformation of multimodal inputs; evidence that this approximates intrinsic multimodal complexity is limited. A calibration study (e.g., correlation with human difficulty ratings or model error rates) would strengthen the claim. 

2. The 0.5 uncertainty threshold, equal bucket quotas, and exemplar count are not stress-tested. Without sensitivity curves, it’s hard to disentangle whether “balanced difficulty” per se or just “not all-hard” drives gains. 

3. Active disagreement over large pools (even with 10k caps) can be expensive; there is no cost vs. accuracy analysis or amortization strategy (e.g., sub-sampling, early stopping, proxy models). 

4. While Auto-CoT, Self-Consistency, and Active Prompt are included, comparisons to complementary planning/verification paradigms (e.g., Least-to-Most, ReAct/ToT in multimodal settings) are absent; also no results with larger closed models to assess headroom.

5. Needs explicit confirmation that official VQA scoring (soft-acc, text normalization) is used; if not, comparisons may be apples-to-oranges. 

6. The construction of exemplars by concatenating dataset “question, reasoning, answer” presumes such rationales exist and are high-quality for each dataset; many VQA datasets lack gold rationales. Scope and handling when rationales are unavailable should be clarified.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. On the Effective Boundary of the Complexity Assessor: We observed in Figure 4 that the performance gain in the language science (LAN) domain was less pronounced than in NAT and SOC. Could this suggest that your complexity assessor, trained by increasing "reasoning steps," is less effective at capturing the "conceptual" or "insight-based" difficulty common in language problems? Following this, how would you anticipate your method performing on high-level reasoning tasks like AIME math problems, where difficulty is almost entirely conceptual? Have you considered alternative or complementary methods to assess complexity for such challenges?

2. On the Reproducibility of "Evol-Complexity": To improve the transparency and reproducibility of your work, would it be possible to provide the specific prompt used to guide ChatGPT in increasing question complexity, along with one or two full examples showing a question's "evolution" through the iterative process?

3. On the Consideration of Computational Cost: Could you provide a brief quantitative analysis of the computational overhead (e.g., approximate extra GPU hours or API calls) required by the CAMS preprocessing pipeline compared to a baseline like FS-CoT? This information is crucial for readers to assess the practical deployment value of your method.
soundness: 3
strengths: 1. The work's primary novelty lies in creatively fusing two distinct concepts—the model-centric view of uncertainty and the data-centric view of complexity—into a single, unified framework for example selection. Elevating prompt selection to "prompt curriculum design" provides a fresh and systematic perspective for the field.

2. The experimental design is  robust. Comprehensive evaluations across five diverse benchmarks and three different mainstream models strongly support the method's generalization ability and model-agnosticism. The thorough ablation studies also clearly validate the necessity of each core component of the framework.

3. The paper is well-structured and highly readable. 

4. This research directly addresses a critical pain point in current large model applications: prompt fragility and instability. It provides a systematic, reproducible solution that significantly enhances the reliability and performance of MCOT for complex reasoning, holding substantial practical value.
summary: This paper introduces a novel framework named CAMS to address the performance instability of Multimodal Chain-of-Thought (MCOT) caused by its reliance on randomly selected examples. Inspired by the principle of "tailored teaching," CAMS reframes prompt selection as a systematic "prompt curriculum design" problem. It innovatively selects examples by considering two dimensions: 1) model-perceived difficulty (uncertainty), measured by the model's own prediction disagreement, and 2) intrinsic sample difficulty (complexity), judged by an independent assessor. By balancing examples from these two dimensions, the method demonstrates significant and stable performance improvements in reasoning across multiple benchmarks.
weaknesses: 1. Limited Scope of the Complexity Assessor and Its Potential Impact on Generalization: The paper's core innovation, the "complexity" dimension, is defined in a way that equates complexity with structural difficulty (i.e., more reasoning steps). This approach is effective for problems where complexity and difficulty are aligned. However, it may fail on tasks where difficulty stems from conceptual insight rather than procedural length. The paper's own results (Figure 4) hint at this, showing a smaller performance gain in the language science (LAN) domain. This raises concerns about the method's effectiveness on higher-level reasoning tasks where a key insight, not structural complexity, is the primary barrier.

2. Lack of Transparency and Reproducibility in the Complexity Assessor's Training: A key step is the "Evol-Complexity" method, which uses ChatGPT to iteratively increase question difficulty to generate training data. However, the paper does not provide the specific prompts and examples used to guide this process. This makes the core module a "black box," hindering reproducibility and preventing a deeper analysis of the biases the assessor might have learned from ChatGPT.

3. High Computational Cost is Undiscussed: The CAMS framework requires a computationally intensive preprocessing stage, involving multiple inference passes for uncertainty estimation and calls to another large model for complexity assessment. The paper focuses exclusively on performance gains without providing any discussion on the trade-off between these gains and the significant increase in computational overhead, which is a critical factor for practical application.