Agri-CM$^3$: A Chinese Massive Multi-modal, Multi-level Benchmark for Agricultural Understanding and Reasoning
Authors: Haotian Wang, Yi Guan, Fanshu Meng, Chao Zhao, Lian Yan, Yang Yang, Jingchi Jiang
Accepted at: acl25

Agri-CM^3: A Chinese Massive Multi-modal, Multi-level Benchmark for Agricultural Understanding and Reasoning

Haotian Wang1, Yi Guan1, Fanshu Meng1, Chao Zhao
Lian Yan1, Yang Yang2, Jingchi Jiang1*
1Harbin Institute of Technology   2Changchun University Of Science And Technology
{wanght1998,zhaochaocs}@gmail.com, {24s003058,23b903008}@stu.hit.edu.cn,
yangyang_hit@cust.edu.cn, {guanyi,jiangjingchi}@hit.edu.cn

Abstract

Multi-modal Large Language Models (MLLMs) integrating images, text, and speech can provide farmers with accurate diagnoses and treatment of pests and diseases, enhancing agricultural efficiency and sustainability. However, existing benchmarks lack comprehensive evaluations, particularly in multi-level reasoning, making it challenging to identify model limitations. To address this issue, we introduce Agri-CM^3, an expert-validated benchmark assessing MLLMs’ understanding and reasoning in agricultural management. It includes 3,939 images and 15,901 multi-level multiple-choice questions with detailed explanations. Evaluations of 45 MLLMs reveal significant gaps. Even GPT-4o achieves only 63.64% accuracy, falling short in fine-grained reasoning tasks. Analysis across three reasoning levels and seven compositional abilities highlights key challenges in accuracy and cognitive understanding. Our study provides insights for advancing MLLMs in agricultural management, driving their development and application. Code and data are available at https://github.com/HIT-Kwoo/Agri-CM3.

1 Introduction

The modernization and intensification of agriculture have led to increasing diversity and adaptability of pests and diseases, making their management increasingly challenging (Singh et al., 2024; Lenné and Wood, 2024). Traditional diagnostic methods primarily rely on manual observation and expert experience, often suffering from delays, low accuracy, and high costs (John et al., 2023). With the rapid advancement of artificial intelligence (AI), particularly multi-modal large language models (MLLMs), agricultural pest and disease management have entered a new era. By integrating multiple data modalities, MLLMs enable intelligent recognition, prediction, and decision support, helping farmers optimize agricultural management, reduce pesticide use, and enhance sustainability (Zhu et al., 2024; Kuska et al., 2024).

Despite the impressive performance of state-of-the-art MLLMs (Hurst et al., 2024; Team et al., 2024; Anthropic, 2025) in vision-language tasks within the agricultural domain (Zhu et al., 2024), their practical effectiveness in pest and disease management remains unclear. A systematic benchmark is essential to evaluate their real-world applicability and reasoning capabilities in agricultural management. Existing benchmarks, such as AgriBench (Zhou and Ryo, 2024), focus primarily on broader agricultural vision tasks (e.g., land segmentation and captioning) but lack a dedicated evaluation framework for pest and disease management. Furthermore, current domain-specific benchmarks (Yue et al., 2024; Zhang et al., 2024b) fail to assess models’ multi-level reasoning abilities, which are critical for complex decision-making in agricultural contexts. As illustrated in Figure 1, agri-

![A diagram showing a question about pathogen characteristics, reasoning process, and a robot labeled MLLM. The diagram illustrates the failure in reasoning and asks which stage leads to the failure.](page_1012_370_388_312.png)

Figure 1: Existing benchmarks fail to identify the specific cause of failure in agricultural complex reasoning, making it difficult to distinguish whether errors stem from perception, mixed perception-cognition reasoning, or knowledge application.
Figure 2: Left: The process of deriving high-level data with clearly specified entities into low-level data. Middle: Distribution of labels across the entire dataset. Right: Performance of 5 MLLMs and experts on the validation set.

cultural multi-modal reasoning tasks involve multiple stages, including crop identification, symptom recognition and disease diagnosis, and knowledge integration and application. Any failure at one stage may lead to an incorrect diagnosis. However, existing benchmarks do not effectively distinguish models’ performance at each reasoning step, making it difficult to diagnose the root causes of errors.

To tackle these challenges, we propose Agri-CM^3, a novel benchmark designed to provide a comprehensive, hierarchical evaluation of MLLMs in agricultural pest and disease management. Specifically, we decompose complex reasoning tasks into three fine-grained sub-reasoning tasks: (1) Crop identification (Perception Stage). (2) Symptom recognition and disease diagnosis (Mixture of Perception-Cognition Reasoning Stage). (3) Agricultural knowledge integration and application (Knowledge Application Stage). These sub-tasks are then systematically organized as a reasoning hierarchy, as illustrated in Figure 2(Left): Level 1 evaluates individual sub-reasoning tasks. Level 2 assesses combinations of two sub-reasoning tasks. Level 3 examines the whole reasoning pipeline involving all three tasks. Each evaluation is structured as multiple-choice questions (MCQs) with detailed explanations, ensuring a rigorous assessment of MLLMs across different reasoning levels. This hierarchical design allows Agri-CM^3 to systematically analyze models’ performance at various reasoning levels and diagnose their specific weaknesses.

To construct Agri-CM^3, we curated 3,939 high-quality images covering diverse pest and disease categories, rigorously filtered through model generation and expert validation. We designed 15,901 MCQs based on these images and conducted extensive evaluations on 38 open-source MLLMs and seven proprietary state-of-the-art MLLMs, revealing several key insights: (1) Open-source MLLMs Surpassing Proprietary Models. (2) Mixture cognition-perception reasoning tasks are the key factor limiting model performance. (3) Multi-modal training boosts performance in single-text tasks.

In summary, the key contributions of our work are as follows:

• To the best of our knowledge, we are the first multi-modal benchmark for agricultural pest and disease management, covering a wide range of pests and diseases. The benchmark is rigorously filtered and validated by experts, ensuring high quality.

• We decompose complex reasoning tasks and combine them based on practical needs, resulting in a multi-level benchmark. This framework allows for a comprehensive assessment of model capabilities at each reasoning stage, helping identify weaknesses in various steps.

• We evaluated various models and summarized several key insights. Our findings provide valuable guidance for future research and developments in this area.

2 Related Work

2.1 Multi-modal Large Language Models

MLLMs have introduced a new paradigm for reasoning, offering a natural and intuitive way to interact with users (Zhu et al., 2024). These models
have demonstrated outstanding potential for applications in various vertical domains (Xie et al., 2024; Xiao et al., 2025). Currently, leading proprietary MLLMs have achieved exceptional performance in multi-modal understanding and reasoning tasks. Meanwhile, the research community has developed various open-source MLLMs with multi-modal instruction-tuning datasets (Zhang et al., 2024a). Notable open-source models include the InternVL series (Chen et al., 2024b), LLaVA series (Liu et al., 2024), Qwen-VL series (Wang et al., 2024b), DeepSeek series (Wu et al., 2024), and MiniCPM-V series (Yao et al., 2024), which have been rapidly adopted across diverse domains.

To address domain-specific challenges, researchers have developed specialized MLLMs trained and fine-tuned on domain-specific datasets. For example, medical MLLMs (Li et al., 2023; Moor et al., 2023) and Financial MLLMs (Xie et al., 2024) have been proposed to enhance AI capabilities in their respective fields. In the agricultural domain, Wang et al. (2024a) fine-tuned a multi-modal agricultural model based on LLaVA (Liu et al., 2023b), demonstrating promising potential in pest and disease diagnosis. These advancements highlight the growing impact of MLLMs in agriculture, yet a standardized evaluation framework remains necessary to assess their real-world performance systematically.

2.2 MLLM Benchmarks

Despite the rapid progress of MLLMs, systematic benchmarking is equally crucial. It helps evaluate strengths and weaknesses, guiding optimization and accelerating real-world application. Several multi-modal benchmarks, such as MMMU (Yue et al., 2024) and CMMMU (Zhang et al., 2024b), cover multiple vertical domains. However, their domain-specific evaluation capabilities are limited. To address domain-specific needs, researchers have developed specialized benchmarks for evaluating MLLMs in fields such as medicine (Sun et al., 2025; Chen et al., 2024a) and industry (Jiang et al., 2025). In the agricultural domain, Zhou and Ryo (2024) proposed AgriBench, a multi-modal benchmark focusing on landscape understanding tasks. While AgriBench provides a general framework for evaluating MLLMs’ multi-modal reasoning in agriculture, it lacks a dedicated assessment for pest and disease management. Furthermore, existing domain-specific benchmarks fail to distinguish which specific reasoning abilities contribute to model failures in complex reasoning tasks. Identifying these deficiencies at a granular level is essential for improving MLLMs and guiding future model development.

To address this gap, we propose Agri-CM$^3$, a benchmark designed to systematically evaluate MLLMs’ hierarchical reasoning abilities in agricultural pest and disease management. Unlike existing benchmarks, Agri-CM$^3$ explicitly evaluates models across multiple reasoning levels, providing fine-grained insights into their strengths and weaknesses.

3 The Agri-CM$^3$ Benchmark

3.1 Design Principal

Existing benchmarks often evaluate complex reasoning tasks as a whole, especially for tasks involving multiple reasoning steps, such as agricultural knowledge reasoning. This approach fails to accurately identify model strengths and weaknesses at each reasoning stage, leading to a lack of clear guidance for future model improvements.

To address this limitation, we propose Agri-CM$^3$, which aims to comprehensively assess the complex reasoning capabilities of models in agricultural pest and disease management through a multi-level evaluation framework. We decompose the complex reasoning task into three key sub-tasks:

i. Perception (P): The ability to identify crops and pests. ii. Mixed Perception-Cognition Reasoning (M): The ability to identify crop symptoms and reason diseases. iii. Knowledge Application (K): The ability to integration and apply agricultural knowledge. These key abilities are then combined into three levels based on task-specific needs:

Level 1: Evaluates model performance on a single sub-reasoning task.

Level 2: Assesses the model’s ability to perform a combination of two sub-reasoning tasks.

Level 3: Evaluates the model’s performance across the full reasoning chain.

Through this layered design, Agri-CM$^3$ provides a comprehensive evaluation of a model’s capabilities at different reasoning stages, offering detailed feedback on performance at each level. The overall data collection and generation process is illustrated in Figure 3.
Figure 3: Overview of the Agri-CM³ generation process. Left: The collection and processing of images and background knowledge. Right: Multi-level data generation, followed by filtering using LLMs, and final validation by agricultural experts.

3.2 Data Collection and Preprocessing

Traditional agricultural visual datasets typically focus on a single crop or pest type. However, an expert in agricultural pest and disease management should be capable of identifying and diagnosing various pests and diseases. To address this, we extensively collected existing agricultural pest and disease image datasets, resulting in 21 datasets, including 11 crops, 45 diseases, and 107 pests.

To ensure the diversity, we applied K-means (Arthur and Vassilvitskii, 2007) clustering to group images and filter out highly similar images. We filtered out low-resolution images to ensure that details are clearly visible, resulting in a final average resolution of \(1800 \times 1500\) for the retained images. Additionally, we enlisted professional agricultural experts to filter out unusable images manually. To prevent data leakage, we did not directly use image-question pairs from existing datasets. Instead, we collected images and their corresponding captions and employed GPT-4o to generate image-text question-answer pairs and other options. Besides, we sourced domain-specific agricultural knowledge from professional agricultural books and reputable encyclopedic websites, ensuring the model generated questions based on accurate, relevant knowledge. See the Appendix A.1 for more details.

3.3 Data Generation and Expert Validation

To ensure the accuracy and diversity of the generated data, we not only provide crop, disease, and pest labels but also select relevant background knowledge to generate high-quality knowledge reasoning data. To facilitate manual validation of the generated data, we include answer explanation in the generated data, ensuring that each question is supported by a clear and justified rationale.

High-level Data Generation. We begin by generating high-level knowledge reasoning data based on agricultural pest and disease images and relevant background knowledge. The tasks include Disease-P-M-K, Disease-P-M, and Pest-P-K. These tasks require the model to perform multiple reasoning steps, including crop and pest identification, symptom recognition and disease diagnosis, followed by final knowledge reasoning. Pest identification involves only perceptual stage, with high-level data considering only P-K.

Low-level Data Derivation. After generating high-level data, we derive low-level data by explicitly replacing entities in the high-level data questions. This method allows us to generate data at different reasoning stages, thus providing more granular tasks that align with the evaluation needs of each reasoning level.

Data Filtering. To ensure that the generated data relies heavily on the image information, we use three LLMs to answer questions based on the question and options. We then filter out the data where two models provide correct answers, as these are deemed to have low dependency on the images. This ensures that the final dataset accurately reflects the image content.
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">Validation</th>
    <th colspan="3">Test</th>
    <th colspan="6">Disease</th>
    <th colspan="3">Pest</th>
  </tr>
  <tr>
    <th>Total</th>
    <th>Test Total</th>
    <th>P</th>
    <th>M</th>
    <th>P-M</th>
    <th>K</th>
    <th>M-K</th>
    <th>P-M-K</th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <td>Random Choice</td>
    <td>23.22</td>
    <td>25.14</td>
    <td>25.44</td>
    <td>26.04</td>
    <td>25.44</td>
    <td>24.15</td>
    <td>24.31</td>
    <td>26.61</td>
    <td>25.04</td>
    <td>24.94</td>
    <td>24.89</td>
  </tr>
  <tr>
    <td>Expert Performance</td>
    <td>79.56</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
</table>

Multi-Modal Large Language Model (MLLMs): Text + Image as Input

<table>
  <tr>
    <th>Model</th>
    <th>Validation</th>
    <th>Test</th>
    <th>Disease</th>
    <th>Pest</th>
  </tr>
  <tr>
    <td>LLaVA-1.5-13B (Liu et al., 2023a)</td>
    <td>41.33</td>
    <td>38.19</td>
    <td>43.87</td>
    <td>53.00</td>
    <td>52.41</td>
    <td>40.52</td>
    <td>42.54</td>
    <td>42.38</td>
    <td>23.40</td>
    <td>33.31</td>
    <td>36.47</td>
  </tr>
  <tr>
    <td>Phi-3.5-Vision (Abdin et al., 2024)</td>
    <td>46.11</td>
    <td>41.69</td>
    <td>39.14</td>
    <td>57.99</td>
    <td>53.85</td>
    <td>44.10</td>
    <td>44.66</td>
    <td>45.18</td>
    <td>30.01</td>
    <td>38.50</td>
    <td>40.29</td>
  </tr>
  <tr>
    <td>DeepseeK-VL2 (Wu et al., 2024)</td>
    <td>46.33</td>
    <td>42.91</td>
    <td>36.18</td>
    <td>47.93</td>
    <td>41.5</td>
    <td>48.12</td>
    <td>48.47</td>
    <td>45.23</td>
    <td>37.28</td>
    <td>44.17</td>
    <td>42.56</td>
  </tr>
  <tr>
    <td>Yi-VL-34B (Ai et al., 2024)</td>
    <td>48.78</td>
    <td>45.98</td>
    <td>49.79</td>
    <td>52.83</td>
    <td>38.46</td>
    <td>51.25</td>
    <td>48.43</td>
    <td>44.25</td>
    <td>42.68</td>
    <td>46.50</td>
    <td>44.76</td>
  </tr>
  <tr>
    <td>CogVLM2 (Hong et al., 2024)</td>
    <td>48.11</td>
    <td>46.73</td>
    <td>76.84</td>
    <td>45.65</td>
    <td>43.11</td>
    <td>39.45</td>
    <td>39.53</td>
    <td>37.24</td>
    <td>58.92</td>
    <td>38.21</td>
    <td>40.91</td>
  </tr>
  <tr>
    <td>Cambrian-34B (Tong et al., 2024)</td>
    <td>56.00</td>
    <td>49.16</td>
    <td>74.13</td>
    <td>54.35</td>
    <td>45.98</td>
    <td>53.13</td>
    <td>45.41</td>
    <td>38.71</td>
    <td>53.99</td>
    <td>47.91</td>
    <td>39.11</td>
  </tr>
  <tr>
    <td>MiniCPM-V2.6 (Yao et al., 2024)</td>
    <td>54.67</td>
    <td>52.24</td>
    <td>63.54</td>
    <td>58.58</td>
    <td>54.35</td>
    <td>49.02</td>
    <td>46.70</td>
    <td>45.96</td>
    <td>64.05</td>
    <td>44.85</td>
    <td>45.42</td>
  </tr>
  <tr>
    <td>GLM4V-9B (GLM et al., 2024)</td>
    <td>56.00</td>
    <td>52.65</td>
    <td>86.73</td>
    <td>55.62</td>
    <td>56.04</td>
    <td>50.72</td>
    <td>47.84</td>
    <td>47.71</td>
    <td>57.79</td>
    <td>43.32</td>
    <td>44.17</td>
  </tr>
  <tr>
    <td>LLaVA-1.6-34B (Liu et al., 2024)</td>
    <td>57.78</td>
    <td>52.71</td>
    <td>59.09</td>
    <td>59.51</td>
    <td>54.35</td>
    <td>57.60</td>
    <td>51.96</td>
    <td>50.47</td>
    <td>54.58</td>
    <td>48.51</td>
    <td>47.76</td>
  </tr>
  <tr>
    <td>Qwen2.5-VL-7B (Bai et al., 2025)</td>
    <td>56.44</td>
    <td>53.76</td>
    <td>76.16</td>
    <td>53.85</td>
    <td>50.38</td>
    <td>49.46</td>
    <td>45.33</td>
    <td>43.75</td>
    <td>73.51</td>
    <td>45.70</td>
    <td>44.87</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B (Li et al., 2024)</td>
    <td>57.00</td>
    <td>53.92</td>
    <td>70.33</td>
    <td>58.66</td>
    <td>53.00</td>
    <td>51.25</td>
    <td>50.78</td>
    <td>50.54</td>
    <td>64.87</td>
    <td>45.86</td>
    <td>46.41</td>
  </tr>
  <tr>
    <td>Molmo-7B (Deitke et al., 2024)</td>
    <td>55.89</td>
    <td>54.76</td>
    <td>54.69</td>
    <td>66.19</td>
    <td>59.93</td>
    <td>52.50</td>
    <td>49.92</td>
    <td>49.77</td>
    <td>67.41</td>
    <td>47.91</td>
    <td>47.54</td>
  </tr>
  <tr>
    <td>InternVL-2.5-8B (Chen et al., 2024b)</td>
    <td>59.67</td>
    <td>55.73</td>
    <td>70.58</td>
    <td>62.13</td>
    <td>57.06</td>
    <td>55.37</td>
    <td>52.43</td>
    <td>50.54</td>
    <td>64.16</td>
    <td>49.20</td>
    <td>48.13</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B (Wang et al., 2024b)</td>
    <td>60.00</td>
    <td>56.32</td>
    <td>85.88</td>
    <td>58.92</td>
    <td>57.82</td>
    <td>54.20</td>
    <td>47.88</td>
    <td>46.27</td>
    <td>70.34</td>
    <td>47.43</td>
    <td>46.26</td>
  </tr>
  <tr>
    <td>InternVL-2.5-26B (Chen et al., 2024b)</td>
    <td>64.78</td>
    <td>61.97</td>
    <td>74.30</td>
    <td>71.09</td>
    <td>66.61</td>
    <td>63.69</td>
    <td>57.06</td>
    <td>56.07</td>
    <td>72.57</td>
    <td>54.91</td>
    <td>51.50</td>
  </tr>
  <tr>
    <td>Molmo-72B (Deitke et al., 2024)</td>
    <td>66.33</td>
    <td>63.13</td>
    <td>63.14</td>
    <td>73.96</td>
    <td>71.01</td>
    <td>66.01</td>
    <td>60.75</td>
    <td>59.56</td>
    <td>68.86</td>
    <td>58.37</td>
    <td>55.61</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B (Li et al., 2024)</td>
    <td>69.00</td>
    <td>64.37</td>
    <td>78.61</td>
    <td>72.19</td>
    <td>68.98</td>
    <td>65.38</td>
    <td>60.99</td>
    <td>58.79</td>
    <td>70.07</td>
    <td>57.32</td>
    <td>57.66</td>
  </tr>
  <tr>
    <td>Qwen2.5-VL-72B (Bai et al., 2025)</td>
    <td>67.78</td>
    <td>64.70</td>
    <td>84.02</td>
    <td>68.64</td>
    <td>67.03</td>
    <td>63.33</td>
    <td>56.86</td>
    <td>54.46</td>
    <td>79.11</td>
    <td>55.47</td>
    <td>53.41</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B (Wang et al., 2024b)</td>
    <td>72.00</td>
    <td>64.81</td>
    <td>80.98</td>
    <td>73.29</td>
    <td>72.36</td>
    <td>65.74</td>
    <td>61.77</td>
    <td>59.95</td>
    <td>72.07</td>
    <td>55.07</td>
    <td>56.27</td>
  </tr>
  <tr>
    <td>InternVL-2.5-78B (Chen et al., 2024b)</td>
    <td>68.11</td>
    <td>65.31</td>
    <td>74.81</td>
    <td>76.67</td>
    <td>70.16</td>
    <td>65.65</td>
    <td>61.30</td>
    <td>60.26</td>
    <td>75.04</td>
    <td>56.84</td>
    <td>56.89</td>
  </tr>
  <tr>
    <td>InternVL-2.5-78B (Chen et al., 2024b)</td>
    <td>69.89</td>
    <td>65.49</td>
    <td>79.54</td>
    <td>76.67</td>
    <td>72.78</td>
    <td>65.56</td>
    <td>60.44</td>
    <td>59.33</td>
    <td>73.04</td>
    <td>58.00</td>
    <td>56.38</td>
  </tr>
  <tr>
    <td>Qwen-VL-PLUS (Wang et al., 2024b)</td>
    <td>51.67</td>
    <td>51.54</td>
    <td>80.56</td>
    <td>47.76</td>
    <td>48.27</td>
    <td>47.14</td>
    <td>45.29</td>
    <td>42.67</td>
    <td>68.39</td>
    <td>40.23</td>
    <td>43.42</td>
  </tr>
  <tr>
    <td>Yi-Vision (Ai et al., 2024)</td>
    <td>61.56</td>
    <td>56.59</td>
    <td>60.27</td>
    <td>58.66</td>
    <td>55.45</td>
    <td>60.82</td>
    <td>53.61</td>
    <td>51.01</td>
    <td>70.31</td>
    <td>51.85</td>
    <td>48.35</td>
  </tr>
  <tr>
    <td>Qwen-VL-MAX (Wang et al., 2024b)</td>
    <td>59.44</td>
    <td>57.97</td>
    <td>76.84</td>
    <td>60.19</td>
    <td>60.19</td>
    <td>62.25</td>
    <td>52.04</td>
    <td>48.83</td>
    <td>66.12</td>
    <td>53.46</td>
    <td>49.67</td>
  </tr>
  <tr>
    <td>GPT-4-mini (Hurst et al., 2024)</td>
    <td>63.00</td>
    <td>58.21</td>
    <td>79.88</td>
    <td>69.99</td>
    <td>67.12</td>
    <td>53.49</td>
    <td>52.28</td>
    <td>48.37</td>
    <td>68.78</td>
    <td>48.87</td>
    <td>47.80</td>
  </tr>
  <tr>
    <td>Gemini-1.5-flash (Team et al., 2024)</td>
    <td>64.67</td>
    <td>61.73</td>
    <td>87.57</td>
    <td>68.55</td>
    <td>69.74</td>
    <td>57.33</td>
    <td>52.51</td>
    <td>52.72</td>
    <td>80.40</td>
    <td>49.92</td>
    <td>47.73</td>
  </tr>
  <tr>
    <td>GPT-4o (Hurst et al., 2024)</td>
    <td>67.33</td>
    <td>63.64</td>
    <td>83.85</td>
    <td>69.48</td>
    <td>68.30</td>
    <td>65.65</td>
    <td>54.71</td>
    <td>49.53</td>
    <td>80.09</td>
    <td>56.84</td>
    <td>51.10</td>
  </tr>
  <tr>
    <td>Gemini-1.5-pro (Team et al., 2024)</td>
    <td>67.78</td>
    <td>64.73</td>
    <td>89.18</td>
    <td>69.82</td>
    <td>70.33</td>
    <td>62.43</td>
    <td>57.30</td>
    <td>54.20</td>
    <td>78.72</td>
    <td>57.36</td>
    <td>52.49</td>
  </tr>
</table>

Large Language Models (LLMs): Only Text as Input

<table>
  <tr>
    <th>Model</th>
    <th>Validation</th>
    <th>Test</th>
    <th>Disease</th>
    <th>Pest</th>
  </tr>
  <tr>
    <td>GLM4-9B (GLM et al., 2024)</td>
    <td>41.89</td>
    <td>35.37</td>
    <td>34.66</td>
    <td>38.55</td>
    <td>27.98</td>
    <td>45.26</td>
    <td>40.03</td>
    <td>38.72</td>
    <td>21.56</td>
    <td>41.67</td>
    <td>36.91</td>
  </tr>
  <tr>
    <td>Qwen2.5-VL-7B (Hui et al., 2024)</td>
    <td>40.67</td>
    <td>35.37</td>
    <td>36.43</td>
    <td>44.04</td>
    <td>19.78</td>
    <td>48.21</td>
    <td>40.89</td>
    <td>37.48</td>
    <td>20.74</td>
    <td>42.04</td>
    <td>36.69</td>
  </tr>
  <tr>
    <td>InternLM2.5-7B (Cai et al., 2024)</td>
    <td>40.44</td>
    <td>36.55</td>
    <td>44.55</td>
    <td>39.73</td>
    <td>32.12</td>
    <td>46.33</td>
    <td>39.72</td>
    <td>38.34</td>
    <td>22.38</td>
    <td>42.12</td>
    <td>35.48</td>
  </tr>
  <tr>
    <td>GPT-3.5-Turbo (OpenAI, 2023)</td>
    <td>39.11</td>
    <td>37.01</td>
    <td>32.04</td>
    <td>44.72</td>
    <td>32.29</td>
    <td>45.80</td>
    <td>43.33</td>
    <td>39.04</td>
    <td>23.28</td>
    <td>41.55</td>
    <td>39.11</td>
  </tr>
  <tr>
    <td>GPT-4-Turbo (Achiam et al., 2023)</td>
    <td>47.11</td>
    <td>41.84</td>
    <td>33.39</td>
    <td>43.20</td>
    <td>35.08</td>
    <td>57.07</td>
    <td>52.83</td>
    <td>46.27</td>
    <td>23.08</td>
    <td>50.36</td>
    <td>44.21</td>
  </tr>
</table>

Table 1: The zero-shot result of different models in Agri-CM^3. The best results are highlighted in bold, while the second-best is underlined. The abbreviation in Agri-CM^3 means P: Perception, M: Mixture of Perception and Cognition Reasoning, and K: Knowledge Application. Test Total represents the average of all test instances.

Expert Validation. To guarantee the quality of the generated data, we employed two agricultural experts for manual validation. Initially, we randomly selected 200 data samples for annotation, including questions, options, answers and answers explanations. By evaluating the consistency of their annotations, if the consistency is below 90%, we retrain the experts on the inconsistent annotated data and reassess their consistency until it exceeds 90%. Only after reaching this threshold can we proceed with the formal annotation process. For detailed annotation procedures, please refer to the Appendix A.3.

3.4 Data Analysis

Base Statistics. The Agri-CM^3 contains 3,939 high-quality images and 15,901 multiple-choice questions across three reasoning levels, six disease tasks, and three pest tasks. 900 samples, 100 from each subclass, form the validation set, with the remaining 15,001 used for testing. More details in Appendix A.4.

Multi-modal diversity. Figure 11 illustrates the semantic space representation of images encoded by CLIP (Radford et al., 2021). The results demonstrate that Agri-CM^3 spans a broad semantic space, highlighting its rich semantic diversity and comprehensive coverage.

4 Experiments

4.1 Experiments Setting

In this study, we conduct a systematic evaluation of various MLLMs, including 38 open-source and
seven proprietary models, to assess their performance on the Agri-CM^3 benchmark. All evaluations are conducted in a zero-shot setting, meaning that the models do not receive any example prompts during reasoning and have not been trained explicitly on tasks related to the benchmark. This approach allows us to measure the generalization ability of MLLMs in agricultural pest and disease management and evaluate their reasoning capabilities on unseen tasks. All experiments are executed on 4 NVIDIA H800 GPUs (80GB memory). Detailed experimental settings and hyperparameter configurations are provided in the Appendix B.

4.2 Main Results

In this section, we comprehensively compare various LLMs and MLLMs using the Agri-CM^3 benchmark. For clarity, we present results from selected models in Table 1, while the complete experimental results and data can be found in the Appendix A.4 and C. Based on these results, we summarize the following key findings:

Agri-CM^3 remains a challenging benchmark. The results from Table 1 show that 26 out of the 35 MLLMs achieved a performance below 60%. The top-performing model on the test set reached 65.49%, demonstrating that real-world applications in agricultural pest and disease management still face significant challenges. Specifically, the models showed limited success in high-level reasoning tasks, with the Disease-P-M-K and Pest-P-K tasks achieving only 59.33% and 56.38%, respectively. These results highlight substantial room for improvement, particularly in complex reasoning.

Open-source MLLMs gradually outperform proprietary models. The top-performing open-source model, InternVL-2.5-78B, achieved 65.49% on the test set, outperforming the best proprietary model, Gemini-1.5-Pro, by 0.73%. This result suggests that the application potential of open-source MLLMs in agricultural tasks is steadily increasing, and in some cases, they have already surpassed proprietary models. It also highlights the importance of open-source technologies in the future of agricultural intelligence.

Optimal MLLMs performance still below expert level. We evaluated the performance of MLLMs and agricultural experts on the validation set. As shown in Table 1 and Figure 2 (right), although the optimal MLLM has demonstrated strong performance on certain tasks, there remains a performance gap of approximately 8% compared to experts. This indicates that while current MLLMs have made notable progress in agricultural tasks, further optimization is needed to integrate high-level reasoning and domain-specific knowledge.

Besides, the experts’ performance did not meet the expected level of excellence. Although agricultural experts have in-depth knowledge of specific crops or pests, their performance decreases when dealing with a broader range of crops and diseases due to the specialized nature of their research. This suggests that while experts have substantial advantages in specific areas, both their performance and that of the models are limited when faced with broader agricultural scenarios.

4.3 Further Analysis

Larger models improve performance, but with diminishing returns. We conducted experiments using InternVL-2.5 with different parameter sizes, as shown in Figure 4. The results indicate that as the model size increases, overall performance improves significantly. For tasks across different reasoning levels, larger models consistently outperform smaller ones, with an average performance gap of 24.78% between the largest and smallest models. However, as the parameter size further increases, the rate of performance improvement gradually slows, exhibiting diminishing returns. This suggests that although larger models enhance accuracy, the benefits of increasing model size become less pronounced.

Mixed-reasoning tasks are the key factor limiting model performance. To explore the impact of different reasoning levels on model performance, we use Disease-P-M-K as a case study. By successively specifying entities in the question: starting

![Scaling law of model size in Agri-CM^3. We use the InternVL-2.5 series as examples.](page_1012_120_410_312.png)

Figure 4: Scaling law of model size in Agri-CM^3. We use the InternVL-2.5 series as examples.
Figure 5: Performance of different models at different reasoning levels.

Figure 6: Performance of different MLLMs and their corresponding LLMs on Disease-K (text modality).

from Disease-P-M-K, then specifying crop to create Disease-M-K, and further specifying disease to obtain Disease-K, we investigate how different reasoning levels influence model performance. The experimental results are shown in Figure 5. The results indicate that eliminating perception reasoning (Disease-M-K) leads to only a modest performance improvement of approximately 2.32% on average. However, when mixed perception-cognition reasoning is further removed (Disease-K), the performance increase becomes more significant, with an average improvement of 6.00%. This suggests that mixed-reasoning tasks are more challenging for the models, and improving their ability in this area is key to further enhancing model performance.

Multi-modal training boosts performance in single-text tasks. In this benchmark, the Disease-K and Pest-K tasks focus on evaluating pest and disease knowledge in a single-text modality. We compared the performance of MLLMs and their corresponding LLMs on these tasks, as shown in Figure 6 and 12. The results demonstrate that, after multi-modal training, MLLMs exhibit significant improvements in single-text modality tasks compared to their corresponding LLMs, with an average performance increase of 5.06% and 4.01%. This phenomenon indicates that multi-modal training not only enhances models’ ability for joint reasoning over images and text but also improves their adaptability and generalization to complex contexts through image information processing, align with the conclusions of Ji et al. (2024).

Figure 7: In-Context Learning performance of InternVL-2.5-38B under different shots.

average performance increase of 5.06% and 4.01%. This phenomenon indicates that multi-modal training not only enhances models’ ability for joint reasoning over images and text but also improves their adaptability and generalization to complex contexts through image information processing, align with the conclusions of Ji et al. (2024).

4.4 Exploration

The structured reasoning process enabled by CoT leads to superior performance. We applied the CoT (Wei et al., 2022) method to enable step-by-step thinking and assessed its impact on performance. As shown in Table 2, GPT-4o demonstrated significant improvement, particularly in complex multi-step reasoning tasks. Compared to other models, GPT-4o effectively follows the multi-step process required by the benchmark: first identifying the crop, diagnosing the disease, and then performing knowledge integration and application based on the options. Other models, lacking such structured reasoning, tend to provide answers without step-by-step logic, resulting in less pronounced CoT performance. A detailed case analysis is provided in Appendix D.1.

In-context learning cannot stably improve performance. In this experiment, we conducted in-context learning with different shot counts using InternVL-2.5-38B, as shown in Figure 7. The results indicate that, despite some fluctuations in accuracy across different shot counts, there is no clear upward trend or consistent performance improvement. The main reason for this may be that as the number of images increases, the model could become confused due to the additional information, leading to potential interference. Therefore, this issue is likely related to the selection of examples in in-context learning. Future work could further
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th rowspan="2">CoT</th>
    <th colspan="2">Validation</th>
    <th colspan="2">Test</th>
    <th colspan="6">Disease</th>
    <th colspan="3">Pest</th>
  </tr>
  <tr>
    <th>Total</th>
    <th>Total</th>
    <th>P</th>
    <th>M</th>
    <th>P-M</th>
    <th>K</th>
    <th>M-K</th>
    <th>P-M-K</th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <td>InternVL-2.5-8B</td>
    <td>x</td>
    <td>59.67</td>
    <td>55.73</td>
    <td>70.58</td>
    <td>62.13</td>
    <td>57.06</td>
    <td>55.37</td>
    <td>52.43</td>
    <td>50.54</td>
    <td>64.16</td>
    <td>49.20</td>
    <td>48.13</td>
  </tr>
  <tr>
    <td>InternVL-2.5-8B</td>
    <td>✓</td>
    <td>58.22<sup>1.5</sup></td>
    <td>55.30<sup>1.4</sup></td>
    <td>69.32<sup>1.3</sup></td>
    <td>61.62<sup>1.6</sup></td>
    <td>59.59<sup>1.5</sup></td>
    <td>50.89<sup>1.5</sup></td>
    <td>52.83<sup>1.4</sup></td>
    <td>51.09<sup>1.6</sup></td>
    <td>64.71<sup>1.5</sup></td>
    <td>46.78<sup>1.4</sup></td>
    <td>48.53<sup>1.4</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-38B</td>
    <td>x</td>
    <td>68.11</td>
    <td>65.31</td>
    <td>74.81</td>
    <td>76.67</td>
    <td>70.16</td>
    <td>65.65</td>
    <td>61.30</td>
    <td>60.26</td>
    <td>75.04</td>
    <td>56.84</td>
    <td>56.89</td>
  </tr>
  <tr>
    <td>InternVL-2.5-38B</td>
    <td>✓</td>
    <td>64.56<sup>1.5</sup></td>
    <td>59.89<sup>1.4</sup></td>
    <td>73.29<sup>1.5</sup></td>
    <td>72.02<sup>1.4</sup></td>
    <td>64.07<sup>1.3</sup></td>
    <td>58.05<sup>1.6</sup></td>
    <td>55.97<sup>1.5</sup></td>
    <td>53.34<sup>1.9</sup></td>
    <td>70.15<sup>1.4</sup></td>
    <td>52.86<sup>1.4</sup></td>
    <td>49.45<sup>1.4</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-78B</td>
    <td>x</td>
    <td>69.89</td>
    <td>65.49</td>
    <td>79.54</td>
    <td>76.67</td>
    <td>72.78</td>
    <td>65.56</td>
    <td>60.44</td>
    <td>59.33</td>
    <td>73.04</td>
    <td>58.00</td>
    <td>56.38</td>
  </tr>
  <tr>
    <td>InternVL-2.5-78B</td>
    <td>✓</td>
    <td>67.56<sup>1.3</sup></td>
    <td>61.71<sup>1.6</sup></td>
    <td>79.37<sup>1.2</sup></td>
    <td>74.22<sup>1.2</sup></td>
    <td>69.23<sup>1.5</sup></td>
    <td>61.81<sup>1.8</sup></td>
    <td>55.42<sup>1.8</sup></td>
    <td>55.52<sup>1.6</sup></td>
    <td>69.47<sup>1.3</sup></td>
    <td>53.86<sup>1.4</sup></td>
    <td>50.70<sup>1.4</sup></td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>x</td>
    <td>60.00</td>
    <td>56.32</td>
    <td>85.88</td>
    <td>58.92</td>
    <td>57.82</td>
    <td>54.20</td>
    <td>47.88</td>
    <td>46.27</td>
    <td>70.34</td>
    <td>47.43</td>
    <td>46.26</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>✓</td>
    <td>60.00<sup>1.0</sup></td>
    <td>55.87<sup>1.3</sup></td>
    <td>79.71<sup>1.3</sup></td>
    <td>62.55<sup>1.3</sup></td>
    <td>61.12<sup>1.3</sup></td>
    <td>51.61<sup>1.8</sup></td>
    <td>48.04<sup>1.2</sup></td>
    <td>47.59<sup>1.1</sup></td>
    <td>69.41<sup>1.3</sup></td>
    <td>45.90<sup>1.3</sup></td>
    <td>46.08<sup>1.3</sup></td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>x</td>
    <td>72.00</td>
    <td>64.81</td>
    <td>80.98</td>
    <td>73.29</td>
    <td>72.36</td>
    <td>65.74</td>
    <td>61.77</td>
    <td>59.95</td>
    <td>72.07</td>
    <td>55.07</td>
    <td>56.27</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>✓</td>
    <td>67.44<sup>1.6</sup></td>
    <td>62.19<sup>1.2</sup></td>
    <td>79.46<sup>1.2</sup></td>
    <td>71.17<sup>1.1</sup></td>
    <td>66.53<sup>1.8</sup></td>
    <td>65.21<sup>1.8</sup></td>
    <td>55.57<sup>1.8</sup></td>
    <td>55.44<sup>1.9</sup></td>
    <td>70.42<sup>1.4</sup></td>
    <td>54.83<sup>1.2</sup></td>
    <td>52.97<sup>1.3</sup></td>
  </tr>
  <tr>
    <td>GPT-4o-mini</td>
    <td>x</td>
    <td>63.00</td>
    <td>58.21</td>
    <td>79.88</td>
    <td>69.99</td>
    <td>67.12</td>
    <td>53.49</td>
    <td>52.28</td>
    <td>48.37</td>
    <td>68.78</td>
    <td>48.87</td>
    <td>47.80</td>
  </tr>
  <tr>
    <td>GPT-4o-mini</td>
    <td>✓</td>
    <td>61.78<sup>1.2</sup></td>
    <td>56.14<sup>1.2</sup></td>
    <td>82.25<sup>1.4</sup></td>
    <td>68.30<sup>1.7</sup></td>
    <td>66.78<sup>1.3</sup></td>
    <td>52.06<sup>1.4</sup></td>
    <td>48.12<sup>1.4</sup></td>
    <td>46.03<sup>1.3</sup></td>
    <td>64.67<sup>1.4</sup></td>
    <td>46.66<sup>1.2</sup></td>
    <td>45.75<sup>1.2</sup></td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>x</td>
    <td>67.33</td>
    <td>63.64</td>
    <td>83.85</td>
    <td>69.48</td>
    <td>68.30</td>
    <td>65.65</td>
    <td>54.71</td>
    <td>49.53</td>
    <td>80.09</td>
    <td>56.84</td>
    <td>51.10</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>✓</td>
    <td>70.56<sup>1.2</sup></td>
    <td>65.43<sup>1.8</sup></td>
    <td>87.91<sup>1.4</sup></td>
    <td>78.95<sup>1.5</sup></td>
    <td>75.23<sup>1.9</sup></td>
    <td>61.45<sup>1.2</sup></td>
    <td>58.48<sup>1.8</sup></td>
    <td>54.04<sup>1.4</sup></td>
    <td>80.13<sup>1.0</sup></td>
    <td>53.94<sup>1.9</sup></td>
    <td>52.49<sup>1.4</sup></td>
  </tr>
</table>

Table 2: Performance comparison of different MLLMs with and without CoT.

![Bar chart comparing average performance of different models on seven image-dependent tasks, with and without images](page_101_1012_495_312.png)

Figure 8: The average performance of different models on seven image-dependent tasks, comparing results with and without the use of images.

![Pie chart showing error distribution for GPT-4o errors](page_101_1342_495_312.png)

Figure 9: Overall error distribution for 200 annotated GPT-4o errors.

explore the selection of examples.

The dataset relies heavily on visual input. To assess the dataset’s dependency on the visual modality, we conducted experiments on datasets requiring image input for responses, comparing model performance with and without image input. Specifically, we evaluated MLLMs of different scales under both settings. The results, shown in Figure 8, indicate a significant performance drop across all models when visual information is removed. This finding demonstrates that the dataset is highly dependent on visual input, as models require image information for practical reasoning.

5 Error Analysis and Future Work

In this section, we analyze the errors of GPT-4o CoT predictions by randomly sampling 200 incorrect predictions from the most challenging pest and disease reasoning tasks. These errors were annotated and categorized by agricultural experts, and the distribution is shown in Figure 9. Analysis reveals that the primary cause of errors is the lack of domain-specific knowledge, particularly in disease reasoning tasks. Additionally, in pest reasoning tasks, the model’s insufficient expertise leads to inaccurate pest identification and noticeable perceptual errors, highlighting the limitations of current models: insufficient training on specialized agricultural knowledge. To address these issues, future work should focus on enhancing domain knowledge. Overcoming these challenges will improve the performance of multi-modal models in agricultural disease and pest reasoning tasks. Detailed case studies and analyses are provided in the Appendix D.2.

6 Conclusion

In this study, we introduce Agri-CM^3, an innovative multi-level benchmark designed to comprehensively evaluate the complex reasoning capabilities of MLLMs in agricultural pest and disease management, filling a gap in the field. This multi-level design not only evaluates the model’s ability in individual sub-reasoning tasks but also tests its performance in combined reasoning tasks and complete
reasoning chains, offering clear direction for future model optimization. Additionally, Agri-CM^3 provides valuable insights for designing and evaluating MLLMs. Our study emphasizes the importance of integrating visual and language modalities and validates the crucial role of domain-specific knowledge in vertical tasks.

Limitations

In this study, we introduced Agri-CM^3, a multi-level benchmark designed to evaluate the complex reasoning abilities of MLLMs in agricultural pest and disease management. Although we have considered various factors, there are still some limitations. Firstly, the current benchmark is based on the Chinese language, which limits its applicability to multilingual tasks. To enhance its usability across language communities, we employed GPT-4o to translate the dataset into English, enabling broader participation from the global research community. However, potential translation biases and semantic shifts introduced by the model may affect a small portion of the content. Secondly, agricultural experts typically specialize in specific areas, which may impact the accuracy of annotations for tasks outside their area of expertise. This specialization bias could lead to annotation inconsistencies, especially for tasks involving different crops or pests. To address this, future efforts may incorporate multi-domain experts or cross-domain knowledge to enhance annotation accuracy and consistency.

Ethical Considerations

In constructing the Agri-CM^3 benchmark, we leveraged GPT-4o to assist in both translating Chinese content into English and generating synthetic question–answer pairs based on agricultural images and structured domain knowledge. While this approach improves scalability and consistency, it also introduces potential ethical concerns.

First, GPT-4o may encode biases from its pre-training data as a proprietary model, particularly regarding domain-specific or culturally nuanced content. Such biases may affect translation fidelity, domain framing, or answer correctness. This is particularly relevant in agriculture, where regional terminology and knowledge may be underrepresented. Second, although all generated content was validated by human experts, the filtering and editing of LLM outputs remain subject to human limitations. As a result, some edge-case errors or subtle biases may persist. Third, since the benchmark was originally constructed in Chinese and then translated, there is a risk of semantic drift or misinterpretation during translation. Lastly, all image data were sourced from public datasets or used under fair academic use, with no personal or sensitive information involved. Nonetheless, we acknowledge the importance of ongoing ethical reflection on data provenance, model transparency, and equitable access.

Future iterations of Agri-CM^3 will incorporate multilingual sources, fairness metrics, and explicit documentation of observed biases to improve transparency and inclusiveness.

Acknowledgements

We thank the anonymous reviewers for their insightful comments. This work was supported in part by the Science and Technology Innovation Award of Heilongjiang Province, China [JD2023GJ01], the Key Research and Development Program of Heilongjiang Province, China [2024ZX01A07], and the National Natural Science Foundation of China [72293584, 72431004]. Jingchi Jiang is the corresponding author of this work.

References

Petchiammal A, Briskline Kiruba S, Murugan D, and Pandarasamy Arjunan. 2022. Paddy doctor: A visual image dataset for automated paddy disease classification and benchmarking.

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219.

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

01. AI, ;, Alex Young, Bei Chen, Chao Li, Chenge Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Tao Yu, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yuchi Xu, Yudong Liu, Yue Wang, Yuxuan Cai, Zhenyu Gu, Zhiyuan Liu, and Zonghong Dai. 2024. Yi: Open foundation models by 01.ai. Preprint, arXiv:2403.04652.
Anthropic. 2025. Claude 3.5 sonnet. Accessed: 2025-02-07.

David Arthur and Sergei Vassilvitskii. 2007. k-means++: the advantages of careful seeding. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’07, page 1027–1035, USA. Society for Industrial and Applied Mathematics.

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1(2):3.

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923.

Zheng Cai, Maosong Cao, Haojing Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297.

Pengcheng Chen, Jin Ye, Guoan Wang, Yanjun Li, Zhongying Deng, Wei Li, Tianbin Li, Haodong Duan, Ziyuan Huang, Yanzhou Su, Benyou Wang, Shaoting Zhang, Bin Fu, Jianfei Cai, Bohan Zhuang, Eric J Seibel, Junjin He, and Yu Qiao. 2024a. GMAI-MMBench: A comprehensive multimodal evaluation benchmark towards general medical AI. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jingou Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024b. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271.

Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024c. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101.

Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. 2024. Molmio and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146.

Fei Gao, Xiaoli Wang, Tingting Liu, Zhuang Li, and Rui Man. 2020. Spectral and imaging datasets of apple leaf disease and insect pests in china in 2015. Journal of Agricultural Big Data, 2(4):120–124.

Hawi Getachew. 2021. Wheat leaf dataset. Mendeley Data, V1. Accessed: 2025-02-14.

Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793.

Lakshay Goyal, Chandra Mani Sharma, Anupam Singh, and Pradeep Kumar Singh. 2021. Leaf and spike wheat disease detection classification using an improved deep convolutional architecture. Informatics in Medicine Unlocked, 25:100642.

Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, et al. 2024. Cogym2: Visual language models for image and video understanding. arXiv preprint arXiv:2408.16500.

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186.

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, Al Oshtor, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.

Jiaming Ji, Jiayi Zhou, Hantao Lou, Boyuan Chen, Donghai Hong, Xuyao Wang, Wenci Chen, Kaile Wang, Rui Pan, Jiahao Li, Mohan Wang, Josef Dai, Tianyi Qiu, Hua Xu, Dong Li, Weipeng Chen, Jun Song, Bo Zheng, and Yaodong Yang. 2024. Align anything: Training all-modality models to follow instructions with language feedback.

Xi Jiang, Jian Li, Hanqiu Deng, Yong Liu, Bin-Bin Gao, Yifeng Zhou, Jialin Li, Chengjie Wang, and Feng Zheng. 2025. MMAD: The first-ever comprehensive benchmark for multimodal large language models in industrial anomaly detection. In The Thirteenth International Conference on Learning Representations.

Matthew Abu John, Ibukunoluwa Bankole, Oluwatayo Ajayi-Moses, Tofunmi Ijila, Oluwatimilehin Jeje, and Patil Lalit. 2023. Relevance of advanced plant disease detection techniques in disease and pest management for ensuring food security and their implication: A review. American Journal of Plant Sciences, 14(11):1260–1295.

Matheus Thomas Kuska, Mirwaes Wahabzada, and Stefan Paulus. 2024. Ai for crop production – where can large language models (llms) provide substantial value? Computers and Electronics in Agriculture, 221:108924.

Jillian Lenné and David Wood. 2024. Crop diversity in agroecosystems for pest management and food production. Plants, 13(8):1164.

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Ueyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. 2023. LLaVA-med: Training a large language-and-vision assistant for biomedicine in one day. In Thirty- seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.

Yanzhong Li. 2023. Atlas of Alfalfa Diseases, Insect Pests, and Natural Enemies and Dataset of Major Alfalfa Diseases in Heilongjiang Province (2015-2016).

Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023a. Improved baselines with visual instruction tuning.

Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. Llava-next: Improved reasoning, ocr, and world knowledge.

Haotian Liu, Chunyuan Li, Qinyang Wu, and Yong Jae Lee. 2023b. Visual instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems.

Md Maful Hasan Matin Mafi, Mst. Arifa Anjum Ava, Md. Golam Moazzam Moazzam, and Mohammad Shorif Uddin. 2023. Eggplant disease recognition dataset. Mendeley Data, V2. Accessed: 2025-02-14.

Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zaz- kka, Eduardo Pontes Reis, and Pranav Rajpurkar. 2023. Med-flamingo: a multimodal medical few-shot learner. In Proceedings of the 3rd Machine Learning for Health Symposium, volume 225 of Proceedings of Machine Learning Research, pages 353–367. PMLR.

OpenAI. 2023. Introducing GPT-3.5 & GPT-4. Accessed: 2025-02-04.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR.

Jitesh Shah, Harshadkumar Prajapati, and Vipul Dabhi. 2017. Rice Leaf Diseases. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5R013.

Amrita Singh, Nikita Shraogi, Rahul Verma, Joel Saji, Aditya Kumar Kar, Sarita Tehlan, Debabrata Ghosh, and Satyakam Patnaik. 2024. Challenges in current pest management practices: Navigating problems and a way forward by integrating controlled release system approach. Chemical Engineering Journal, 498:154989.

Davinder Singh, Naman Jain, Pranjali Jain, Pratik Kayal, Sudhakar Kumawat, and Nipun Batra. 2020. Plantdoc: A dataset for visual plant disease detection. In Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, CoDS COMAD 2020, page 249–253, New York, NY, USA. Association for Computing Machinery.

Yuxuan Sun, Hao Wu, Chenglu Zhu, Sunyi Zheng, Qizi Chen, Kai Zhang, Yunlong Zhang, Dan Wan, Xiaoxiao Lan, Mengye Zheng, Jingxiong Li, Xinheg Lyu, Tao Lin, and Lin Yang. 2025. Pathmmu: A massive multimodal expert-level benchmark for understanding and reasoning in pathology. In Computer Vision – ECCV 2024, pages 56–73, Cham. Springer Nature Switzerland.

Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530.

Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. 2024. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. arXiv preprint arXiv:2406.16860.

Liqiong Wang, Teng Jin, Jinyu Yang, Ales Leonardis, Fangyi Wang, and Feng Zheng. 2024a. Agri-llava: Knowledge-infused large multimodal assistant on agricultural pests and diseases. arXiv preprint arXiv:2412.02158.

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024b. Owen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichtier, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc.

Tyr Wiesner-Hanks, Ethan L Stewart, Nicholas Kaczmar, Chad DeChant, Harvey Wu, Rebecca J Nelson, Hod Lipson, and Michael A Gore. 2018. Image set for deep learning: field images of maize annotated with disease symptoms. BMC research notes, 11:1–3.

Xiaoping Wu, Chi Zhan, Yu-Kun Lai, Ming-Ming Cheng, and Jufeng Yang. 2019. Ipl02: A large-scale benchmark dataset for insect pest recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8787–8796.

Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. 2024. Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302.
Hanguang Xiao, Feizhong Zhou, Xingyue Liu, Tianqi Liu, Zhipeng Li, Xin Liu, and Xiaoxuan Huang. 2025. A comprehensive survey of large language models and multimodal large language models in medicine. Information Fusion, 117:102888.

Qianqian Xie, Dong Li, Mengxi Xiao, Zihao Jiang, Ruoyu Xiang, Xiao Zhang, Zhengyu Chen, Yueru He, Weiguang Han, Yuzhe Yang, et al. 2024. Open-fimllms: Open multimodal large language models for financial applications. arXiv preprint arXiv:2408.11878.

Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. 2024. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800.

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556–9567.

Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024a. MM-LLMs: Recent advances in MultiModal large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12401–12430, Bangkok, Thailand. Association for Computational Linguistics.

Ge Zhang, Xinrun Du, Bei Chen, Yiming Liang, Tongxu Luo, Tianyu Zheng, Kang Zhu, Yuyang Cheng, Chunpu Xu, Shuyue Guo, et al. 2024b. Cmmmu: A chinese massive multi-discipline multi-modal understanding benchmark. arXiv preprint arXiv:2401.11944.

Xianghe Zhang, Xiaoli Wang, Tingting Liu, Lin Hu, and Jingchao Fan. 2022. Image data set of six common orchard pests such as bactrocera dorsalis. Journal of Agricultural Big Data, 4(1):114–118.

Yutong Zhou and Masahiro Ryo. 2024. Agribench: A hierarchical agriculture benchmark for multi-modal large language models. arXiv preprint arXiv:2412.00465.

Hongyan Zhu, Shuai Qin, Min Su, Chengzhi Lin, Anjie Li, and Junfeng Gao. 2024. Harnessing large vision and language models in agriculture: A review. arXiv preprint arXiv:2407.19679.

A Dataset Agri-CM^3 Details

A.1 Data Collection

To ensure high-quality image data, we collected a large number of images from existing agricultural pest and disease datasets and related websites, resulting in 21 distinct pest and disease datasets (see Table 3). We filtered the images based on resolution and excluded those with fewer instances. Additionally, we applied K-means clustering to eliminate highly similar images, ensuring dataset diversity. In the end, we retained 5,727 high-quality images. To further validate the images, two agricultural experts were hired to review and filter out the following categories of unusable images:

i. Images containing crop, disease, pest names, or symptom-related terms.

ii. Images with large watermarks obscuring key features of pests and diseases.

iii. Images unrelated to pests and diseases.

After filtering, we obtained 3,939 high-quality agricultural pest and disease images with crop and pest labels. These images formed the foundation for generating subsequent question-answer data.

To ensure the quality of the generated content and prevent hallucinations, we gathered domain-specific background knowledge related to pests and diseases. The sources included professional agricultural books, Baidu Baike\footnote{https://baike.baidu.com/}, and Wikipedia\footnote{https://zh.wikipedia.org/}, totaling 5.2 million words of specialized content. This knowledge was structured into a database for reference by GPT-4o during data generation, providing a solid foundation for generating precise and reliable question-answer data.

A.2 Data Generation

Firstly, we use GPT-4o to generate high-level complex reasoning data based on agricultural pest and disease images, crop, pest, and disease captions, along with relevant agricultural background knowledge. Based on this high-level data, we progressively derive low-level complex reasoning data, with the derivation sequence shown in Table 4 and the prompts provided in Prompt A.1, A.2 A.3, and A.4. To ensure the accuracy of the generated data, we provide an example to guide the model towards generating more accurate results.

It is important to note that the pest recognition task only involves the perception stage, and therefore, the mixture of perception-cognition reasoning stage is not included in the generation of high-level pest data.

A.3 Expert Validation

To ensure the quality of the generated data, we incorporated external domain-specific background
<table>
  <tr>
    <th>Dataset Name</th>
    <th>Dataset Type</th>
    <th># Crop</th>
    <th># Disease/Pest</th>
    <th>Link</th>
  </tr>
  <tr>
    <td>PlantDoc-Dataset (Singh et al., 2020)</td>
    <td>Disease</td>
    <td>13</td>
    <td>28</td>
    <td>https://github.com/pratikkayal/PlantDoc-Dataset</td>
  </tr>
  <tr>
    <td>Rice disease image classification</td>
    <td>Disease</td>
    <td>1</td>
    <td>5</td>
    <td>https://www.kaggle.com/nurnoboi1/rice-disease</td>
  </tr>
  <tr>
    <td>Rice Diseases</td>
    <td>Disease, Pest</td>
    <td>1</td>
    <td>4</td>
    <td>https://www.kaggle.com/renzmarrionperera/ricedisease</td>
  </tr>
  <tr>
    <td>ai_challenger_pdf2018</td>
    <td>Disease</td>
    <td>10</td>
    <td>40</td>
    <td>https://aistudio.baidu.com/datasetdetail/76075</td>
  </tr>
  <tr>
    <td>Agricultural Pest Research Base</td>
    <td>Disease, Pest</td>
    <td>8</td>
    <td>40</td>
    <td>http://www.icgroupcas.cn/website_bchtk/fuwu.html</td>
  </tr>
  <tr>
    <td>CGIAR Computer Vision for Crop Disease</td>
    <td>Disease</td>
    <td>1</td>
    <td>3</td>
    <td>https://www.kaggle.com/datasets/shadabhusain/cgia-r-computer-vision-for-crop-disease</td>
  </tr>
  <tr>
    <td>IP102 v1.1 (Wu et al., 2019)</td>
    <td>Pest</td>
    <td>8</td>
    <td>102</td>
    <td>https://github.com/xpu95/IP102</td>
  </tr>
  <tr>
    <td>Wheat Leaf Dataset (Getachew, 2021)</td>
    <td>Disease</td>
    <td>1</td>
    <td>3</td>
    <td>https://data.mendeley.com/datasets/wgd6f6n8h/1</td>
  </tr>
  <tr>
    <td>Large Wheat Disease Classification Dataset (Goyal et al., 2021)</td>
    <td>Disease</td>
    <td>1</td>
    <td>4</td>
    <td>https://drive.google.com/drive/folders/10HKtwO1UrdmhqxrQeE_f_X_pGkQtXRQG</td>
  </tr>
  <tr>
    <td>Spectral and image dataset of apple leaf diseases and insect pests in China in 2015 (Gao et al., 2020)</td>
    <td>Disease, Pest</td>
    <td>1</td>
    <td>3</td>
    <td>https://www.agridata.cn/data.html#/datadetail?id=4363</td>
  </tr>
  <tr>
    <td>Image dataset of six common orchard pests including Bactrocera dorsalis (Zhang et al., 2022)</td>
    <td>Pest</td>
    <td>0</td>
    <td>6</td>
    <td>https://www.agridata.cn/data.html#/datadetail?id=286640</td>
  </tr>
  <tr>
    <td>Corn Disease Image Database</td>
    <td>Disease</td>
    <td>1</td>
    <td>5</td>
    <td>https://www.agridata.cn/data.html#/datadetail?id=288848</td>
  </tr>
  <tr>
    <td>Field images of maize annotated with disease symptoms (Wiesner-Hanks et al., 2018)</td>
    <td>Disease</td>
    <td>1</td>
    <td>1</td>
    <td>https://osf.io/p67rz/</td>
  </tr>
  <tr>
    <td>Database of rice pests and diseases in the field in South China and Southwest China from 2018 to 2020</td>
    <td>Disease, Pest</td>
    <td>1</td>
    <td>20</td>
    <td>https://www.agridata.cn/data.html#/datadetail?id=291593</td>
  </tr>
  <tr>
    <td>Data on the occurrence of insect pests in major crop fields in China from 2013 to 2018</td>
    <td>Pest</td>
    <td>5</td>
    <td>53</td>
    <td>https://www.agridata.cn/data.html#/datadetail?id=4437</td>
  </tr>
  <tr>
    <td>Data on the occurrence of major crop field diseases in China from 2013 to 2018</td>
    <td>Disease</td>
    <td>5</td>
    <td>34</td>
    <td>https://www.agridata.cn/data.html#/datadetail?id=4436</td>
  </tr>
  <tr>
    <td>Atlas of Alfalfa Diseases, Insect Pests, and Natural Enemies and Dataset of Major Alfalfa Diseases in Heilongjiang Province (2015-2016) (Li, 2023)</td>
    <td>Pest</td>
    <td>1</td>
    <td>203</td>
    <td>https://www.scidb.cn/en/detail?dataSetId=D5_455c0351d04b4a7e912b6d01a66ee5f7</td>
  </tr>
  <tr>
    <td>Rice Leaf Disease (Shah et al., 2017)</td>
    <td>Disease</td>
    <td>1</td>
    <td>3</td>
    <td>https://archive.ics.uci.edu/dataset/486/rice+leaf+diseases</td>
  </tr>
  <tr>
    <td>Eggplant Disease Recognition Dataset (Mafi et al., 2023)</td>
    <td>Disease, Pest</td>
    <td>1</td>
    <td>13</td>
    <td>https://data.mendeley.com/datasets/r3tb5mzn4d/2</td>
  </tr>
  <tr>
    <td>paddy-doctor-diseases-medium (A et al., 2022)</td>
    <td>Disease, Pest</td>
    <td>1</td>
    <td>13</td>
    <td>https://ieee-dataport.org/competitions/design-classifier-classify-diseases-paddy-based-leaf-color</td>
  </tr>
  <tr>
    <td>Baidu Encyclopedia Data</td>
    <td>Pest</td>
    <td>1</td>
    <td>5</td>
    <td>https://baike.baidu.com/</td>
  </tr>
</table>

Table 3: Sources for agricultural pest and disease image data collection.

<table>
  <tr>
    <th colspan="3">Derivation Sequence</th>
  </tr>
  <tr>
    <td>Disease-P-M-K</td>
    <td><i>Specified Crop</i>, Disease-M-K</td>
    <td>Disease-K</td>
  </tr>
  <tr>
    <td>Disease-P-M</td>
    <td><i>Specified Crop</i>, Disease-M</td>
    <td></td>
  </tr>
  <tr>
    <td>Pest-P-K</td>
    <td><i>Specified Pest</i></td>
    <td>Pest-K</td>
  </tr>
  <tr>
    <td>Disease-P</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Disease-P</td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 4: Derivation sequence of high to low level complex reasoning data.

knowledge during the generation process. However, due to the inherent hallucination issue of MLLMs, errors in the data are unavoidable. Therefore, we hired two agricultural experts to annotate the data and filter out erroneous entries. To ensure annotation consistency, the experts underwent training, followed by the annotation of 200 samples for preliminary labeling. The annotation consistency was calculated, and when it fell below 90%, further training was provided to address discrepancies, followed by reannotation of another 200 samples. The detailed process is shown in Figure 10. After two training rounds, the experts achieved over 90% consistency, with the results in Table 5.

<table>
  <tr>
    <th>Metrics</th>
    <th>Round 1</th>
    <th>Round 2</th>
  </tr>
  <tr>
    <td># Consistency of Question</td>
    <td>92.5</td>
    <td>94.0</td>
  </tr>
  <tr>
    <td># Consistency of Options</td>
    <td>90.5</td>
    <td>93.0</td>
  </tr>
  <tr>
    <td># Consistency of Answer</td>
    <td>87.5</td>
    <td>92.5</td>
  </tr>
  <tr>
    <td># Consistency of Answer Explanation</td>
    <td>87.5</td>
    <td>93.0</td>
  </tr>
</table>

Table 5: Consistency among experts in different labeling rounds.

A.4 Data Statistics

Through rigorous expert review, we obtained 3,939 high-quality images and 15,901 multiple-choice questions. A total of 900 high-resolution data entries were selected from each category, with 100
Prompt A.1: High-Level Data Generation

【指令】你是一名农学领域的教师，你需要依据作物疾病某个方面的背景知识生成考察知识的推理逻辑图。然后依据图片和逻辑图再生成多项单选题。通过逻辑图生成问题是为了引导学生从识别图中的作物开始，逐步深入到判断是该作物的哪种疾病、该疾病的相关特性是什么，从全面考察他们在农学领域的知识运用与逻辑思维能力。
【示例】
【信息】
作物：{ Crop Name }
疾病：{ Disease or Pest Name }
背景知识：{ Background Knowledge }
【要求】
1. 逻辑图的前两个节点固定为“作物”、“疾病”，逻辑图至少包含4个节点。
2. 根据图片和逻辑图生成问题和选项，保证问题需要根据图片才能回答。
3. 问题中不应出现作物名称、疾病名称、害虫名称、致病因子的名称、致病因子的类型（如孢子、细菌等）。
4. 每个选项由一个词或一个短语组成。
5. 答案中给出正确选项的字母。
6. 根据逻辑图和背景知识给出解题逻辑，每个步骤对应逻辑图中的一个节点，每一步中要有推理的结果。
7. 对于每个选择题输出生成选择题用到的背景知识。
8. 输出选项解析，逐个分析选项。
9. 生成3个多项选择题。

<table>
  <tr>
    <th rowspan="2">Metric</th>
    <th colspan="2">Validation Overall</th>
    <th colspan="2">Test Overall</th>
    <th colspan="6">Disease</th>
    <th colspan="3">Pest</th>
  </tr>
  <tr>
    <th>P</th>
    <th>M</th>
    <th>P-M</th>
    <th>K</th>
    <th>M-K</th>
    <th>P-M-K</th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <td>Avg. Question Length</td>
    <td>19.43</td>
    <td>19.40</td>
    <td>12.61</td>
    <td>18.56</td>
    <td>22.70</td>
    <td>17.63</td>
    <td>24.80</td>
    <td>22.59</td>
    <td>13.37</td>
    <td>21.36</td>
    <td>21.95</td>
  </tr>
  <tr>
    <td>Avg. Option Length</td>
    <td>3.66</td>
    <td>3.55</td>
    <td>2.03</td>
    <td>5.11</td>
    <td>3.55</td>
    <td>5.11</td>
    <td>3.54</td>
    <td>3.54</td>
    <td>3.39</td>
    <td>3.28</td>
    <td>3.28</td>
  </tr>
  <tr>
    <td># Question / # Images</td>
    <td>900 / 200</td>
    <td>15001 / 3767</td>
    <td>1283 / 1283</td>
    <td>1283 / 1283</td>
    <td>1283 / 1283</td>
    <td>1218 / -</td>
    <td>1375 / 1160</td>
    <td>1389 / 1160</td>
    <td>2656 / 2656</td>
    <td>2586 / -</td>
    <td>2828 / 2332</td>
  </tr>
  <tr>
    <td>Avg. Image Resolution</td>
    <td>2081 × 1642</td>
    <td>1815 × 1489</td>
    <td>2128 × 1783</td>
    <td>2128 × 1783</td>
    <td>2128 × 1783</td>
    <td>-</td>
    <td>2171 × 1808</td>
    <td>2186 × 1819</td>
    <td>1444 × 1140</td>
    <td>-</td>
    <td>1449 × 1139</td>
  </tr>
</table>

Table 6: Detailed statistics on the Agri-CM^3 benchmark.

entries per category, to form the Validation set. The remaining 15,001 entries were used as the Test set. The dataset includes 11 crops, 45 diseases, and 107 pests. Figure 14 shows data cases for different tasks, and Figure 13 illustrates the label distribution across the global, disease, and pest levels, indicating a balanced distribution across all labels.

Further statistical analysis of the dataset is provided in Table 6, demonstrating the high image quality of Agri-CM^3. The average resolution is 1800 × 1500 pixels, ensuring high clarity and detail. Additionally, homogenized images were filtered out automatically by the model, ensuring diversity. All images were also manually annotated and validated by agricultural experts to guarantee data reliability and usability.

A.5 Diversity Analysis of Agri-CM^3

Image Diversity Analysis. Figure 11 illustrates the spatial representations of images encoded using CLIP (Radford et al., 2021). The wide distribution of image representations across the space indicates rich semantic content and intense coverage, suggesting that the dataset has a high diversity and comprehensive representation. This diversity enables a comprehensive evaluation of the model’s ability across various agricultural pest and disease management tasks.

Text Diversity Analysis. Further analysis of phrase frequencies in questions and options is presented in the word clouds in Figure 15. In the question word cloud, terms like “害虫 (Pest)”, “病害 (Disease)”, “特征 (Characteristic)” and “防治 (Prevention)” dominate, reflecting the benchmark’s focus on agricultural pest and disease management, aligning with our objectives. In the option word cloud, terms like “水稻 (Rice)”, “小麦 (Wheat)” and “稻瘟病 (Rice Blast)” are prominent, validating the targeted approach of our dataset in reflecting key agricultural issues.

A.6 Multilingual Dataset

Given the strong machine translation capabilities of current large language models—particularly proprietary models like GPT-4o—we utilized GPT-4o to translate our Chinese dataset into English. This not only ensures high-quality translations but also enhances reproducibility and facilitates community-driven extensions.
Prompt A.2: High-Level Data Generation (Translation)

[Instruction] You are a teacher in the field of agronomy. Based on the background knowledge of a crop disease in a specific aspect, you are required to generate a reasoning logic diagram to assess the knowledge. Then, based on the diagram, generate multiple-choice questions. The purpose of generating questions from the logic diagram is to guide students from identifying the crop to gradually delving into determining the type of disease affecting the crop, its related characteristics, and comprehensively assess their knowledge application and logical thinking abilities in the field of agronomy.

[Example]
[Example]

【Information】
Crop: {Crop Name}
Disease: {Disease or Pest Name}
Background Knowledge: {Background Knowledge}

【Requirements】
1. The first two nodes of the logic diagram should be fixed as “Crop” and “Disease,” and the diagram should include at least four nodes.
2. Generate questions and options based on the image and logic diagram, ensuring that the questions can only be answered by referring to the diagram.
3. The question should not mention the crop name, disease name, pest name, pathogen name, or the type of pathogen (such as spores, bacteria, etc.).
4. Each option should be a word or a short phrase.
5. Provide the correct option in the answer key.
6. Based on the logic diagram and background knowledge, provide the reasoning for solving the question, where each step corresponds to a node in the diagram. Each step should have a reasoning outcome.
7. Provide the background knowledge used to generate the multiple-choice questions.
8. Output option analysis, analyzing each option individually.
9. Generate three multiple-choice questions.

Prompt A.3: Low-Level Data Derivation

【指令】你是一名农学老师，你现在要修改一个农学多项选择题的问题，将原先没有明确作物和疾病名称、结合图片进行的提问，改为在题目中给出作物名称、结合图片进行提问。
【示例】
[Example]

【信息】
作物：{Crop Name}
需要修改的问题：{Question}

B Evaluation

For closed-source models, we directly call the corresponding APIs. For open-source models, if the model supports vLLM\(^3\), we deploy it in the OpenAI API format and test it according to the API call procedure. If the model does not support vLLM, we use the official inference method provided by the model developers to perform reasoning. During the evaluation, to ensure the stability of the results, we set the temperature parameter to 0.0 and the maximum output length to 1024.

See Prompt B.1, B.3, B.5 for the prompts in the experiment. The English Prompt after translation is B.2, B.4, and B.6.

\(^3\)https://github.com/vllm-project/vllm

B.1 Evaluation Metric

Our benchmark consists entirely of multiple-choice questions. For models that directly respond with an option, we evaluate their performance by matching the answer string with the correct option. For CoT methods, we consider the final answer provided after the reasoning process. The evaluation metric used is accuracy (ACC), calculated using the formula:

\[
\text{ACC} = \frac{n}{N}.
\]

where n denotes the amount of data answered correctly and N denotes the amount of all data.

B.2 Evaluation Models

In this study, we evaluated 45 MLLMs and 5 LLMs on Agri-CM\(^3\). The specific parameters of these
Prompt A.4: Low-Level Data Derivation (Translation)

[Instruction] You are an agronomy teacher. You are now required to modify a multiple-choice question in agronomy by revising the original question, which previously did not specify the crop and disease names and was based on images, to a version where the crop name is provided in the question along with the image for reference.
[Example]
[Example]
[Information]
Crop: {Crop Name}
Question to be modified: {Question}

![Flowchart showing expert annotation training process](page_328_613_795_181.png)

Figure 10: Expert annotation training process. To ensure annotation quality and consistency, formal annotation begins only when expert agreement exceeds 90%.

![Scatter plot of image spatial representation via CLIP encoding and PCA dimensionality reduction](page_120_820_495_312.png)

Figure 11: Image spatial representation via CLIP encoding and PCA dimensionality reduction.

models are listed in Table 7.

C Supplementary Results

In this section, we present the complete experimental results for all models on the benchmark in Table 10, including 45 MLLMs and 5 LLMs. Additionally, Table 11 provides the full CoT experimental results for 11 models of varying scales, covering both open-source and proprietary models.

Expert Performance Analysis. To systematically evaluate expert performance across tasks, we present each expert’s scores on individual tasks in Table 8. Experimental results indicate that current agricultural experts typically possess deep expertise in specific crops or pest and disease categories, with their knowledge being relatively domain-focused rather than broadly distributed. As a result, when confronted with the diverse crop and pest categories included in our benchmark, their overall performance exhibits noticeable variability, highlighting limitations in cross-domain generalization.

Mixed reasoning poses a greater challenge than perceptual complexity for current models. To systematically evaluate deficiencies in different model capabilities, we introduce two diagnostic metrics using the Disease-P-M-K task as a case
<table>
  <tr>
    <th>Models</th>
    <th>Size</th>
    <th>Vision Encoder</th>
    <th>LLM</th>
    <th>Parameter Link</th>
  </tr>
  <tr>
    <td>LLaVA1.5-7B</td>
    <td>7.06B</td>
    <td>CLIP ViT-L/14</td>
    <td>Vicuna-v1.5-7B</td>
    <td>https://huggingface.co/llava-hf/llava-1.5-7b-hf</td>
  </tr>
  <tr>
    <td>LLaVA1.5-13B</td>
    <td>13.4B</td>
    <td>CLIP ViT-L/14</td>
    <td>Vicuna-v1.5-13B</td>
    <td>https://huggingface.co/llava-hf/llava-1.5-13b-hf</td>
  </tr>
  <tr>
    <td>LLaVA-1.6-7B</td>
    <td>7.57B</td>
    <td>CLIP ViT-L/14</td>
    <td>Mistral-7B</td>
    <td>https://huggingface.co/llava-hf/llava-v1-mistral-7b-hf</td>
  </tr>
  <tr>
    <td>LLaVA-1.6-13B</td>
    <td>13.98B</td>
    <td>CLIP ViT-L/14</td>
    <td>Vicuna-v1.5-13B</td>
    <td>https://huggingface.co/llava-hf/llava-v1-13b-hf</td>
  </tr>
  <tr>
    <td>LLaVA-1.6-34B</td>
    <td>34.8B</td>
    <td>CLIP ViT-L/14</td>
    <td>Nous-Hermes-Yi-34B</td>
    <td>https://huggingface.co/llava-hf/llava-1.6-34b-hf</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>8.03B</td>
    <td>SigLIP-SO400M</td>
    <td>Qwen-2-7B</td>
    <td>https://huggingface.co/llava-hf/llava-onevision-7b-ov-hf</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>73.2B</td>
    <td>SigLIP-SO400M</td>
    <td>Qwen-72B</td>
    <td>https://huggingface.co/llava-hf/llava-onevision-72b-ov-hf</td>
  </tr>
  <tr>
    <td>Phi-3.5-vision-instruct</td>
    <td>4.15B</td>
    <td>CLIP ViT-L/14</td>
    <td>Phi-3-Mini</td>
    <td>https://huggingface.co/microsoft/phi-3.5-vision-instruct</td>
  </tr>
  <tr>
    <td>Qwen-2-7B-chat</td>
    <td>7.4B</td>
    <td>ViT-ResG</td>
    <td>Qwen-2-7B</td>
    <td>https://huggingface.co/ques/Qwen-2-7B-chat</td>
  </tr>
  <tr>
    <td>Qwen-2-VL-7B-Instruct</td>
    <td>7.34B</td>
    <td>ViT-ResG</td>
    <td>Qwen-2-7B</td>
    <td>https://huggingface.co/openemb/Qwen2-VL-7B-Instruct</td>
  </tr>
  <tr>
    <td>MiniCPM-Llama3-V2.5</td>
    <td>8.54B</td>
    <td>SigLip-400M</td>
    <td>Llama3-8B-Instruct</td>
    <td>https://huggingface.co/openemb/MiniCPM-Llama3-V2.5</td>
  </tr>
  <tr>
    <td>MiniCPM-V2.6</td>
    <td>8.1B</td>
    <td>SigLip-400M</td>
    <td>Qwen-2-7B</td>
    <td>https://huggingface.co/openemb/MiniCPM-V2.6</td>
  </tr>
  <tr>
    <td>MiniCPM-o-2.6</td>
    <td>8.67B</td>
    <td>SigLip-400M et al.</td>
    <td>Qwen-2-5.7B</td>
    <td>https://huggingface.co/openemb/MiniCPM-o-2.6</td>
  </tr>
  <tr>
    <td>CogVLM-19B</td>
    <td>19.5B</td>
    <td>EVA CLIP-E</td>
    <td>LLaMA-8B</td>
    <td>https://huggingface.co/TIMI/cogvlm-19b</td>
  </tr>
  <tr>
    <td>GLMA-48B</td>
    <td>48.3B</td>
    <td>EVMA-CLIP-E</td>
    <td>GLMA-48B</td>
    <td>https://huggingface.co/TIMI/GLMA-48B</td>
  </tr>
  <tr>
    <td>InternLM-2.1B</td>
    <td>938M</td>
    <td>InternVIT-300M-448px</td>
    <td>Qwen-0.5B-Instruct</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.1B</td>
  </tr>
  <tr>
    <td>InternLM-2.2B</td>
    <td>2.21B</td>
    <td>InternVIT-300M-448px</td>
    <td>internLM-2.1B-Chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.2B</td>
  </tr>
  <tr>
    <td>InternLM-2.4B</td>
    <td>4.15B</td>
    <td>InternVIT-300M-448px</td>
    <td>Phi-3-mini-128k-instruct</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.4B</td>
  </tr>
  <tr>
    <td>InternLM-2.8B</td>
    <td>8.08B</td>
    <td>InternVIT-300M-448px</td>
    <td>internLM-2.7B-chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.8B</td>
  </tr>
  <tr>
    <td>InternLM-2.8B</td>
    <td>8.08B</td>
    <td>InternVIT-300M-448px-V2.5</td>
    <td>internLM-2.8B-chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.8B</td>
  </tr>
  <tr>
    <td>InternLM-2.5-1B</td>
    <td>2.38B</td>
    <td>InternVIT-300M-448px-V2.5</td>
    <td>Qwen-2.5-5.8B-Instruct</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.5-1B</td>
  </tr>
  <tr>
    <td>InternLM-2.5-2B</td>
    <td>2.21B</td>
    <td>InternVIT-300M-448px-V2.5</td>
    <td>internLM-2.5-1B-chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.5-2B</td>
  </tr>
  <tr>
    <td>InternLM-2.5-4B</td>
    <td>3.71B</td>
    <td>InternVIT-300M-448px-V2.5</td>
    <td>Qwen-2.5-3.8B-Instruct</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.5-4B</td>
  </tr>
  <tr>
    <td>InternLM-2.5-8B</td>
    <td>8.08B</td>
    <td>InternVIT-300M-448px-V2.5</td>
    <td>internLM-2.5-7B-chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.5-8B</td>
  </tr>
  <tr>
    <td>InternLM-2.6B</td>
    <td>25.5B</td>
    <td>InternVIT-48B-448px-V2.5</td>
    <td>internLM-2.6B-chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.6B</td>
  </tr>
  <tr>
    <td>InternLM-2.8B</td>
    <td>8.08B</td>
    <td>InternVIT-48B-448px-V2.5</td>
    <td>internLM-2.8B-chat</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.8B</td>
  </tr>
  <tr>
    <td>InternLM-2.7B</td>
    <td>7.84B</td>
    <td>InternVIT-48B-448px-V2.5</td>
    <td>Qwen-2.5-7.8B-Instruct</td>
    <td>https://huggingface.co/OpenVLab/InternLM-2.7B</td>
  </tr>
  <tr>
    <td>Yi-VL-6B</td>
    <td>6.6B</td>
    <td>CLIP ViT-H/14</td>
    <td>Yi-6B-Chat</td>
    <td>https://huggingface.co/01-ai/Yi-VL-6B</td>
  </tr>
  <tr>
    <td>Yi-VL-34B</td>
    <td>34B</td>
    <td>CLIP ViT-H/14</td>
    <td>Yi-34B-Chat</td>
    <td>https://huggingface.co/01-ai/Yi-VL-34B</td>
  </tr>
  <tr>
    <td>Deepseek-v12-tiny</td>
    <td>1.08B</td>
    <td>SigLIP-SO400M</td>
    <td>DeepSeek-MoE</td>
    <td>https://huggingface.co/deepseek-ai/deepseek-v12-tiny</td>
  </tr>
  <tr>
    <td>Deepseek-v12-small</td>
    <td>2.83B</td>
    <td>SigLIP-SO400M</td>
    <td>DeepSeek-MoE</td>
    <td>https://huggingface.co/deepseek-ai/deepseek-v12-small</td>
  </tr>
  <tr>
    <td>Deepseek-v12</td>
    <td>4.05B</td>
    <td>SigLIP-SO400M</td>
    <td>DeepSeek-MoE</td>
    <td>https://huggingface.co/deepseek-ai/deepseek-v12</td>
  </tr>
  <tr>
    <td>Molmo-7B</td>
    <td>8.02B</td>
    <td>CLIP ViT-L/14</td>
    <td>Qwen-2-7B</td>
    <td>https://huggingface.co/allenai/Molmo-7B-9224</td>
  </tr>
  <tr>
    <td>Molmo-72B</td>
    <td>73.3B</td>
    <td>CLIP ViT-L/14</td>
    <td>Qwen-72B</td>
    <td>https://huggingface.co/allenai/Molmo-72B-9224</td>
  </tr>
  <tr>
    <td>Cambrian-8B</td>
    <td>8.33B</td>
    <td>SVA</td>
    <td>Llama3-8B-Instruct</td>
    <td>https://huggingface.co/nyu-vision/cambrian-8b</td>
  </tr>
  <tr>
    <td>Cambrian-13B</td>
    <td>13.4B</td>
    <td>SVA</td>
    <td>Vicuna-v1.5-13B</td>
    <td>https://huggingface.co/nyu-vision/cambrian-13b</td>
  </tr>
  <tr>
    <td>Cambrian-34B</td>
    <td>34.8B</td>
    <td>SVA</td>
    <td>Hermes-11-34B</td>
    <td>https://huggingface.co/nyu-vision/cambrian-34b</td>
  </tr>
  <tr>
    <td>GPT-4-0-mini</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Gemini-1.5-flash</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Gemini-1.5-pro</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Yi-Vision</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Qwen-PLUS</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Qwen-VL-MAX</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td><b>GLMS-7B</b></td>
    <td>7.4B</td>
    <td>-</td>
    <td>-</td>
    <td>https://huggingface.co/TIMI/GLMS-7B</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>7.62B</td>
    <td>-</td>
    <td>-</td>
    <td>https://huggingface.co/qwen/Qwen2.5-7B-Instruct</td>
  </tr>
  <tr>
    <td>InternLM-2.5-7B</td>
    <td>7.74B</td>
    <td>-</td>
    <td>-</td>
    <td>https://huggingface.co/internlm/internlm2.5-7b-chat</td>
  </tr>
  <tr>
    <td>GPT-3.5-Turbo</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>GPT-4-Turbo</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
</table>

Table 7: The model architecture of 45 MLLMs and 5 LLMs evaluated on Agri-CM³.

Prompt B.1: Zero-Shot Normal

以下是一道农学单项选择题。不需要做任何分析和解释，直接输出答案。

问题：{question}
选项：{options_str}
答案：

Prompt B.2: Zero-Shot Normal (Translation)

The following is a single-choice question in Agronomy. No analysis or explanation is required, just output the answer.

Question: {question}
Options: {options_str}
Answer:

study:

• Perception Drop: The percentage of samples where the model correctly answers both Disease-K and Disease-M-K, but fails on Disease-P-M-K. This reflects performance degradation due to increased perceptual complexity.

• Mixed Reasoning Drop: The percentage of samples where the model performs well on Disease-K but fails on Disease-M-K, capturing the impact of added reasoning complexity.

We compute these metrics across a range of representative open-source and proprietary models. As shown in Table 9, Mixed Reasoning Drop consistently exceeds Perception Drop, highlighting mixed reasoning as a more significant challenge for current models.
<table>
  <tr>
    <th rowspan="2">Expert</th>
    <th rowspan="2">Validation</th>
    <th colspan="4">Disease</th>
    <th colspan="3">Pest</th>
  </tr>
  <tr>
    <th>P</th>
    <th>M</th>
    <th>P-M</th>
    <th>K</th>
    <th>M-K</th>
    <th>P-M-K</th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <td>1</td>
    <td></td>
    <td>80.00</td>
    <td>100.00</td>
    <td>85.00</td>
    <td>83.00</td>
    <td>66.00</td>
    <td>71.00</td>
    <td>69.00</td>
    <td>95.00</td>
    <td>76.00</td>
    <td>75.00</td>
  </tr>
  <tr>
    <td>2</td>
    <td></td>
    <td>79.11</td>
    <td>100.00</td>
    <td>73.00</td>
    <td>77.00</td>
    <td>73.00</td>
    <td>73.00</td>
    <td>71.00</td>
    <td>94.00</td>
    <td>76.00</td>
    <td>75.00</td>
  </tr>
  <tr>
    <td>Average</td>
    <td></td>
    <td>79.56</td>
    <td>100.00</td>
    <td>79.00</td>
    <td>80.00</td>
    <td>69.50</td>
    <td>72.00</td>
    <td>70.00</td>
    <td>94.50</td>
    <td>76.00</td>
    <td>75.00</td>
  </tr>
</table>

Table 8: Expert performance across agricultural tasks.

Prompt B.3: Zero-Shot CoT
以下是一道农学单项选择题，请请一步一步地思考，并在最后给出答案。
问题: {question}
选项: {options_str}
答案:

Prompt B.4: Zero-Shot CoT (Translation)
The following is a single-choice question in Agronomy. Please think step by step and provide the answer at the end.
Question: {question}
Options: {options_str}
Answer:

Prompt B.5: Few-Shot Normal
以下是一道农学单项选择题，不需要做任何分析和解释，直接输出答案。
以下是一些示例:
{examples}
现在，请根据以下题目输出答案:
问题: {question}
选项: {options_str}
答案:

Prompt B.6: Few-Shot Normal (Translation)
The following is a single-choice question in Agronomy. No analysis or explanation is required, just output the answer.
Here are some examples:
{examples}
Now, please output the answer based on the following question:
Question: {question}
Options: {options_str}
Answer:

D Case Study

D.1 GPT-4o CoT Case Study
Through analyzing the CoT responses of GPT-4o and InternVL-2.5-78B, we observed that GPT-4o effectively follows the multi-level reasoning process required by the benchmark, as shown in Figure 16. Specifically, GPT-4o diagnoses the disease, analyzes key features, and performs knowledge integration and application, ultimately selecting the correct answer through option analysis. In contrast, InternVL-2.5-78B lacks this structured reasoning process. These findings demonstrate that GPT-4o’s structured multi-step reasoning significantly enhances CoT performance. This highlights the importance of multi-level structured reasoning in improving model performance and provides valuable insights for future model optimization.

D.2 Error Analysis and Case Study
We randomly selected 200 erroneous predictions from GPT-4o’s CoT responses in the most challenging disease and pest reasoning tasks. These errors were annotated by two agricultural experts and categorized into five types:

• Question Misunderstanding: Errors caused by incorrect comprehension of the question.

• Perceptual Error: Errors in interpreting visual information from images.

• Lack of Knowledge: Errors due to incorrect or insufficient application of domain-specific knowledge.

• Irrelevant Response: Model responses unrelated to the question.

• Refuse to Answer: The model refuses to respond due to privacy or safety concerns.

Through the analysis of error distributions, we identified the following characteristics: In disease reasoning tasks, which involve a relatively small number of crop types (only 11), perceptual tasks are relatively simple, and most errors are attributed to a lack of domain knowledge. This indicates that
<table>
  <tr>
    <th>Model</th>
    <th>PD</th>
    <th>MRD</th>
  </tr>
  <tr>
    <td>internvl2.5-8B</td>
    <td>3.50</td>
    <td>14.33</td>
  </tr>
  <tr>
    <td>Internvl2.5-78B</td>
    <td>2.76</td>
    <td>15.71</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>4.43</td>
    <td>14.27</td>
  </tr>
  <tr>
    <td>Qwen2.5-VL-7B</td>
    <td>3.17</td>
    <td>13.33</td>
  </tr>
  <tr>
    <td>Qwen2.5-VL-72B</td>
    <td>4.54</td>
    <td>16.2</td>
  </tr>
  <tr>
    <td>Qwen-VL-MAX</td>
    <td>4.99</td>
    <td>18.05</td>
  </tr>
  <tr>
    <td>Gemini-1.5-flash</td>
    <td>3.72</td>
    <td>15.71</td>
  </tr>
  <tr>
    <td>Gemini-1.5-pro</td>
    <td>4.67</td>
    <td>17.09</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>6.66</td>
    <td>21.99</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>4.21</th>
    <th>15.96</th>
  </tr>
</table>

Table 9: Comparative analysis of Perception Drop (PD) and Mixed Reasoning Drop (MRD) across models.

![Bar chart comparing accuracy (%) of different MLLMs and their corresponding LLMs on Pest-K (text modality)](page_320_670_480_320.png)

Figure 12: Performance of different MLLMs and their corresponding LLMs on Pest-K (text modality).

models require further enhancement in domain-specific knowledge reasoning to handle disease-related tasks effectively. In contrast, pest reasoning tasks, with 107 pest types, exhibit significantly higher perceptual complexity, making perceptual errors the primary cause of failures. Analysis reveals that the perceptual errors in pest reasoning tasks typically arise from the model’s failure to accurately identify pests, which stems from a lack of domain-specific knowledge.

Future efforts should strengthen models’ ability to combine visual perception with domain-specific knowledge to improve performance in complex reasoning tasks. Additionally, expanding datasets to cover a broader range of pest and disease types and with improving multi-modal fusion techniques will be crucial for reducing errors and enhancing reasoning accuracy. Detailed case analyses for the five error types are provided in Figures 18, 19, 20, 21, 22, and 23.
Figure 13: Label distribution across the global, disease, and pest levels.

<table>
  <tr>
    <th>Disease-P-M-K</th>
    <th>Disease-P</th>
    <th>Pest-P-K</th>
    <th>Pest-K</th>
  </tr>
  <tr>
    <td>Question: 图片中叶片上的黄色孢子堆通常会有怎样的排列特征？<br>Options: A. 随机 B. 零星 C. 成行 D. 集中<br>Answer: C</td>
    <td>Question: 图片中出现的是哪种作物？<br>Options: A. 大豆 B. 玉米 C. 水稻 D. 小麦<br>Answer: D</td>
    <td>Question: 哪个时期最可能促使图片中害虫对作物的严重危害？<br>Options: A. 气温较高时 B. 干旱季节 C. 霜冻期结束时 D. 湿润初春<br>Answer: A</td>
    <td>Question: 哪个时期最可能促使麦长管蚜对作物的严重危害？<br>Options: A. 气温较高时 B. 干旱季节 C. 霜冻期结束时 D. 湿润初春<br>Answer: A</td>
  </tr>
  <tr>
    <td>Question: 图片中小麦叶片上的黄色孢子堆通常会有哪些排列特征？<br>Options: A. 随机 B. 零星 C. 成行 D. 集中<br>Answer: C</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Question: 小麦条锈病中的黄色孢子堆通常具有怎样的排列特征？<br>Options: A. 随机 B. 零星 C. 成行 D. 集中<br>Answer: C</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Question: 这片叶子可能感染了哪种疾病？<br>Options: A. 小麦条锈病 B. 水稻稻瘟病 C. 小麦白粉病 D. 玉米黑粉病<br>Answer: A</td>
    <td>Question: 这张图中的害虫是什么？<br>Options: A. 苹果蠹蛾 B. 草地贪夜蛾 C. 麦长管蚜 D. 小菜蛾<br>Answer: C</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Question: 这片小麦叶子可能感染了哪种疾病？<br>Options: A. 小麦条锈病 B. 水稻稻瘟病 C. 小麦白粉病 D. 玉米黑粉病<br>Answer: A</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Figure 14: Data cases in the Agri-CM^3 benchmark. The abbreviation in Agri-CM^3 means P: Perception, M: Mixture of Perception and Cognition, and K: Knowledge Application.

(a) Word Cloud of All Questions

(b) Word Cloud of All Options

Figure 15: The word frequency information for both questions and options is separately calculated and presented in the form of word clouds.
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">Validation Total</th>
    <th colspan="6">Test Total</th>
    <th colspan="2">Disease<sup>1</sup></th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <th>Total</th>
    <th>Test</th>
    <th>Total</th>
    <th>P</th>
    <th>M</th>
    <th>P-M</th>
    <th>K</th>
    <th>M-K</th>
    <th>P+M-K</th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <td>Random Choice</td>
    <td>23.22</td>
    <td>-</td>
    <td>25.14</td>
    <td>25.44</td>
    <td>26.04</td>
    <td>25.44</td>
    <td>24.15</td>
    <td>24.31</td>
    <td>26.61</td>
    <td>25.04</td>
    <td>24.94</td>
    <td>24.89</td>
  </tr>
  <tr>
    <td>Expert Performance</td>
    <td>-</td>
    <td>-</td>
    <td>79.56</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
</table>

Multi-Modal Large Language Model(MLLMs): Text + Image as Input

<table>
  <tr>
    <th>Model</th>
    <th colspan="100">BioChem/Somatotropin/Code</th>
    <th colspan="100">Pest</th>
  </tr>
  <tr>
    <th></th>
    <th colspan="21">Text + Image as Input</th>
    <th colspan="21">(Text only outperforms Image-only)</th>
  </tr>
  <tr>
    <td>LFLASH-GLM (GLM et al., 2024)</td>
    <td>35.32</td>
    <td>6.43</td>
    <td>34.66</td>
    <td>38.55</td>
    <td>27.98</td>
    <td>45.26</td>
    <td>40.03</td>
    <td>38.72</td>
    <td>21.56</td>
    <td>41.67</td>
    <td>36.91</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Jviance(z白 nh, 2024)</td>
    <td>40.67</td>
    <td>35.37</td>
    <td>36.43</td>
    <td>44.04</td>
    <td>19.78</td>
    <td>48.21</td>
    <td>40.89</td>
    <td>37.48</td>
    <td>20.74</td>
    <td>42.04</td>
    <td>36.69</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>InternLM.2.5-BH (Cat et al., 2024)</td>
    <td>40.44</td>
    <td>36.55</td>
    <td>44.55</td>
    <td>39.73</td>
    <td>32.12</td>
    <td>46.33</td>
    <td>39.72</td>
    <td>38.34</td>
    <td>22.38</td>
    <td>42.12</td>
    <td>35.48</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>GPT-3.5-TurBO (OpenAI, 2023)</td>
    <td>39.11</td>
    <td>37.01</td>
    <td>32.04</td>
    <td>44.72</td>
    <td>32.29</td>
    <td>45.80</td>
    <td>43.33</td>
    <td>39.04</td>
    <td>23.28</td>
    <td>41.55</td>
    <td>39.11</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>GPT -4-TurBO (Achim et at., 2023)</td>
    <td>47.11</td>
    <td>41.84</td>
    <td>33.39</td>
    <td>43.20</td>
    <td>35.08</td>
    <td>57.07</td>
    <td>52.83</td>
    <td>46.27</td>
    <td>23.08</td>
    <td>50.36</td>
    <td>45.21</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 10: The zero-shot result of 45 MLLMs and 5 LLMs in Agri-CM^3. The abbreviation in Agri-CM^3 means P: Perception, M: Mixture of Perception and Cognition Reasoning, K: Knowledge Application. Test total represents the average of all test instances.
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th rowspan="2">CoT</th>
    <th colspan="2">Validation</th>
    <th colspan="2">Test</th>
    <th colspan="6">Disease</th>
    <th colspan="3">Pest</th>
  </tr>
  <tr>
    <th>Total</th>
    <th>Total</th>
    <th>P</th>
    <th>M</th>
    <th>P-M</th>
    <th>K</th>
    <th>M-K</th>
    <th>P-M-K</th>
    <th>P</th>
    <th>K</th>
    <th>P-K</th>
  </tr>
  <tr>
    <td>InternVL-2.5-1B</td>
    <td></td>
    <td>44.33</td>
    <td>40.71</td>
    <td>50.55</td>
    <td>48.01</td>
    <td>39.39</td>
    <td>35.78</td>
    <td>38.78</td>
    <td>36.70</td>
    <td>44.41</td>
    <td>35.64</td>
    <td>39.85</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>39.33<sup>±0.8</sup></td>
    <td>35.01<sup>±1.1</sup></td>
    <td>39.05<sup>±1.5</sup></td>
    <td>41.50<sup>±1.8</sup></td>
    <td>31.87<sup>±2.3</sup></td>
    <td>34.97<sup>±2.8</sup></td>
    <td>35.01<sup>±3.8</sup></td>
    <td>32.89<sup>±3.8</sup></td>
    <td>36.62<sup>±3.8</sup></td>
    <td>31.54<sup>±4.1</sup></td>
    <td>34.49<sup>±5.4</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-2B</td>
    <td></td>
    <td>50.56</td>
    <td>46.14</td>
    <td>55.79</td>
    <td>59.34</td>
    <td>52.41</td>
    <td>46.24</td>
    <td>42.54</td>
    <td>39.27</td>
    <td>48.63</td>
    <td>42.04</td>
    <td>39.81</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>46.00<sup>±4.6</sup></td>
    <td>41.14<sup>±8.0</sup></td>
    <td>40.41<sup>±14.4</sup></td>
    <td>53.09<sup>±4.2</sup></td>
    <td>44.46<sup>±9.0</sup></td>
    <td>40.07<sup>±6.2</sup></td>
    <td>40.58<sup>±8.0</sup></td>
    <td>38.55<sup>±8.4</sup></td>
    <td>44.48<sup>±4.2</sup></td>
    <td>37.69<sup>±4.4</sup></td>
    <td>38.05<sup>±1.8</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-4B</td>
    <td></td>
    <td>55.33</td>
    <td>51.76</td>
    <td>64.92</td>
    <td>58.24</td>
    <td>51.23</td>
    <td>55.19</td>
    <td>52.04</td>
    <td>49.38</td>
    <td>54.03</td>
    <td>46.30</td>
    <td>45.93</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>52.78<sup>±2.5</sup></td>
    <td>48.76<sup>±3.8</sup></td>
    <td>57.31<sup>±7.4</sup></td>
    <td>53.34<sup>±4.9</sup></td>
    <td>44.38<sup>±6.8</sup></td>
    <td>53.67<sup>±1.5</sup></td>
    <td>49.84<sup>±2.2</sup></td>
    <td>45.26<sup>±4.1</sup></td>
    <td>49.73<sup>±4.3</sup></td>
    <td>46.54<sup>±6.2</sup></td>
    <td>45.20<sup>±6.7</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-8B</td>
    <td></td>
    <td>59.67</td>
    <td>55.73</td>
    <td>70.58</td>
    <td>62.13</td>
    <td>57.06</td>
    <td>55.37</td>
    <td>52.43</td>
    <td>50.54</td>
    <td>64.16</td>
    <td>49.20</td>
    <td>48.13</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>58.22<sup>±1.5</sup></td>
    <td>55.30<sup>±8.4</sup></td>
    <td>69.32<sup>±1.3</sup></td>
    <td>61.62<sup>±8.5</sup></td>
    <td>59.59<sup>±2.5</sup></td>
    <td>50.89<sup>±4.5</sup></td>
    <td>52.83<sup>±8.4</sup></td>
    <td>51.09<sup>±6.6</sup></td>
    <td>64.71<sup>±7.5</sup></td>
    <td>46.78<sup>±2.4</sup></td>
    <td>48.53<sup>±8.4</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-26B</td>
    <td></td>
    <td>64.78</td>
    <td>61.97</td>
    <td>74.30</td>
    <td>71.09</td>
    <td>66.61</td>
    <td>63.69</td>
    <td>57.06</td>
    <td>56.07</td>
    <td>72.57</td>
    <td>54.91</td>
    <td>51.50</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>62.78<sup>±2.0</sup></td>
    <td>61.97<sup>±8.0</sup></td>
    <td>74.30<sup>±0.0</sup></td>
    <td>73.37<sup>±2.3</sup></td>
    <td>68.30<sup>±1.7</sup></td>
    <td>56.89<sup>±8.8</sup></td>
    <td>55.10<sup>±2.0</sup></td>
    <td>52.41<sup>±3.7</sup></td>
    <td>71.67<sup>±0.9</sup></td>
    <td>50.52<sup>±4.4</sup></td>
    <td>49.85<sup>±1.6</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-38B</td>
    <td></td>
    <td>68.11</td>
    <td>65.51</td>
    <td>74.81</td>
    <td>76.67</td>
    <td>70.16</td>
    <td>65.65</td>
    <td>61.30</td>
    <td>60.26</td>
    <td>75.04</td>
    <td>56.84</td>
    <td>56.89</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>64.56<sup>±3.5</sup></td>
    <td>59.89<sup>±5.4</sup></td>
    <td>73.29<sup>±1.5</sup></td>
    <td>72.02<sup>±4.7</sup></td>
    <td>64.07<sup>±6.1</sup></td>
    <td>58.05<sup>±7.6</sup></td>
    <td>55.97<sup>±5.3</sup></td>
    <td>53.34<sup>±6.9</sup></td>
    <td>70.15<sup>±4.9</sup></td>
    <td>52.86<sup>±4.4</sup></td>
    <td>49.45<sup>±7.4</sup></td>
  </tr>
  <tr>
    <td>InternVL-2.5-78B</td>
    <td></td>
    <td>69.89</td>
    <td>65.49</td>
    <td>79.54</td>
    <td>76.67</td>
    <td>72.78</td>
    <td>65.56</td>
    <td>60.44</td>
    <td>59.33</td>
    <td>73.04</td>
    <td>58.00</td>
    <td>56.38</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>67.56<sup>±3.1</sup></td>
    <td>61.71<sup>±3.8</sup></td>
    <td>79.37<sup>±0.2</sup></td>
    <td>74.22<sup>±5.5</sup></td>
    <td>69.23<sup>±3.5</sup></td>
    <td>61.81<sup>±8.8</sup></td>
    <td>55.42<sup>±5.0</sup></td>
    <td>55.52<sup>±3.8</sup></td>
    <td>69.87<sup>±3.2</sup></td>
    <td>53.86<sup>±4.1</sup></td>
    <td>50.70<sup>±5.7</sup></td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td></td>
    <td>60.00</td>
    <td>56.32</td>
    <td>85.88</td>
    <td>58.92</td>
    <td>57.82</td>
    <td>54.20</td>
    <td>47.88</td>
    <td>46.27</td>
    <td>70.34</td>
    <td>47.43</td>
    <td>46.26</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>60.00<sup>±0.0</sup></td>
    <td>55.87<sup>±0.0</sup></td>
    <td>79.71<sup>±6.2</sup></td>
    <td>62.55<sup>±3.4</sup></td>
    <td>61.12<sup>±3.3</sup></td>
    <td>51.61<sup>±8.6</sup></td>
    <td>48.04<sup>±9.2</sup></td>
    <td>47.59<sup>±1.3</sup></td>
    <td>69.41<sup>±0.8</sup></td>
    <td>45.90<sup>±1.5</sup></td>
    <td>46.08<sup>±8.2</sup></td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td></td>
    <td>72.00</td>
    <td>64.81</td>
    <td>80.98</td>
    <td>73.29</td>
    <td>72.36</td>
    <td>65.74</td>
    <td>61.77</td>
    <td>59.95</td>
    <td>72.07</td>
    <td>55.07</td>
    <td>56.27</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>67.44<sup>±6.6</sup></td>
    <td>62.19<sup>±1.6</sup></td>
    <td>79.46<sup>±1.5</sup></td>
    <td>71.17<sup>±2.1</sup></td>
    <td>66.33<sup>±5.6</sup></td>
    <td>65.21<sup>±5.5</sup></td>
    <td>55.57<sup>±4.2</sup></td>
    <td>55.44<sup>±4.2</sup></td>
    <td>70.42<sup>±6.6</sup></td>
    <td>54.83<sup>±8.2</sup></td>
    <td>52.97<sup>±8.3</sup></td>
  </tr>
  <tr>
    <td>GPT-4o-mini</td>
    <td></td>
    <td>63.00</td>
    <td>58.21</td>
    <td>79.88</td>
    <td>69.99</td>
    <td>67.12</td>
    <td>53.49</td>
    <td>52.28</td>
    <td>48.37</td>
    <td>68.78</td>
    <td>48.87</td>
    <td>47.80</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>61.78<sup>±3.1</sup></td>
    <td>56.14<sup>±1.1</sup></td>
    <td>82.25<sup>±2.4</sup></td>
    <td>68.30<sup>±1.7</sup></td>
    <td>66.78<sup>±8.3</sup></td>
    <td>52.06<sup>±1.4</sup></td>
    <td>48.12<sup>±1.1</sup></td>
    <td>46.03<sup>±3.5</sup></td>
    <td>64.67<sup>±1.1</sup></td>
    <td>46.66<sup>±1.3</sup></td>
    <td>45.75<sup>±1.0</sup></td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td></td>
    <td>67.33</td>
    <td>63.64</td>
    <td>83.85</td>
    <td>69.48</td>
    <td>68.30</td>
    <td>65.65</td>
    <td>54.71</td>
    <td>49.53</td>
    <td>80.09</td>
    <td>56.84</td>
    <td>51.10</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>70.56<sup>±3.2</sup></td>
    <td>65.43<sup>±1.8</sup></td>
    <td>87.91<sup>±4.1</sup></td>
    <td>78.95<sup>±9.5</sup></td>
    <td>75.23<sup>±4.9</sup></td>
    <td>61.45<sup>±8.2</sup></td>
    <td>58.48<sup>±3.8</sup></td>
    <td>54.04<sup>±4.5</sup></td>
    <td>80.13<sup>±0.8</sup></td>
    <td>53.94<sup>±9.9</sup></td>
    <td>52.49<sup>±1.4</sup></td>
  </tr>
</table>

Table 11: All experimental results for performance comparison of different mlms with and without CoT.

Disease-P-M-K: Wheat Stem Rust

Question: 图片中的作物病害萌发所需的湿度是多少?
Options: A: 40%   B: 55%   C: 75%   D: 100%
Answer: D
Background Knowledge: 小麦秆锈病菌对湿度要求较高，需要水滴或100%湿度才能萌发。

InternVL-2.5-78B CoT Response:
要确定图片中的作物病害萌发所需的湿度，我们需要分析图片中的植物病害特征。图片中可以看到植物叶片上有明显的锈斑，这通常是锈病的症状。锈病是由真菌引起的，常见于高湿度环境中。
锈病的病原菌通常在相对湿度较高时萌发和传播。根据植物病理学的知识，锈病的病原菌在相对湿度达到75%以上时，会更有利于其孢子的萌发和侵染。
因此，结合图片中的病害特征和锈病的湿度需求，答案是：C. 75%

GPT-4o CoT Response:
从图片中可以看出，该作物的病害为一种锈病。锈病通常由真菌引起，而这些病原体在萌发和传播时需要一定的湿度。
分析过程：
1. 锈病的特点：锈病是由真菌引起的，病菌孢子在萌发和侵入宿主植物时，需要高湿度的环境。
2. 真菌孢子萌发的湿度需求：一般来说，锈病病菌孢子的萌发通常需要空气相对湿度达到95%-100%的水平。
因此，这样的病害通常在潮湿、雨雾多的环境下容易爆发。
3. 选项分析：
   • A: 40%：湿度较低，不可能支持真菌孢子的萌发。
   • B: 55%：仍然偏低，也不适合锈病病菌的萌发。
   • C: 75%：虽然湿度较高，但对于锈病来说通常不足以确保孢子萌发。
   • D: 100%：这是典型的适合锈病病菌萌发的湿度范围。
答案：D. 100%

Figure 16: Comparison of CoT responses between GPT-4o and InternVL-2.5-78B. GPT-4o demonstrates superior performance due to its reasoning logic, which better aligns with the benchmark’s evaluation criteria. Red bold highlights the wrong content.
(a) Overall error distribution for 200 annotated GPT-4o errors.

(b) Error distribution for Disease-P-M-K annotated GPT-4o errors.

(c) Error distribution for Pest-P-K annotated GPT-4o errors.

Figure 17: Error distribution on different complex reasoning tasks.

<table>
  <tr>
    <th colspan="5">Disease-P-M-K: Lack of Knowledge</th>
  </tr>
  <tr>
    <td>Disease:</td>
    <td>苹果黑星病</td>
  </tr>
  <tr>
    <td>Question:</td>
    <td>这种疾病在什么温度下生长最为理想?</td>
  </tr>
  <tr>
    <td>Options:</td>
    <td>A: 30°C  B: 10°C  C: 20°C  D: 25°C</td>
  </tr>
  <tr>
    <td>Answer:</td>
    <td>C</td>
  </tr>
  <tr>
    <td>GPT-4o Response:</td>
    <td>从图片看，这种疾病显然是由真菌病害引起的，<b>看起来很像霉菌或白粉病。</b>真菌病害通常在温暖潮湿的环境中生长最为理想。真菌普遍在20°C到25°C的温度下生长得很好。30°C可能会太高，抑制一些真菌的生长，10°C则可能太低而不利于活跃生长。因此，答案可能是在20°C或25°C。<b>在选择20°C或25°C之间时，考虑到许多霉菌类真菌在较温暖的环境中活跃，因此25°C可能更为合理。</b><br>答案：D. 25°C</td>
  </tr>
  <tr>
    <td>Error Analysis:</td>
    <td>模型缺乏专业知识答错题目，如果准确诊断为<b>苹果黑星病</b>，便可以根据专业知识确定<b>苹果黑星病病原菌最适生长温度为20°C。</b></td>
  </tr>
</table>

Figure 18: A lack of knowledge example in Disease-P-M-K. <b>Red bold</b> highlights the wrong content.
Pest-P-K: Lack of Knowledge

Pest: 叶蝉科
Question: 图片中展示的害虫通常活跃在哪种环境区域？
Options: A: 亮色区域  B: 阴影处  C: 水面  D: 干燥处
Answer: A

GPT-4o Response:
这是一只叶蝉科的害虫，通常以吸食植物汁液为生，并且其颜色鲜艳显眼，如红色和蓝色，这表明它可能有捕食者防御的特性。以下是对问题的逐步分析：

第一步：害虫特征分析
叶蝉类害虫通常生活在植物表面，尤其是叶片表面，这与图片展示的虫子形态和它站在叶片上的姿态是吻合的。

第二步：与环境的关系
叶蝉类害虫一般会选择适宜其生存和繁殖的环境区域，具体特点如下：
- 亮色区域 (A): 叶蝉可能会暴露于阳光下，但这并不是其最偏好的环境，因为阳光直射可能会导致过热。
- 阴影处 (B): 叶蝉多数活跃在植物叶片的背面或阴影较多的地方，以躲避阳光直射以及捕食者。
- 水面 (C): 叶蝉并不生活在水面，它们与植物密切相关。
- 干燥处 (D): 干燥的环境并不是叶蝉的主要活动场所，因为它们需要植物汁液以维持生存。

Error Analysis:
模型缺乏专业知识答错题目，根据相关的农学专业知识可知叶蝉科的成虫具有强趋光性，因此相对于阴影处，该害虫更喜欢在较亮的区域活动。

Figure 19: A lack of knowledge example in Pest-P-K. Red bold highlights the wrong content.

Disease-P-M-K: Perceptual Error

Disease: 茄子病毒病
Question: 这种病害在什么环境条件下更容易发生？
Options: A: 阴凉湿润  B: 短日照  C: 低温多湿  D: 高温干旱
Answer: D

GPT-4o Response:
根据图片中植物叶片上的白色粉状物质，这种病害很可能是白粉病。白粉病是由子囊菌目真菌引起的一种常见作物病害，通常表现为叶片上出现白色粉状霉层。
白粉病通常在以下环境条件下更容易发生：
- 高湿度：高湿度有利于真菌孢子的萌发和繁殖。
- 温度适中：白粉病在温暖的天气中更为常见。
根据这些信息，以及选项来看：
- A. 阴凉湿润：不完全符合，因为虽然湿润有利于病害发生，但阴凉的环境不太符合白粉病的高发条件。
- B. 短日照：与白粉病的发生没有直接关系。
- C. 低温多湿：低温环境通常不利于白粉病的发展。
- D. 高温干旱：干旱不利于白粉病，但略高的温度可能会促进白粉病在温暖湿润环境中的发展。
虽然题目提供的选项可能不完全匹配白粉病的典型环境条件，但在这些选项中，“阴凉湿润”相对于其他选项来说可能更能引发白粉病的发生，尽管它并不是最完美的描述。因此，选择：
A. 阴凉湿润

Error Analysis:
模型感知图片中病害类型发生错误，由图片中茄子叶片发生皱缩卷曲、黄化可以感知、推断可能感染为茄子病毒病，但是模型错误感知为白粉病，导致后续判断错误。

Figure 20: An example of perceptual error in Disease-P-M-K. Red bold highlights the wrong content.
Pest-P-K: Perceptual Error

Pest: 豆芫菁
Question: 图片中的昆虫若以幼虫形式度过冬天，会在哪里越冬呢？
Options: A: 土表越冬  B: 土中越冬  C: 植物越冬  D: 水越冬
Answer: B

GPT-4o Response:
这是一只黑肩绿天牛（Atanycolus cappaerti），通常称为“红头黑肩天牛”，成虫通体黝黑，头部呈明显红色。

在思考这道题目时，重点在于确定这种昆虫的幼虫习性。天牛科（Cerambycidae）的昆虫幼虫主要以木材或植物组织为食，这通常意味着它们会在木质植物中度过一部分或全部幼虫阶段。因此，这类昆虫的幼虫经常会在植物组织中（如枯死或衰弱的树干、树枝中）越冬。

所以，考虑到黑肩绿天牛属昆虫的一般生活习性，幼虫最有可能是在植物组织中度过冬天。

因此，图片中的昆虫若以幼虫形式度过冬天，会在哪里越冬呢？选择C. 植物越冬。

Error Analysis:
模型感知错误，没有正确判断出图中害虫的种类是豆芫菁，因此未能正确推断害虫的越冬场所（土中越冬）。

Figure 21: An example of perceptual error in Pest-P-K. Red bold highlights the wrong content.

Disease-P-M-K: Question Misunderstanding

Disease: 小麦叶锈病
Question: 为减缓图片中病害的扩展，采用哪种栽培防病策略最为合适？
Options: A: 喷施杀菌剂  B: 及时排水  C: 增施氮肥  D: 深耕灭茬
Answer: B

GPT-4o Response:
这张图片中显示的病害是小麦锈病，其特征是叶片上出现黄色至橙色的小斑点。这种病害是由真菌引起的，主要通过空气传播，在较潮湿的条件下容易扩散。
为了减缓这种病害的扩展，我们需要采取一种有效的栽培策略。让我们依次分析各个选项：
A. 喷施杀菌剂：这是应对真菌性病害的常见方法，通过直接杀灭病原体，可以有效控制病害的扩散。
B. 及时排水：虽然过多的潮湿可能促进锈病的发展，但仅仅排水通常不足以有效控制病害。
C. 增施氮肥：过分施用氮肥可能导致植株过于茂盛，透风性变差，反而可能加重病害。
D. 深耕灭茬：这是一种清除土壤表面作物残留的方法，可以减少病原菌的越冬体积，但对于已经发病的植株来说，短期内效果有限。
综上所述，对于已经出现的小麦锈病，喷施杀菌剂是最直接和有效的防治措施。因此，正确答案是：
A. 喷施杀菌剂

Error Analysis:
题目中的“栽培防病策略”是指农业防治措施，而模型给出的答案是“喷施杀菌剂”。尽管喷施杀菌剂也能有效防治病害，但它属于化学防治措施，与题目要求不符，反映出模型对问题的理解存在偏差。

Figure 22: A question misunderstanding example in Disease-P-M-K. Red bold highlights the wrong content.
Pest-P-K: Question Misunderstanding

<table>
  <tr>
    <th>Pest:</th>
    <td>蚜虫</td>
  </tr>
  <tr>
    <th>Question:</th>
    <td>图中所示害虫特征——腹管——首次出现在什么地质年代？</td>
  </tr>
  <tr>
    <th>Options:</th>
    <td>A:侏罗纪  B:二叠纪  C:三叠纪  D:白垩纪</td>
  </tr>
  <tr>
    <th>Answer:</th>
    <td>D</td>
  </tr>
</table>

GPT-4o Response:
这道题目要求我们确定图中所示的“腹管”这一害虫特征首次出现的地质年代。
- 图中展示的是蚜虫（Aphid），这些昆虫广为人知的特征之一就是腹部有突出的腹管。
- 根据化石记录，蚜虫的祖先被认为出现在二叠纪（Permian）时代后期，并在三叠纪（Triassic）时期得以发展。
- 尤其是蚜虫在三叠纪已经出现，它们的化石已在这段时期的地层中被发现。
因此，考虑到问题中问的是“首次出现在什么地质年代”，答案是C: 三叠纪。

Error Analysis:
模型的回复的问题与题目中描述的问题不一致：问题中提问的是蚜虫的“腹管”结构是什么时候出现的，而模型回复的是蚜虫出现的时期。

Figure 23: A question misunderstanding example in Pest-P-K. Red bold highlights the wrong content.

Reviewer paper summaries: The paper introduces a benchmark assessing MLLMs' understanding and reasoning in agricultural management, especially in pest and disease management. The authors decompose complex agricultural reasoning tasks into three sub-tasks and create the dataset for each task. They evaluate various MLLMs in the benchmark and provide several insights for future research.

This article constructs a Chinese multimodal dataset for evaluating the model's ability in the diagnoses and treatment of pests and diseases. Specifically, the article points out that existing benchmarks only verify the correctness of the results end-to-end, ignoring the perception, perception-cognition, and knowledge application stages in the multimodal reasoning process. Therefore, the authors have separately evaluated the model's three abilities by constructing test data at different levels. The method of constructing the dataset is reasonable and effective, and it has high value for other multimodal benchmarks in the community to learn from. The article also conducted a large number of experiments, presented the performance of current mainstream open-source and closed-source models on this benchmark, and provided specific analysis. The paper is well written and is a good paper, but it can still add some experiments to further improve the quality of the paper.
Summary of strengths: The main strengths of this paper are listed:

- The paper fills the pest and disease management gap in the agricultural field. The proposed benchmark facilitates a comprehensive assessment of MLLMs' reasoning skills in the research problem of interest.
- Clear presentation and nice-looking visualization of the paper.
- The paper experiments with lots of MLLMs and provides some interesting insights.

1. The article splits the end-to-end evaluation into three stages, which I think is very reasonable. This more detailed evaluation of model capabilities will help researchers in the community to adjust the model's training methods in a targeted manner, thereby further improving the overall performance of the model.
2. The generation process introduced in Figure 3 effectively meets the evaluation needs of the three abilities of MLLM mentioned in the motivation: using GPT-4o to rewrite public materials, construct data in a specified format, and ensure the quality of data in the benchmark through data cleaning and expert validation. This process is universal and should be able to be migrated to other MLLM benchmarks at a lower cost, thereby evaluating the abilities of models at different stages in other tasks (P-M-K).
3. Although I am not an expert in the agricultural field, from the data distribution in Figure 2, it can be seen that the benchmark should cover a large part of the problems in pests and diseases, and can serve as an effective benchmark in the agricultural field.
4. The authors conducted a large number of experiments, tested the performance of mainstream models under this benchmark, to demonstrate that this benchmark is still challenging at present.
5. The experimental conclusions of the article on CoT, In-context learning, and Multi-modal are detailed and credible, which should also be of reference significance for other related research (MLLM).
Summary of weaknesses: The main weaknesses of this paper are as follows, some of which have been discussed in Limitation:

- Concerns about the data quality and the question setting. Although I am not an agricultural expert, I am concerned about the sufficiency of the information provided in the constructed dataset. Pest and disease management is a complicated domain requiring a wide range of knowledge and even fieldwork experience. Maybe the given image cannot provide enough information to make decisions. Moreover, Table 1 reports expert performance, which is not very high.
- Also, why can the "Only Text as Input" models of Table 1 achieve above random performance? Although Section 4.4 points out that the dataset relies heavily on visual input, the models can still achieve above-random performance without any visual input. Further clarification is needed.
- Potential data bias from GPT-4o. The dataset might include potential bias since it is annotated by GPT-4o. I am not saying that the authors should not use GPT-4o for annotation, but a relevant statement is necessary.

1. The benchmark is only for Chinese data, which limits its scope of application. The authors may construct the corresponding English dataset by direct translation, which will effectively dispel the doubts of readers and users about the scope of application.
2. Lack of testing for some latest models, such as Qwen2.5-VL (significantly improved compared to Qwen2-VL) and Deepseek-R1 (pure text problem)
Comments, suggestions and typos: See "Summary Of Weaknesses"

1. The expert performance data in Table 1 is insufficient. The accuracy data of human experts at different levels will also effectively help to judge the ability of the model. Can you supplement the corresponding data?
2. Although models like Deepseek-R1 that require long reasoning currently do not have the ability to input images, they can also be used to solve Disease-K and Pest-K problems. Could you supplement the results of long reasoning models under these two problems?
3. Since each question in the benchmark should be able to find the corresponding level version, could the author count the scores of different models on different abilities as additional evaluation metrics? For example, the model can answer a certain question correctly in K and M-K, but it is wrong in P-M-K, which shows that the model's perception ability is defective in this question.
Overall assessment scores: 3.5

4