Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
Authors: Mengliang He, Jiayi Zeng, Yankai Jiang, Wei Zhang, Zeming Liu, Xiaoming Shi, Aimin Zhou
Accepted at: acl25

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability

Mengliang He1, Jiayi Zeng1, Yankai Jiang2, Wei Zhang1, Zeming Liu3*, Xiaoming Shi1*, Aimin Zhou1
1 East China Normal University, Shanghai, China 2 Shanghai AI Lab, Shanghai, China 3 Beihang University, Beijing, China
{51255901020, 51265901055}@stu.ecnu.edu.cn; zhangwei.thu2011@gmail.com; zmliu@buaa.edu.cn; {xmshi, amzhou}@cs.ecnu.edu.cn

Abstract
While large language models (LLMs) show promise in code generation, existing benchmarks neglect the flowchart-based code generation. To promote further research on flowchart-based code generation, this work presents Flow2Code, a novel benchmark for flowchart-based code generation evaluation. The evaluation dataset spans 15 programming languages and includes 5,622 code segments paired with 16,866 flowcharts of three types: code, UML, and pseudocode. Extensive experiments with 13 multimodal LLMs reveal that current LLMs can not generate code based on flowcharts perfectly. Besides, experiment results show that the supervised fine-tuning technique contributes greatly to the models’ performance. We publicly release our code and datasets at https://github.com/hml-github/Flow2Code.

1 Introduction
The code generation task aims to convert specific requirements into executable code (Nuseibeh and Easterbrook, 2000), which attracts interest and focus from the academic and industrial communities. Recently, for automatic code generation, large language models (LLMs) (OpenAI, 2023; Team, 2024; Nijkamp et al., 2023; DeepSeek-AI et al., 2024; Hui et al., 2024) exhibit substantial potential and show alluring application value for enhancing productivity, minimizing human error.

To comprehensively evaluate and understand the code generation capabilities of emerging LLMs, substantial efforts have been devoted to establishing and refining code generation benchmarks. Specifically, as shown in Figure 1, current code generation benchmarks can be classified into two categories: those based on textual descriptions, such as HumanEval (Chen et al., 2021) and MBPP (Austin et al., 2021), and those based on images of programming problems or matplotlib plots, such as MMCode (Li et al., 2024) and Plot2Code (Wu et al., 2024a). Despite the potential value, these works fundamentally suffer from a critical flaw: The lack of flowchart-based code generation evaluation.

Compared to textual descriptions, images of programming problems, or matplotlib plots, flowcharts offer a more effective and intuitive way to understand and visualize program logic such as decisions, loops, and conditionals, making them accessible to both programmers and non-programmers alike (Xinogalos, 2013). Flowcharts mainly consist of three types: basic code flowcharts, Unified Modeling Language (UML) flowcharts, and pseudocode flowcharts (Chapin, 2003). Specifically, basic code flowcharts represent the step-by-step execution of a program, UML flowcharts are a formal graphical representation of an object-oriented system’s structure, and pseudocode flowcharts are a high-level abstraction of program logic and are represented by natural Language. Despite these advantages, there is a notable gap in code generation benchmarks, with no dedicated datasets or frameworks for generating code from flowcharts. This limitation hinders the ability of LLMs to fully utilize flowcharts for code generation.

To address this gap, this work first introduces Flow2Code, a comprehensive code generation benchmark that includes three types of flowcharts and corresponding code in 15 programming languages. The construction of Flow2Code consists of three parts: the creation of code and UML flowcharts, pseudocode conversion, and data checking. After the construction of the dataset, a two-step human evaluation process is employed to ensure data quality, which includes both code verification and validation of pseudocode flowchart transformations. Finally, the dataset is obtained with a total of 5,622 code segments, 16,866 flowcharts and includes 15 programming languages, offering a rich resource for evaluating code generation tasks.

To further conduct the evaluation of current
<table>
  <tr>
    <th>Datasets</th>
    <th>Code Flowchart</th>
    <th>UML Flowchart</th>
    <th>Pseudocode Flowchart</th>
    <th>Multimodal</th>
    <th>Multilingual</th>
    <th>Samples</th>
  </tr>
  <tr>
    <td>APPS (Hendrycks et al., 2021)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>X</td>
    <td>10,000</td>
  </tr>
  <tr>
    <td>HumanEval (Chen et al., 2021)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>X</td>
    <td>164</td>
  </tr>
  <tr>
    <td>MBPP (Austin et al., 2021)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>X</td>
    <td>974</td>
  </tr>
  <tr>
    <td>DS-1000 (Lai et al., 2023)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>X</td>
    <td>1,000</td>
  </tr>
  <tr>
    <td>CodeContests (Li et al., 2022)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>✓ (3)</td>
    <td>13,610</td>
  </tr>
  <tr>
    <td>MBXP (Athiwaratkun et al., 2023)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>✓ (13)</td>
    <td>12,425</td>
  </tr>
  <tr>
    <td>ClassEval (Du et al., 2023)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>X</td>
    <td>100</td>
  </tr>
  <tr>
    <td>CoderEval (Yu et al., 2024)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>✓ (2)</td>
    <td>460</td>
  </tr>
  <tr>
    <td>HumanEval-X (Zheng et al., 2023)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>✓ (5)</td>
    <td>820</td>
  </tr>
  <tr>
    <td>MCEVAL (Chai et al., 2024)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>✓ (40)</td>
    <td>16,031</td>
  </tr>
  <tr>
    <td>BigCodeBench (Zhuo et al., 2024)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>X(text)</td>
    <td>X</td>
    <td>1,140</td>
  </tr>
  <tr>
    <td>Plot2Code (Wu et al., 2024a)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>✓ (image, text)</td>
    <td>X</td>
    <td>132</td>
  </tr>
  <tr>
    <td>MMCode (Li et al., 2024)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>✓ (image, text)</td>
    <td>X</td>
    <td>3,548</td>
  </tr>
  <tr>
    <td>HumanEval-V (Zhang et al., 2024)</td>
    <td>X</td>
    <td>X</td>
    <td>X</td>
    <td>✓ (image, text)</td>
    <td>X</td>
    <td>253</td>
  </tr>
  <tr>
    <td>Flow2Code</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓ (image, text)</td>
    <td>✓ (15)</td>
    <td>16,866</td>
  </tr>
</table>

Table 1: Comparison of the Flow2Code dataset with other code generation benchmarks, where the number in the Multilingual column represents the amount of programming code contained in the dataset.

LLMs, we conduct comprehensive benchmarking experiments on the Flow2Code dataset using 13 LLMs under the settings of zero-shot and supervised fine-tuning. The experimental results demonstrate that: (1) Current LLMs lack sufficient flowchart-based code generation capability, particularly on the UML and pseudocode flowcharts; (2) Supervised fine-tuning technique is effective in improving LLMs’ flowchart-based code generation. These findings highlight areas for improvement and provide guidance for future research in code generation.

This work makes the following contributions:

• We identify a key challenge for current LLMs as flowchart-based code generation.

• To promote further research on flowchart-based code generation, we introduce a novel benchmark, termed Flow2Code, and perform extensive evaluation experiments on various LLMs, providing a standardized platform for assessing flowchart-based code generation.

• Experimental results show that supervised fine-tuning on flowchart-based code generation datasets is an effective technique to improve LLMs’ performance.

2 Related work

This Section first reviews the current research on code generation benchmarks (Section 2.1), followed by an overview of research on multimodal benchmarks (Section 2.2).

2.1 Code Generation Benchmarks

Traditional code generation benchmarks, such as HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021), DS-1000 (Lai et al., 2023), and APPS (Hendrycks et al., 2021), focus on text-based tasks and a single programming language. Although newer benchmarks like MBXP (Athiwaratkun et al., 2023), MCEVAL (Chai et al., 2024), MultiPL-E (Cassano et al., 2023), humaneval-X (Zheng et al., 2023), and HumanEval-XL (Peng et al., 2024) cover multiple languages and tasks, they still focus on text-based code generation. In contrast, Flow2Code incorporates flowchart-based representations, offering a more comprehensive evaluation of LLMs in multimodal code generation tasks.

2.2 Multimodal Benchmarks

Recent multimodal benchmarks such as MM-Bench (Xu et al., 2023), MMMU (Yue et al., 2024), MMStar (Chen et al., 2024a), and Web2Code (Yun et al., 2024) evaluate MLLMs on tasks involving text and images, mainly focusing on visual reasoning and general multimodal capabilities. MMCode (Li et al., 2024) and Plot2Code (Wu et al., 2024a) extend this by targeting code generation, with MMCode focusing on Python problems with visual aids and Plot2Code on plots-to-code generation. However, these datasets are domain-specific, with limited coverage of broader code generation tasks. In contrast, the Flow2Code benchmark specifically evaluates the task of translating various flowchart types—code, UML, and pseudocode—into executable code, offering a more specialized and structured evaluation across diverse programming languages.
Figure 1: Overview of the flowchart generation process. The source code is initially converted into DOT code and code, UML flowcharts using Visustin. GPT-4o is then employed to transform the DOT code into a natural language pseudocode (highlighted in gray text). The generated DOT code is first validated through a Gemini-2.0 check, after which the pseudocode flowcharts are created. Finally, the pseudocode, code, and UML flowcharts are subjected to a comprehensive human review to ensure the accuracy and quality of the transformations.

3 Dataset Construction

This section first describes the data source and selection for the Flow2Code dataset (Section 3.1), followed by the flowchart construction process (Section 3.2), and concludes with an analysis of the dataset (Section 3.3).

3.1 Data Selection

The Flow2Code dataset is constructed from four key datasets: HumanEval-X (Zheng et al., 2023), MBXP (Athiwaratkun et al., 2023), MCEval (Chai et al., 2024), and ClassEval (Du et al., 2023). These datasets are selected for their diverse programming languages, task complexities, and availability of solution code segments, offering a comprehensive foundation for code generation tasks. Compared to other datasets like DS-1000 (Lai et al., 2023) or APPS (Hendrycks et al., 2021) and HumanEval (Chen et al., 2021), which often focus on isolated, single-language problems, the selected datasets in Flow2Code offer a better balance of multilingual coverage and task variety. Furthermore, the inclusion of class-level tasks from ClassEval allows for a more holistic evaluation, challenging models with more complex code generation.

Moreover, only those problems with official solution code segments are kept. The availability of official solutions ensures the reliability and consistency of the flowchart generation process, enabling us to evaluate flowchart-based code generation tasks effectively.

Finally, the selected data is obtained. These selections ensure a rich and varied dataset that tests code generation models across a wide range of pro-
gramming languages and task complexities.

3.2 Flowchart Construction
In the Flow2Code dataset construction process, a crucial step is transforming the raw data from the original datasets into a format suitable for code generation. This transformation process relies on an automated pipeline designed to convert solution code segments into flowchart representations, facilitating their subsequent use in code generation tasks.

3.2.1 Code Flowchart and UML Flowchart
The construction progress of code flowchart and UML flowchart can be divided into three steps: format conversion, flowchart conversion, and human review.

Format conversion: The first step in this transformation process involves converting the solution code segments into code files. The solution code segments contain code in various programming languages, covering diverse task complexities.

Flowchart conversion: As shown in Figure 1, this work uses Visustin software\footnote{https://www.aivosto.com/visustin.html}, a widely-used tool for code-to-flowchart conversion, to generate code flowcharts and UML flowcharts from these code files. Visustin processes the code into flowcharts that visually represent the control flow, function calls, conditionals, and other programmatic steps, providing a clear and intuitive view of the underlying logic.

Human review: To ensure the correctness and fidelity of the generated flowcharts, human evaluation is conducted. Five evaluators with master’s degrees in computer science who have over four years of practical programming experience are employed. Each evaluator is proficient in multiple programming languages and software engineering practices. The evaluation is conducted in two phases. For code and UML flowcharts, the evaluators verify the logical correctness of the generated flowcharts against the source code using the following criteria: Logical consistency: Do flowchart execution paths, conditions, and loops exactly reflect the original code? Completeness: Are there missing or extraneous elements that alter the intended program logic? Semantic accuracy: Are variable assignments, function calls, and control statements correctly captured? Each instance is scored in binary form: **1**: The flowchart is logically and semantically correct and complete. **0**: The flowchart contains critical inaccuracies or omissions. All instances are independently double-blind reviewed by two evaluators. In case of disagreement, the third expert adjudicates. The Krippendorff’s Alpha (Krippendorff, 2011) is employed to assess inter-annotator agreement, achieving a reliability score of \( \alpha = 0.88 \). This high agreement ensures the trustworthiness of the manual verification.

3.2.2 Pseudocode Flowchart
Pseudocode flowcharts provide a human-readable and language-agnostic description of the program’s logic, which complements the visual structure of code and UML flowcharts. This combination of textual pseudocode with visual flowchart representations enables more comprehensive and interpretable model evaluations. It helps to bridge the gap between abstract flow control and practical code generation by providing an additional level of clarity that may not be captured fully by flowcharts or UML diagrams alone.

A particularly important step in the dataset construction process is the conversion of the raw DOT files into a pseudocode flowchart. The pseudocode flowchart construction can be divided into six steps: format conversion, Dot code generation, GPT-4o transformation, Gemini-2.0 check, human review, and automatic flowchart conversion.

Format conversion: The format conversion is conducted the same as the format conversion in code flowchart and UML flowchart conversion.

Dot code generation: Furthermore, Visustin also converts the flowcharts into raw DOT files, which are descriptions of graphs in a specific format used by the Graphviz visualization tool (Ellson et al., 2004). In these DOT files, each node corresponds to a specific step or operation in the code’s flow, such as a variable assignment or a loop, while the edges between nodes represent the relationships between these steps. The DOT format provides a flexible, machine-readable structure that can be used for various types of graph visualizations.

GPT-4o transformation: The GPT model is given the task of converting each node label in the DOT file into a natural language description. The prompt is carefully designed to ensure the labels clearly describe the logic of each node in plain English (the prompt is in Appendix Figure 11). As shown in Figure 1, a label like “a, b = b, a % b” might be transformed into “Assign ‘b’ to ‘a’, and the remainder of ‘a’ divided by ‘b’ to ‘b’,” a conditional like “b ?” might be converted into
"check if 'b' is equal to zero."

Gemini check: Since GPT-4o is used to generate the pseudocode DOT files, using the same model to also validate the results could introduce the risk of potential biases or errors in the verification process. To mitigate this concern, Gemini-2.0 is tasked with determining whether the content of the generated DOT files has been correctly converted into natural language descriptions by GPT-4o. The prompt used for Gemini-2.0’s verification is detailed in Appendix Figure 10. If Gemini-2.0 determines that the content is accurate, the corresponding data is retained in the dataset. If Gemini-2.0 identifies errors in an instance, the instance is re-generated by GPT-4o.

To maintain the integrity of the dataset, this regeneration process is repeated up to five times. If the data still fails to meet the accuracy criteria after five attempts, it is excluded from the final dataset to prevent erroneous or unreliable data from affecting the benchmark. This approach guarantees that only high-quality, accurately converted data is included in Flow2Code, ensuring the reliability of the dataset for subsequent code generation evaluations.

Human review: In verifying the pseudocode flowcharts derived from natural language transformations of DOT representations, evaluators conduct a structured assessment based on: Semantic accuracy: Whether natural language node labels precisely match the intended semantics of the original DOT nodes. Clarity: Whether the labels are concise and easily interpretable. Consistency: Whether the number and structure of descriptions accurately correspond to the graph structure. Scoring follows a binary scheme: 1: Label is accurate, clear, and consistent with the original structure. 0: Label is ambiguous, incorrect, or inconsistent. Each pseudocode flowchart is reviewed independently by at least two evaluators, with a third reviewer resolving conflicts. As with the other flowcharts, the scoring achieves Krippendorff’s Alpha of 0.88.

Automatic flowchart conversion: Finally, Graphviz is used to automatically convert the corresponding pseudocode flowcharts from the generated DOT files.

3.3 Dataset Analysis

3.3.1 Data Statistics

Through this transformation process, the Flow2Code dataset ultimately includes three types of flowcharts—code flowcharts, UML flowcharts, and pseudocode flowcharts—and contains 15 different programming languages, 5,622 code segments, and 16,866 flowcharts, as shown in Table 2. The distribution of samples across these languages reflects the inherent characteristics of the original datasets, where some languages are more represented than others, as illustrated in Figure 2. We offer a well-rounded multimodal approach that better tests and pushes the code generation capabilities of MLLMs. This ensures that models are evaluated in a more thorough, balanced manner, improving their ability to generate accurate and meaningful code from diverse input formats.

To support fine-tuning and evaluation experiments, the Flow2Code dataset is divided into training and test sets using a 9:1 split ratio. This partitioning is conducted within each language subset of the original source datasets (e.g., HumanEval-X, MBXP, McEval, and ClassEval). For instance, in HumanEval-X, which contains code samples across multiple programming languages, we select 10% of the instances from each language as the test set and retain the remaining 90% for training. This approach ensures proportional representation across all languages and prevents the exclusive use of any source dataset as the test set. Such balanced partitioning enhances the fairness and generalizability of our experimental evaluations.

![Pie chart showing the number and proportion of each programming language in the Flow2Code dataset.](page_1012_172_410_312.png)

Figure 2: The number and proportion of each programming language in the Flow2Code dataset.
<table>
  <tr>
    <th># of Programming Languages</th>
    <td>15</td>
  </tr>
  <tr>
    <th># of Code Segments</th>
    <td>5,622</td>
  </tr>
  <tr>
    <th>Avg. # of Characters in Code Segments</th>
    <td>315.93</td>
  </tr>
  <tr>
    <th>Max. # of Characters in Code Segments</th>
    <td>3914</td>
  </tr>
  <tr>
    <th>Min. # of Characters in Code Segments</th>
    <td>10</td>
  </tr>
  <tr>
    <th># of Code Flowchart</th>
    <td>5,622</td>
  </tr>
  <tr>
    <th># of UML Flowchart</th>
    <td>5,622</td>
  </tr>
  <tr>
    <th># of Pseudocode Flowchart</th>
    <td>5,622</td>
  </tr>
  <tr>
    <th>Avg. # of Units in Flowcharts</th>
    <td>11.44</td>
  </tr>
  <tr>
    <th>Max. # of Units in Flowcharts</th>
    <td>84</td>
  </tr>
  <tr>
    <th>Min. # of Units in Flowcharts</th>
    <td>3</td>
  </tr>
</table>

Table 2: Statistics of the Flow2Code dataset.

degrees in computer science and over four years of programming experience, are selected to carry out the evaluation.

The evaluation includes two steps:

**Code and UML Flowchart Verification:** Evaluators checked whether the flowcharts accurately represented the solution code segment’s logic. Each correct conversion is assigned a score of “1” and an incorrect one “0”.

**Pseudocode Flowchart Verification:** Evaluators judge whether GPT correctly transforms the DOT file labels into natural language descriptions. Each correct conversion is assigned a score of “1” and an incorrect one “0”.

We randomly sampled 100 instances from each flowchart type for evaluation. The average score across all evaluations is 0.94, indicating that the dataset contains high-quality flowcharts suitable for code generation tasks. This process ensures the reliability of Flow2Code for further research and evaluation.

4 Experiments and Results

This section first presents the experimental setting (Section 4.1) and evaluation metrics (Section 4.2), followed by the model baselines (Section 4.3) and experimental results (Section 4.4).

4.1 Experimental Setting

**Implementation Details:** In this work, the local model deployment frameworks LMDeploy (version 0.6.5) and SGLang (version 0.4.1.post4) are used, with model fine-tuning conducted via the Llama-Factory framework (version 0.9.2.dev0).

**Computing Platform:** Experiments are carried out on a server featuring dual Intel Xeon Gold 5320 CPUs, 377 GB RAM, and eight NVIDIA A100 GPUs, running Ubuntu 20.04.6 LTS. The setup provides a robust environment for efficient model training and deployment.

**Fine-tuning Details:** Qwen2-VL-7B is fine-tuned on the Flow2Code dataset using Low-Rank Adaptation (LoRA) (Hu et al., 2021), chosen for its efficient adaptation of large models to multi-modal tasks. The fine-tuning employs a learning rate of 5.0e-5, a batch size of 4, and 1 training epoch with a cosine learning rate scheduler. The AdamW (Loshchilov, 2017) optimizer is used with gradient accumulation steps of 8 to ensure stability (Details can be found in the Appendix Section D).

4.2 Evaluation Metrics

The evaluation uses test samples from the four core datasets selected for Flow2Code. The testing procedure follows the execution-based evaluation method, where candidate code is executed against a set of test cases, and success is determined by passing those tests. Unlike traditional n-gram evaluations, execution-based evaluation allows for functional correctness even if the generated solution differs in implementation from the reference solution. This flexibility is crucial for code generation tasks, as it accommodates different code styles and approaches that still achieve the desired functionality.

Following Athiwaratkun et al. (2023), we use Pass@k scores (Kulal et al., 2019) with the unbiased estimate from Chen et al. (2021) as the evaluation metrics, where a task is deemed successful if any of the top k samples are correct. For the evaluation, k is set as 1, 3, and 5, as the key metrics, which provide a comprehensive view of the model’s ability to generate correct code given flowchart.

4.3 Baselines

For the experiments, we evaluate a range of MM-LLMs with a focus on their ability to generate code. The specific information of the model is shown in Appendix Table 4.

Based on the research of Shiri et al. (2024), Das et al. (2024), and Ai et al. (2024), The following LLMs are evaluated:

**Claude-3.5-Sonnet** (Anthropic, 2024) is known for its strong performance in reasoning, math, and coding tasks, particularly with multimodal inputs.

**DeepSeek-VL2** (Wu et al., 2024b) is a Mixture-of-Experts model designed for visual understanding, ideal for tasks involving complex visual inputs.

**Gemini-2.0-Flash-Exp** (Int, 2024) offers low-latency, high-performance multimodal capabilities, particularly excelling in video understanding.
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th colspan="4">McEval (50)</th>
  </tr>
  <tr>
    <th>Python</th><th>CPP</th><th>Java</th><th>JS</th>
    <th>Python</th><th>C#</th><th>Java</th><th>PHP</th>
    <th>Python</th><th>Ruby</th><th>C</th><th>C#</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td><td>26.00</td><td>32.32</td><td>48.78</td><td>39.02</td><td>49.39</td><td>43.86</td><td>51.23</td><td>50.90</td><td>51.06</td><td>100.00</td><td>57.77</td><td>49.26</td><td>48.00</td><td>70.00</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td><td>1.00</td><td>1.83</td><td>11.59</td><td>29.88</td><td>40.24</td><td>12.77</td><td>11.29</td><td>9.00</td><td>39.12</td><td>85.27</td><td>42.06</td><td>45.01</td><td>28.00</td><td>36.00</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td><td><b>70.00</b></td><td><b>85.37</b></td><td><b>90.24</b></td><td><b>87.20</b></td><td><b>87.80</b></td><td><b>90.67</b></td><td><b>82.98</b></td><td><b>89.20</b></td><td><u>92.14</u></td><td><u>91.49</u></td><td><b>94.27</b></td><td><b>94.11</b></td><td><b>78.00</b></td><td><b>96.00</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td><td>18.00</td><td>54.27</td><td>60.37</td><td>78.66</td><td>72.56</td><td>76.60</td><td>68.58</td><td>75.12</td><td>81.83</td><td>98.36</td><td>79.38</td><td>84.78</td><td>68.00</td><td>72.00</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>47.00</td><td>43.90</td><td>82.93</td><td>85.98</td><td>82.93</td><td>74.14</td><td>67.92</td><td>71.85</td><td>85.76</td><td>99.67</td><td>87.23</td><td>89.20</td><td>76.00</td><td>90.00</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td><td>14.00</td><td>12.80</td><td>21.95</td><td>56.70</td><td>59.75</td><td>31.75</td><td>14.40</td><td>21.44</td><td>60.07</td><td>99.51</td><td>68.74</td><td>72.01</td><td>38.00</td><td>30.00</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td><td>35.00</td><td>53.05</td><td>68.29</td><td>79.27</td><td>84.15</td><td>72.01</td><td>71.85</td><td>58.10</td><td>81.01</td><td>97.87</td><td>85.11</td><td>89.85</td><td>60.00</td><td>78.00</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td><td>0.20</td><td>0.00</td><td>6.71</td><td>3.66</td><td>10.37</td><td>0.00</td><td>0.00</td><td>1.47</td><td>18.99</td><td>65.47</td><td>9.33</td><td>13.75</td><td>2.00</td><td>2.00</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td><td>7.00</td><td>9.15</td><td>26.22</td><td>34.15</td><td>42.07</td><td>23.24</td><td>13.75</td><td>18.99</td><td>46.64</td><td>100.00</td><td>45.34</td><td>52.05</td><td>26.00</td><td>36.00</td>
  </tr>
  <tr>
    <td>MiniCPM-V2_6</td><td>2.00</td><td>1.22</td><td>14.02</td><td>21.34</td><td>34.76</td><td>5.56</td><td>10.47</td><td>3.44</td><td>45.50</td><td>83.47</td><td>42.23</td><td>54.17</td><td>4.00</td><td>2.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td><td>41.00</td><td>54.88</td><td>74.39</td><td>74.39</td><td>79.27</td><td>76.60</td><td>65.47</td><td>70.21</td><td>84.94</td><td>100.00</td><td>83.63</td><td>88.38</td><td>66.00</td><td>84.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td><td>14.00</td><td>1.83</td><td>38.41</td><td>50.00</td><td>64.63</td><td>10.15</td><td>3.60</td><td>14.89</td><td>54.99</td><td>66.28</td><td>54.34</td><td>28.48</td><td>32.00</td><td>44.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td><td>25.00</td><td>49.39</td><td>75.61</td><td>71.34</td><td>79.88</td><td>84.12</td><td>81.34</td><td>82.65</td><td>81.83</td><td>86.42</td><td>78.56</td><td>83.63</td><td>64.00</td><td>76.00</td>
  </tr>
  <tr>
    <td>Variance</td><td>4.10</td><td>9.52</td><td>9.23</td><td>7.49</td><td>5.67</td><td>11.55</td><td>11.28</td><td>11.14</td><td>5.09</td><td>1.47</td><td>5.93</td><td>6.65</td><td>6.36</td><td>9.92</td>
  </tr>
  <tr>
    <td>Avg</td><td>24.09</td><td>34.80</td><td>50.57</td><td>56.88</td><td>62.46</td><td>48.99</td><td>44.90</td><td>46.61</td><td>64.94</td><td>89.77</td><td>65.38</td><td>66.40</td><td>46.57</td><td>56.86</td>
  </tr>
</table>

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="10">McEval (50)</th>
  </tr>
  <tr>
    <th>CPP</th><th>Fortran</th><th>HTML</th><th>Java</th><th>JS</th><th>Pascal</th><th>Perl</th><th>PHP</th><th>Python</th><th>Ruby</th><th>Shell</th><th>Tcl</th><th>VB</th><th>Avg</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td><td>48.00</td><td>32.00</td><td>12.00</td><td>32.08</td><td>50.00</td><td>52.00</td><td>54.00</td><td>62.00</td><td>60.00</td><td>68.00</td><td>48.00</td><td>56.00</td><td>70.00</td><td>50.43</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td><td>26.00</td><td>12.00</td><td>6.00</td><td>33.96</td><td>34.00</td><td>4.00</td><td>32.00</td><td>38.00</td><td>46.00</td><td>32.00</td><td>12.00</td><td>10.00</td><td>32.00</td><td>26.53</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td><td><b>86.00</b></td><td>24.00</td><td>52.00</td><td>35.84</td><td><b>92.00</b></td><td><b>70.00</b></td><td><b>92.00</b></td><td><b>94.00</b></td><td>84.00</td><td><b>90.00</b></td><td>48.00</td><td><b>82.00</b></td><td><b>86.00</b></td><td><b>80.48</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td><td>64.00</td><td>46.00</td><td>30.00</td><td>30.19</td><td>68.00</td><td>48.00</td><td>52.00</td><td>86.00</td><td>74.00</td><td>78.00</td><td>42.00</td><td>54.00</td><td>52.00</td><td>63.49</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>64.00</td><td>68.00</td><td>60.00</td><td><b>35.85</b></td><td>84.00</td><td>66.00</td><td>74.00</td><td>92.00</td><td><b>94.00</b></td><td>82.00</td><td><b>66.00</b></td><td><b>74.00</b></td><td><b>92.00</b></td><td><b>75.43</b></td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td><td>44.00</td><td>20.00</td><td>14.00</td><td>24.53</td><td>38.00</td><td>24.00</td><td>38.00</td><td>32.00</td><td>48.00</td><td>52.00</td><td>14.00</td><td>22.00</td><td>36.00</td><td>37.34</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td><td>58.00</td><td>60.00</td><td><b>66.00</b></td><td>26.42</td><td>68.00</td><td>46.00</td><td>76.00</td><td>60.00</td><td>82.00</td><td>80.00</td><td>50.00</td><td>66.00</td><td>74.00</td><td>67.71</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td><td>4.00</td><td>0.00</td><td>0.00</td><td>5.66</td><td>10.00</td><td>4.00</td><td>8.00</td><td>20.00</td><td>68.00</td><td>50.00</td><td>2.00</td><td>4.00</td><td>0.00</td><td>11.47</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td><td>24.00</td><td>4.00</td><td>2.00</td><td>33.96</td><td>30.00</td><td>24.00</td><td>30.00</td><td>24.00</td><td>42.00</td><td>60.00</td><td>12.00</td><td>20.00</td><td>48.00</td><td>30.91</td>
  </tr>
  <tr>
    <td>MiniCPM-V2_6</td><td>10.00</td><td>0.00</td><td>0.00</td><td>1.89</td><td>14.00</td><td>0.00</td><td>28.00</td><td>12.00</td><td>34.00</td><td>48.00</td><td>8.00</td><td>4.00</td><td>0.00</td><td>18.31</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td><td>66.00</td><td><b>72.00</b></td><td>40.00</td><td>30.19</td><td>74.00</td><td>50.00</td><td>74.00</td><td>78.00</td><td>84.00</td><td>78.00</td><td>64.00</td><td>66.00</td><td>80.00</td><td>70.35</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td><td>42.00</td><td>22.00</td><td>4.00</td><td>20.42</td><td>48.00</td><td>18.00</td><td>24.00</td><td>56.00</td><td>64.00</td><td>32.00</td><td>20.00</td><td>52.00</td><td>35.05</td><td>35.05</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td><td>70.00</td><td>50.00</td><td>20.00</td><td>20.75</td><td>68.00</td><td>26.00</td><td>56.00</td><td>78.00</td><td>64.00</td><td>70.00</td><td>46.00</td><td>56.00</td><td>68.00</td><td>62.85</td>
  </tr>
  <tr>
    <td>Variance</td><td>5.81</td><td>6.32</td><td>5.25</td><td>1.09</td><td>6.31</td><td>5.60</td><td>5.73</td><td>7.59</td><td>3.49</td><td>2.56</td><td>5.32</td><td>8.05</td><td>8.58</td><td>5.30</td>
  </tr>
  <tr>
    <td>Avg</td><td>47.71</td><td>32.57</td><td>24.14</td><td>26.01</td><td>52.86</td><td>35.00</td><td>49.43</td><td>57.00</td><td>64.29</td><td>65.86</td><td>36.57</td><td>43.29</td><td>54.71</td><td>49.95</td>
  </tr>
</table>

Table 3: Pass@1 results for 13 LLMs on the code flowchart generation task in the Flow2Code benchmarks. ClassEval, HumanEval-X, MBXP, and McEval represent the code source of the instances in Flow2chart, respectively. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).

GLM-4V-Plus (GLM et al., 2024) specializes in image and video recognition, making it suitable for tasks requiring advanced visual comprehension.

GPT-4o (Team, 2024) is a leading multimodal model capable of processing text, images, and audio, with strong code generation abilities.

InternVL2.5-MPO (8B and 78B) (Chen et al., 2024b) is optimized for multimodal reasoning tasks, this model excels in complex, multimodal code generation scenarios.

LLaVA-OneVision-Qwen2-OV (7B and 72B) (Li et al., 2025) is known for strong performance across a range of visual tasks, including image and video understanding.

MiniCPM-V2_6 (Yao et al., 2024) is optimized for video understanding and multimodal reasoning with efficient processing of visual data.

Qwen2-VL-Instruct (7B and 72B) (Wang et al., 2024) provides strong performance in understanding visual content across varying resolutions.

These models are selected based on their demonstrated capabilities in multimodal tasks and their potential for generating code from both visual and textual inputs.

4.4 Experimental Results

As shown in Table 3 and Figure 3, the benchmark evaluation of 13 multimodal models across three flowchart types (code, UML, and pseudocode) reveals distinct performance patterns.

Model Performance Disparity: Gemini-2.0 consistently maintains a significant advantage across all three tasks, with an average pass rate improvement of 4.78% over the next best models, i.e., GPT-4o, GPT-4o and Qwen2-VL-72B perform similarly on the code flowchart task, but their performance decreases by about 5% and 10% respectively on UML and pseudocode tasks. Smaller models (such as LLaVA-OneVision-7B and MINICPM-V2_6) exhibit overall weaker performance, with average pass rates only 14.3%-22.35% of those achieved by the top-performing Gemini-2.0.

Programming Language Sensitivity: The generation ability for PHP is notably strong, with three
Figure 3: Stacked diagram of Pass@1, 3, and 5 of all the evaluation models on the benchmark. The suffixes of the model names represent different flowchart types: C represents code flowchart, P represents pseudocode flowchart, and U represents UML flowchart.

Figure 4: The average Pass@1 rate of the three difficulty levels of the samples in the Flow2Code’s McEval part on the code flowchart.

models achieving perfect scores. In contrast, the average pass rates for Fortran and HTML are 57.2%–65.63% lower compared to PHP. The disparity in generation capabilities between models is most pronounced for CPP, C#, and JavaScript, which exhibit the largest variance in performance. PHP, Python, and Ruby languages exhibit the least variation across models.

Model Scale Effect: Increasing the model size results in systematic improvements. For example, Qwen2-VL-72B shows a 103.8% performance boost over its 7B version in the code flowchart task. Similarly, the Intern-VL2.5-78B version outperforms its 8B counterpart in the UML task by 75.1%.

Flowchart Type Difference Results: The code flowchart task shows the highest average pass rate, followed by the UML task, with the pseudocode task proving to be the most challenging, showing a decrease of 10.63% and 7.48% in average pass rates compared to the first two tasks. Some models exhibit task-specific performance, such as Claude-3.5-Sonnet, which performs exceptionally well in generating Tcl code for pseudocode tasks, but its ability to generate Fortran code decreases by 6% compared to the code flowchart task.

Fine-Tuning Strategy Effectiveness: As shown in Figure 5, after fine-tuning, significant performance improvements are observed in Qwen2-VL-7B-FT, with the most pronounced in object-oriented languages (C#, Java) and web languages (JS, PHP), suggesting the fine-tuning data emphasized modern programming paradigms. The modest performance increase in legacy languages (Fortran, Pascal) and markup languages (HTML) suggests either data scarcity in these domains or architectural limitations in handling non-OOP paradigms.

Code Generation with Different Difficulty Levels: Figure 4 presents the average Pass@1 performance of 13 multimodal models across 15 programming languages, categorized by difficulty level (Easy, Middle, Hard). Several key observations can be made.

(1) A consistent trend across most languages and models is the inverse correlation between task dif-
Figure 5: Average Pass@1 results for Qwen2-VL-7B and the fine-tuned Qwen2-VL-7B-FT (on the Flow2Code dataset) on code flowchart tasks. The dataset labels in the bottom-right corner of the image show which subclass corresponds to each dimension of the radar chart.

ficulty and Pass@1. Performance is highest for “Easy” tasks, decreases for “Middle” tasks, and is lowest for “Hard” tasks. This is expected, as the complexity of the code generation task directly impacts the likelihood of generating a correct solution on the first attempt (Pass@1).

(2) There is significant variation in performance across different programming languages. For instance, models generally achieve higher Pass@1 scores in Python and Ruby, even at higher difficulty levels, compared to languages like HTML, Java, or Fortran. The superior performance of Python and Ruby may be attributed to a combination of factors, including the prevalence of these languages in training data, their relatively simpler syntax and higher-level abstractions, and the rich availability of libraries and corresponding descriptions within the dataset.

5 Conclusion

We introduce Flow2Code, a multimodal dataset combining flowcharts and code in 15 programming languages, designed to advance research in code generation. The dataset includes three flowchart types—code, UML, and pseudocode—providing a rich resource for training and evaluating MLLMs. We propose a new benchmark, Flow2Code, to assess MLLMs on code generation from flowchart inputs. Extensive evaluations of 13 MLLMs highlight their strengths and weaknesses, providing valuable insights for future research.

Acknowledgments

We want to thank the School of Computer Science and Technology and the Institute of AI Education at East China Normal University for providing the computational platform. We also thank the reviewers for their insightful comments.

Limitations

While this study introduces the Flow2Code dataset and the Flow2Code benchmark as valuable resources for code generation, the limitations remain that should be addressed in future work. The current evaluation framework focuses on end-to-end code generation tasks but does not account for the potential complexities involved in real-world code maintenance tasks, such as debugging or refactoring. Extending the benchmark to include tasks beyond initial code generation, such as identifying and fixing bugs or optimizing existing code based on flowchart representations, could better reflect the broader utility of multimodal models in software development.

Ethics Statement

We ensure that the Flow2Code dataset is constructed in full compliance with the terms of use of the original datasets (HumanEval-X, MBXP, MCEval, and ClassEval) and strictly respect the intellectual property rights of their authors. All code segments and flowchart transformations are derived from publicly available solutions or generated programmatically, with no inclusion of sensitive, proprietary, or personally identifiable information.

For human reviewers involved in the flowchart verification and pseudocode transformation processes, we guarantee fair treatment, including appropriate compensation and adherence to ethical labor practices. Reviewers participated voluntarily with full awareness of their tasks and associated requirements, and their contributions were anonymized to protect privacy.

The Flow2Code dataset focuses on flowchart-based code generation tasks and does not involve socially sensitive topics, biased content, or ethically controversial material. The generated flowcharts and code segments are purely algorithmic and logic-driven, posing no foreseeable risks of misuse or harmful societal consequences. All flowchart generation tools (Visustin, Graphviz) and AI models (GPT-4o, Gemini-2.0) were used in accordance with their respective licenses and API terms. The
dataset will be released under open licenses that align with the original data sources’ policies.

References

2024. Introducing Gemini 2.0: Our new AI model for the agentic era. https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/.

Qihang Ai, Jiafan Li, Jincheng Dai, Jianwu Zhou, Lemao Liu, Haiyun Jiang, and Shuming Shi. 2024. Advancement in graph understanding: A multimodal benchmark and fine-tuning of vision-language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7485–7501, Bangkok, Thailand. Association for Computational Linguistics.

AI Anthropic. 2024. Claude 3.5 sonnet model card addendum. Claude-3.5 Model Card, 3:6.

Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, et al. 2023. Multi-lingual evaluation of code generation models. In ICLR.

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, H. Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. Program Synthesis with Large Language Models. ArXiv.

Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, Arjun Guha, Michael Greenberg, and Abhinav Jangda. 2023. MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation. IEEE Transactions on Software Engineering, 49(7):3675–3691.

Linzhen Chai, Shukai Liu, Jian Yang, Yuwei Yin, Ke Jin, Jiaheng Liu, Tao Sun, Ge Zhang, Changyu Ren, Hongcheng Guo, Zekun Wang, Boyang Wang, Xianjie Wu, Bing Wang, Tongliang Li, Liquan Yang, Sufeng Duan, and Zhoujun Li. 2024. McEval: Massively Multilingual Code Evaluation. Preprint, arXiv:2406.07436.

Ned Chapin. 2003. Flowchart, page 714–716. John Wiley and Sons Ltd., GBR.

Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. 2024a. Are we on the right way for evaluating large vision-language models? CoRR.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidi Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, F. Such, D. Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William H. Guss, Alex Nichol, Igor Babuschkin, S. Balaji, Shan-tanu Jain, A. Carr, J. Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, M. Knight, Miles Brundage, Mira Murati, Katie Mayer, P. Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, I. Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. ArXiv.

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. 2024b. Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24185–24198.

Rocktim Das, Simeon Hristov, Haonan Li, Dimitar Dimitrov, Ivan Koychev, and Preslav Nakov. 2024. EXAMS-V: A multi-discipline multilingual multimodal exam benchmark for evaluating vision language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7768–7791, Bangkok, Thailand. Association for Computational Linguistics.

DeepSeek-Al, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y. Wu, Yukun Li, Huazuo Gao, Shirong Ma, Wangding Zeng, Xiao Bi, Zihui Gu, Hanwei Xu, Damai Dai, Kai Dong, Liyue Zhang, Yishi Piao, Zhibin Gou, Zhenda Xie, Zhenwen Hao, Bingxuan Wang, Junxiao Song, Deli Chen, Xin Xie, Kang Guan, Yuxiang You, Aixin Liu, Qiushi Du, Wenjun Gao, Xuan Lu, Qinyu Chen, Yaohui Wang, Chengqi Deng, Jiashi Li, Cheng-gang Zhao, Chong Ruan, Fulu Luo, and Wenfeng Liang. 2024. DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence. Preprint, arXiv:2406.11931.

Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2023. ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation. Preprint, arXiv:2308.01861.

John Ellson, Emden R Gansner, Eleftherios Koutsofios, Stephen C North, and Gordon Woodhull. 2004. Graphviz and dynagraph—static and dynamic graph drawing tools. Graph drawing software, pages 127–148.

Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chen-hui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi
Gui, Jie Tang, Jing Zhang, Jingyu Sun, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang, Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang. 2024. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. Preprint, arXiv:2406.12793.

Dan Hendrycks, Steven Basart, Saurav Kadavath, Man-tas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. 2021. Measuring coding challenge competence with apps. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).

Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2021. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations.

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xinzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. 2024. Owen2.5-Coder Technical Report. Preprint, arXiv:2409.12186.

Klaus Krippendorff. 2011. Computing krippendorff’s alpha-reliability.

Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy S Liang. 2019. Spoc: Search-based pseudocode to code. Advances in Neural Information Processing Systems, 32.

Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pages 18319–18345. PMLR.

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2025. LLaVA-onevision: Easy visual task transfer. Transactions on Machine Learning Research.

Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, Zhiyong Huang, and Jing Ma. 2024. Mmcode: Benchmarking multimodal large language models for code generation with visually rich programming problems. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 736–783.

Yuji Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittweisler, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. 2022. Competition-Level Code Generation with AlphaCode. Science, 378(6624):1092–1097.

Zeming Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che, and Ting Liu. 2020. Towards conversational recommendation over multi-type dialogs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1036–1049.

I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.

Erik Nijkamp, Hiroaki Hayashi, Caiming Xiong, Silvio Savarese, and Yingbo Zhou. 2023. CodeGen2: Lessons for Training LLMs on Programming and Natural Languages.

Bashar Nuseibeh and Steve Easterbrook. 2000. Requirements engineering: a roadmap. In Proceedings of the Conference on the Future of Software Engineering, pages 35–46.

OpenAI. 2023. GPT-4 Technical Report. Preprint, arXiv:2303.08774.

Qiwei Peng, Yekun Chai, and Xuhong Li. 2024. Humaneval-xl: A multilingual code generation benchmark for cross-lingual natural language generalization. In LREC/COLING.

Fatemeh Shiri, Xiao-Yu Guo, Mona Golestani Far, Xin Yu, Reza Haf, and Yuan-Fang Li. 2024. An empirical analysis on spatial reasoning capabilities of large multimodal models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21440–21455, Miami, Florida, USA. Association for Computational Linguistics.

OpenAI Team. 2024. GPT-4o System Card. Preprint, arXiv:2410.21276.

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Owen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution. Preprint, arXiv:2409.12191.

Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhixuan Liang, Zeyu Lu, Ying Shan, and Ping Luo. 2024a. Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. 2024b. DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding. Preprint. arXiv:2412.10302.

Stelios Xinoglou. 2013. Using flowchart-based programming environments for simplifying programming and software engineering processes. In 2013 IEEE Global Engineering Education Conference (EDUCON), pages 1313–1322.

Cheng Xu, Xiaofeng Hou, Jiacheng Liu, Chao Li, Tianhao Huang, Xiaozi Zhu, Mo Niu, Lingyu Sun, Peng Tang, Tongqiao Xu, Kwang-Ting Cheng, and Minyi Guo. 2023. MMbench: Benchmarking End-to-End Multi-modal DNNs and Understanding Their Hardware-Software Implications. In 2023 IEEE International Symposium on Workload Characterization (IISWC), pages 154–166.

Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhibui He, et al. 2024. Minicpm-v: A gpt-4v level mllm on your phone. CoRR.

Hao Yu, Bo Shen, Dezhi Ran, Jiaxin Zhang, Qi Zhang, Yuchi Ma, Guangtai Liang, Ying Li, Qianxiang Wang, and Tao Xie. 2024. CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pages 1–13.

Xiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang, Ruqii Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Rubin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhua Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhui Chen. 2024. MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9556–9567.

Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, et al. 2024. Web2code: A large-scale webpage-to-code dataset and evaluation framework for multimodal llms. CoRR.

Fengji Zhang, Linquan Wu, Huiyu Bai, Guancheng Lin, Xiao Li, Xiao Yu, Yue Wang, Bei Chen, and Jacky Keung. 2024. Humaneval-v: Evaluating visual understanding and reasoning abilities of large multimodal models through coding tasks. CoRR.

Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, et al. 2023. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5673–5684.

Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadiira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. 2024. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions. CoRR.

A Flowchart Example

Due to page limitations in the main text, it is not feasible to include relatively complex flowcharts. Therefore, a set of representative flowchart examples is provided in the appendix. These examples illustrate three different forms of flowchart representations for the same code segment.

B Prompts to Use

This section presents the prompts used during the flowchart generation process and Flow2Code benchmark evaluation, as well as the LLM message code templates.

C Baselines Details

This section provides detailed information about the 13 MLLMs used for the Flow2Code benchmark evaluation.

D Fine-tuning Setting

The parameters used for fine-tuning are as follows:
bf16: true; cutoff_len: 2048; ddp_timeout: 180000000; do_train: true; eval_steps: 100; eval_strategy: steps; finetuning_type: lora; flash_attn: auto; gradient_accumulation_steps: 8; learning_rate: 5.0e-05; logging_steps: 5; lora_alpha: 16; lora_dropout: 0; lora_rank: 8; lora_target: all; lr_scheduler_type: cosine; max_grad_norm: 1.0; max_samples: 100000; num_train_epochs: 1.0; optim: adamw_torch; packing: false; per_device_eval_batch_size: 4; per_device_train_batch_size: 4; plot_loss: true; preprocessing_num_workers: 16; report_to: none; save_steps: 100; stage: sft; template: qwen2_vl; trust_remote_code: true; val_size: 0.1; warmup_steps: 0.

E Additional Results

This section presents the detailed results obtained from evaluating 13 MLLMs using the
Prompt for DOT Code Transformation

from typing import List

def separate_paren_groups(paren_string: str) -> List[str]:

    # Remove spaces from the input string
    paren_string = paren_string.replace(" ", "")

    # Initialize variables to store current group and stack to track parentheses
    current_group = ""
    stack = []
    result = []

    # Iterate over each character in the input string
    for char in paren_string:
        # If it's an opening parenthesis, push to stack and add to current group
        if char == "(":
            stack.append(char)
            current_group += char
        # If it's a closing parenthesis, pop from stack and add to current group
        elif char == ")" and stack:
            stack.pop()
            current_group += char
        # If stack is empty, it means we have a complete group
        if not stack:
            result.append(current_group)
            current_group = ""  # Reset current group for the next one

    return result

Figure 6: The code segment example.
from typing import List

def separate_paren_groups(paren_string: str) -> List[str]:
    Remove spaces from the input string
    paren_string = paren_string.replace(" ", "")
    Initialize variables to store current group and stack to track parentheses
    current_group = ""
    stack = []
    result = []
    Iterate over each character in the input string
    for char in paren_string:
        If it's an opening parenthesis, push to stack and add to current group
        char == "(" ?
            stack.append(char)
            current_group += char
        No
        If it's a closing parenthesis, pop from stack and add to current group
        char == ")" and stack ?
            stack.pop()
            current_group += char
        No
        If stack is empty, it means we have a complete group
        not stack ?
            Yes
                result.append(current_group)
                Reset current group for the next one
                current_group = ""
            No
    End for
    return result
End

Figure 7: The code flowchart based on Figure 6 the code segment example.
Figure 8: The UML flowchart based on Figure 6 the code segment example.
Figure 9: The pseudocode flowchart based on Figure 6 the code segment example.

Flow2Code benchmark, including the specific Pass@1, Pass@3, and Pass@5 data for each of the three flowchart types.
Prompt for Dot Code Check

You will be given two inputs:

1. A new DOT representation of the control flow for a piece of code.
2. An old DOT representation of the control flow for the same piece of code.

Your task is to evaluate if the transformation from the old DOT code to the new DOT code adheres to the following rules:

---

**Example:**
**Old DOT code:**
```
dot
digraph FlowChart {
    1 [shape=ellipse, label="import math"];
    2 [shape=ellipse, label="def calculate_area(radius)"];
    3 [shape=box, label="area = math.pi * radius * radius"];
    4 [shape=Msquare, label="End"];

    1->2;
    2->3;
    3->4;
}
```

**New DOT code:**
```
dot
digraph FlowChart {
    1 [shape=ellipse, label="Import the math module"];
    2 [shape=ellipse, label="Define function 'calculate_area' with parameter 'radius'"];
    3 [shape=box, label="Calculate the area using the formula 'pi * radius * radius' and assign it to 'area'"];
    4 [shape=Msquare, label="End"];

    1->2;
    2->3;
    3->4;
}
```

**Explanation:**
- The import statement is translated into "Import the math module".
- The function definition is described as "Define function 'calculate_area' with parameter 'radius'".
- The calculation step is explained as "Calculate the area using the formula 'pi * radius * radius' and assign it to 'area'".
---

**Final Note:**
Respond only with 'yes' or 'no'.
---

Figure 10: The prompt used by Gemini-2.0 to check the DOT code generated by GPT-4o.
Prompt for DOT Code Transformation

---

You will be given one input:
1. A 'DOT' representation of the control flow for a piece of code.

Your task is to generate a natural language-oriented flowchart in 'DOT' syntax. This flowchart must:
- Clearly describe the logic in plain English.
- Follow 'DOT' syntax accurately.
- Include key operations, conditions, and outputs.
- Be mindful of the specific characteristics of the programming language used.

Instructions:
1. **Do not change the structure of the 'DOT' code**. Only modify the 'label' content of the nodes to describe the code's logic in natural language.
2. For the "label" fields, describe the code inside them in clear and concise natural language, while considering the characteristics of the given programming language. In particular, the definitions of various data types and function types should be clarified.
3. If the 'label' represents an import statement, describe it as "Import module XYZ".
4. For function definitions, describe the function name and its input parameters in plain English, e.g., "Define function 'function_name' with parameters 'param1', 'param2' of type 'type1'". If a parameter has a default value, mention it explicitly, e.g., "Parameter 'param1' has a default value of '10'."
5. For operations like assignments or calculations, describe them in natural language, e.g., "Assign the value of 'radius * radius * pi' to 'area'."
6. For conditionals, describe them in natural language, e.g., "Check if 'distance' is less than 'threshold'."
7. For loops, describe them as "Iterate through the list 'numbers'."
8. If a function has a return statement, describe the returned value, e.g., "Return 'result'."
9. **Do not modify the rest of the 'DOT' syntax** like nodes, edges, or graph attributes; only modify the labels as instructed.
10. **Only output the 'DOT' code inside code blocks** (i.e., between the triple backticks '```').

---

**Example:**
**Given DOT Input:**
```dot
digraph FlowChart {
  1 [shape=ellipse, label="import math"];
  2 [shape=ellipse, label="def calculate_area(radius)"];
  3 [shape=box, label="area = math.pi * radius * radius"];
  4 [shape=Msquare, label="End"];

  1->2;
  2->3;
  3->4;
}
```

**Desired Output:**
```dot
digraph FlowChart {
  1 [shape=ellipse, label="Import the math module"];
  2 [shape=ellipse, label="Define function 'calculate_area' with parameter 'radius'"];
  3 [shape=box, label="Calculate the area using the formula 'pi * radius * radius' and assign it to 'area'"];
  4 [shape=Msquare, label="End"];

  1->2;
  2->3;
  3->4;
}
```

**Explanation:**
- The import statement is translated into "Import the math module".
- The function definition is described as "Define function 'calculate_area' with parameter 'radius'".
- The calculation step is explained as "Calculate the area using the formula 'pi * radius * radius' and assign it to 'area'".
...

**Final Note:**
Next is the dot code you need to convert, and you should modify the labels based on the programming language's characteristics. It is converted from XXX programming language into dot code.
...

Figure 11: The prompt used by GPT-4o to convert DOT code into pseudocode DOT code.
Prompt for Code Generation

---

**Prompt:**

You will be given an image representing a **XXX flowchart**. The flowchart describes the logic and structure of a program in terms of nodes and edges.

Your task is to generate the **complete code** that matches the flowchart. Follow these instructions:

1. Write the **complete code** based on the flowchart's structure. Ensure that the program is functional and adheres to the flowchart's steps.
2. The generated code must match the flowchart's described logic:
   - Define any necessary functions.
   - Implement loops, conditionals, and assignments as indicated by the flowchart.
   - Follow the flow and operations of the program as described by the nodes and edges.
3. The code should **match the intended programming language**: `<insert_language_name_here>`.
4. You must **only output the complete code** surrounded by **code blocks** (i.e., between triple backticks ````).
5. Do not include any other explanations or markdown outside the code block.
6. Do not include any comments or explanations within the generated code. The code should be solely composed of the functional elements from the flowchart, without any descriptive or explanatory annotations.
7. Do not add any additional code that is not represented in the flowchart. For example, if there is no main function in the flowchart, do not add a main function without authorization.

---

Figure 12: The prompt used to evaluate LLMs on flowchart-based code generation tasks using the Flow2Code benchmark.

<html>
  <table>
    <tr>
      <th>Unified Format for LLMs 'messages' Code</th>
    </tr>
    <tr>
      <td>
        <pre>
completion = client.chat.completions.create(
    model = model,
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{base64_image}"},
                },
            ],
        }
    ]
)
        </pre>
      </td>
    </tr>
  </table>
</html>

Figure 13: The LLM message code template used for evaluating large models on flowchart-based code generation tasks using the Flow2Code benchmark.
<table>
  <tr>
    <th>Model</th>
    <th>Language Model</th>
    <th>Vision Model</th>
    <th>time</th>
  </tr>
  <tr>
    <td rowspan="4">Closed-source</td>
    <td>Claude-3.5-Sonnet (Anthropic, 2024)</td>
    <td>-</td>
    <td>-</td>
    <td>2024.10</td>
  </tr>
  <tr>
    <td>Gemini-2.0 (Int, 2024)</td>
    <td>-</td>
    <td>-</td>
    <td>2024.12</td>
  </tr>
  <tr>
    <td>GLM-4V-plus (GLM et al., 2024)</td>
    <td>-</td>
    <td>-</td>
    <td>2024.08</td>
  </tr>
  <tr>
    <td>GPT-4o (Team, 2024)</td>
    <td>-</td>
    <td>-</td>
    <td>2024.05</td>
  </tr>
  <tr>
    <td rowspan="7">Open-source</td>
    <td>DeepSeek-VL2 (Wu et al., 2024b)</td>
    <td>DeepSeekMoE-27B</td>
    <td>SigLIP-400M</td>
    <td>2025.01</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO (Chen et al., 2024b)</td>
    <td>InternLM2.5-7B</td>
    <td>InternViT-300M-v2.5</td>
    <td>2024.12</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO (Chen et al., 2024b)</td>
    <td>Qwen-2.5-72B</td>
    <td>InternViT-6B-v2.5</td>
    <td>2024.12</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B (Li et al., 2025)</td>
    <td>Qwen2-7B</td>
    <td>SigLIP-400M</td>
    <td>2024.09</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B (Li et al., 2025)</td>
    <td>Qwen2-72B</td>
    <td>SigLIP-400M</td>
    <td>2024.09</td>
  </tr>
  <tr>
    <td>MiniCPM-V-2_6 (Yao et al., 2024)</td>
    <td>Qwen2-7B</td>
    <td>SigLIP-400M</td>
    <td>2024.08</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B (Wang et al., 2024)</td>
    <td>Qwen2-72B</td>
    <td>QwenViT</td>
    <td>2024.10</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B (Wang et al., 2024)</td>
    <td>Qwen2-7B</td>
    <td>QwenViT</td>
    <td>2024.10</td>
  </tr>
</table>

Table 4: Details of the baseline model.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th colspan="2">McEval (50)</th>
  </tr>
  <tr>
    <th>Python</th>
    <th>CPP</th>
    <th>Java</th>
    <th>JS</th>
    <th>Python</th>
    <th>CPP</th>
    <th>C#</th>
    <th>Java</th>
    <th>JS</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
    <th>C</th>
    <th>C#</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>24.00</td>
    <td>29.88</td>
    <td>40.85</td>
    <td>22.56</td>
    <td>50.61</td>
    <td>48.61</td>
    <td>55.16</td>
    <td>53.36</td>
    <td>50.25</td>
    <td><b>100.00</b></td>
    <td>57.61</td>
    <td>51.23</td>
    <td>44.00</td>
    <td>72.00</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>3.00</td>
    <td>2.44</td>
    <td>9.76</td>
    <td>40.85</td>
    <td>35.98</td>
    <td>21.44</td>
    <td>9.98</td>
    <td>8.84</td>
    <td>38.13</td>
    <td>83.63</td>
    <td>43.54</td>
    <td>48.61</td>
    <td>24.00</td>
    <td>20.00</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>60.00</b></td>
    <td><b>71.34</b></td>
    <td><b>79.88</b></td>
    <td><b>89.02</b></td>
    <td><b>82.93</b></td>
    <td><b>87.89</b></td>
    <td><b>88.05</b></td>
    <td><b>79.54</b></td>
    <td><b>90.02</b></td>
    <td>86.42</td>
    <td><b>91.65</b></td>
    <td><b>93.45</b></td>
    <td><b>76.00</b></td>
    <td><b>88.00</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>16.00</td>
    <td>50.61</td>
    <td>57.32</td>
    <td>81.10</td>
    <td>73.78</td>
    <td>77.58</td>
    <td>64.98</td>
    <td>73.32</td>
    <td>81.34</td>
    <td>98.85</td>
    <td>76.92</td>
    <td>84.94</td>
    <td>62.00</td>
    <td>62.00</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>43.00</td>
    <td>37.80</td>
    <td>73.17</td>
    <td>80.49</td>
    <td>79.27</td>
    <td>60.56</td>
    <td>58.27</td>
    <td>69.07</td>
    <td>76.92</td>
    <td>99.18</td>
    <td>82.16</td>
    <td>85.43</td>
    <td>66.00</td>
    <td><b>90.00</b></td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td>
    <td>13.00</td>
    <td>13.41</td>
    <td>26.83</td>
    <td>65.24</td>
    <td>54.27</td>
    <td>38.95</td>
    <td>23.73</td>
    <td>29.13</td>
    <td>61.37</td>
    <td>98.20</td>
    <td>63.01</td>
    <td>67.27</td>
    <td>46.00</td>
    <td>28.00</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>37.00</td>
    <td>42.68</td>
    <td>64.63</td>
    <td>82.32</td>
    <td>75.00</td>
    <td>72.01</td>
    <td>71.69</td>
    <td>62.03</td>
    <td>80.20</td>
    <td>99.84</td>
    <td>83.96</td>
    <td>87.07</td>
    <td>60.00</td>
    <td>80.00</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>0.20</td>
    <td>0.00</td>
    <td>3.05</td>
    <td>9.15</td>
    <td>5.49</td>
    <td>0.00</td>
    <td>0.00</td>
    <td>3.11</td>
    <td>19.15</td>
    <td>62.36</td>
    <td>10.80</td>
    <td>15.22</td>
    <td>4.00</td>
    <td>0.00</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>9.00</td>
    <td>7.93</td>
    <td>23.78</td>
    <td>39.02</td>
    <td>37.20</td>
    <td>20.46</td>
    <td>13.91</td>
    <td>19.64</td>
    <td><b>100.00</b></td>
    <td>45.01</td>
    <td>49.92</td>
    <td>28.00</td>
    <td>32.00</td>
    <td>32.00</td>
  </tr>
  <tr>
    <td>MiniCPM-V-2_6</td>
    <td>1.00</td>
    <td>1.22</td>
    <td>10.37</td>
    <td>39.63</td>
    <td>32.93</td>
    <td>15.22</td>
    <td>16.31</td>
    <td>2.13</td>
    <td>41.73</td>
    <td>88.87</td>
    <td>42.23</td>
    <td>57.28</td>
    <td>24.00</td>
    <td>12.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>39.00</td>
    <td>50.01</td>
    <td>73.17</td>
    <td>73.61</td>
    <td>76.22</td>
    <td>77.38</td>
    <td>67.59</td>
    <td>71.19</td>
    <td>82.00</td>
    <td>98.04</td>
    <td>82.00</td>
    <td>88.71</td>
    <td>60.00</td>
    <td>80.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>8.00</td>
    <td>3.66</td>
    <td>23.78</td>
    <td>54.27</td>
    <td>54.88</td>
    <td>27.50</td>
    <td>4.75</td>
    <td>7.36</td>
    <td>57.01</td>
    <td>73.63</td>
    <td>57.28</td>
    <td>36.66</td>
    <td>38.00</td>
    <td>48.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>20.00</td>
    <td>48.78</td>
    <td>71.34</td>
    <td>60.98</td>
    <td>65.24</td>
    <td>81.51</td>
    <td><b>82.65</b></td>
    <td><b>81.18</b></td>
    <td>81.51</td>
    <td>84.78</td>
    <td>71.52</td>
    <td>83.47</td>
    <td>56.00</td>
    <td>0.00</td>
  </tr>
</table>

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="14">McEval (50)</th>
  </tr>
  <tr>
    <th>CPP</th>
    <th>Fortran</th>
    <th>HTML</th>
    <th>Java</th>
    <th>JS</th>
    <th>Pascal</th>
    <th>Perl</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
    <th>Shell</th>
    <th>Tcl</th>
    <th>VB</th>
    <th>Avg</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>46.00</td>
    <td>34.00</td>
    <td>32.00</td>
    <td>26.42</td>
    <td>48.00</td>
    <td>38.00</td>
    <td>62.00</td>
    <td>68.00</td>
    <td>60.00</td>
    <td>28.00</td>
    <td>52.00</td>
    <td>56.00</td>
    <td>48.61</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>22.00</td>
    <td>10.00</td>
    <td>10.00</td>
    <td>26.42</td>
    <td>34.00</td>
    <td>8.00</td>
    <td>20.00</td>
    <td>24.00</td>
    <td>38.00</td>
    <td>8.00</td>
    <td>10.00</td>
    <td>28.00</td>
    <td>24.33</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>78.00</b></td>
    <td>14.00</td>
    <td>52.00</td>
    <td><b>37.74</b></td>
    <td><b>80.00</b></td>
    <td><b>62.00</b></td>
    <td><b>90.00</b></td>
    <td>78.00</td>
    <td>76.00</td>
    <td><b>90.00</b></td>
    <td>50.00</td>
    <td><b>74.00</b></td>
    <td><b>82.00</b></td>
    <td><b>75.56</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>62.00</td>
    <td>44.00</td>
    <td>36.00</td>
    <td>28.30</td>
    <td>68.00</td>
    <td>48.00</td>
    <td>50.00</td>
    <td>70.00</td>
    <td>66.00</td>
    <td>76.00</td>
    <td>40.00</td>
    <td>54.00</td>
    <td>52.00</td>
    <td>61.34</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>54.00</td>
    <td>58.00</td>
    <td>54.00</td>
    <td>33.96</td>
    <td>74.00</td>
    <td>56.00</td>
    <td>80.00</td>
    <td><b>98.00</b></td>
    <td><b>92.00</b></td>
    <td><b>90.00</b></td>
    <td><b>68.00</b></td>
    <td>70.00</td>
    <td><b>92.00</b></td>
    <td>71.18</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td>
    <td>32.00</td>
    <td>18.00</td>
    <td>32.08</td>
    <td>48.00</td>
    <td>22.00</td>
    <td>36.00</td>
    <td>38.00</td>
    <td>50.00</td>
    <td>38.00</td>
    <td>10.00</td>
    <td>24.00</td>
    <td>24.00</td>
    <td>37.77</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>58.00</td>
    <td>52.00</td>
    <td><b>58.00</b></td>
    <td>28.30</td>
    <td>60.00</td>
    <td>46.00</td>
    <td>74.00</td>
    <td>70.00</td>
    <td>74.00</td>
    <td>84.00</td>
    <td>50.00</td>
    <td>58.00</td>
    <td>72.00</td>
    <td>66.03</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>4.00</td>
    <td>0.00</td>
    <td>0.00</td>
    <td>5.66</td>
    <td>10.00</td>
    <td>8.00</td>
    <td>12.00</td>
    <td>4.00</td>
    <td>76.00</td>
    <td>60.00</td>
    <td>2.00</td>
    <td>0.00</td>
    <td>4.00</td>
    <td>11.78</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>20.00</td>
    <td>14.00</td>
    <td>2.00</td>
    <td>28.30</td>
    <td>28.00</td>
    <td>14.00</td>
    <td>30.00</td>
    <td>14.00</td>
    <td>62.00</td>
    <td>44.00</td>
    <td>14.00</td>
    <td>22.00</td>
    <td>38.00</td>
    <td>29.61</td>
  </tr>
  <tr>
    <td>MiniCPM-V-2_6</td>
    <td>24.00</td>
    <td>0.00</td>
    <td>4.00</td>
    <td>15.09</td>
    <td>32.00</td>
    <td>6.00</td>
    <td>20.00</td>
    <td>20.00</td>
    <td>36.00</td>
    <td>24.00</td>
    <td>10.00</td>
    <td>8.00</td>
    <td>18.00</td>
    <td>22.56</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>74.00</td>
    <td><b>68.00</b></td>
    <td>48.00</td>
    <td>32.08</td>
    <td>70.00</td>
    <td>48.00</td>
    <td>76.00</td>
    <td>70.00</td>
    <td>74.00</td>
    <td>84.00</td>
    <td>66.00</td>
    <td>62.00</td>
    <td>78.00</td>
    <td>69.61</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>34.00</td>
    <td>12.00</td>
    <td>6.00</td>
    <td>24.53</td>
    <td>40.00</td>
    <td>6.00</td>
    <td>38.00</td>
    <td>38.00</td>
    <td>48.00</td>
    <td>46.00</td>
    <td>20.00</td>
    <td>14.00</td>
    <td>50.00</td>
    <td>32.55</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>64.00</td>
    <td>36.00</td>
    <td>38.00</td>
    <td>20.75</td>
    <td>62.00</td>
    <td>18.00</td>
    <td>56.00</td>
    <td>70.00</td>
    <td>50.00</td>
    <td>68.00</td>
    <td>48.00</td>
    <td>40.00</td>
    <td>0.00</td>
    <td>54.08</td>
  </tr>
</table>

Table 5: Pass@1 results for 13 LLMs on UML flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th colspan="3">McEval (50)</th>
  </tr>
  <tr>
    <th>Python</th><th>CPP</th><th>Java</th><th>JS</th>
    <th>Python</th><th>CPP</th><th>C#</th><th>Java</th><th>JS</th><th>PHP</th><th>Python</th><th>Ruby</th><th>C</th><th>C#</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td><td>2.00</td><td>62.80</td><td><b>71.34</b></td><td>43.29</td><td>73.17</td><td>51.72</td><td>66.12</td><td>48.12</td><td>80.85</td><td><b>100.00</b></td><td>80.69</td><td>81.51</td><td>58.00</td><td>66.00</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td><td>1.00</td><td>0.00</td><td>4.27</td><td>19.51</td><td>36.59</td><td>1.80</td><td>0.82</td><td>1.80</td><td>36.82</td><td>60.23</td><td>36.99</td><td>40.92</td><td>14.00</td><td>12.00</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td><td><b>44.00</b></td><td><b>78.05</b></td><td>70.73</td><td><b>86.59</b></td><td><b>85.37</b></td><td>50.74</td><td><b>79.54</b></td><td><b>80.52</b></td><td><b>90.67</b></td><td>77.09</td><td><b>92.80</b></td><td><b>87.23</b></td><td><b>76.00</b></td><td><b>88.00</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td><td>1.00</td><td>20.12</td><td>7.93</td><td>29.88</td><td>73.17</td><td>28.97</td><td>21.11</td><td>4.91</td><td>76.27</td><td>71.69</td><td>66.61</td><td>79.54</td><td>50.00</td><td>40.00</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>9.00</td><td>56.71</td><td>57.93</td><td>35.98</td><td><u>85.37</u></td><td>63.83</td><td>52.21</td><td>36.66</td><td>81.01</td><td><u>98.20</u></td><td>87.56</td><td>82.16</td><td>66.00</td><td>80.00</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td><td>7.00</td><td>0.00</td><td>3.66</td><td>43.29</td><td>54.88</td><td>1.80</td><td>11.95</td><td>5.07</td><td>60.56</td><td>93.94</td><td>61.70</td><td>60.07</td><td>26.00</td><td>8.00</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td><td>26.00</td><td>51.22</td><td>7.93</td><td>79.27</td><td>82.93</td><td>41.90</td><td>58.27</td><td>7.86</td><td>79.38</td><td>82.65</td><td>82.98</td><td>82.49</td><td>58.00</td><td>56.00</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td><td>0.20</td><td>0.00</td><td>2.44</td><td>7.93</td><td>27.44</td><td>0.00</td><td>0.00</td><td>0.98</td><td>36.66</td><td>25.53</td><td>23.24</td><td>31.76</td><td>4.00</td><td>8.00</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td><td>9.00</td><td>4.88</td><td>12.20</td><td>41.46</td><td>58.54</td><td>6.71</td><td>17.84</td><td>4.75</td><td>59.74</td><td><b>100.00</b></td><td>53.68</td><td>60.07</td><td>20.00</td><td>34.00</td>
  </tr>
  <tr>
    <td>MiniCPM-V_2_6</td><td>0.00</td><td>0.00</td><td>0.61</td><td>6.71</td><td>37.80</td><td>0.00</td><td>0.82</td><td>0.49</td><td>38.30</td><td>35.68</td><td>37.64</td><td>39.44</td><td>12.00</td><td>8.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td><td>19.00</td><td>64.63</td><td>23.78</td><td>73.17</td><td>84.15</td><td>40.75</td><td>67.43</td><td>9.66</td><td>83.80</td><td>98.04</td><td>84.62</td><td>81.83</td><td>62.00</td><td>74.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td><td>2.00</td><td>0.00</td><td>22.56</td><td>39.02</td><td>59.15</td><td>1.47</td><td>1.47</td><td>12.27</td><td>62.19</td><td>65.47</td><td>59.57</td><td>28.81</td><td>26.00</td><td>42.00</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td><td>4.00</td><td>46.95</td><td>57.32</td><td>53.05</td><td>61.59</td><td><u>79.05</u></td><td>79.38</td><td>70.83</td><td>75.61</td><td>71.36</td><td>75.29</td><td>40.00</td><td>54.00</td><td>54.00</td>
  </tr>
</table>

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="10">McEval (50)</th>
  </tr>
  <tr>
    <th>CPP</th><th>Fortran</th><th>HTML</th><th>Java</th><th>JS</th><th>Pascal</th><th>Perl</th><th>PHP</th><th>Python</th><th>Ruby</th><th>Shell</th><th>Tcl</th><th>VB</th><th>Avg</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td><td>52.00</td><td>26.00</td><td>12.00</td><td>30.19</td><td>72.00</td><td>52.00</td><td>72.00</td><td>70.00</td><td>80.00</td><td>80.00</td><td>50.00</td><td><b>78.00</b></td><td>80.00</td><td>60.73</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td><td>22.00</td><td>2.00</td><td>0.00</td><td>18.87</td><td>34.00</td><td>0.00</td><td>26.00</td><td>28.00</td><td>32.00</td><td>34.00</td><td>8.00</td><td>4.00</td><td>18.00</td><td>18.69</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td><td><b>72.00</b></td><td>30.00</td><td>20.00</td><td><b>15.09</b></td><td><b>78.00</b></td><td><b>68.00</b></td><td>84.00</td><td>88.00</td><td>86.00</td><td>44.00</td><td>74.00</td><td><b>82.00</b></td><td><b>70.98</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td><td>50.00</td><td>24.00</td><td>16.00</td><td>45.28</td><td>56.00</td><td>38.00</td><td>46.00</td><td>70.00</td><td>52.00</td><td>76.00</td><td>22.00</td><td>32.00</td><td>66.00</td><td>44.17</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>48.00</td><td>40.00</td><td>22.00</td><td><u>28.30</u></td><td>72.00</td><td>54.00</td><td><b>74.00</b></td><td><b>92.00</b></td><td><b>90.00</b></td><td><b>90.00</b></td><td>44.00</td><td>68.00</td><td><b>86.00</b></td><td>63.05</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td><td>36.00</td><td>12.00</td><td>8.00</td><td>26.42</td><td>42.00</td><td>10.00</td><td>26.00</td><td>28.00</td><td>28.00</td><td>36.00</td><td>14.00</td><td>10.00</td><td>30.00</td><td>27.77</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td><td>48.00</td><td><b>48.00</b></td><td>16.00</td><td>52.83</td><td>56.00</td><td>38.00</td><td>66.00</td><td>64.00</td><td>70.00</td><td>70.00</td><td>46.00</td><td>56.00</td><td>72.00</td><td>56.19</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td><td>8.00</td><td>0.00</td><td>0.00</td><td>9.43</td><td>20.00</td><td>2.00</td><td>6.00</td><td>20.00</td><td>36.00</td><td>56.00</td><td>4.00</td><td>4.00</td><td>12.00</td><td>12.78</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td><td>24.00</td><td>20.00</td><td>8.00</td><td>58.49</td><td>48.00</td><td>24.00</td><td>38.00</td><td>36.00</td><td>54.00</td><td>58.00</td><td>34.00</td><td>22.00</td><td>44.00</td><td>35.24</td>
  </tr>
  <tr>
    <td>MiniCPM-V_2_6</td><td>12.00</td><td>0.00</td><td>0.00</td><td>13.00</td><td>22.00</td><td>2.00</td><td>10.00</td><td>36.00</td><td>28.00</td><td>18.00</td><td>18.00</td><td>16.00</td><td>6.00</td><td>16.43</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td><td>60.00</td><td>32.00</td><td><b>26.00</b></td><td><b>66.04</b></td><td>66.00</td><td>46.00</td><td>60.00</td><td>78.00</td><td>66.00</td><td>80.00</td><td><b>52.00</b></td><td>56.00</td><td>58.00</td><td>59.81</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td><td>30.00</td><td>2.00</td><td>10.00</td><td>22.48</td><td>48.00</td><td>4.00</td><td>30.00</td><td>48.00</td><td>44.00</td><td>42.00</td><td>24.00</td><td>12.00</td><td>30.00</td><td>29.12</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td><td>42.00</td><td>16.00</td><td>4.00</td><td>15.09</td><td>52.00</td><td>10.00</td><td>38.00</td><td>56.00</td><td>42.00</td><td>56.00</td><td>32.00</td><td>20.00</td><td>52.00</td><td>47.79</td>
  </tr>
</table>

Table 6: Pass@1 results for 13 LLMs on the pseudocode flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th>Avg</th>
  </tr>
  <tr>
    <th>Python</th><th>CPP</th><th>Java</th><th>JS</th>
    <th>Python</th><th>CPP</th><th>C#</th><th>Java</th><th>JS</th><th>PHP</th><th>Python</th><th>Ruby</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td><td>33.60</td><td>45.24</td><td>56.10</td><td>49.27</td><td>59.02</td><td>58.64</td><td>63.24</td><td>60.03</td><td>59.07</td><td><b>100.00</b></td><td>68.56</td><td>59.20</td><td>59.33</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td><td>3.40</td><td>3.72</td><td>32.26</td><td>54.70</td><td>65.61</td><td>35.30</td><td>29.30</td><td>27.99</td><td>71.05</td><td>99.12</td><td>69.84</td><td>72.98</td><td>47.15</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td><td><b>73.90</b></td><td><b>90.67</b></td><td><b>93.54</b></td><td><b>90.85</b></td><td><b>91.65</b></td><td><b>92.39</b></td><td><b>89.80</b></td><td><b>92.08</b></td><td><b>94.53</b></td><td>96.15</td><td><b>95.22</b></td><td><b>96.63</b></td><td><b>91.77</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td><td>20.70</td><td>61.46</td><td>65.55</td><td>80.55</td><td>76.77</td><td>78.33</td><td>72.68</td><td>77.94</td><td>84.14</td><td>98.81</td><td>81.85</td><td>86.84</td><td>73.90</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>64.10</td><td>57.99</td><td>91.22</td><td>89.27</td><td>89.09</td><td>83.29</td><td>85.16</td><td>85.17</td><td>90.18</td><td><b>100.00</b></td><td>93.29</td><td>95.27</td><td>85.33</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td><td>21.90</td><td>24.94</td><td>30.49</td><td>65.37</td><td>67.68</td><td>46.25</td><td>28.43</td><td>31.11</td><td>69.38</td><td>99.98</td><td>74.12</td><td>79.26</td><td>53.24</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td><td>37.70</td><td>60.98</td><td>75.06</td><td>83.05</td><td>86.83</td><td>78.90</td><td>78.87</td><td>67.58</td><td>85.09</td><td>99.66</td><td>88.72</td><td>92.03</td><td>77.90</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td><td>0.60</td><td>0.00</td><td>10.06</td><td>9.15</td><td>16.77</td><td>0.00</td><td>0.00</td><td>5.06</td><td>33.57</td><td>88.71</td><td>19.44</td><td>28.72</td><td>17.67</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td><td>10.60</td><td>17.68</td><td>35.67</td><td>46.65</td><td>50.55</td><td>37.92</td><td>28.27</td><td>34.65</td><td>57.68</td><td><b>100.00</b></td><td>58.25</td><td>64.71</td><td>45.22</td>
  </tr>
  <tr>
    <td>MiniCPM-V_2_6</td><td>3.00</td><td>1.10</td><td>22.50</td><td>35.18</td><td>44.57</td><td>12.93</td><td>19.39</td><td>9.95</td><td>58.92</td><td>96.73</td><td>52.70</td><td>70.61</td><td>35.88</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td><td>50.90</td><td>62.74</td><td>77.56</td><td>79.02</td><td>81.71</td><td>81.46</td><td>76.50</td><td>75.58</td><td>86.25</td><td><b>100.00</b></td><td>86.46</td><td>90.33</td><td>79.04</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td><td>20.40</td><td>4.39</td><td>59.94</td><td>67.44</td><td>77.20</td><td>20.90</td><td>9.17</td><td>33.88</td><td>75.22</td><td>88.90</td><td>71.42</td><td>51.82</td><td>49.11</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td><td>36.30</td><td>67.01</td><td>82.68</td><td>83.11</td><td>86.95</td><td><u>91.67</u></td><td><u>90.57</u></td><td>91.00</td><td>90.29</td><td>92.00</td><td>88.66</td><td>92.18</td><td>82.74</td>
  </tr>
</table>

Table 7: Pass@3 results for 13 LLMs on the code flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th rowspan="2">Avg</th>
  </tr>
  <tr>
    <th>Python</th>
    <th>CPP</th>
    <th>Java</th>
    <th>JS</th>
    <th>Python</th>
    <th>CPP</th>
    <th>C#</th>
    <th>Java</th>
    <th>JS</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>30.70</td>
    <td>36.59</td>
    <td>51.71</td>
    <td>28.66</td>
    <td>61.59</td>
    <td>60.98</td>
    <td>66.22</td>
    <td>63.96</td>
    <td>59.87</td>
    <td><b>100.00</b></td>
    <td>67.56</td>
    <td>61.42</td>
    <td>57.44</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>8.50</td>
    <td>7.38</td>
    <td>28.78</td>
    <td>61.77</td>
    <td>61.04</td>
    <td>43.65</td>
    <td>29.31</td>
    <td>26.94</td>
    <td>68.92</td>
    <td>98.59</td>
    <td>65.55</td>
    <td>73.26</td>
    <td>47.88</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>67.40</b></td>
    <td><b>78.60</b></td>
    <td><b>83.72</b></td>
    <td>89.57</td>
    <td><b>88.84</b></td>
    <td>90.07</td>
    <td><b>91.33</b></td>
    <td>86.99</td>
    <td><b>92.50</b></td>
    <td>91.90</td>
    <td><b>93.76</b></td>
    <td><b>95.02</b></td>
    <td><b>88.15</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>18.30</td>
    <td>57.56</td>
    <td>63.54</td>
    <td>82.99</td>
    <td>78.23</td>
    <td>79.77</td>
    <td>69.87</td>
    <td>76.74</td>
    <td>83.26</td>
    <td>99.15</td>
    <td>79.38</td>
    <td>85.73</td>
    <td>72.95</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>60.30</td>
    <td>60.00</td>
    <td>80.91</td>
    <td><b>94.82</b></td>
    <td>88.11</td>
    <td>83.99</td>
    <td>87.35</td>
    <td>84.35</td>
    <td>88.92</td>
    <td>99.98</td>
    <td>92.23</td>
    <td>94.81</td>
    <td>84.65</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td>
    <td>18.00</td>
    <td>23.11</td>
    <td>34.21</td>
    <td>73.96</td>
    <td>63.54</td>
    <td>51.03</td>
    <td>37.25</td>
    <td>42.18</td>
    <td>70.38</td>
    <td>99.74</td>
    <td>72.19</td>
    <td>74.75</td>
    <td>55.05</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>44.00</td>
    <td>48.17</td>
    <td>74.02</td>
    <td>86.65</td>
    <td>81.10</td>
    <td>78.23</td>
    <td>80.16</td>
    <td>70.28</td>
    <td>85.22</td>
    <td>99.77</td>
    <td>87.73</td>
    <td>91.00</td>
    <td>77.21</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>0.60</td>
    <td>0.00</td>
    <td>5.37</td>
    <td>17.38</td>
    <td>12.80</td>
    <td>0.20</td>
    <td>0.00</td>
    <td>5.97</td>
    <td>33.27</td>
    <td>87.87</td>
    <td>18.25</td>
    <td>25.66</td>
    <td>17.28</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>10.90</td>
    <td>13.17</td>
    <td>33.54</td>
    <td>51.52</td>
    <td>50.12</td>
    <td>31.10</td>
    <td>29.48</td>
    <td>36.76</td>
    <td>56.43</td>
    <td><b>100.00</b></td>
    <td>57.12</td>
    <td>62.64</td>
    <td>44.40</td>
  </tr>
  <tr>
    <td>MiniCPM-V-2_6</td>
    <td>5.30</td>
    <td>3.05</td>
    <td>19.88</td>
    <td>53.66</td>
    <td>43.35</td>
    <td>23.40</td>
    <td>29.85</td>
    <td>4.12</td>
    <td>56.55</td>
    <td>97.94</td>
    <td>55.34</td>
    <td>69.95</td>
    <td>38.69</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>44.10</td>
    <td>58.05</td>
    <td>76.71</td>
    <td>78.90</td>
    <td>78.29</td>
    <td>81.26</td>
    <td>76.07</td>
    <td>77.58</td>
    <td>85.24</td>
    <td>99.93</td>
    <td>84.39</td>
    <td>91.00</td>
    <td>77.63</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>20.60</td>
    <td>10.06</td>
    <td>51.10</td>
    <td>66.16</td>
    <td>67.87</td>
    <td>50.98</td>
    <td>13.85</td>
    <td>21.19</td>
    <td>73.26</td>
    <td>91.62</td>
    <td>73.58</td>
    <td>57.87</td>
    <td>50.32</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>30.00</td>
    <td>65.43</td>
    <td>80.79</td>
    <td>71.52</td>
    <td>79.45</td>
    <td><u>91.72</u></td>
    <td><u>90.36</u></td>
    <td><u>90.07</u></td>
    <td>88.71</td>
    <td>93.08</td>
    <td>83.11</td>
    <td>91.41</td>
    <td>79.68</td>
  </tr>
</table>

Table 8: Pass@3 results for 13 LLMs on the UML flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th rowspan="2">Avg</th>
  </tr>
  <tr>
    <th>Python</th>
    <th>CPP</th>
    <th>Java</th>
    <th>JS</th>
    <th>Python</th>
    <th>CPP</th>
    <th>C#</th>
    <th>Java</th>
    <th>JS</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>3.00</td>
    <td>71.10</td>
    <td>79.76</td>
    <td>50.61</td>
    <td>81.89</td>
    <td>62.90</td>
    <td>73.00</td>
    <td>69.13</td>
    <td>84.55</td>
    <td><b>100.00</b></td>
    <td>86.24</td>
    <td>85.30</td>
    <td>70.62</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>1.20</td>
    <td>3.29</td>
    <td>8.84</td>
    <td>43.66</td>
    <td>53.29</td>
    <td>5.94</td>
    <td>3.85</td>
    <td>7.86</td>
    <td>65.74</td>
    <td>87.69</td>
    <td>63.42</td>
    <td>67.09</td>
    <td>35.07</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>52.50</b></td>
    <td><b>87.13</b></td>
    <td><b>84.51</b></td>
    <td><b>87.32</b></td>
    <td><b>88.48</b></td>
    <td><b>58.81</b></td>
    <td><b>83.06</b></td>
    <td><b>88.63</b></td>
    <td><b>92.90</b></td>
    <td>79.59</td>
    <td><b>94.60</b></td>
    <td><b>89.36</b></td>
    <td><b>83.94</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>1.00</td>
    <td>31.40</td>
    <td>12.20</td>
    <td>31.71</td>
    <td>76.28</td>
    <td>36.37</td>
    <td>25.52</td>
    <td>8.43</td>
    <td>78.09</td>
    <td>76.01</td>
    <td>69.53</td>
    <td>81.33</td>
    <td>45.98</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>18.00</td>
    <td>79.02</td>
    <td>79.21</td>
    <td>50.55</td>
    <td><b>91.10</b></td>
    <td>80.77</td>
    <td>77.66</td>
    <td>69.44</td>
    <td>87.86</td>
    <td>99.93</td>
    <td>92.05</td>
    <td>88.59</td>
    <td>76.19</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td>
    <td>13.60</td>
    <td>0.37</td>
    <td>8.72</td>
    <td>55.18</td>
    <td>65.55</td>
    <td>4.17</td>
    <td>21.59</td>
    <td>12.14</td>
    <td>69.12</td>
    <td>98.77</td>
    <td>69.59</td>
    <td>72.08</td>
    <td>41.00</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>33.00</td>
    <td>67.80</td>
    <td>16.34</td>
    <td>83.54</td>
    <td>83.23</td>
    <td>50.54</td>
    <td>68.41</td>
    <td>17.04</td>
    <td>83.65</td>
    <td>93.06</td>
    <td>86.45</td>
    <td>86.38</td>
    <td>64.70</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>0.00</td>
    <td>0.00</td>
    <td>7.74</td>
    <td>22.26</td>
    <td>41.46</td>
    <td>0.00</td>
    <td>0.00</td>
    <td>3.34</td>
    <td>52.37</td>
    <td>50.00</td>
    <td>38.67</td>
    <td>48.90</td>
    <td>22.13</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>9.30</td>
    <td>10.49</td>
    <td>31.34</td>
    <td>53.84</td>
    <td>69.51</td>
    <td>10.74</td>
    <td>35.34</td>
    <td>14.27</td>
    <td>67.91</td>
    <td><b>100.00</b></td>
    <td>65.86</td>
    <td>74.06</td>
    <td>45.22</td>
  </tr>
  <tr>
    <td>MiniCPM-V-2_6</td>
    <td>1.50</td>
    <td>0.00</td>
    <td>1.71</td>
    <td>16.16</td>
    <td>44.33</td>
    <td>0.10</td>
    <td>1.59</td>
    <td>0.88</td>
    <td>56.60</td>
    <td>62.19</td>
    <td>53.32</td>
    <td>54.75</td>
    <td>27.49</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>21.10</td>
    <td>69.09</td>
    <td>30.12</td>
    <td>75.85</td>
    <td>86.52</td>
    <td>43.73</td>
    <td>72.27</td>
    <td>15.14</td>
    <td>86.56</td>
    <td>99.31</td>
    <td>87.33</td>
    <td>84.75</td>
    <td>64.37</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>4.80</td>
    <td>0.37</td>
    <td>39.09</td>
    <td>58.48</td>
    <td>73.72</td>
    <td>2.55</td>
    <td>3.63</td>
    <td>29.35</td>
    <td>78.85</td>
    <td>82.06</td>
    <td>79.60</td>
    <td>50.62</td>
    <td>42.90</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>9.00</td>
    <td>61.59</td>
    <td>73.90</td>
    <td>64.09</td>
    <td>79.63</td>
    <td><b>87.74</b></td>
    <td><b>89.75</b></td>
    <td><b>87.04</b></td>
    <td>88.99</td>
    <td>85.40</td>
    <td>88.35</td>
    <td>86.84</td>
    <td>75.22</td>
  </tr>
</table>

Table 9: Pass@3 results for 13 LLMs on the pseudocode flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th rowspan="2">Avg</th>
  </tr>
  <tr>
    <th>Python</th>
    <th>CPP</th>
    <th>Java</th>
    <th>JS</th>
    <th>Python</th>
    <th>CPP</th>
    <th>C#</th>
    <th>Java</th>
    <th>JS</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>37.00</td>
    <td>50.61</td>
    <td>58.54</td>
    <td>53.05</td>
    <td>61.59</td>
    <td>63.50</td>
    <td>66.78</td>
    <td>63.67</td>
    <td>62.68</td>
    <td><b>100.00</b></td>
    <td>71.85</td>
    <td>63.18</td>
    <td>62.70</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>4.00</td>
    <td>4.88</td>
    <td>42.07</td>
    <td>60.98</td>
    <td>71.34</td>
    <td>47.14</td>
    <td>39.61</td>
    <td>38.46</td>
    <td>78.07</td>
    <td>99.84</td>
    <td>76.43</td>
    <td>80.20</td>
    <td>53.60</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>75.00</b></td>
    <td><b>92.07</b></td>
    <td><b>93.90</b></td>
    <td><b>91.46</b></td>
    <td><b>92.07</b></td>
    <td>93.13</td>
    <td>91.16</td>
    <td>92.96</td>
    <td><b>95.25</b></td>
    <td>98.04</td>
    <td><b>95.58</b></td>
    <td><b>97.22</b></td>
    <td><b>92.48</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>22.00</td>
    <td>64.02</td>
    <td>66.46</td>
    <td>81.10</td>
    <td>77.44</td>
    <td>78.89</td>
    <td>73.65</td>
    <td>78.56</td>
    <td>84.62</td>
    <td>99.02</td>
    <td>82.65</td>
    <td>87.40</td>
    <td>74.73</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>66.00</td>
    <td>67.07</td>
    <td>92.68</td>
    <td>90.24</td>
    <td>90.24</td>
    <td>86.91</td>
    <td>87.40</td>
    <td>87.89</td>
    <td>91.49</td>
    <td><b>100.00</b></td>
    <td>94.27</td>
    <td>96.40</td>
    <td>87.55</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-8B-MPO</td>
    <td>26.00</td>
    <td>29.27</td>
    <td>35.37</td>
    <td>67.68</td>
    <td>69.51</td>
    <td>51.55</td>
    <td>33.88</td>
    <td>35.84</td>
    <td>72.67</td>
    <td><b>100.00</b></td>
    <td>75.94</td>
    <td>81.67</td>
    <td>56.62</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>41.00</td>
    <td>64.02</td>
    <td>78.05</td>
    <td>84.76</td>
    <td>87.80</td>
    <td>80.85</td>
    <td>70.87</td>
    <td>86.58</td>
    <td>99.67</td>
    <td>98.01</td>
    <td>63.01</td>
    <td>69.07</td>
    <td>49.63</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>1.00</td>
    <td>0.00</td>
    <td>12.80</td>
    <td>13.41</td>
    <td>20.12</td>
    <td>0.00</td>
    <td>0.00</td>
    <td>7.36</td>
    <td>40.43</td>
    <td>93.78</td>
    <td>24.22</td>
    <td>35.19</td>
    <td>20.69</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>12.00</td>
    <td>21.95</td>
    <td>39.63</td>
    <td>50.61</td>
    <td>54.88</td>
    <td>44.84</td>
    <td>34.70</td>
    <td>43.37</td>
    <td>61.54</td>
    <td><b>100.00</b></td>
    <td>63.01</td>
    <td>69.07</td>
    <td>49.63</td>
  </tr>
  <tr>
    <td>MiniCPM-V-2_6</td>
    <td>3.00</td>
    <td>1.83</td>
    <td>26.83</td>
    <td>41.46</td>
    <td>48.17</td>
    <td>17.35</td>
    <td>25.20</td>
    <td>14.89</td>
    <td>63.83</td>
    <td>98.20</td>
    <td>57.12</td>
    <td>76.10</td>
    <td>39.64</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>55.00</td>
    <td>65.24</td>
    <td>79.27</td>
    <td>80.49</td>
    <td>82.93</td>
    <td>83.14</td>
    <td>78.40</td>
    <td>78.23</td>
    <td><b>100.00</b></td>
    <td>87.73</td>
    <td>91.00</td>
    <td>80.72</td>
    <td></td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>26.00</td>
    <td>7.32</td>
    <td>68.90</td>
    <td>71.95</td>
    <td>81.71</td>
    <td>29.79</td>
    <td>13.09</td>
    <td>46.15</td>
    <td>80.69</td>
    <td>94.27</td>
    <td>76.60</td>
    <td>61.05</td>
    <td>55.20</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>41.00</td>
    <td>71.34</td>
    <td>85.98</td>
    <td>84.76</td>
    <td>89.63</td>
    <td><b>93.45</b></td>
    <td><b>91.98</b></td>
    <td><b>92.14</b></td>
    <td>92.80</td>
    <td>91.00</td>
    <td>93.62</td>
    <td>85.12</td>
    <td></td>
  </tr>
</table>

Table 10: Pass@5 results for 13 LLMs on the code flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th rowspan="2">Avg</th>
  </tr>
  <tr>
    <th>Python</th>
    <th>CPP</th>
    <th>Java</th>
    <th>JS</th>
    <th>Python</th>
    <th>CPP</th>
    <th>C#</th>
    <th>Java</th>
    <th>JS</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>35.00</td>
    <td>40.24</td>
    <td>55.49</td>
    <td>31.10</td>
    <td>63.41</td>
    <td>64.98</td>
    <td>70.38</td>
    <td>67.76</td>
    <td>63.99</td>
    <td><b>100.00</b></td>
    <td>70.87</td>
    <td>64.98</td>
    <td>60.68</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>11.00</td>
    <td>10.37</td>
    <td>36.59</td>
    <td>66.46</td>
    <td>66.46</td>
    <td>53.19</td>
    <td>39.12</td>
    <td>36.50</td>
    <td>77.25</td>
    <td>99.51</td>
    <td>71.52</td>
    <td>80.03</td>
    <td>54.03</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>68.00</b></td>
    <td><b>81.10</b></td>
    <td><b>84.76</b></td>
    <td><b>90.24</b></td>
    <td><b>90.24</b></td>
    <td><b>90.83</b></td>
    <td><b>92.14</b></td>
    <td><b>88.87</b></td>
    <td><b>93.13</b></td>
    <td>94.44</td>
    <td><b>94.27</b></td>
    <td><b>95.58</b></td>
    <td><b>89.10</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>19.00</td>
    <td>60.37</td>
    <td>64.63</td>
    <td>83.54</td>
    <td>79.27</td>
    <td>80.52</td>
    <td>71.36</td>
    <td>77.74</td>
    <td>84.12</td>
    <td>99.35</td>
    <td>80.20</td>
    <td>85.92</td>
    <td>73.89</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>64.00</td>
    <td>67.68</td>
    <td>83.54</td>
    <td><b>96.34</b></td>
    <td>89.63</td>
    <td>87.23</td>
    <td>89.53</td>
    <td>86.74</td>
    <td>90.83</td>
    <td><b>100.00</b></td>
    <td>93.29</td>
    <td><b>96.07</b></td>
    <td>87.07</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-BB-MPO</td>
    <td>21.00</td>
    <td>27.44</td>
    <td>37.80</td>
    <td>76.83</td>
    <td>67.68</td>
    <td>55.97</td>
    <td>43.21</td>
    <td>48.12</td>
    <td>74.80</td>
    <td>99.84</td>
    <td>75.45</td>
    <td>77.09</td>
    <td>58.78</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>47.00</td>
    <td>50.00</td>
    <td>76.22</td>
    <td>87.80</td>
    <td>82.32</td>
    <td>80.69</td>
    <td>82.49</td>
    <td>73.16</td>
    <td>87.07</td>
    <td>99.84</td>
    <td>89.03</td>
    <td>91.98</td>
    <td>78.98</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>1.00</td>
    <td>0.00</td>
    <td>7.93</td>
    <td>22.56</td>
    <td>15.24</td>
    <td>0.33</td>
    <td>0.00</td>
    <td>8.67</td>
    <td>40.10</td>
    <td>93.78</td>
    <td>21.93</td>
    <td>31.75</td>
    <td>20.27</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>12.00</td>
    <td>16.46</td>
    <td>37.80</td>
    <td>56.71</td>
    <td>55.49</td>
    <td>36.01</td>
    <td>36.01</td>
    <td>45.01</td>
    <td>61.05</td>
    <td><b>100.00</b></td>
    <td>61.21</td>
    <td>67.92</td>
    <td>48.81</td>
  </tr>
  <tr>
    <td>MiniCPM-V2_6</td>
    <td>7.00</td>
    <td>3.05</td>
    <td>23.78</td>
    <td>58.54</td>
    <td>48.17</td>
    <td>28.31</td>
    <td>36.17</td>
    <td>6.22</td>
    <td>62.19</td>
    <td>99.02</td>
    <td>60.07</td>
    <td>75.29</td>
    <td>42.40</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>47.00</td>
    <td>61.59</td>
    <td>78.66</td>
    <td>79.88</td>
    <td>79.27</td>
    <td>82.65</td>
    <td>77.74</td>
    <td>79.38</td>
    <td>86.42</td>
    <td><b>100.00</b></td>
    <td>85.60</td>
    <td>91.65</td>
    <td>79.15</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>25.00</td>
    <td>13.41</td>
    <td>60.98</td>
    <td>70.12</td>
    <td>71.95</td>
    <td>59.57</td>
    <td>19.15</td>
    <td>30.44</td>
    <td>77.58</td>
    <td>95.91</td>
    <td>79.05</td>
    <td>66.45</td>
    <td>56.09</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>32.00</td>
    <td>69.51</td>
    <td><b>84.76</b></td>
    <td>74.39</td>
    <td>84.76</td>
    <td><b>93.62</b></td>
    <td><b>92.14</b></td>
    <td><b>90.67</b></td>
    <td>95.09</td>
    <td>86.58</td>
    <td>93.13</td>
    <td>82.45</td>
    <td></td>
  </tr>
</table>

Table 11: Pass@5 results for 13 LLMs on the UML flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">ClassEval (100)</th>
    <th colspan="4">HumanEval-X (164)</th>
    <th colspan="4">MBXP (611)</th>
    <th rowspan="2">Avg</th>
  </tr>
  <tr>
    <th>Python</th>
    <th>CPP</th>
    <th>Java</th>
    <th>JS</th>
    <th>Python</th>
    <th>CPP</th>
    <th>C#</th>
    <th>Java</th>
    <th>JS</th>
    <th>PHP</th>
    <th>Python</th>
    <th>Ruby</th>
  </tr>
  <tr>
    <td>Claude-3.5-Sonnet</td>
    <td>3.00</td>
    <td>73.17</td>
    <td>81.71</td>
    <td>52.44</td>
    <td>84.15</td>
    <td>66.78</td>
    <td>74.30</td>
    <td>75.12</td>
    <td>85.92</td>
    <td><b>100.00</b></td>
    <td>88.22</td>
    <td>86.58</td>
    <td>72.62</td>
  </tr>
  <tr>
    <td>DeepSeek-VL2</td>
    <td>2.00</td>
    <td>5.49</td>
    <td>13.41</td>
    <td>53.05</td>
    <td>59.15</td>
    <td>9.17</td>
    <td>5.89</td>
    <td>12.27</td>
    <td>73.98</td>
    <td>95.25</td>
    <td>70.38</td>
    <td>74.30</td>
    <td>39.86</td>
  </tr>
  <tr>
    <td>Gemini-2.0</td>
    <td><b>55.00</b></td>
    <td><b>89.63</b></td>
    <td><b>87.20</b></td>
    <td><b>88.41</b></td>
    <td><b>89.02</b></td>
    <td><b>62.68</b></td>
    <td><b>83.80</b></td>
    <td><b>90.67</b></td>
    <td><b>93.45</b></td>
    <td>80.52</td>
    <td><b>94.93</b></td>
    <td><b>90.02</b></td>
    <td><b>85.40</b></td>
  </tr>
  <tr>
    <td>GLM-4V-plus</td>
    <td>1.00</td>
    <td>33.54</td>
    <td>15.24</td>
    <td>32.32</td>
    <td>76.83</td>
    <td>39.28</td>
    <td>27.17</td>
    <td>9.66</td>
    <td>78.56</td>
    <td>77.58</td>
    <td>70.21</td>
    <td>81.67</td>
    <td>47.12</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>21.00</td>
    <td><u>82.93</u></td>
    <td><u>84.76</u></td>
    <td><u>55.49</u></td>
    <td><u>92.07</u></td>
    <td><u>84.94</u></td>
    <td><u>80.36</u></td>
    <td><u>78.56</u></td>
    <td><u>89.20</u></td>
    <td><b>100.00</b></td>
    <td>93.29</td>
    <td>89.53</td>
    <td>79.34</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-BB-MPO</td>
    <td>16.00</td>
    <td>0.61</td>
    <td>10.37</td>
    <td>60.37</td>
    <td>69.51</td>
    <td>5.40</td>
    <td>25.53</td>
    <td>17.18</td>
    <td>71.36</td>
    <td>99.18</td>
    <td>72.50</td>
    <td>75.12</td>
    <td>43.66</td>
  </tr>
  <tr>
    <td>Intern-VL2.5-78B-MPO</td>
    <td>36.00</td>
    <td>70.73</td>
    <td>20.12</td>
    <td><u>85.98</u></td>
    <td>84.15</td>
    <td>53.68</td>
    <td>70.70</td>
    <td>23.57</td>
    <td>84.94</td>
    <td>95.74</td>
    <td>87.40</td>
    <td>87.56</td>
    <td>67.07</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-7B</td>
    <td>1.00</td>
    <td>0.00</td>
    <td>11.59</td>
    <td>30.49</td>
    <td>48.17</td>
    <td>0.00</td>
    <td>0.00</td>
    <td>4.91</td>
    <td>59.41</td>
    <td>61.05</td>
    <td>44.03</td>
    <td>57.12</td>
    <td>26.48</td>
  </tr>
  <tr>
    <td>LLaVA-OneVision-72B</td>
    <td>12.00</td>
    <td>14.63</td>
    <td>41.46</td>
    <td>57.93</td>
    <td>75.00</td>
    <td>13.42</td>
    <td>42.39</td>
    <td>21.11</td>
    <td>70.87</td>
    <td><b>100.00</b></td>
    <td>70.38</td>
    <td>78.23</td>
    <td>49.79</td>
  </tr>
  <tr>
    <td>MiniCPM-V2_6</td>
    <td>2.00</td>
    <td>0.00</td>
    <td>2.44</td>
    <td>21.95</td>
    <td>49.39</td>
    <td>0.16</td>
    <td>2.29</td>
    <td>1.47</td>
    <td>63.67</td>
    <td>72.18</td>
    <td>59.41</td>
    <td>60.88</td>
    <td>30.31</td>
  </tr>
  <tr>
    <td>Qwen2-VL-72B</td>
    <td>22.00</td>
    <td>70.73</td>
    <td>32.93</td>
    <td>76.83</td>
    <td>87.20</td>
    <td>44.68</td>
    <td>73.98</td>
    <td>17.68</td>
    <td>87.23</td>
    <td>99.51</td>
    <td>88.05</td>
    <td>85.60</td>
    <td>65.58</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B</td>
    <td>6.00</td>
    <td>0.61</td>
    <td>47.56</td>
    <td>65.24</td>
    <td>78.05</td>
    <td>3.60</td>
    <td>5.89</td>
    <td>40.75</td>
    <td>82.65</td>
    <td>87.89</td>
    <td>80.03</td>
    <td>60.39</td>
    <td>47.50</td>
  </tr>
  <tr>
    <td>Qwen2-VL-7B-FT</td>
    <td>11.00</td>
    <td>67.68</td>
    <td>78.05</td>
    <td>69.51</td>
    <td>85.37</td>
    <td><b>89.69</b></td>
    <td><b>91.98</b></td>
    <td><b>89.85</b></td>
    <td><u>91.82</u></td>
    <td>87.23</td>
    <td>91.82</td>
    <td><b>89.85</b></td>
    <td>78.70</td>
  </tr>
</table>

Table 12: Pass@5 results for 13 LLMs on the pseudocode flowchart generation task in the Flow2Code benchmarks. The results in bold are the optimal results, while the underlined results represent the suboptimal results. The results are represented in percentage (%).