Code Generation From Flowcharts with Texts: A Benchmark Dataset and An Approach (2022)

Abstract: Currently, researchers focus on generating codes from the requirement documents. However, current approaches still perform poorly on some requirements needing complex problem-solving skills. In reality, to tackle such complex requirements, instead of directly translating requirement documents into codes, software engineers write codes via unified modeling language diagrams, such as flowcharts, an intermediate tool to analyze and visualize the system. Therefore, we propose a new source code generation task, that is, to generate source code from flowcharts with texts. We manually construct a benchmark dataset containing 320 flowcharts with their corresponding source codes. Obviously, it is not straightforward to employ the current approaches for the new source code generation task since (1) the flowchart is a graph that contains various structures, including loop, selection, and others which is different from texts; (2) the connections between nodes in the flowchart are abundant and diverse which need to be carefully handled. To solve the above problems, we propose a two-stage code generation model. In the first stage, a structure recognition algorithm is employed to transform the flowchart into pseudo-code containing the structural conventions of a typical programming language such as while, if. In the second stage, a code generation model is employed to convert the pseudo-code into code. Experimental results show that the proposed approach can achieve some improvement over the baselines.

## introduction recently, automated source code generation from requirements documents has been a fashionable task that maps specific descriptions to various executable codes in software engineering and artificial intelligence. developments in deep learning in practice, to solve complex code generation tasks, rather than translating requirements documents directly into code, software engineers write code through unified modelling language(uml), which intends to provide a standard way of visualizing the design of a system @xcite . for all uml, flowchart plays an important role in the analysis of system requirements, preliminary design, and detailed design @xcite . before writing code, drawing a flowchart to illustrate the algorithm to be used and the steps to follow can significantly reduce task's difficulty. there is evidence that, in industrial practice, flowcharts have been widely used for problem understanding (i.e. analysis) and documentation. thus, we propose a new task to generating executable code from a flowchart with text. there have also been several studies on converting flowcharts to code. @xcite proposed rule-based methods that can identify the loop and selection semantics in the flowchart and automatically convert it into pseudo-code which use structural conventions of a normal programming language such as while, if, but the details in the pseudo-code are still in text, which makes these codes unable to execute directly on the computer. in contrast, we tackle the task of executable code(python) generation from flowcharts with text. most closely related to our task is @xcite , they propose a task to generate code from pseudocode. however, each line of pseudo-code in their dataset's is independent of the other, models can generate code snippets for each line regardless of other lines. however, in our dataset, the connections between nodes in the flowchart are abundant and diverse which need to be carefully handled. thus, although flowcharts and pseudo-codes can be converted into each other, our dataset is different from theirs. we constructed a benchmark dataset, fc2code, which contains 320 flowcharts with natural language and code pairs. we obtained the code from the programming competition platform leetcode and manually drew the flowcharts with natural language. previous models cannot be used directly for the new source code generation task for the following reasons. firstly, a flowchart is a graph containing various structures, including loops and selections, which is different from the text. secondly, in a natural scene, each node of the flowchart is not independent. as seen from to solve these problems above, we propose a two-stage code generation model. in the first stage, the structure recognition model transforms the flowchart into a pseudo-code that containing some structure information such as while, if. in the second stage, a code generation mode is employed to merge the information in the node of the flowchart and convert the pseudo-code into executable code. the experiments show that it is necessary to enhance each node's representation with its neighbors according to the structure of the flowchart. ## dataset construction we created a new dataset called fc2code (flowchart to code), which consists of code and flowchart pairs. as can be seen in figure 1 . the basic process of dataset construction can be divided into 3 steps: 1) code extraction. 2) flowchart sketch generation. 3) nature language annotation. the following sections describe the above three steps in detail. code extraction. firstly, we extract 320 codes from leetcode 2 , an online programming competition platform. the code crawled in public available code repositories such as github often has projectrelated operations or global variables and the logic of some source code is often not clear enough, and even has bugs. thus, we extract the python codes from an online programming competition platform, leetcode. leetcode provides open-domain tasks with high-quality official solutions, each solution contains well-tested code with a detailed explanation. because each function in the code will be converted into a flowchart, we filter out the code containing multiple functions to make sure each flowchart contains complete information that can generate executable code. we select 253 problems with 320 codes from leetcode. flowchart sketch generation. secondly, we use pyflowchart 3 to automatically convert the code 2 https://leetcode.com/problemset/all/ 3 https://github.com/cdfmlr/pyflowchart/ into flowchart sketches. specifically speaking, each line of code is transformed into a node in the flowchart, then the tool automatically connects edges to each node according to their execution order in the code. note that the content of the nodes in the flowchart sketches is still code snippets. nature language annotation. thirdly, we translate each code in the node into natural languages in chinese. we set the following principle for annotators: annotating nodes in flowcharts is a timeconsuming process. labeling one flowchart takes us about 30 minutes of labor on average. and annotating 320 flowcharts costs us around 160 hours of human labor. to verify the quality of the annotation, we let annotators exchange their samples and infer the code based on the annotation results. finally, we sampled 50 flowcharts with 771 nodes, and find that 82.00% flowcharts and 98.44% nodes from our dataset are solvable. ## conclusion in this paper, we introduced the task of generating source code from flowcharts with texts. structure of the flowchart. and the ablation experiments further show that considering the direction of edges in the flowchart will improve the model's performance, and when fusing the information of neighbor nodes, compared with pseudo-code, calculating the distance of two nodes using a flowchart is better, which means that the flowchart is important for the second stage.