# An Empirical Study of the Code Generation of Safety-Critical Software Using LLMs
## 6 Discussions
### Prompt Engineering for Code Generation of Safety-Critical Software  
From the perspective of code generation by LLMs, the above experiments indicate that the most crucial aspect is to provide the most appropriate prompts for the models. This paper conducts experiments combining two mainstream prompt methods and proposes a new prompt method. The three prompt methods yield different effects, as shown in Table 4.  
By comparison, the augmented prompt achieves the best domain code generation results. The generated code size increases from 63 lines to 144 lines, the completeness of the generated code function improves from 30% to 100%, and the correctness of the generated code function improves from partially correct to completely correct. The code comment rate increases to 26.3%, and the static analysis errors and warnings decrease from 19 to 5 warnings. Furthermore, better results are achieved in terms of code compliance, code readability, and code usability. The intuitive comparison results are shown in Figure 3. This prompt engineering framework integrates basic functional requirements, domain feature generalization, and standard specification constraints. We refer to it as Prompt-FDC. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Comparison Item** & **LLMs Code Generation** & **SCADE Code Generation** \\ \hline Input & Software Requirements & Software Models \\ \hline Development Time & \(1\) h & 24 h \\ Investment & \(1\) h & 1000+ \\ \hline Code Size & 100+ & 1000+ \\ \hline Learning Curves & Almost no learning curve & Long-term specialized \\  & required & learning \\ \hline Verification & Comprehensive verification is & Model correctness implies \\  & required & code correctness; model needs \\  & & to be verified carefully \\ \hline Domain Compliance & Complies with standards & Complies with standards \\ \hline \hline \end{tabular} Note: cruise control software is an example.  
\end{table} 
Table 3: Code generation comparison between LLMs and SCADE. Different from other prompt methods, Prompt-FDC starts with the requirements of safety-critical software. This well-established development practice can also be applied to code generation based on large language models. This paper does not provide a detailed description of the requirement analysis process but starts with the requirement document and proposes the new prompt engineering framework, Prompt-FDC. As shown in Figure 4, the Prompt-FDC method can be divided into five phases, from the requirement document to prompts. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Comparison Item** & **Prompt-Simple [27; 28]** & **Prompt-Specific [7]** & \begin{tabular}{c} **Prompt-FDC** \\ **(Proposed in This Paper)** \\ \end{tabular} \\ \hline Input & \begin{tabular}{c} Prompt of overall \\ requirements \\ \end{tabular} & \begin{tabular}{c} Prompt of specific \\ requirements \\ \end{tabular} & \begin{tabular}{c} Prompt of augmented \\ requirements \\ \end{tabular} \\ \hline Code size & 63 & 116 & 144 \\ \hline Code comment rate & 11.1\% & 10\% & 26.3\% \\ \hline Completeness of generated code function & 30\%, function missing & 80\%, domain function missing & 100\%, complete function \\ \hline Correctness of generated code function & 60\%, partially correct & 80\%, partially correct & 100\%, completely correct \\ \hline Static analysis results * & Style warnings: 16 & \begin{tabular}{c} Errors: 4 \\ Style warnings: 15 \\ \end{tabular} & \
\begin{tabular}{c} Style warnings: 5 \\ \end{tabular} \\ \hline Code compliance & Non-compliant & Non-compliant &
\begin{tabular}{c} Compliant with \\ MISRA-C:2004 standard \\ \end{tabular} \\ \hline Code readability & Moderate & Moderate & High \\ \hline Code usability & Not directly usable & Not directly usable & Directly usable \\ \hline \hline \end{tabular}  
* Cruise control software is an example. Static analysis results are checked using Cppcheck (2.11) [41].  
\end{table} 
Table 4: Code generation Comparison between different Prompts. Figure 3: Code generation comparison between different prompts. Phase 1: Extraction.