# Optimizing Large Language Models: A Deep Dive into Effective Prompt Engineering Techniques
## 5 Simulation Result
### Winogrande  
Tables 14 and 15 present the performance of Llama3, Gemma2, and Mistral on the Winogrande dataset, evaluated using the BLEU, ROUGE, METEOR, BLEURT, and BERTScore metrics under various combinations of Prompt Engineering techniques. As observed in these tables, the Winogrande dataset focuses on contextual reasoning abilities concerning common-sense knowledge, where techniques such as ToT, CoT, ICL, and SSR generally result in strong performance. Furthermore, the optimal combination of these Prompt Engineering techniques varies depending on the LLM and the evaluation metric. For Llama3, a combination of ToT, CoT, and ICL techniques typically yields the best performance. For Gemma2, SSR combined with CoT tends to deliver the highest performance, while Mistral shows the best results when combining CoT and ICL techniques. Fundamentally, as seen in Tables 14 and 15, Gemma2 generally exhibits the best performance in four out of the five evaluation metrics when no Prompt Engineering techniques are applied. However, when applying a comprehensive Prompt Engineering strategy, Gemma2 outperforms both Llama3 and Mistral across all five evaluation metrics. This indicates that a well-implemented Prompt Engineering strategy can further amplify the strengths of an LLM. Following Gemma2, Llama3 demonstrates slightly better performance compared to Mistral. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Method** & **Llama3** & **Gemma2** & **Mistral** & **Method** & **Llama3** & **Gemma2** & **Mistral** & **Method** & **Llama3** & **Gemma2** & **Mistral** \\ \hline \multicolumn{8}{c}{**BLEU**} & \multicolumn{8}{c}{**ROUGE**} & \multicolumn{8}{c}{**METEOR**} \\ \hline
**B** & 6.1499 & 2.8829 & 0.5264 & **B** & 0.3990 & 0.5460 & 0.2813 & **B** & 0.2067 & 0.2858 & 0.1667 \\ \hline
**I** & 0.0000 & 21.1627 & 0.6066 & **I** & 0.3757 & 0.5440 & 0.2641 & **I** & 0.1908 & 0.2842 & 0.1466 \\ \hline
**C** & 0.0000 & 0.0000 & 9.1291 & **C** & 0.2126 & 0.6772 & 0.5354 & **C** & 0.1080 & 0.3428 & 0.2701 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Performances of BLUE, ROUGE, METEOR according to LLMs and Prompt Engineering techniques on the Winogrande dataset. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Method** & **Llama3** & **Gemma2** & **Mistral** & **Method** & **Llama3** & **Gemma2** & **Mistral** \\ \hline \multicolumn{8}{c}{**BLEURT**} & \multicolumn{8}{c}{**BERTScore**} \\ \hline
**T, C** & 0.2696 & 0.2542 & 0.3557 & **T, C** & 0.4861 & 0.4639 & 0.5800 \\ \hline
**T, C, I** & 0.4077 & 0.2837 & 0.3792 & **T, C, I** & 0.5991 & 0.5051 & 0.5881 \\ \hline
**T, R(B)** & 0.2981 & 0.2731 & 0.3361 & **T, R(B)** & 0.5103 & 0.4669 & 0.4832 \\ \hline
**T, R(C)** & 0.4508 & 0.3842 & 0.4214 & **T, R(C)** & 0.6359 & 0.5758 & 0.6142 \\ \hline
**S, B** & 0.2188 & 0.2662 & 0.3445 & **S, B** & 0.4113 & 0.4693 & 0.5326 \\ \hline
**S, I** & 0.1690 & 0.2573 & 0.3939 & **S, I** & 0.3438 & 0.4733 & 0.5579 \\ \hline
**S, C** & 0.2628 & 0.2946 & 0.3826 & **S, C** & 0.4861 & 0.5022 & 0.5912 \\ \hline
**S, C, I** & 0.3925 & 0.2969 & 0.3937 & **S, C, I** & 0.5993 & 0.4949 & 0.5548 \\ \hline
**S, R(B)** & 0.3241 & 0.3012 & 0.3635 & **S, R(B)** & 0.5216 & 0.5125 & 0.5694 \\ \hline
**S, R(C)** & 0.3792 & 0.3810 & 0.4153 & **S, R(C)** & 0.5851 & 0.5741 & 0.5923 \\ \hline
**S, T, B** & 0.2235 & 0.2464 & 0.3194 & **S, T, B** & 0.4207 & 0.4536 & 0.4880 \\ \hline
**S, T, I** & 0.3046 & 0.2281 & 0.3542 & **S, T, I** & 0.5106 & 0.4332 & 0.5607 \\ \hline
**S, T, C** & 0.2899 & 0.2719 & 0.3860 & **S, T, C** & 0.5093 & 0.4852 & 0.5937 \\ \hline
**S, T, C, I** & 0.4290 & 0.2913 & **0.4377** & **S, T, C, I** & 0.6248 & 0.5138 & 0.6190 \\ \hline
**S, T, R(B)** & 0.3130 & 0.2647 & 0.3480 & **S, T, R(B)** & 0.5110 & 0.4594 & 0.5684 \\ \hline
**S, T, R(C)** & **0.4631** & **0.3935** & 0.4285 & **S, T, R(C)** & **0.6401** & **0.5877** & **0.6232** \\ \hline \hline \end{tabular}
\end{table} 
Table 13: _Cont._