# Optimizing Large Language Models: A Deep Dive into Effective Prompt Engineering Techniques
## 2 Related Works
The Prompt Engineering techniques described above are summarized in Table 1.  
Lastly, there have been studies addressing issues related to model alignment, demonstration formatting, and the theoretical underpinnings of prompting using the aforementioned Prompt Engineering techniques [30; 31; 32]. The study by [30] proposed efficient prompt construction methods such as Instruction+Question, Instruction+Input, and Question+Answer. It also identified the limitations of LLMs and suggested strategies to overcome these limitations using CoT-based Prompt Engineering. Similarly, ref. [31] explored effective prompt construction techniques and discussed optimization strategies for LLMs through approaches such as CoT, Self-Consistency, General Knowledge, Least-to-Most \begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Techniques** & **Key Characteristics** \\ \hline ICL [13; 19; 20; 21; 22; 23] & Focuses on few-shot learning by providing demonstrations within prompts. Includes derivatives such as PICL (contextual data collection and reconstruction), MetaICL (continual training phase), KATE (demonstration example selection), SG-ICL (format of demonstrations), and GlobalE\&LocalE (demonstration order). \\ \hline CoT [30; 31] & Encourages intermediate reasoning steps for solving complex problems. Demonstrated to improve logical reasoning capabilities in LLMs. \\ \hline Self-Consistency [31] & Optimizes the model to produce consistent and reliable outputs across varied inputs conveying the same meaning. \\ \hline SSR [31] & Promotes systematic problem-solving by explicitly breaking down the reasoning process into individual steps. \\ \hline ToT [17; 31] & Represents reasoning as a tree structure with branches exploring diverse hypotheses or possibilities, aiding in logical exploration and plausibility evaluation. \\ \hline RAG [24; 25; 26; 27; 28; 29] & Addresses hallucination issues by retrieving external information (e.g., from knowledge graphs or search engines) and augmenting prompts with this information. Includes variants such as the LLM-Augmenter (retrieval before generation), Knowledge Retrieval (retrieval during generation), RARR (retrieval after generation), and CoVe (self-refinement via feedback). \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of Prompt Engineering techniques. Prompting, ToT, GoT, Decomposed Prompting, and Active Prompting. The study by [32] provided a SWOT analysis of LLM technologies and presented the mathematical theoretical underpinnings of various Prompt Engineering techniques. However, none of these studies have focused on deriving the optimal Prompt Engineering techniques based on performance analysis using diverse metrics tailored to the characteristics of LLMs and datasets. The main contributions of this paper are as follows:  
* A technical analysis of various LLMs, Prompt Engineering techniques for performance enhancement, related datasets, and evaluation metrics. * A comparative performance analysis of key Prompt Engineering techniques across different LLMs and datasets, using various evaluation metrics. * Derivation of performance optimization strategies based on the comparative analysis, tailored to specific application domains. In Section 3, we describe the LLMs utilized in this study. Section 4 provides an overview of the benchmark datasets employed, while Section 3.2 explains the performance metrics used for evaluation. In Section 5, we present the experimental results based on these parameters and propose a PE Strategy for optimizing LLM performance. Finally, Section 6 offers the conclusion.