# Domain-Specific Few-Shot Table Prompt Question Answering via Contrastive Exemplar Selection
## 3 Methodology
### Exemplar Retrieval Approach
The model's hyperparameter settings are shown in Table 3. Due to the constraints of text length and hardware conditions, the batch size was set to 120. In the preliminary experiments to prevent overfitting, various hyperparameter combinations were tested. To prevent overfitting and achieve optimal performance, the optimizer used was Adam, with a learning rate of \(1\times 10^{-4}\), and the number of epochs was set to 30. In this study, we did not employ fine-tuning. Instead, we used a contextual learning approach, which does not suffer from the low generalization issue associated with the fine-tuning paradigm, where parameters cannot be reused. In this approach, the model parameters remain frozen, and the input consists of given samples along with data to be predicted, allowing the model to output the predicted values. Since the model parameters do not change during this process, there is no need to update them when applying to downstream tasks, making it effectively applicable to the tasks addressed in this paper. To evaluate the performance of the proposed approach, this paper uses Vicuna-13B as the selected LM and compares prompt exemplar selection approaches, including the following: (1) ZERO SHOT: zero-shot setting. This method supports diverse application scenarios, but its performance cannot be guaranteed. (2) RANDOM: random sampling with replacement from the training set D. This method is easy to understand and involves random sampling, but it still faces challenges in handling complex tasks. (3) BM25 [45]: TF-IDF-based sparse retrieval method. This method easily handles documents containing query terms, but it is not good at dealing with semantic relatedness of words. (4) EPR: prompt exemplar retrieval approach proposed in this paper. To maintain consistency with existing research [23], this paper uses text-to-SQL task standard evaluation metrics logical form accuracy (LF) and execution result accuracy (EX) to measure performance. LF compares the synthesized SQL queries by the model with the actual SQL queries, and its calculation method is shown in Formulas (1) and (2). \[Score_{lf}=\left\{ \begin{array}{l}1,SQL^{\prime}=SQL\\ 0,SQL^{\prime}\
eq SQL\end{array} \right. \tag{1}\]

\[Acc_{lf}=\frac{1}{N}{\sum_{i=1}^{N}{Score_{lf}^{i}}} \tag{2}\]

where \(SQL^{\prime}\) represents the predicted \(SQL\) statement, and \(SQL\) represents the true \(SQL\) statement. \(ACC_{lf}\) represents the logical form matching accuracy, and \(N\) is the total number of questions. EX compares the execution result of the generated \(SQL\) statement with the correct answer. Its calculation method is shown in Formulas (3) and (4). \[Score_{ex}=\left\{ \begin{array}{l}1,Y^{\prime}=Y\\ 0,Y^{\prime}\
eq Y\end{array} \right. \tag{3}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **1 Shot** & **Test** \\ \hline ESQL & 1 & 1500 \\ SMI-SQL & 1 & 1500 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: ESQL, SMI-SQL dataset few-shot setting. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Hyperparameter** & **Optimizer** & **Batch Size** & **Learning Rate** & **Epoch** \\ \hline Value & Adam & 120 & \(1\times 10^{-4}\) & 30 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameter settings. \[Acc_{ex}=\frac{1}{N}\sum_{i=1}^{N}Score_{ex}^{i} \tag{4}\]

where \(Y^{\prime}\) and \(Y\) represent the execution results of the predicted \(SQL\) statement and the execution results of the true \(SQL\) statement, respectively, \(Acc_{ex}\) represent the accuracy of the execution results, with \(N\) being the total number of questions. Therefore, the experiment comprehensively evaluated the performance of nine tasks, including the syntax legality of the generated SQL (Syn-SQL), LF, and EX, as well as the performance of six sub-tasks (SC, SA, WN, WC, WO, and WV). It is important to note that Syn-SQL is constructed through an exception handling mechanism; that is, if an invalid statement occurs, the system will throw an exception.