# Towards Harnessing the Most of ChatGPT for Korean Grammatical Error Correction
## 5 Experimental Results
### Quantitative Analysis  
In Table 6, we present illustrative examples of K-GEC tasks performed by ChatGPT. The'source' column illustrates sentences containing errors, while the 'target' column provides the corresponding corrected versions. The table further categorizes the results based on varying 'temperature' and'shots' parameters. One noteworthy category of error is the 'punctuation' type, where the performance variation is particularly pronounced across different shots. In the exemplified case, the inadvertent insertion of exclamation marks in the midst of sentences results in an error. With a consistent temperature of 0.2, the performance exhibits significant variations based on the number of shots. In the zero-shot instance, ChatGPT successfully removes the misplaced exclamation marks but overcorrects by introducing superfluous terms such as '(written)'. Similar overcorrections are observed in both 1-shot and 4-shot scenarios, with additions like '(written)' and '(written it)'. However, the 8-shot scenario demonstrates a seamless correction aligning precisely with the target sentence. This evidence underscores the considerable performance variability of ChatGPT depending on the number of shots. In particular, it is noteworthy that such results emerged even when the temperature was set low, which is generally expected to produce more grammatically correct and deterministic \begin{table}
\begin{tabular}{l l c c} \hline \hline  & **Sentence** & **Temperature** & **\# Shots** \\ \hline \multicolumn{4}{c}{**Error type = punctuation**} \\ \hline Source & \begin{tabular}{c} (She can identify who wrote it by looking at the sender of the envelope) \\ \end{tabular} & - & - \\ \hline Target & \begin{tabular}{c} (She can identify who wrote it by looking at the sender of the envelope) \\ \end{tabular} & - & - \\ \hline \multirow{4}{*}{\begin{tabular}{l} ChatGPT \\ \end{tabular} } & \begin{tabular}{c} (She can identify who wrote it by looking at the sender of the envelope) \\ \end{tabular} & 0.2 & 0 \\ \cline{2-4}  & \begin{tabular}{c} (She can identify who wrote it by looking at the sender written on the envelope) \\ \end{tabular} & 0.2 & 1 \\ \cline{2-4}  & \begin{tabular}{c} (She can identify who wrote it by looking at the sender written on the envelope) \\ \end{tabular} & 0.2 & 4 \\ \cline{2-4}  & \begin{tabular}{c} (She can identify who wrote it by looking at the sender of the envelope) \\ \end{tabular} & 0.2 & 8 \\ \cline{2-4}  & \begin{tabular}{c} (She can identify who wrote it by looking at the sender of the envelope) \\ \end{tabular} & 0.2 & 16 \\ \hline \hline Source & \begin{tabular}{c} (What time do you think it was over tomorrow?) \\ \end{tabular} & - & - \\ \hline Target & \begin{tabular}{c} (What time do you think it will be over tomorrow?) \\ \end{tabular} & - & - \\ \hline \multirow{4}{*}{\begin{tabular}{l} ChatGPT \\ \end{tabular} } & \begin{tabular}{c} (What time do you think it will be over tomorrow?) \\ \end{tabular} & 0.2 & 1 \\ \cline{2-4}  & \
\begin{tabular}{c} (What time do you think it will be over tomorrow?) \\ \end{tabular} & 0.5 & 1 \\ \cline{2-4}  &
\begin{tabular}{c} (What time do you think it will be finished tomorrow?) \\ \end{tabular} & 0.8 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Qualitative analysis results. Source refers to the original or input text that contains spelling errors or mistakes. Target represents the desired or corrected version of the text. ChatGPT shows the results of performing K-GEC based on temperature and shots. For the 'tense' type of errors, there's a notable difference in performance depending on the temperature setting, even when the number of shots remains consistent. In the given examples, the K-GEC outcomes based on varying temperatures are demonstrated under the same 1-shot setting. With temperatures set at 0.2 and 0.5, the corrections align closely with the target sentence. However, a setting of 0.8 for the temperature appears to introduce changes in the sentence structure.