# Exploring GPT-4 Capabilities in Generating Paraphrased Sentences for the Arabic Language
## 3 Methodology
### Quality Ranking Framework for Paraphrasing
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Evaluator \#1**} & \multicolumn{4}{c}{**Evaluator \#2**} & \multicolumn{4}{c}{**Evaluator \#3**} \\ \cline{2-13}  & **0 * & 1** & 2** & 3** & 0 & 1 & 2 & 3 & 0 & 1 & 2 & 3 \\ \hline Culture & 67.14 & 16.43 & 12.86 & 3.57 & 0.00 & 0.71 & 1.43 & 97.86 & 57.86 & 10.00 & 21.43 & 10.71 \\ Sport & 69.29 & 14.29 & 12.14 & 4.29 & 1.43 & 0.71 & 2.14 & 95.71 & 50.00 & 8.57 & 20.71 & 20.71 \\ Technology & 71.43 & 16.43 & 9.29 & 2.86 & 0.71 & 2.86 & 5.71 & 90.71 & 43.57 & 12.86 & 18.57 & 25.00 \\ \hline \end{tabular}
\end{table} 
Table 12: Criterion of changing word form. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Evaluator \#1**} & \multicolumn{4}{c}{**Evaluator \#2**} & \multicolumn{4}{c}{**Evaluator \#3**} \\ \cline{2-13}  & **0 * & 1** & 2 & 3 & 0 & 1 & 2 & 3 & 0 & 1 & 2 & 3 \\ \hline Culture & 0.71 & 0.00 & 7.86 & 91.43 & 0.00 & 0.00 & 2.14 & 97.86 & 2.86 & 2.86 & 7.86 & 86.43 \\ Sport & 0.00 & 1.43 & 5.71 & 92.86 & 0.71 & 0.71 & 1.43 & 97.14 & 2.14 & 1.43 & 15.00 & 81.43 \\ Technology & 2.14 & 2.14 & 10.00 & 85.71 & 2.86 & 1.43 & 8.57 & 87.14 & 11.43 & 3.57 & 10.00 & 75.00 \\ \hline \end{tabular}
\end{table} 
Table 10: Criterion of following Arabic grammar and rules. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Evaluator \#1**} & \multicolumn{4}{c}{**Evaluator \#2**} & \multicolumn{4}{c}{**Evaluator \#3**} \\ \cline{2-13}  & **0 * & 1** & 2 & 3 & 0 & 1 & 2 & 3 & 0 & 1 & 2 & 3 \\ \hline Culture & 0.71 & 1.43 & 12.86 & 85.00 & 1.43 & 2.86 & 6.43 & 89.29 & 4.29 & 26.43 & 65.00 \\ Sport & 0.71 & 1.43 & 17.14 & 80.71 & 2.14 & 2.86 & 2.86 & 92.14 & 2.86 Regarding semantic similarity, all three evaluators highly rated paraphrased sentences for the different categories. We conclude that GBT 4 produces paraphrased sentences that are semantically equivalent, as presented in Table 9. Regarding fluency, most sentences are rated as excellent or good for all categories based on Table 10; thus, GBT 4 can generate sentences that look fluent and natural. The evaluators evaluate sentences as grammatically correct, as shown in Table 11. Evaluators find that GPT-4 follows Arabic grammar and rules because the sentence components are correctly formed. In Table 12, the first and third evaluators find the paraphrased sentences are not much restructured compared to the original text's structure. More specifically, GPT does not change sentence structure much and preserves its original structure. In contrast, the second evaluator believes there are many changes in sentence structure and rates most of them as excellent. Based on the first and third evaluators' opinions in Table 13, most sentences do not use different forms of words and paraphrasing. Thus, GPT does not change word forms to create paraphrased sentences. Still, GPT sometimes uses synonym substitution, as illustrated in Table 14, which reveals synonyms appearing in half of the pairs and rated as excellent or good. The overall quality of paraphrased pairs is ranked as excellent for all categories, as provided in Table 15.

### Regular NLP Metrics

The newly generated paraphrased text is evaluated for the entire corpus using BLEU and ROUGE. Table 16 shows the BLEU, ROUGE, and LD scores. The average BLEU score is 0.5, ROUGE-1 precision is 0.7, ROUGE-1 recall is 0.6, and ROUGE-1 F1 is 0.6. The overall performance of the metric shows a high degree of overlap between the Ori_sentence and Par_sentence, with scores of 0.5 or higher. That performance means the quality of paraphrased sentences is satisfactory. In other words, for the three categories, BLEU and ROUGE scores achieved a moderate level of word unigram overlap between the Ori_sentence and Par_sentence, which is considered a sensible degree of similarity due to the use of synonyms. On the other hand, the average LD score is almost 0.8 for all three categories, which means that both Ori_sentence and Par_sentence are rich in vocabulary and phrases.