## 4 Experiments and Results
### Experiments
#### 4.4.2 Comparison with the Original Model  
Table 3 presents a comparison of the overall results between the RL-Fusion LLM and baseline models on the MMLU benchmark. To evaluate the model's language processing capabilities, we selected seven sub-tasks from the MMLU benchmark: Formal Logic, Jurisprudence, International Law, Logical Fallacies, High School Government and Politics, Marketing, and Management. These sub-tasks require a significant level of information extraction and processing ability. The model must extract key information from the given text, understand the context, and conduct logical reasoning and analysis. We observe that the three LLMs exhibit varying performance across the seven MMLU tasks. On average, the RL-Fusion LLM demonstrates a 1.75% relative performance gain over the original best-performing llama-3-8b-Instruct across all seven tasks. In specific tasks, the enhancement achieved by RL-Fusion LLM is substantial, such as an increase from 81.0% to 86.8% on the International Law task. There are two possible reasons for the degradation of the performance of RL-Fusion LLM in tasks such as formal logic and management. First, the poor performance of the base model in these tasks affects the fusion results. Second, continuous pre-training of Chinese instructions and their relevance to downstream tasks on the llama-3-chinese-8b-instruct-v3 model also leads to performance degradation.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Task** & **llama-3-8b-Instruct** & **llama-3-Chinese-8b-Instruct-v3** & **RL–Fusion LLM** \\ \hline formal\_logic & 54.0\% & 50.8\% & 53.2\% \\ \hline high\_school\_government\_and\_politics & 92.2\% & 90.2\% & **94.8\%** \\ \hline international\_law & 81.0\% & 80.2\% & **86.8\%** \\ \hline jurisprudence & 79.6\% & 76.9\% & **82.0\%** \\ \hline logical\_fallacies & 74.8\% & 74.2\% & **76.1\%** \\ \hline management & 84.5\% & 77.7\% & 82.5\% \\ \hline marketing & 91.0\% & 90.2\% & **91.5\%** \\ \hline Weighted\_accuracy & 79.6\% & 77.2\% & **81.0\%** \\ \hline \hline \end{tabular}
\end{table}
Table 3: The overall results of RL–Fusion LLM and the baseline for the MMLU 7 subtask inference evaluation.  
Table 4 presents a comparison of the overall results between the RL-Fusion LLM and baseline models on the C-Eval benchmark. We observe that the three LLMs exhibit varying performance across the C-Eval tasks. On average, the RL-Fusion LLM demonstrates a 1.8% relative performance gain over the original best-performing llama-3-chinese-8b-instruct-v3 across all tasks. In specific tasks, the improvement achieved by RL-Fusion LLM is substantial, with an increase from 57.4% to 60.4% on the Average (Hard) task.  
Table 5 presents a comparison of the overall results between the RL-Fusion LLM and baseline models on the Named Entity Recognition (NER) data from Yayi. The evaluation results were computed using SOTA LLM evaluation. We observe that the three LLMs exhibit varying performance on the NER task. The RL-Fusion LLM demonstrates a 16% relative performance gain in Chinese NER compared to the original best-performing llama-3-chinese-8b-instruct-v3.  
In summary, the results of this study unequivocally validate the effectiveness of the RL-Fusion fusion method. By integrating reinforcement learning with real-time model evaluation feedback to optimize the model fusion process, RL-Fusion not only improves model performance across diverse tasks but also provides a robust framework for enhancing the performance of large language models, particularly for tasks requiring complex reasoning and entity recognition. These findings confirm that RL-Fusion is an effective method for enhancing the capabilities of LLMs, positioning it as a valuable tool for a wide range of natural language processing applications.