## 5 Experiments
### Experimental Results

Table 8: _Cont._Qwen2.5 demonstrates a noticeably larger area under all perturbation levels, indicating its superior performance. The GLM model follows as the second-best performer, while the Llama model and Falcon3-Mamba show relatively poor performance, with substantially smaller radar chart areas. Figure 8 presents radar charts that provide a detailed comparison of the ACC of four LLMs across various datasets and perturbation levels. The area of each radar chart represents the ACC, with larger areas indicating a higher ACC. We observe a consistent trend across all models: lower ACC levels are achieved on the LSAT and LogiQA-en datasets, whereas higher ACC levels are observed on the SAT-en and MMLU datasets. Among the models evaluated, Qwen2.5 consistently exhibits significantly larger areas in all  
Figure 6: Line charts visualize the performance decline trends of different LLMs under 4 perturbation levels (0%, 20%, 50%, 100%). The plots demonstrate how LLMs’ ACC changes across three perturbation types (SwapAlpha, SwapWord, and CaseAlpha) on 4 datasets (MMLU, LogiQA-EN, LSAT, and SAT-EN). Each line represents one LLM’s average performance decline, with the baseline performance (0% perturbation) normalized to 100%. Figure 7: The ACC variations are visualized via radar charts under 4 perturbation levels (0%, 20%, 50%, 100%) for each LLM. This demonstrates how LLMs’ robustness changes across various perturbation types concerning 12-dimensional properties. In total, 4 datasets under 3 different perturbations are considered. perturbation datasets and levels, indicating its remarkable ability to solve the perturbation task. It should be noted that although the Falcon3-Mamba model exhibits relatively poor performance under perturbations, the areas formed at different perturbation levels in the radar chart are very similar in size and shape. This suggests that the Falcon3-Mamba model is less affected by perturbations, demonstrating a high degree of robustness when faced with perturbations. As shown in Tables 4 and 7, the perturbations significantly reduced both MSCS and ACC, with the degree of reduction being proportional to the strength of the perturbations. For the same model evaluated on the same perturbation type, the degree of reduction in MSCS also corresponds to the decrease in ACC. For instance, the MSCS reduction for the GLM4 model in the SAT-en dataset (from 1 to 0.9506, 0.8956, and 0.8243 under 20%, 50%, and 100% SwapAlpha perturbations, respectively) was notably greater than that on other datasets (for example, in the LogiQA-en dataset, MSCS decreased from 1 to 0.9984, 0.9955, and 0.9874 under the same perturbation levels). We believe that comparing the correlation between MSCS and ACC for the same model under different types of perturbation is not meaningful. Generally, MSCS reductions caused by the same type of perturbation are consistent, which means that a lower MSCS for a given perturbation type within the same model typically corresponds to a lower ACC. However, in the same model, a reduction in MSCS across different perturbation types does not always lead to a corresponding reduction in ACC. Consider the following questions: \"What is the last letter of the word 'One?\" and \"What is the last letter of the word 'Apple?\" Although the semantic differences measured by MSCS are evident, the answers produced  
Figure 8: Illustration of ACC performance under four perturbation intensities (0%, 20%, 50%, 100%), showing how different LLMs maintain their robustness across various test scenarios. by the model, as evaluated by ACC, remain the same. This results in a significant drop in MSCS while ACC remains nearly unchanged. To effectively mitigate input perturbations, we incorporated self-correction reflection, allowing the models to respond more accurately to the perturbed input. Through this method, we further evaluated the model performance.