## 4 Results
### Quantitative Analysis of Model Interpretability
#### 4.3.2 PG-Acc  
The localization-based metric Point Game [38] is commonly used to measure the precision of interpretable regions, which we refer to as PG-Acc. We call this the Pointing Game, as it asks the CNN model to point at an object of a designated category in the image. Thus, we perform this \"pointing\" action by extracting the maximum value from the weights and checking if it aligns with the theoretical expectations, to assess whether the model's interpretable results accurately pinpoint the target regions. The Pointing Game does not require highlighting the entire extent of objects and does not consider the CNN model's classification accuracy, making it a fair assessment for different types of interpretable techniques. The final accuracy is defined by considering all the data and all the categories within it, as shown in the following formula:  
\[PG-Acc(all\ classes)=\frac{Hits}{Hits+Misses} \tag{11}\]  
Figure 10 presents the results for all modulation formats on the constellation phase histogram dataset. Overall, except for the poor performance of LIME, the other three methods achieved relatively high accuracy levels, with PG-Acc for AMI-CNN, Grad-CAM, LIME, and SHAP being 1, 0.9945, 0.4833, and 0.9944, respectively. For the QPSK modulation format, all methods except LIME reached a PG-Acc of 1, whereas LIME's PG-Acc was only 0.05. As shown in Figure 7, LIME emphasizes phase features primarily distributed in the background region, opposite to the phase clustering points, resulting in a lower PG-Acc. For the 8 PSK modulation format, all four methods achieved a PG-Acc of 1. In the most complex 16 QAM modulation format, Grad-CAM and SHAP had a PG-Acc of 0.9833, slightly lower than that of AMI-CNN, while LIME's performance remained poor, with a PG-Acc of only 0.4.  
Figure 11 presents the results of the PSD dataset across all modulation formats based on PG-Acc. It can be observed that the AMI-CNN model achieves a PG-Acc of 100% consistently across all modulation formats, demonstrating the best and most stable performance among all models. Grad-CAM performs well with PG-Acc scores of 100% in QPSK and 8 PSK but experiences a significant drop to 0% in 16 QAM. As shown in Figure 6, Grad-CAM indeed annotates features correctly within the main lobe regions for QPSK and 8 PSK but exhibits a substantial change in 16 QAM, where annotated areas appear on both sides. This result correlates with the PG-Acc findings, suggesting Grad-CAM's less precise recognition of smaller targets. In contrast, both LIME and SHAP, two interpretable techniques, appear to perform poorly in this task. Specifically, LIME achieves PG-Acc scores of 0.1333 in QPSK, 0.1 in 8 PSK, and 0.2 in 16 QAM, while SHAP scores 0.05 in QPSK, 0.2667 in 8 PSK, and 0.2 in  
Figure 9: MSE of different interpretability methods for three modulation formats on the constellation phase histogram dataset: QPSK, 8 PSK, and 16 QAM.  
16 QAM. According to Figure 6, these interpretable techniques also seem not to focus on the main lobe positions in the PSD dataset.  
In summary, across both the PSD dataset and the constellation phase histogram dataset, AMI-CNN exhibits the best PG-Acc performance. In terms of this metric, the interpretability demonstrated by AMI-CNN is superior.