## 6 Conclusions  
In this paper, we proposed novel ways to replace the ad-hoc thresholding of attribution scores in activation-based input attribution approaches, which can cause the quality degradation of input attribution maps. First, we proposed the threshold fine-tuning (TFT) procedure to optimize the cut-off thresholds of attribution scores, showing that applying fine-tuned thresholds can significantly improve the quality of attribution. Secondly, we provided the activation fine-tuning (AFT) strategy using a tuner network trained by the output of TFT as auxiliary training data to regulate the activations of a CNN. As a result, AFT-tuned CNN produces activations that do not require further per-input attribution thresholding to generate activation-based input attribution. Furthermore, we have shown that AFT does not sacrifice the prediction accuracy of the target CNN. Therefore, AFT can make activation-based input attribution methods more plausible whenever providing input attribution is necessary to get more information about decision-making by CNN models.  
The effectiveness of our method indicates that the activation-based attribution methods may assign nonzero relevance scores to some class-irrelevant features. We think that several factors could be involved and deserve further investigation. First, gradient computation can be noisy. The non-differentiability of activation functions such as ReLU may have introduced noise in gradient computation. Second, the inaccuracy of the underlying CNN classifier may have resulted in an incorrect evaluation of activation values. Since the effect of these factors can be combined, further research will be needed to determine the exact causes and find proper remedies to improve attribution methods.  
Several aspects of this study can be improved in future works. First, our TFT and AFT methods consider only a single activation layer of a CNN--the last convolutional layer in particular. The reason is that most activation-based input attribution methods use the output of the last convolutional layer to generate input attribution maps. However, a few recent techniques, such as Layer-CAM [9], suggest that information from multiple convolutional layers helps improve input attribution. Therefore, extending our AFT approach to tune multiple convolutional layers together will be a worthwhile research direction. Second, there are a few hyperparameters in our approach, such as the number of threshold values \(T\) in Algorithm 1, the number of training examples to be used for activation fine-tuning \(n_{\text{AFT}}\), and the balancing parameter \(\lambda\) in (8). Although these hyperparameters have not been very sensitive in our experiment, it would require further investigation into other datasets and neural networks than those we have tried in this study.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Grad-CAM** & **Grad-CAM++** & **Ablation-CAM** & **Layer-CAM** \\ \hline Without & 0.017 & 0.017 & 1.691 & 0.018 \\ With & 0.045 & 0.046 & 1.719 & 0.047 \\ \hline \(\Delta\) & 0.028 & 0.029 & 0.028 & 0.029 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Average runtime (in seconds) of generating an attribution map with and without our uncovering procedure using VGG-16. The \(\Delta\) indicates the difference between the two cases.  
**Author Contributions:** Conceptualization, S.H. and S.L.; methodology, S.H., S.L. and J.L.; validation, S.H., S.L. and J.L.; writing--original draft preparation, S.H. and S.L.; supervision, S.L. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research was supported by a Korea University Grant.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** All data used in this paper are publicly available. We refer to Section 5 for details.  
**Conflicts of Interest:** The authors declare no conflict of interest.