## 1 Introduction  
The wide acceptance of artificial intelligence (AI) in various application areas has increased the need to understand the mechanisms of AI better. One motive behind this can be to build more intelligent autonomous systems by analyzing their strengths and weaknesses, thereby improving the effectiveness of AI systems [1]. Another motive can be to protect human beings from the possible abuse of automated decisions. The EU General Data Protection Regulation (GDPR) is an exemplary action toward this direction, which grants the subject of an automated decision the right to obtain an explanation about the decision when it has a legal effect [2].  
According to [PERSON] [1], XAI techniques can be grouped into three categories: new explainable deep learning (DL) models, improving the prediction accuracy and interpretability of pre-DL models, and model induction. We focus on the model induction approaches since they can be computed without modifying the deep neural networks to be investigated, where modifications often result in prediction performance degradation. Among many model induction approaches, we focus on the popular activation-based attribution methods [3, 4, 5, 6, 7, 8, 9, 10] which make use of both the activation and the gradients of the class score function of a classifier with respect to the activation. These methods are computationally efficient and known to pass the sanity check [7, 11]. Furthermore, these methods generate a so-called class activation map (CAM), an attribution map (often scaled in the \([0,1]\) range, stretched to match the input dimensions, and presented as a heat map) that indicates the relative importance of all features in a given input with respect to a particular class. We also focus on convolutional neural networks (CNNs) and image-based classification tasks.  
Despite their success, we found that many activation-based approaches use arbitrarily chosen thresholds to mute some of their relevance scores (discussed more in Sections 2 and 3), and the quality of input attribution can be significantly improved by optimizing the thresholds. Therefore, we propose a simple but effective mechanism for finding an optimal threshold on the per-input basis that improves attribution quality. In addition, to remove the burden of computing the optimal threshold for each image, we suggest an activation fine-tuning framework that regularizes the penultimate activations of a target CNN with the masks created with optimal thresholding as auxiliary data, making the activations better suited for computing CAM-based input attribution. Our contribution can be summarized as follows:  
* We suggest a threshold fine-tuning (TFT) procedure that finds the optimal threshold values to cut-off relevance scores in a class activation map. TFT uses our new measure, called the relative probability increase (RPI), to evaluate the quality of each thresholded attribution map. We show in the experiment that when we apply the optimized thresholds according to RPI, attribution maps can bring a significant improvement in the average increase [5] and average drop [5].
* We provide a new activation fine-tuning (AFT) strategy that fine-tunes the activation layer of a CNN at which activation-based input attribution is created. AFT consists of a tuner network and a new loss term to minimize the gap between transformed activations by the tuner and the masks as auxiliary data generated by the optimal thresholding of TFT. Our experiment demonstrates that a CNN fine-tuned by AFT can produce attribution maps of much better quality than the original CNN and with a similar quality to the result of applying TFT without computing optimal per-input thresholds.
* We demonstrate in experiments the effectiveness of TFT and AFT for the popular activation-based input attribution methods, namely Grad-CAM [4], Grad-CAM++ [5], Ablation-CAM [8], and Layer-CAM [9], on the ImageNet [12] and Pascal VOC [13] datasets.