## 3 Method
### Mathematical Equation for the Main Processes  
In this section, we describe the main process of CAM-NAS in terms of mathematical equations and we define the method for calculating the similarity score. Since we combine Grad-CAM and NAS methods in this paper, some of the formulas are derived from Grad-CAM [15].  
\[\alpha_{k}^{c}=GP(\frac{\partial y^{c}}{\partial A_{l}^{k}}) \tag{1}\]  
For a convolutional layer, l denotes the layer, and \(A_{l}^{k}\) denotes the activation map of the \(k\)th channel. \(GP()\) denotes the global pooling operation. First, we compute the gradient of the score for the class \(c\), \(y_{c}\), with respect to the feature maps \(A^{k}\) by (1). Then we use the global average pooling operation to obtain the importance weights of the neurons \(\alpha_{k}^{c}\), \(\alpha_{k}^{c}\) denotes a partial linearization of the deep network downstream of \(A\), and represents the confidence of a feature map \(k\) for a target class \(c\).  
\[L_{Grad-CAM}^{c}=ReLU(\sum_{k}\alpha_{k}^{c}A_{l}^{k}) \tag{2}\]  
To obtain the class activation map (CAM), we perform a weighted combination of activation maps. Then, we applied the ReLU function to focus only on the positive part of the activation map, as in (2). The positive part better reflects the model's ability to discriminate objects.  
\[A_{Target} =\sum_{i}^{N}L_{Grad-CAM(i)}^{ci} \tag{3}\] \[A_{Search} =\sum_{i}^{N}L_{Grad-CAM(i)}^{ci} \tag{4}\]  
In the NAS algorithm, we first randomly select \(N\) images from the dataset and input them into the teacher model, and then use (3) to obtain \(A_{Target}\). For the search submodel,  
Figure 3: Similarity scores obtained by applying various CAM techniques to the ResNet model. The similarity score unit is a unit-less proportional value (with a maximum similarity of 1) that measures the degree of similarity between two saliency maps, with larger values indicating greater similarity.  
we input the same \(N\) images into the search submodel and then use (4) again to obtain \(A_{Search}\).  
\[Score=\frac{1}{\sqrt{(A_{Target}-A_{Search})^{2}}} \tag{5}\]  
Formula (5) defines how the similarity score is calculated. The calculation used is the inverse of the Euclidean distance. The higher the similarity score, the closer the CAM of the teacher model is to the CAM of the searched submodel. It is assumed that this means that the submodel and the teacher model focus on the same images in a similar way, and that the submodel will eventually perform more similar to the teacher model on the dataset. The CAM-NAS process is shown in Algorithm 1.  
```
0: Search space \(S\), inference budget \(B\), maximal depth \(L\), total number of iterations \(T\), evolutionary population size \(N\), initial structure \(F_{0}\).
1: Initialize population \(P=\{F_{0}\}\)
2:for\(i=1;i<T;i++\)do
3: Randomly select \(F_{i}\in P\)
4: Uniformly select a block \(h\) in \(F_{i}\)
5: Uniformly alternate the block type, kernel size, width and depth of \(h\) within some range.
6: Obtain the mutated structure \(F_{m}\)
7:if\(F_{m}\) exceeds inference budget or has more than \(L\) layers then
8: Do nothing
9:else
10: Obtain Similarity-Score, score = \(Score(F_{m})\)
11: Append \(F_{m}\) to \(P\)
12:endif
13:if the length of \(P>N\)then
14: Delete the submodel of the smallest Similarity-Score
15:endif
16:endfor
17:Return the submodel of the highest Similarity-Score in \(P\)
```  
**Algorithm 1**CAM-NAS