# Automated Segmentation to Make Hidden Trigger Backdoor Attacks Robust against Deep Neural Networks
## 4 Materials and Methods
### Phase 2: Perturbation of Images for Testing the Smoothed Classifier  
Our proposed Algorithm 1 ascertains the next phase as the perturbation of images of the smoothed classifier. The perturbation is denoted as generating the adversarial example, as shown in step 2 of the algorithm, where a targeted backdoor attack is generated. We generate a targeted adversarial attack, so the _argmin_ function is applied along with its parameters, where \(y\) is the target class, and we want \(f(x_{adv})\
eq\ y\). As mentioned in [7], the colors become more saturated in local color regions when larger perturbations are applied, e.g., \(\epsilon=20/60\). However, this is contradictory to proposals where smaller perturbations are usually tested to generate adversarial examples, as explained by [PERSON] et al. [13]. ``` Input: A poisoned classifier \(f\), the denoised smoothing mechanism \(DS\), the dataset (\(x\),\(y\)). 1. Conversion of an already poisoned classifier \(f\) to a robust smoothed classifier. 2. Generate adversarial examples (smoothed): \(x_{adv}=argmin\)\(\mathcal{L}(DS\cdot f(x^{*}),y)\



### Phase 3: Examining Adversarial Samples

The colors made by the association between local color areas and the backdoor usually relate to the color of the backdoor. These color regions provide us with important knowledge regarding the initial trigger. Figure 3 shows different poisoning results in the form of dissimilar color spots, which were exposed after the denoised smoothing was applied to these images. The purpose of showing these irregular color spots was to highlight our obliviousness to the original backdoor and the targeted class. However, we had to examine the images as proposed in our algorithm and generate the untargeted adversarial examples. Therefore, the poisoned classes became evident after generating the adversarial example. ### Phase 4: Generating the New Effective Triggers

To generate adversarial samples, we used basic image operations. There are two common methods to generate the next triggers, either (i) picking a representative color from a backdoor pattern, or (ii) cropping an image with a backdoor pattern. However, we employed both these methods, since the novelty in our approach is the selection method of representative pixels. In previous works, the representative pixels were chosen manually; however, here, we automated the whole procedure of alternate trigger creation. The patches were extracted by the non-local means (NL-Means) technique to patch-wise filter the sub-images. The estimation of NL-Means \(\left[v\right]_{i}\) for any pixel \(i\) was calculated as follows:

\[NL-Means\left[v\right]_{i}=\sum_{j\in I}\omega\left(i,\ j\right)\left[v\right] _{j} \tag{7}\]

Here, \(\left[v\right]_{i}\) and \(\left[v\right]_{j}\) represent the intensities of the pixels at the locations \(i\) and \(j\), respectively. Similarly, \(\omega\) (\(i\), \(j\)) is the measure for managing the similarity between pixels \(i\) and \(j\). It is worth noting here that the similarity index satisfies the equations \(0\leq\omega\) (\(i\), \(j\)) \(\leq 1\) and \(\sum_{j}\omega\) (\(i\), \(j\)) = 1. In the same way, the similarity of weights depends upon two things: first, the similarity at the gray level, and second, the Euclidean distance between the two vectors, i.e., \(\left.N[v\right]_{i}\) and \(\left.N[v\right]_{j}\), while \(\left.N[v\right]_{k}\) denotes a fixed-size square vicinity centered at pixel \(k\)[33]. The pixel locations were extracted through image segmentation using color spaces in OpenCV. The image was read to find the connected components using the 'bwlabel' function, which helped to identify the colors prominent in the backdoor patterns. The bounding box was drawn around the connected regions to determine our image's most dominating colors. To display the most dominating color in the image, the area was calculated for each connected region, and the region with a large area was selected.