# Adversarial Validation in Image Classification Datasets by Means of Cumulative Spectral Gradient
## 4 Results and Discussion
### Comparison with State-of-the-Art Methods  
One of the recent initiatives addressing the problem of distribution shifts in image datasets is MetaShift, a collection of 12,868 natural image sets of 410 classes, which takes the image context into account in its structuring [19]. In addition, the work performed by these researchers includes a score (\(d\)) that measures the distance between any two subsets.  
MetaShift has many subsets of a class, and each subset corresponds to the class in a different context. In this way, MetaShift facilitates the evaluation of changes in these subsets taking into account the context of the change [19]. In order to compare the results of the present paper with the results illustrated in the aforementioned paper [19], some MetaShift  
Figure 9: Example of a CIFAR image with the four types of modifications that have the greatest impact on the distribution of data (**b**–**d**) and the four types of modifications that have the least impact (**e**–**h**).  
subsets were selected and the similarity between these subsets is compared with respect to the values obtained with the adversarial validation methodology of the present paper.  
The selected subsets correspond to a binary classification task, and the distance evaluation between a class under a given context and that same class under a different context has been taken into account. Four different training subsets were taken from the Dog class: _Dog (cabinet + bed)_, _Dog (bag + box)_, _Dog (bench + bike)_ and _Dog (boat + surfboard)_. Each of these subsets are compared against the _Dog (shelf)_ class using a distance metric, given by both MetaShift and the proposed methodology.  
The two distance metrics shown in Table 1 (_d_ and CSG) quantitatively indicate the degree of similarity between the two contexts evaluated. The two metrics coincide in identifying that the dataset with the highest affinity with the subset _Dog (shelf)_ is _Dog (cabinet + bed)_, and that the dataset with the lowest affinity is _Dog (boat + surfboard)_. The degree of difficulty for the other two subsets is also consistent between the two metrics. Here, it is important to take into account two aspects. The first one relates to the fact that the trends of the metrics are inverse, i.e., high values in the MetaShift distance and low values with CSG correspond to a higher degree of distribution shift between the subsets. The second aspect relates to the fact that the distance in MetaShift does not have a maximum value, so it can be difficult to evaluate the degree of distribution shift. In the case of CSG-based adversarial validation, the results are easier to interpret since the metric ranges between 0.0 and 1.0 since two classes are compared.  
Although the MetaShift publication does not provide information on the computational cost, information on the execution time using the proposed methodology is added. This time corresponds to the computation of the distance between subsets using CSG on a machine running on Google Colaboratory using a CPU with 2 logical cores (1 physical core), CPU operating frequency 2200 MHz, 12,978 MB of RAM, and without using GPU.