On the Robustness of Deep Clustering Models: Adversarial Attacks and Defenses
Anonymous Author(s)
AffiliationAddress email
Abstract
Clustering models constitute a class of unsupervised machine learning methodswhich are used in a number of application pipelines, and play a vital role inmodern data science.
With recent advancements in deep learning– deep clusteringmodels have emerged as the current state-of-the-art over traditional clusteringapproaches, especially for high-dimensional image datasets.
While traditionalclustering approaches have been analyzed from a robustness perspective, no priorwork has investigated adversarial attacks and robustness for deep clustering modelsin a principled manner.
To bridge this gap, we propose a blackbox attack using
Generative Adversarial Networks (GANs) where the adversary does not knowwhich deep clustering model is being used, but can query it for outputs.
Weanalyze our attack against multiple state-of-the-art deep clustering models and realworld datasets, and find that it is highly successful.
We then employ some naturalunsupervised defense approaches, but find that these are unable to mitigate ourattack.
Finally, we attack Face++ , a production-level face clustering API service,and find that we can significantly reduce its performance as well.
Through thiswork, we thus aim to motivate the need for truly robust deep clustering models.
Introduction
Clustering models are utilized in many data-driven applications to group similar samples together, anddissimilar samples separately.
They constitute a powerful class of unsupervised Machine Learning(ML) models which can be employed in many cases where labels for data samples are either hard (orimpossible) to obtain.
Note that there are a multitude of different approaches to accomplishing theaforementioned clustering task and an important differentiation can be made between traditional anddeep clustering approaches.
Traditional clustering generally aims to minimize a clustering objective function defined using a givendistance metric [1].
These include approaches such as k-means [2], DBSCAN [3], spectral methods[4], among others.
Such models generally fail to perform satisfactorily on high-dimensional data (i e., image datasets), or incur huge computational costs that make the problem intractable [5].
To improve upon these approaches, initial deep clustering models sought to decompose the high-dimensional data to a cluster-friendly low-dimensional representation using deep neural networks.
Clustering was then undertaken on this latent space representation [6, 7].
Since then, deep clustering models have become considerably advanced, with state-of-the-art models outperforming traditional clustering models by significant margins on a number of real-world datasets [8, 9]
Despite these successes deep clustering models have not been sufficiently analyzed from a robustness perspective.
Recently, traditional clustering models have been shown to be vulnerable to adversarial attacks that can reduce clustering performance significantly [10, 11] 1 .
However, no such adversarialattacks exist for deep clustering methods.
Furthermore, no work investigates generalized blackboxattacks, where the adversary has zero knowledge of the deep clustering model being used.
This isthe most realistic setting under which a malicious adversary could aim to disrupt the working ofthese models.
While there is a multitude of work in this domain for supervised learning models, deepclustering approaches have not received the same attention from the community.
The closest work to ours proposes robust deep clustering [9, 12], by retraining models with adversarially perturbed inputs to improve clustering performance.
However, this line of work has manyshortcomings: 1) it lacks fundamental analysis on attacks specific to deep clustering models (fore.g., the state-of-the-art robust deep clustering model RUC [9] only considers images perturbed via
FGSM/BIM [13, 14] attacks, which are common attack approaches for supervised learning), 2) noclearly defined threat models for the adversary are proposed, and 3) there is no transferability [15]analysis.
Thus in this work we seek to bridge this gap by proposing generalized blackbox attacks thatoperate at the input space under a well-defined adversarial threat model.
We also conduct empiricalanalysis to observe how effectively adversarial samples transfer between different clustering models.
[Figure]
We utilize Generative Adversarial Networks (GANs) [16] for our attack, inspired by previous approaches (AdvGAN [17], AdvGAN++ [18], etc) forsupervised learning.
We also utilizea number of defense approaches (essentially deep learning based anomalydetection [19] and state-of-the-art "robust" deep clustering models [9]) todetermine if our adversarial samplescan be mitigated by an informed defender.
One of the major findings of our work is that theseapproaches are unable to prevent our adversarial attacks.
Through our work, we seek to promote thedevelopment of better defenses for adversarial attacks against deep clustering.
Finally, to truly showcase how powerful our attacks can be, we attack a production-level ML-as-aService (MLaaS) API that performs a clustering task (and possibly utilizes deep clustering models inthe backend).
We find that our attack can also significantly reduce the functioning of such MLaaSclustering API services.
To summarize, we make the following contributions:
• We propose the first blackbox adversarial attack against deep clustering models.
We show that ourattacks can significantly reduce the performance of these models while requiring a minimal numberof queries.
We also undertake a transferability analysis to demonstrate the magnitude of our attack.• We undertake a thorough experimental analysis of most state-of-the-art (SOTA) deep clusteringmodels, on a number of real-world datasets, such as CIFAR-10 [20], CIFAR-100 [21], and STL-10[22] which shows that our attack is applicable across a number of models and datasets.•
We show that existing (unsupervised) defense approaches (such as anomaly detection and robustnessvia adversarial retraining) cannot thwart our attack samples, thus prompting the need for better defenseapproaches for adversarial attacks against deep clustering models.•
We also attack a production-level MLaaS clustering API to showcase the extent of our attack.
Wefind that our attack is highly successful for this real-world task, underscoring the need for makingdeep clustering models truly robust.
Figure 1 shows some of the adversarial samples generated by our attack for the SPICE [8] and RUC[9] deep clustering models on the STL-10 dataset, and their corresponding pre-attack and post-attackpredicted cluster/class labels 2 .
To the human eye, these samples appear indistinguishable from each
Related Works
Deep Clustering.
To leverage clustering algorithms on high-dimensional data, early work on deepclustering [6, 7], aimed to learn a latent low-dimensional cluster-friendly representation that couldthen be clustered on.
This task was achieved using AutoEncoders (AEs) [7, 23, 24], Variational
AEs [25, 26], or GANs [27–29].
More recently, current SOTA deep clustering models employself-supervised and contrastive learning instead of the earlier AE/GAN based approaches to performclustering [8, 9, 30].
We consider these in the paper due to their superlative performance.
Adversarial Attacks Against Clustering.
Recently, a few works have proposed blackbox adversarialattacks against clustering [10, 11].
These seek to poison a small number of samples in the input data, sothat when clustering is undertaken on the poisoned dataset, other unperturbed samples change clustermemberships.
This problem is significantly different from ours– the traditional clustering algorithmretrains on the poisoned input data constituting an attack at training time.
For deep clustering, themodel does not train again once deployed, so we have to generate adversarial images that the modelmisclusters at test time .
There have also been whitebox attacks proposed for traditional clustering,but these can only be employed for the specific traditional clustering algorithm considered [31–33].
Robust Deep Clustering.
A natural in-processing defense approach to preventing adversarial attackson deep neural networks is to utilize some form of adversarial retraining which seeks to jointlyoptimize an adversarial objective along with the original loss function used to train the network.
Thistechnique has been shown to vastly improve the adversarial robustness of traditional deep neuralnetworks [34, 35].
In the context of deep clustering, to the best of our knowledge, there are twoworks that aim to make models robust to adversarial noise in this manner 3 : RUC [9] and ALRDC[12].
Moreover, RUC is considered to be the SOTA robust deep clustering model.
Even in termsof performance metrics (NMI, ACC, ARI) it is second only to the SOTA model SPICE [39].
Preliminaries and Notation
In this section, we introduce notation and preliminary knowledge regarding the clustering task andthe proposed adversarial attack.
Note that for a matrix A of size u × v , we can index into row i as
A i , ∀ i ∈ [ u ] , and index into a single value at row i and column j as A i,j , ∀ i ∈ [ u ] , j ∈ [ v ] .
Moreover,|| .
|| denotes the Euclidean (2-norm) norm of a vector.
Deep Clustering.
Since we are proposing blackbox attacks in our paper, we will be defining deepclustering models in a more generalized manner that abstracts their inner functioning.
We deferthe reader to [5] for more details on the models analyzed in the paper.
A deep clustering model isdenoted as C and operates on samples of the given dataset X consisting of n samples and maps themto one of k clusters.
As deep clustering models utilize deep neural networks internally, they generatea set of softmax probabilities indicating cluster memberships, denoted as M ∈ [0 , 1] n × k .
From this,the cluster labels can be obtained by taking the maximum of the cluster probabilities.
For a datasetsample X i ∈ X , the cluster label can be obtained as l = argmax j ∈ [ k ] { M i,j } .
We denote the vectorof cluster labels computed in this manner as L ∈ N n × 1 .
Note that unlike supervised learning problems, the ground truth labels Y ∈ N n × 1 are not utilizedfor training the model in deep clustering.
However, these ground truth labels are used for evaluatingthe performance of the model.
That is, the output cluster labels L are evaluated in comparisonwith the ground truth Y .
Performance metrics such as Normalized Mutual Information (NMI) [40],
Adjusted Rand Index (ARI) [41], and Unsupervised Accuracy (ACC) [42] are commonly used indeep clustering literature for this purpose.
These metrics are defined analytically in Appendix A.6.
Threat Model.
We now define the threat model for the adversary:
The attacker has knowledge of the dataset X .
This is a commonly made assumption in adversarial attacks against traditional clustering literature [10, 11].
The attacker carries out a blackbox attack and does not know which deep clustering model C is being used, but can query it to observe M (softmax cluster memberships), and hence, L .
In adversarial attacks on (un)supervised models, this is often what constitutes a blackbox attack [43–46].
The goal of the attack is to provide minimally perturbed images (there is a noise thresholdthat cannot be exceeded) as input to the deep clustering model.
Upon doing so, the model shouldmiscluster these samples and the performance measured via the evaluation metrics (ACC, NMI, ARI,etc) should significantly reduce post the attack.
Adversary’s Objective.
The goal of the attacker is to input adversarial images to the model andlead to a performance decrease, measured using metrics such as NMI, ACC, and ARI.
We cannotdirectly optimize post-attack M or L over these metrics to generate this adversarial noise as theyare non-differentiable.
Moreover, cluster labels generated by the model may not be the same asthe ground truth labels, thus requiring some notion of a mapping function, further complicating theproblem.
Instead we will indirectly achieve this goal through another objective function.
Let an input image be X i ∈
X and through a single query the cluster probabilities M i can be obtainedbefore the attack.
To attack, the adversary will introduce a carefully crafted perturbation/noise δspecific to this sample.
The attacker queries the deep clustering model and obtains the set of clusterprobabilities for this sample.
Abusing notation slightly, these are denoted as C ( X i + δ ) .
Assuming the original unperturbed cluster labels/probabilities accurately depict the cluster representing the sample’s ground truth label, the attacker can simply generate the adversarial noise via thefollowing optimization problem:
[Equation]
Here the constraint with ϵ is simply to ensure that the adversarial sample does not have unboundednoise and remains realistic to a human observer.
The objective function above ensures that the clusterprobabilities post the attack C
( X i + δ ) are as distinct as possible to the cluster probabilities M iobtained before the attack.
Then, assuming that prior to the attack the cluster labels were the correctrepresentative of the ground truth label Y i 4 , the deep clustering model will miscluster the sampleafter the attack and performance (NMI, ACC, ARI) should reduce.
If the adversary introduces manysuch adversarial samples, the performance can thus be significantly affected 5 .
Attacking Deep Clustering Models
4.1 Blackbox Attack Using GANs
We utilize a simple GAN based architecture for generating the adversarial perturbation δ and solvingthe attack problem delineated in the previous section.
There are many different variations to using
GANs for generating adversarial samples, such as AdvGAN [17], AdvGAN++ [18], WPAdvGAN[47], CycleAdvGAN [25], among many others.
For our attack, we employ a vanilla GAN architectureconsisting of deep neural networks for both the Generator and Discriminator, similar to AdvGAN.
We utilize the Generator model G to generate the adversarial perturbation δ for a given input image
X i ∈
X , i.e., G ( X i ) → δ .
The Discriminator model D plays a similar role as for the original GANmodel [16] as it aims to ensure that the perturbed image is similar to the distribution of input images.
We then rewrite our attack optimization problem in the context of the GAN architecture.
The loss forthe attack objective can be written directly as in the optimization problem:
[Equation]
And we can simply reformulate the constraint on the norm of the adversarial noise as:
[Equation]
We also write the vanilla minimax GAN loss [16] as:
[Equation]
To train the Generator G and Discriminator D , we then optimize these combined losses by solvingthe following saddle-point problem (where α a , α c are hyperparameters to control tradeoff):
[Equation]
Upon obtaining the trained Generator G , we can generate the adversarial perturbation as δ = G
( X i )for any image
X i ∈
X provided as input.
We then provide these adversarial images
X i + δ as inputto the pre-trained deep clustering model C to obtain cluster membership confidence scores as M ′ i =
C ( X i + δ ) after the attack.
From M i we know the original cluster label as L i = argmax j ∈
[ k ] M i,jand similarly, we can obtain the cluster label of the adversarial image as L ′ i = argmax j ∈ [ k ] M ′
i,j .
If the optimization problem was solved successfully and the distance between M i and C ( X i + δ )is sufficiently large enough, we can have L i ̸
=
L ′ i .
Moreover, assuming that a mapping function ϕexists that maps the output cluster labels to the ground truth labeling, we know that: ϕ
( L i )
=
Y i .
Since L i ̸ =
L ′ i , we can conclude that ϕ
( L ′ i )
̸
= Y i leading to misclustering and a drop in performance.
Remark.
Note that since our attack objective is indirectly formulated, it is possible that M i and
M ′ i are not sufficiently far apart even after the attack, leading to L i =
L ′ i .
However, as we findand our experiments show, this does not happen frequently in practice as our blackbox attack ishighly successful at generating adversarial samples that disrupt the performance of all deep clusteringmodels considered in the paper (Section 6.1).
We have considered the following open-source deepclustering models in experiments: CC [30], SPICE [8], SCAN [48], MiCE [49], NNM [50], RUC [9].
Attacking Face++ : A Production-level MLaaS Clustering API
To showcase the disruptive capability of our attacks, we attack Face++ 6 , which provides an extremelywell-performing face clustering REST API service.
At a high-level, the API performs a basic faceclustering task where it takes in as input a face dataset and seeks to cluster images belonging to thesame person together.
We do not know the clustering algorithm/model being utilized in the backend.
Note that there is one significant difference here compared to the threat model for open-source deepclustering models previously considered– we do not have access to cluster memberships, but just thefinal set of labels.
To overcome this problem, we attack Face++ by training a surrogate open-sourcedeep clustering model on the dataset and then generate adversarial samples for this model using our
GAN based attack pipeline.
We then use these adversarial samples as input to the Face++ API toconduct the attack.
Hence, our attack constitutes a transferability attack via a surrogate model.
The Face++ API service functions as follows: we first create a new face album using thecreateAlbum endpoint which gives us a facealbum_token as a response.
Using this token, we addthe images for our dataset using the addimage endpoint, and then call the groupFace endpoint to perform the clustering task.
Finally, we obtain cluster labels for each image using the getAlbumDetailendpoint, which returns group_ids as the integer cluster labels.
We provide the results of our attackand experiment details in Section 6.3 as well as additional implementation details in Appendix C.
Mitigating the Attack: Possible Defense Approaches
In-processing Defense: Robust Deep Clustering.
Adversarially retrained models (where thelearning process incorporates a joint adversarial loss along with the original loss) have been shown toconsiderably mitigate adversarial attacks, and thus constitute a natural in-processing defense approach.
In the context of deep clustering, RUC and ALRDC utilize adversarial learning to improve modelrobustness.
However, we only consider RUC in this paper for the following reasons: 1) RUC is anadd-on module, and can be applied to any deep clustering model, 2) in contrast, the ALRDC approachspecifically works only to make the latent space robust to perturbations (which most deep clusteringmodels considered in our paper do not possess), restricting its use, and 3) unlike RUC, from ourearly experiments with ALRDC we found that it did not work well with high-dimensional real-worldimage datasets such as CIFAR-10/CIFAR-100/STL-10, which we have considered in this paper 7 .
In our experiments in Section 6.2, we show that using our GAN based attack (Section 4.1) and
RUC as the deep clustering model C , we can disrupt its performance significantly as well.
Once the generator has learnt how to generate adversarial noise specific to RUC, it can easily degrade
RUC’s clustering ability.
These results clearly indicate that there is a deficiency in current robustdeep clustering strategies, and the problem requires different solutions 8 .
Pre-processing Defense: Deep Learning Based Anomaly Detection.
With advancements in deeplearning, the long-standing field of anomaly/outlier detection has also seen major advancements.
Further, due to the unsupervised nature of the deep clustering task (no labels), anomaly detection is asuitable pre-processing approach for detecting adversarial samples.
In particular, most deep learningbased anomaly detection models are trained on specific datasets and can detect out-of-distributionsamples with extremely high precision.
For our experiments, we use a recently proposed selfsupervised deep anomaly detection approach SSD [19] which achieves state-of-the-art performanceon benchmarks when labeled training data is not present [53].
While a detailed description of SSD is beyond the scope of our work, the authors employ contrastiveself-supervised representation learning combined with a Mahalanobis distance based thresholddetector in the feature space to detect anomalies.
In our experiments in Section 6.2 we show that even
SSD is unable to detect a large majority of our adversarial samples.
We further show that this is likelydue to the distribution of our adversarial samples in space, as they mimic the original samples fairlywell due to the norm constraint on the generator’s output.
We use Principal Component Analysis(PCA)
[54] to analyze the adversarial and benign samples and find that their principal componentstend to be very similar.
In this regard, anomaly detection approaches are also unable to detect most ofour generated adversarial samples (less than 1% in the best case, and ≈ 14% on average!).
Experiments and Results
6.1 Blackbox Attack Against Deep Clustering Models
For our experiments on the open-source models, we utilize the following real-world image datasets:
CIFAR-10 [20], CIFAR-100 [21], and STL-10 [22].
These are commonly utilized in a majority of deepclustering literature.
For the models, we consider a number of state-of-the-art deep clustering models:
SPICE [8], SCAN [48], NNM [50], MiCE [49], and CC [30].
We also consider RUC [9], but discussits results as part of the defense Section 6.2.
Notably, SPICE and RUC (add-on to SCAN) are SOTA#1 and #2 for performance on the aforementioned datasets.
We show a large number of adversarialimages generated by our attack model and provide additional experiment details in Appendix A.
[Table]
As part of our experiment, we generate adversarial images (adversarial set) using our attack for allthe images in the original test set.
We show the performance metrics (ACC, NMI, ARI), pre-attack(original images) and post-attack (adversarial images) for all the aforementioned models and datasetsin Table 1.
Quite evidently, it can be observed that performance for all models is significantly reducedpost the attack.
Note that since the GAN network generates fixed noise for the same input, there is novariance in the obtained results.
We have presented hyperparameter ( α a , α c , ϵ , etc.) values for allexperiments in this section in Appendix A.1 due to space limitations.
Effects of the Attack.
For all the deep clustering models, we find a general trend emerges in how themodels’ post-attack clusters differ from the clusters generated on the original test set.
In particular, performance on the benign images is generally very good– this can be observed in the confusionmatrix for the SPICE model and STL-10 dataset in Fig 2a.
Note that most of the clusters are correctlydefined and there are a few miclusterings.
However, for the adversarial images, we notice that thereis a complete clustering breakdown – most images tend to get lumped in a small number of selectclusters, and the remaining few images create small-sized clusters.
This can also be seen in theconfusion matrix in Fig 2b for SPICE on adversarial STL-10 images.
Most of the adversarial imagesare clustered as part of the "cat" cluster, despite being clustered accurately before the attack.
Thistrend is prevalent for the attack on all the models, and tends to be more drastic for less performantdeep clustering models, such as CC.
This can be seen in the confusion matrices shown in Figs 2c and2d.
We present the confusion matrices for the other models in Appendix A.3 due to limited space.
[Figure]
A major takeaway from these results is that while deep clustering models tend to work very well on“clean” data, simple adversarial samples exist that can completely degrade performance.
It is thusimperative that deep clustering model designers evaluate their models against adversarial samples. [Table] [Figure] [Table]
[Figure] [Table] [Figure] Query Complexity and Perturbation Analysis.
Wealso measure query complexity of our approach to analyze the cost associated with carrying out our attackin the real world.
In this blackbox attack scenario, thequery complexity is defined as the number of timesthe deep clustering model is queried by the adversary.
Thus, we can measure the number of times the Generator G queries the clustering model C with an inputbatch, before the loss of the GAN network converges.
We present these results in Table 2 wherethe batch size is 256.
It can be seen that the query complexity of our attack is quite minimal for allmodels and datasets.
In particular, the values obtained for our method are comparable to the querycomplexity rates obtained by existing blackbox attacks against supervised learning models [43, 45].
Moreover, a smaller query complexity is doubly beneficial in our case, because once the generatorhas been trained, we can use it to generate the optimal perturbation for any images that belong to thatinput distribution without requiring any additional queries to the clustering model.
[Figure]
We also analyze the effect of varying the noise penalty (via ϵ ) on the extent to which the attackdegrades the performance of the deep clustering model.
We depict the NMI, ARI, ACC for all themodels on the STL-10 dataset as a function of the norm of the generated perturbation in Fig 3.
Itcan be observed that as the norm threshold is increased the attack becomes more successful andthe performance of the models is worsened, eventually plateauing close to 0.
To conserve space,we defer these results with similar trends for the other datasets to Appendix A.4.
Transferability Analysis.
We undertake a transferability analysis to see whether adversarial samplesgenerated for one model transfer to other deep clustering models.
We present these results in Fig 4for all our datasets as transferability matrices.
In these, we show the post-attack NMI for source andtarget models, along with the pre-attack NMI for each of the deep clustering models.
It can be seenthat the overall transferability of most adversarial samples is high.
Note that SPICE (SOTA #1) is ingeneral a much better deep clustering model than CC in terms of performance.
Interestingly however,we can see that SPICE’s adversarial samples do not transfer well to CC and vice versa.
It would beinteresting to investigate the reasons for this occurrence in future theoretical or empirical work.
Dueto space limitations, we show the transferability matrices with ACC and ARI in Appendix A.5.
[Figure]
6.2 Defense Approaches
Robust Deep Clustering.
As mentioned before [Figure] we utilize the RUC add-on module as a first defenseagainst our attack.
As in the original paper
[Figure]
we utilize RUC with SCAN as the deep clustering modeland then carry out our GAN based attack against it.
Similar to the experiments on the open-sourcemodels in Section 6. [Figure] we measure the performance on a benign test set and then on a correspondingadversarial set generated using our GAN attack.
We find that our attack is also successful against
RUC, thus proving that RUC is not entirely robust to adversarial noise.
This is shown in Table 1.
Itcan be seen that RUC is not a satisfactory defense approach for our attack.
However, note that therobustness of RUC is somewhat observable in our perturbation norm versus performance analysisshown in Fig 3.
In general, it takes higher norm thresholds to reduce the performance of RUC,although it is achievable without adding too much noise to the images.
For the transferability analysisshown in Fig 4, we find RUC behaves similar to SPICE, as RUC’s adversarial samples are highlytransferable across all models and datasets except for CC on the CIFAR-10 and CIFAR-100 datasets.
[Figure]
Deep Learning Based Anomaly Detection.
We now present results for SSD, the deep learning basedanomaly detection approach, as a pre-processing defense to our attack.
In our experiments, we startwith a test set of 8000 images for all of our datasets.
For each benign image X i , we uniformly randomly either add adversarial noise to it or let it remain benign.
We do this over 100 trials and averageour results to observe how many adversarial samples were detected by the anomaly detection approach.
We present these results in Fig 5a.
We find that on average the method detects 11.69%, 19.41%, and11.71%, of adversarial samples for the CIFAR-10, CIFAR-100, and STL-10 datasets, respectively.
However, this is negligible compared to the large number of adversarial samples being injected andremaining undetected.
To analyze the reasons for this occurrence, we compute the first 3 principalcomponents of the adversarial set and the benign set using PCA and present these in Fig 5b for SPICE.
Visually, the principal components of a large majority of adversarial samples are superimposed andinterspersed amongst those of the benign samples, showcasing the potency of our attack.
6.3 Attacking Face++ (MLaaS API)
[Figure]
We now present our results for attacking Face++ , a productionlevel face clustering API service.
Since the service only workswith face images, we use the Extended Yale Face B [55] dataset forthese experiments.
There are 28 persons in this dataset, i.e., k = 28and 500 images for each person.
However, due to rate limits andthe latency associated with uploading images using REST APIs, wetest the service by sending only 10 images per person, for a total of280 images.
Then, as only cluster labels are outputted by the API,we resort to using a surrogate model for the attack via transferability.
We train a CC model as the surrogate on the entire Extended
Yale Face B dataset
and then train our GAN network to generate the adversarial noise for a givenimage from the dataset.
Thus, we generate adversarial counterparts to the original 280 benign imagesand carry out the attack.
Subsequently, we observe the NMI
[Figure]
and post [Figure] [Figure] Since the results obtained can be affected bywhich 10 representative images were picked foreach person in the test set, we randomize thisselection process and take the average over 10runs.
As can be seen in Fig 6, before the attack, the Face++ clustering API boasts stellarperformance on the test set, with average NMI≈ 0 . 97 .
However, for the set of adversarial images, performance of the API service degradessignificantly, with the NMI dropping down to≈ 0 . 62 on average.
This shows that our generated adversarial examples can disrupt the working of a high-performance real-world service,even when we cannot query the model for memberships.
We also show some of the adversarialsamples generated from original images, and how they have been misclustered in Fig 7.
Before theattack, most of the faces are clustered correctly, but after the attack they only majorly belong to the
B12 or B18 face clusters.
Here too, we are observing the clustering breakdown effect previously notedfor the attack against open-source deep clustering models.
We provide additional implementationdetails and empirical results for this attack in Appendix C.
Conclusion
In this paper, we propose the first blackbox adversarial attack against deep clustering models, using asimple GAN based architecture (Section 4.1).
We evaluate our attack against a number of open-source
SOTA deep clustering models and real-world image datasets, and find that it significantly reducesthe performance of these models, while having minimal query complexity (Section 6.1).
To furtherexamine the robustness of deep clustering networks, we utilize two unsupervised defense approaches:adversarially trained "robust" deep clustering models, and deep learning based anomaly detection.
Wenote that even these approaches are unable to detect our adversarial samples and mitigate the attack(Section 6.2).
Finally, we use a surrogate model to attack a production-level face clustering API
Face++ , proving that our attacks are also effective against real-world MLaaS models (Section 6.3).
Broader Impact and Limitations.
Our findings are important as they underscore the need tomaking deep clustering models robust to real-world blackbox adversaries.
As we show, currentrobustness/defense approaches are unsatisfactory, and allow an informed adversary to reduce theperformance of models nonetheless.
Another takeaway is that robust deep clustering model designersshould evaluate their models against adversarial samples crafted specifically for deep clusteringmodels (and not those crafted for supervised models, as in [9]).
In the wrong hands our attackapproach can have negative societal impacts, but we believe through this work we can actually drivethe development of truly robust clustering models, thus offsetting the impact of such threats.
We arepositive that improved defenses and robust models will arise as a result of this work.
There are a fewlimitations of our work as well– for the most effective attack, all the conditions for the threat modelneed to be met. However, as we show for the Face++ API, even when some conditions are not met(for Face++ softmax cluster memberships could not be obtained), our attack can be successful.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] We have justified the claims made in the introductionand abstract with a extensive results section (Section 6) and provide additional detailsand experiments in the Appendix as well.(b) Did you describe the limitations of your work?
[Yes] We describe limitations in Section7.(c) Did you discuss any potential negative societal impacts of your work? [Yes] We describe broader impact of our work as well as negative societal impacts in the Conclusion(Section 7). (d) Have you read the ethics review guidelines and ensured that your paper conforms to
them? [Yes] We have read through them, and ensured that our paper conforms to the guidelines. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Yes, we include
code, dataset, and implementation details in the Appendix D. We also provide actual
code along with reproducibility instructions in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they were chosen)? [Yes] We provide a complete list of all experiment details and hyperpa- rameter values in Appendix A.1 for the attack on open-source models.
For the defense
approaches, we provide additional experiment details in Appendix B. For the attack on
Face++ we provide additional experiment details in Appendix C.
(c) Did you report error bars (e
g., with respect to the random seed after running exper-
iments multiple times)?
[Yes] We report error bars for experiments where results
will have variance (such as in the figures in Sections 6.2 and 6.3).
Additionally, we
provide the exact values used to create these figures in Appendix B and Appendix C,
respectively.
(d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)? [Yes] We provide details on implementation
and resources used in Appendix D.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses
existing assets, did you cite the creators?
[Yes] We only use existingdatasets in this paper, and cite the respective authors in the main text.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A] If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions
given to
participants and screenshots if applicable? [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]