Supplementary Materials : Deep Subspace Clustering with Data Augmentation

In this section, we provide the details of the image operations that were used in our search space for augmentation policies, a sensibility test for the EMA decay parameter α in our model, and our procedure of reducing the search space of augmentation policies is explained. Finally, we provide some additional experiments for evaluation of our proposed method. 1 Details of Image Operations Used in Our Augmentation Policies As mentioned in Section 5.1, our image operation search space for augmentation policies is the following set: {FlipLR, ShearX, FlipUD, SearY, Posterize, Rotate, Invert, Brightness, Equalize, Solarize, Contrast, TranslateY, TranslateX, AutoContrust, Sharpness, Cutout}. Table 1 lists these transformation techniques. The description of the magnitude is shown in the second column. In addition, the ranges of the magnitudes are presented in the third column. There are some transformations such as Invert and Equalize that do not use the magnitude information. 2 EMA Decay Parameter In all of our experiments, we used the training settings of MLRDSC described in [2] for our MLRDSCDA model. This includes the same architecture for networks and values for the hyper-parameters λ1, λ2, λ3 in different experiments as well as the initial values for the membership matrix Q̂, the coefficient matrices G and Dl s at the iteration t = 0. Our model introduces one additional regularization parameter to MLRDSC (the EMA decay α in (5)). In this section, we analyze the sensibility of the proposed method to determine the α value. Table 4 shows the effect of this parameter on the performance of the EMA decay on the Extended Yale-B dataset. In Table 4, we test different value of α over the range of [0.09, 0.9999]. As can be seen from the table, higher values generally resulted in a better performance. This is inline with the choice of EMA decay value in previous works [3]. 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada. Table 1: List of the image transformations in our search space. Operation Name Description Magnitudes’ range ShearX(Y) Shearing the image along the horizontal ( or vertical) axis with rate magnitude. [-0.3,0.3] TranslateX(Y) Translating the image in the horizontal (or vertical) direction by magnitude number of pixels. [-15,15] FlipRL(UD) Flipping in the right-left (Up-down) direction. Rotate Rotating the image by magnitude degrees. [-30,30] AutoContrast Maximizing the the image contrast, by making the darkest pixel black and lightest pixel white. Invert Inverting the pixels of the image. Equalize Equalizing the image histogram. Solarize Inverting all pixels above a threshold value of magnitude. [0,256] Posterize Reducing the number of bits for each pixel to magnitude bits. [4,8] Contrast Controlling the contrast of the image. A magnitude=0 gives a gray image, whereas magnitude=1 gives the original image. [0.1,1.9] Brightness Adjusting the brightness of the image. By the scaled magnitude, where magnitude=0 gives a black image, whereas magnitude=1 gives the original image. [0.1,1.9] Sharpness Adjusting the sharpness of the image. A magnitude=0 gives a blurred image, whereas magnitude=1 gives the original image. [0.1,1.9] Cutout [1] Setting a random square patch of side-length magnitude pixels to gray. [0,6] Table 2: Clustering error (%) on Extended Yale B with different α values of EMA decay for MLRDSC-DA. The α Values: α = 0.09 α = 0.9 α = 0.99 α = 0.999 α = 0.9999 Extended YaleB 3.04 2.88 0.9 0.82 0.86 3 Reducing The Search Space of Augmentation Policies As mentioned in Section 5.2, to reduce the computations, we used a greedy search [4] in the search space of augmentation policies with the maximum number of sub-policies `max = 2 (i.e. up to two sub-policies can be combined to construct a policy), and set the probability to p = 0.1 and the magnitude to m = 0.3 × r where r = (max−min) is the magnitude range that image operations accept. The range of each image operation is shown in Table 1. We selected the values p = 0.1 and m = 0.3 × r, for magnitude and probability of augmentation polices by searching in a larger search space of augmentation policies for the first two subjects in the Extended Yale B dataset. In particular, we created a subset of Extended Yale B dataset with the first 128 samples corresponding to the first two subjects of the dataset. We ran the search algorithm for a range of different values for p and m, and chose the median value of p and m that resulted in the best mean Silhouette scores across different image operations. For both p and m we searched through values of [0.1, 0.2, . , 1.0]. That means, for each probability, there are 10 number of magnitudes. In most of the different image operations, p = 0.1 and m = 0.3 resulted in better performances. 4 Additional Evaluations Runtime: In Table 2, we report the runtime for the policy search algorithm on the COIL-20 dataset. Silhouette Coefficient v.s. Ground-truth: We tested our augmentation algorithm with accuracy as the Score on the COIL-20 dataset, and observed that for a set of 9 best augmentation policies it achieves the same clustering error rate of 1.79. Our found augmentations in Table 1 of the main