## 4 Optimization for ISMLP
The MNIST contains 70,000 grayscale handwritten digital images from ten classes, whereas Fashion-MNIST consists of 70,000 images from ten fashion objects. MNIST and Fashion-MNIST are wildly used in the field of semi-supervised learning. The latter two datasets CUB-200 and Cars-196 are two fine-grained visual recognition datasets. The detailed information on the datasets is listed in Table 1.

Figure 1: The variations of the normalized objective function values of the proposed method with the number of iterations on MNIST, Corel 5K, and Cars-196. Obviously, the loss value decreases with the number of iterations and inclines to become stable after several iterations. (**a**) MNIST; (**b**) Corel 5K; (**c**) Cars-196. The pixel value of MNIST and Fashion-MNIST datasets are served as the image feature, which provides us with a 784-dimensional feature. As for the other datasets, the VGG-19 network [47] pre-trained on ImageNet is utilized to extract the features. Since the dimensionality of the features is extremely high, PCA is adopted to reduce the dimensionality of each feature to a 150-dimensional subspace. **Evaluation protocol:** Given that each dataset comes with a default partition of training/testing set, we adopt the same strategy for consistency. Additionally, for each dataset, we set aside 1000 instances from the training data to form the validation set (Since Corel 5K has a default partition of the validation set, we exclude it). The specific number of instances used for validation can be found in Table 1. For the MNIST and Fashion-MNIST datasets, we adopt 3-nearest neighbors to quantify the performance of each compared method, whereas, for the CUB-200 and Cars-196 datasets, we report the Recall@K (abbreviation as R@K) performance, where R@K reflects the proportion of the correct samples in the return \(K\) samples. More specifically, R@1, R@2, R@4, and R@8 index is utilized to measure the performance of each method. We report the R@K of each method under different labeling rates, namely 5%, 10%, and 30%, the rest samples in the training set serve as the unlabeled data. **Compared methods:** We compare the proposed ISMLP with several state-of-the-art semi-supervised metric learning methods including, LSeMML [22], SERAPH [28], S-RLMM [29], LRML [33], SLRML [19], APIT [21], CMM [1], and APLLR [21]. One supervised metric learning method LMNN [48], one deep semi-supervised metric learning entitled as SSML-DR [49], the Euclidean distance denoted as EUCLID is also adopted for a baseline method. The hyper-parameters of all the methods are tuned on the validation set and we choose those parameters that achieve the best results on the validation set. For example, for LMNN, we tune \(\lambda\) from the set \(\{0.1,0.2,\cdots,0.9\}\). As for the proposed method, we empirically set \(p\) as 50, and choose \(\mu\) from the range \(\{0.00001,0.0001,\cdots,1000\}\), and tune the \(\lambda\) from the range \(\{0.0001,0.001,\cdots,100,1000\}\), the number for the proxy vector is chosen from \(\{\#\text{Class},2\#\text{Class},3\#\text{Class}\}\). For SSML-DR, to make a fair comparison, a three-layer full-connected neural network whose nodes are \(\{128,256,128\}\) is incorporated as the backbone network by SSML-DR. The batch size is set as 100, and the number of epochs is set as 50. ### Classification Experimental Result on MNIST and Fashion-MNIST Datasets

In this section, we test the classification performance of the compared methods based on 3-nearest neighbors on MNIST and Fashion-MNIST datasets. To mitigate the influence of the random partition of the dataset, we repeat each task 30 times, and the mean accuracy and standard deviation are recorded to quantify the performance of each method. Tables 2 and 3 record the mean error rate and standard deviation of all the methods on MNIST and Fashion-MNIST datasets, respectively. It is readily seen that the metric learning methods can boost the performance of \(k\)-nearest neighbors classification, and all the methods can benefit from the amount of labeled data.