O RTHOGONAL S UBSPACE D ECOMPOSITION : A NEW PERSPECTIVE OF LEARNING DISCRIMINATIVE FEATURES FOR FACE CLUSTERING
Anonymous authors
Paper under double-blind review
A BSTRACT
Face clustering is an important task, due to its wide applications in practice.
Graph-based face clustering methods have recently made a great progress and achieved new state-of-the-art results.
Learning discriminative node features is the key to further improve the performance of graph-based face clustering.
To this end, we propose subspace learning as a new way to learn discriminative node features, which is implemented by a new orthogonal subspace decomposition (OSD) module.
In graph-based face clustering, OSD leads to more discriminative node features, which better reﬂect the relationship between each pair of faces, thereby boosting the accuracy of face clustering.
Extensive experiments show that OSD outperforms state-of-the-art results with a healthy margin.
I NTRODUCTION
Face clustering has been explored for several years, because of its wide applications, such as online photo albums (Zhu et al., 2011), automatic data labeling or cleaning for dataset construction (Nech & Kemelmacher-Shlizerman, 2017;
Guo et al., 2016; Zhan et al., 2018), and face image or video management Klontz & Jain (2013).
Several traditional face clustering algorithms have been proposed and applied to the extracted hand-crafted or deep features of faces (Lloyd, 1982; Shi &
Malik, 2000; Ester et al., 1996; Zhao et al., 2006; You et al., 2016).
However, these algorithms usually rely on different data distribution assumptions, which limits their applicability.
Therefore, some graph-based clustering methods (Wang et al., 2019; Yang et al., 2019; 2020; Zhang et al., 2020) are proposed in recent years, which make no assumptions on the data distribution.
These graph-based methods achieve new state-of-the-art results on face clustering benchmarks.
To further improve the performance of graph-based clustering methods, learning discriminative node representations can be an effective way.
Most previous methods propose to increase the inter-class distance or to decrease the intra-class distance for learning discriminative features, such as center loss (Wen et al., 2016) and AM-Softmax (Wang et al., 2018b;a).
However, these methods are directly used on neural networks during training without considering to ﬁlter the noise or the redundant information of the learned features.
But removing the noise or redundant information from features is a good strategy to obtain more discriminative representations (Lin et al., 2016; Chinta & Murty, 2012; Shang et al., 2019; Sun et al., 2019), and this process is called feature selection.
Therefore, we propose a new method to learn discriminative features from the perspective of feature selection.
Subspace learning is a classical method for feature selection, which can eliminate redundancy and noise effectively.
However, previous subspace learning methods for feature selection (Wang et al., 2015a; Shang et al., 2020; Wang et al., 2015b; Shang et al., 2019) cannot be combined with neural networks and trained end-to-end, because these methods are unable to be optimized via gradientbased algorithms.
Thus, we propose an orthogonal subspace decomposition (OSD) module that can be combined with neural networks for feature selection and learning discriminative representations.
In this work, we choose to leverage OSD to graph convolutional networks (GCNs) and the instance pivot subgraphs (IPSs) proposed by Wang et al.
IPSs are built around each instance (pivot), to depict the local context, with each node modeling a pivot-neighbor pair.
As for each pivotneighbor pair, if the neighbor node and the pivot node have the same identity, then they should be linked.
Therefore, the face clustering task can also be regarded as a prediction problem on face local
[Figure]
graphs, where faces indicating the same identity are linked together.
In this paper, the “positive” nodes refer to the neighbor nodes that should be linked to the pivot, while the “negative” nodes refer to the neighbor nodes that should not be linked.
A simple example in Fig.
1 explains the main ideas behind OSD:
the gray and yellow points denote input node features in an IPS, whose ground truth are positive and negative, respectively, and they lie in the input space.
Link prediction (LP) is used to transform the input node features into l 2normalized node features and to classify them into two categories.
The objective of OSD is to learn a new space that can be decomposed into two orthogonal subspaces and to make l 2 -normalized node features (i.e., the green and red points) lie in the two subspaces separately.
In the “two orthogonal subspaces”, any base vector in the basis of one subspace is orthogonal to any base vector in the basis of the other subspace, and at the same time, the base vectors in the basis of each subspace are orthogonal to each other.
For example, we assume that a newly learned space is an Euclidean space R 3 , where each point can be denoted by ( x , y , z ).
Intuitively, the Euclidean space R 3 can be decomposed into two orthogonal subspaces, i.e., ( x , y , 0 ) and ( 0 , 0 , z ).
The l 2 -normalized positive and negative node features learned from OSD lie in these two orthogonal subspaces, respectively.
By doing this, OSD implements the feature selection process.
As OSD is required to zero out some dimensions of learned l 2 -normalized node features, it has to select the redundancy or noise to ﬁlter out, and as a result, only important and useful information can be preserved.
As OSD performs feature selection, the new learned l 2 -normalized features will be easier to be classiﬁed or clustered closely, because the loss functions (e.g., cross-entropy loss with softmax) only perform on the dimensions that are important, resulting in more discriminative representations.
Moreover, the orthogonality property in OSD also introduces a ﬁxed margin to better differentiate the two types of nodes, especially for some hard samples.
As illustrated in Fig.
1, the learned l 2 -normalized node features make the Euclidean distance of a pair of nodes fall into a range [0 , 2] , and as for two nodes located on the two subspaces, the distance between them should be, in theory, around √ 2 .
The main contributions of this paper are brieﬂy summarized as follows:
• We propose orthogonal subspace decomposition (OSD), which to our knowledge is the ﬁrst approach to combine subspace learning and feature selection with neural networks.
• We theoretically prove that, when combined with some regularization terms, matrix addition is a good approximation to the direct sum.
As a result, we propose a new loss function, called space reconstruction (SR) loss , which enables OSD to be combined with graph convolutional networks (GCNs) and trained in an end-to-end manner.
• OSD not only outperforms some classical metric learning methods that are designed for learning discriminative features, namely center loss (Wen et al., 2016) and AM-Softmax (Wang et al., 2018b;a), but also surpasses state-of-the-art results on IJB-512, IJB-1024, IJB-1845 (Whitelam et al., 2017), and VoxCeleb2 (Chung et al., 2018).
R ELATED W ORK
Face Clustering.
During the past few years, different types of traditional clustering technologies were proposed.
Some of them are not very suitable for face embeddings, for their rigid assumptions on the data distributions, such as K-means (Lloyd, 1982) and spectral clustering (Shi & Malik, 2000).
Several new clustering methods try to overcome their shortcomings.
Zhu et al.
(2011) designed a new afﬁnity metric, called rank-order distance, which is more robust to noise and outliers.
Lin et al.
(2017) proposed the proximity-aware hierarchical clustering (PAHC) method that employs a linear
SVM to classify local positive and negative instances.
Some agglomerative hierarchical clustering (AHC) algorithms (e.g., (Zhu et al., 2011; Lin et al., 2017; 2018)) were also proposed to tackle clustering problem with complex data distributions.
Zhan et al.
(2018) trained a multi-layer perceptron (MLP) classiﬁer to aggregate information and discover robust linkages, and then ﬁnd connected components to acquire ﬁnal clusters.
Encoder-decoder architectures (e.g., (Dizaji et al., 2017; Xie et al., 2016; Yang et al., 2017)) were proposed to learn low-dimensional embeddings and cluster assignments.
Yang et al.
(2016) designed a recurrent paradigm to jointly update the clusters and feature representations.
Shi et al.
(2017) proposed a ConPac algorithm, which is optimized by using loopy belief propagation to maximize the posterior probability of the adjacency matrix.
Recently, Yang et al.
(2019) proposed a top-down method, based on a detection-segmentation paradigm, which is suitable for clusters with complicated structures.
Wang et al.
(2019) built the IPS and applied GCNs to the prediction of linkage among a pivot face embedding and its kNN neighbors.
Yang et al.
(2020) proposed a new clustering framework composed by two components.
One component is to estimate the conﬁdence of a vertex, while the other is to estimate the edge connectivity.
Zhang et al.
(2020) proposed a two-stage GCN framework, consisting of a global and a local GCN.
The former classiﬁes all nodes globally, while the latter performs on the low conﬁdence nodes locally.
Subspace Learning for Feature Selection.
There are many works related to subspace learning, and here we only focus on those designed for feature selection in recent years.
Wang et al.
(2015a) proposed a subspace learning method based on matrix factorization for unsupervised feature selection, which considers subspace learning as a matrix decomposition problem, and uses the learned feature selection matrix to select important features.
Wang et al.
(2015b) proposed an unsupervised method to obtain an indicator matrix for feature selection via minimizing the projection error and redundancy.
Zhou et al.
(2016) proposed a global and local structure preserving sparse subspace learning (GLoSS) algorithm, which preserves the geometric structure information in data.
Cai et al.
(2010) designed a multi-cluster feature selection (MCFS) method by incorporating sparse subspace learning with feature selection.
MCFS introduces spectral analysis techniques and measures the correlations between different features for feature selection, so that the selected features can best preserve the cluster structure of the data.
Gu et al.
(2011) further improved the MCFS method by using the L 2 , 1 -norm to enforce the row sparsity of the transformation matrix.
Shang et al.
(2019) proposed a local discriminative sparse subspace learning method for feature selection, which aims to exploit local discriminative and geometry information contained in the original features for feature selection, since such information is important for differentiating data samples.
Shang et al.
(2020) proposed a new algorithm called subspace learning for unsupervised feature selection via adaptive structure learning and rank approximation (SLASR), which learns the manifold structure adaptively to make the preserved local geometric structure more accurate and more robust to noise.
M ETHODOLOGY
Preliminaries for OSD module.
The subspace decomposition in the OSD module is implemented by the direct sum (Kreyszig, 1978).
Formally, a vector space Z is the direct sum of two subspaces
X and Y of Z, denoted Z = X ⊕ Y , if each z ∈ Z has a unique representation z
= x + y , where x ∈ X and y ∈ Y .
We call Y an algebraic complement of X in Z and vice versa.
The direct sum combines the basis of two subspaces to form a new whole space.
In our method, we assume that the space of the l 2 -normalized node features learned by OSD is the whole space.
Our goal is to learn two subspaces, and the direct sum of the two subspaces is the whole space.
In this way, we can also regard the two subspaces as decomposed from the whole space.
However, the direct sum cannot be implemented and trained with neural networks in an end-to-end manner, which causes difﬁculties for learning the two subspaces.
Therefore, we provide the following theorem (with proof in Appendix A), which reveals a relationship between the direct sum and subspace matrix addition.
(In this paper, we organize the base vectors of a subspace as a matrix by row for simplicity).
Theorem 1.
Given two subspaces V 1 and V 2 of V .
If V = V 1 + V 2 , then V 1 ∩ V 2 = { (cid:126) 0 } is equivalent to V being the direct sum of V 1 and V 2 .
The above result provides the fundament for our subspace decomposition, as we can simply perform the addition operation on two subspace matrices to approximate the direct sum of them by introducing regularization terms to fulﬁll the condition in this theorem, i.e., making the intersection of
[Figure]
the two learned subspaces only contain (cid:126) 0 .
In fact, how to design regularization terms to fulﬁll this condition is an open question.
Here, we make the basis of one subspace orthogonal to the basis of the other subspace, as the inner product of any two vectors from the two subspaces is zero.
This orthogonality can easily be implemented by adding a regularization term that enforces the product of the two subspace matrices to be the zero matrix.
With this regularization term and matrix addition, we can learn the basis of the two decomposed subspaces and obtain the basis of the whole space.
As the learned l 2 -normalized node features are represented in the whole space, each of them can be uniquely decomposed into two feature vectors that lie in the two subspaces, which is immediate by the uniqueness representation property of the direct sum: by the deﬁnition of the direct sum, if V is the direct sum of V 1 and V 2 , then any v ∈ V can be uniquely decomposed into v 1 ∈ Vand v 2 ∈ V 2 through v = v 1 + v 2 .
Thus, the decomposition of features into each subspace is unique, and the projected features disjoint from each other.
As for a l 2 -normalized node feature, there are many choices in respect of decomposing it into two feature vectors that lie in these two subspaces.
Considering that OSD aims to implement feature selection, which is a process that zeros out some dimensions, we can design some regularization terms that make the base vectors in the basis of each subspace linearly independent of each other, and also make the representation of positive l 2 -normalized node features in the subspace of negative ones to be (cid:126) 0 and vice versa.
In this way, some dimensions of l 2 -normalized positive and negative node features are compulsorily ﬁltered out by OSD, and thus, it should select the important information to preserve and zero out useless or redundant information.
For example, the representation of a l 2 -normalized node feature in a subspace can be considered as a multiplication of the feature and the subspace matrix, which is a linear combination of the basis of the subspace based on the feature vector.
Due to the linearly independent property of the subspaces, the basis of each subspace only contains the zero base vector and a set of non-zero base vectors.
If the representation of the l 2 -normalized positive node features in the subspace that negative node features lie in is (cid:126) 0 , the dimensions of the l 2 -normalized positive node features that are corresponding to the non-zero base vectors should be equal to zero.
In addition, as discussed above, when the two types of l 2 -normalized node features only lie in their own type of subspace, orthogonality introduces a ﬁxed margin between these two types of l 2 -normalized node features, resulting in a more discriminative representation of them.
LP Module.
The LP module, denoted LP(GCN-M), is a GCN constructed by stacking four graph convolutional layers (GCLs), following the GCN-M by Wang et al.
by using the GCL with the mean aggregator.
In detail, the ﬁrst GCL performs on the input node features X ∈ R N × d in , while the other GCLs receive the transformed features from the previous layer.
The output node features from the last graph convolution layer are l 2 -normalized and form a matrix Y ∈ R N × d out , which is fed to the classiﬁcation (softmax) layer for link prediction.
Here, N is the number of nodes, while d and d out are the dimensions of input and output node features, respectively
[Table]
OSD Module.
The OSD module (see Fig. 2) aims to learn the two decomposed orthogonal subspaces, which are implemented by two learnable matrices, denoted S 1 , S 2 ∈ R d out × d out .
Hence,
OSD merely introduces extremely tiny additional computational overhead to the whole framework during training.
We consider each row of the subspaces S 1 and S 2 as a base vector of their basis.
By the deﬁnition of the direct sum, the original whole space S ori where the learned l 2 -normalized node features lie in can be regarded as the direct sum of two subspaces S 1 and S 2 :
[Equation]
To learn the decomposed subspaces, the problem lies in optimizing:
[Equation]
where E [ · ] takes the expectation or overall mean value of an input matrix or batch of input matrices.
( · ) ◦ 2 takes the Hadamard square of the input matrix.
Here, Y ∈ R N × d out is the representation in the original whole space, and Y S 1 ∈ R N × d out and Y S 2 ∈ R N × d out are the representation of Y in the subspaces S 1 and S 2 , respectively.
To implement the direct sum, we try to approximate the subspace matrix addition to the direct sum, as the addition operation enables the gradient to be backpropagated from the top to the bottom.
Therefore, based on Theorem 1, we propose the following regularization term and combine it with the addition operation to implement the direct sum:
[Equation]
Minimizing the ﬁrst term of Eq.
(3) encourages the two subspaces to be orthogonal to each other, which ensures that the intersection of the two subspaces can only contain the zero vector, satisfying
Theorem 1.
Minimizing the second term and the third term of Eq.
(3) penalizes both diagonal and off-diagonal elements of S 1 S T 1 and S 2 S T 2 .
Penalizing off-diagonal elements ensures that the base vectors of each subspace are linearly independent of each other.
Penalizing the diagonal elements encourages OSD to learn low-rank subspace matrices, and by doing this, there will not be too many non-zero base vectors in the two subspaces, avoiding ﬁltering out too much information from Y .
Moreover, as what we have discussed above, we make the representation of learned node features on the other subspace be (cid:126) 0 by introducing the following regularization term:
[Equation]
Here, without loss of generality, we use Y (+) and Y ( − ) to denote a positive l 2 -normalized node feature and a negative l 2 -normalized node feature respectively.
In addition, we also assume that S 1 and
S 2 are the subspaces of positive and negative node features, respectively.
Minimizing Eq.
(4) leads the network to learning features that are only distributed in their own subspace, thereby implementing the feature selection.
Finally, according to the simple example in Sec. 1, the trivial solution I (i.e., the identity matrix) is a good solution for the original space S ori , but not the best.
I restricts the search space of the two decomposed subspaces, as the addition of non-diagonal elements of them u k        92.
F-measure
NMI (%) k(%)
[Figure]
[Table]
[Figure]
[Figure]
should be zero.
Therefore, we propose to penalize the diagonal elements to avoid the trivial solution by designing:
[Equation]
where Diag ( · ) takes the diagonal elements of the input matrix.
Hence, in the real implementation, we can directly add the two subspaces to approximate the direct sum of them by combining the above regularization terms together, resulting in a space reconstruction (SR) loss:
[Equation]
Finally, the total loss of the whole framework is:
[Equation]
Here, L entropy refers to the cross-entropy loss with softmax for classifying the positive and negative l 2 -normalized node features.
In the testing phase, the two subspace matrices are no longer needed, and only LP(GCN-M) is used, so that no additional computation burden is put on the inference.
We follow Wang et al.
(2019) to construct IPS by looping over all the instances.
LP(GCN-M) predicts the likelihood of a linkage between the involved instances and their related pivot nodes.
After the loop, we accordingly adopt the pseudo-label propagation strategy by Zhan et al.
(2018) to merge the links and form clusters.
Optimization.
The OSD module and LP(GCN-M) are jointly trained, and the whole optimization process is summarized in Algorithm 1 .
From now on, we use “OSD-LP(GCN-M)” to denote the whole framework that consists of LP(GCN-M) and the OSD module.
[Table] [Table]
E XPERIMENTS
Datasets.
We conducted extensive experiments on the following datasets.
As for face clustering, we used CASIA-Webface (Yi et al., 2014) as our training set, which consists of 10,575 identities with around 494,414 face images.
FaceScrub (Ng & Winkler, 2014) is a celebrity face dataset that comprises a total of 106,863 face images.
We randomly picked 517 identities
[Table]
and we removed duplicates to construct a subset with 33607 face images.
We used this subset as our validation set.
IJB-B (Whitelam et al., 2017), which has a clustering protocol, was applied to our testing set.
The protocol has 7 subtasks, and we chose the three largest subtasks.
In particular, those three subtasks include 18,171, 36,575, and 68,195 face images from 512, 1,024, and 1,845 identities respectively, denoted IJB-B-512, IJB-B-1024, and IJB-B-1845.
As for multi-view testing, we used
VoxCeleb2 (Chung et al., 2018), which has 145K videos of 5,994 different identities.
We followed the experimental setting of multi-view testing in (Wang et al., 2019) for a fair comparison, i.e., constructing two testing sets by picking 512 and 2048 identities.
The other data were used for the training set.
The implementation details are given in Appendix A.
Ablation Studies.
We now report on our ablation studies on FaceScrub (Ng & Winkler, 2014).
Firstly, we explored how the hyperparameters of IPS impact the performance of OSD-LP(GCN-M), which include the nearest neighbors of the pivot ( k 1 ), the nearest neighbors of each 1-hop node ( k 2 ), and the linked nearest neighbors for picking edges ( u ).
The speciﬁc meaning and function of these hyperparameters can be found in (Wang et al., 2019).
It is not necessary to use the same conﬁguration of constructing IPS as in the training phase.
So, we ﬁrst set k 1 = 40 and only studied different u and k 2 , and our results are reported in Table 1.
We conclude that, for any ﬁxed u , increasing k 2 always leads to a performance gain until k 2 equals to u .
Once k 2 is larger than u , a signiﬁcant drop can be observed in terms of the F-measure.
In contrast, given a ﬁxed k 2 , larger u results in a lower BCubed F-measure in most cases.
The highest BCubed F-measure is obtained when k 2 = u
= 5 .
Then, using this setting, we tested the effect brought by different k 1 .
According to the results in Fig. 3, we found that the F-measure and NMI reach the peak when k 1 = 70 .
Therefore, from now on, we ﬁx k 1 = 70 , k 2 = 5 , and u = 5 in all of the following experiments.
Secondly, we compare the OSD module with two other classical metric learning methods, which are widely used for learning discriminative features, as well as the SR loss.
In detail, in the ﬁrst part of Table 2, we report the result of LP(GCN-M) combined with center loss (Wen et al., 2016) and AMsoftmax (Wang et al., 2018b;a).
Although center loss and AM-softmax can improve the performance
Method
Face
Audio
[Figure]
Audio
F
NMI
F
NMI
F
NMI
[Table]
[Figure]
[Table] [Table]
[Figure]
[Table] [Table]
ARO
[Table] [Table]
of the baseline, i.e., LP(GCN-M), the improvement on LP(GCN-M) brought by OSD is higher than that brought by either of the two methods (last line of Table 2), indicating OSD’s superiority in helping with learning discriminative representations.
The second part of Table 2 shows the results of different combinations of subterms in the SR loss to explore the effect of the SR loss on the OSD module: once we remove any subterm of the SR loss, a decline can be observed with regard to
F-measure and NMI, conﬁrming the correctness of the theory and the importance of the SR loss.
Finally, we explored the weights of the loss terms in L total , and the results are shown in Fig.
Note that each curve in Fig.
4 reﬂects the weight of a loss term, and when the weight changes, the weight of the other loss terms are ﬁxed to 1.
We can conclude that increasing or decreasing the weight of L entropy and L norm leads to a performance degradation.
As for the other loss terms, declining their weights from 1.0 to 0.001 will hurt the performance of the model.
By contrast, further increasing the weights does not affect the performance too much.
Concerning that, it is difﬁcult to do a comprehensive grid search of those hyperparameters in order to ﬁnd the best ones, thus we chose equal weights for those loss terms during training.
Comparison with State-of-the-Art Methods.
We now compare our method to previous ones.
The ﬁrst part of Table 3 shows some traditional clustering methods on the IJB-B dataset.
Due to their restricted assumptions on features (except for AHC), most of those methods always perform worse than ours with a signiﬁcant margin.
As for AHC, it performs best among traditional methods, reaching approximately 79.5% F-measure and 0.920 NMI on the test set.
However, OSD-LP(GCNM) still outperforms AHC with a large margin.
In the second part of Table 3, we list the state-of-the-art methods proposed in recent years.
It is obvious that our method surpasses previous state-of-the-art methods on the three subtasks with regard to both F-measure and NMI score.
Note that OSD is additively used on the LP(GCN-M), and therefore the fact that OSD-LP(GCN-M) outperforms GCN-M in (Wang et al., 2019) directly demonstrates the availability of the proposed method.
Besides, we followed Wang et al.
and tested the OSD module on the face embeddings from
ResNet-50 (He et al., 2016) model trained with softmax and cross-entropy loss.
The results are shown in Table 4, and we can conclude that OSD-LP(GCN-M) can still outperform previous methods based on different face embeddings, validating the versatility of the OSD module.
Multi-View Testing.
To further verify the generalization of our OSD module, we followed Wang et al.
and also evaluated our model on multi-view data.
We utilized face embeddings and audio embeddings to evaluate OSD on video face data clustering.
According to the results in Tablesand 6, although the data distribution of face embeddings is quite different from that of audio embeddings, OSD can improve the performance on both of them, which means that OSD is a general module, and it works well when used for different data modalities.
Moreover, OSD also achieves new state-of-the-art results in the combination of multi-view data.
S UMMARY AND O UTLOOK
In this work, we have proposed orthogonal subspace decomposition (OSD), which implements a new way to help the network to learn discriminative features via subspace learning and feature selection.
We have also reported on extensive experiments, which verify the effectiveness of the proposed method.
Moreover, to our knowledge, this is the ﬁrst work to simultaneously combine subspace learning and feature selection with neural networks.
Finally, this work provides a new research direction, i.e., how to leverage feature selection and subspace learning to encourage neural networks to learn more discriminative representations.
A A PPENDIX A. P ROOF OF T HEOREM
Proof.
Given the two subspaces V 1 and V 2 of V .
Sufﬁciency: Suppose V = V 1 + V 2 and V 1 ∩ V 2 = { (cid:126) 0 } .
We treat a subspace as a set of vectors.
As for the addition of two sets, V 1 + V 2 = { v 1 + v 2 : v 1 ∈ V 1 , v 2 ∈ V 2 } , which means that any vector in V is the sum of a vector from V 1 and a vector from V 2 , and then we should prove the uniqueness of this addition.
As for a vector v ∈ V , we assume v 1 ∈ V 1 , v 2 ∈ V 2 , v = v 1 + v 2 , and v 3 ∈ V 1 , v 4 ∈ V 2 , v = v 3 + v 4 .
Since V 1 ∩ V 2 = { (cid:126) 0 } , v 1 + v 2 = v 3 + v 4 → v 1 − v 3 = v 4 − v 2 .
Due to v 1 − v 3 ∈ V 1 and v 4 − v 2 ∈ V 2 , v 1 − v 3 = v 4 − v 2 = (cid:126) 0 .
Thus, v 1 = v 3 and v 4 = v 2 .
Necessity: Suppose V is the direct sum of V 1 and V 2 .
By deﬁnition, V = V 1 + V 2 holds.
If v ∈ V 1 ∩ V 2 , considering (cid:126) 0 ∈ V , we obtain v +( − v )
= (cid:126)
with v ∈ V 1 and − v ∈ V 2 by the deﬁnition of the direct sum.
If v is not a (cid:126) 0 , (cid:126) 0 can be represented by (cid:126)
+ (cid:126)
and v + ( − v ) , which is not a unique representation and violates the deﬁnition of the direct sum.
So, V 1 ∩ V 2 = { (cid:126) 0 } .
A. I MPLEMENTATION D ETAILS
Initially, the embeddings of faces in CASIA-Webface (Yi et al., 2014), FaceScrub (Ng & Winkler, 2014), and IJB-B (Whitelam et al., 2017) were obtained by using the ArcFace model (Deng et al., 2019) with ResNet-101.
The model was trained on the union of MS-Celeb-1M (Guo et al., 2016) and VGGFace2 (Cao et al., 2018).
Then, we used the method in (Wang et al., 2019) to construct the instance pivot subgraph (IPS) as the input of whole framework, and we followed their training setup by setting k 1 = 80 , k 2 = 8 , and u = 8 .
The LP(GCN-M) is the same as the link prediction model constructed by GCN-M in (Wang et al., 2019).
In particular, the dimensions of the output features of the ﬁrst three graph convolution layers were all 512 , while that of the last one was 256 .
As for OSD, the dimensions of the two learned subspace matrices were all 256 × 256 .
The whole framework was optimized by stochastic gradient descent (SGD) with 0.9 momentum and 0.0005 weight decay.
We set the batch size to 16, and the model was trained for 3 epochs.
The learning rate was set to 10 − 3 at ﬁrst, and then multiplied by 0.1 after each epoch.
The last saved model was used for evaluation, and the evaluation metrics that we used are the same as those in (Wang et al., 2019), namely, normalized mutual information (NMI) and the BCubed F-measure (Amig ´ o
et al., 2009).
As for the multi-view testing, we simply concatenated face embeddings with matched audio embeddings and constructed
IPS to conduct the “Face + Audio” experiment, using the same training process as described above.
A. G ENERALITY OF OSD
OSD is designed to learn discriminative representations that can better differentiate images from different categories.
Hence, OSD may have the potential to be further extended in other computer vision tasks, such as object classiﬁcation and face identiﬁcation.
We further studied the generality of OSD by conducting an experiment with respect to object classiﬁcation on CIFAR-10 and FashionMNIST.
According to the results in Table 7, when the neural networks were equipped with OSD, the performance of them was improved.
The improvement brought by OSD is even slightly higher than the most common margin-based method (i.e., AM-Softmax), demonstrating the generality of OSD and its potential in terms of learning superior representations.
[Table]
A. V ISUALIZATION OF NODE FEATURES LEARNED BY OSD
To better explain what OSD learns, we visualize the node features learned by OSD.
We randomly sampled 1000 nodes consisting of positive or negative ones to their pivot nodes from the training set, and used t-SNE (Maaten & Hinton, 2008) to perform dimension-reduction and visualize the node features in Fig.
Without OSD
(Fig. 5, left), the representation of positive and negative nodes is not discriminative enough; the distribution of the negative node features is dispersed, and some positive and negative node features are even mixed.
Such learned features may have bad effects on the performance of graph-based face clustering.
With OSD-LP(GCN-M) (Fig. 5, right), there is still a clear gap between the learned node features, although we do not speciﬁcally design penalties or loss functions to maximize the margin between the two types of nodes.