G ENERALIZED STRUCTURE - AWARE MISSING VIEW COMPLETION NETWORK FOR INCOMPLETE MULTIVIEW CLUSTERING
Anonymous authors
Paper under double-blind review
A
BSTRACT
In recent years, incomplete multi-view clustering has been widely regarded as a challenging problem.
The missing views inevitably damage the effective information of the multi-view data itself.
To date, existing methods for incomplete multi-view clustering usually bypass invalid views according to prior missing information, which is considered as a second-best scheme based on evasion.
Other methods that attempt to recover missing information are mostly applicable to speciﬁc two-view datasets.
To handle these problems, we design a general structure-aware missing view completion network (SMVC) for incomplete multiview clustering.
Concretely, we build a two-stage autoencoder network with the self-attention structure to synchronously extract high-level semantic representations of multiple views and recover the missing data.
In addition, we develop a recurrent graph reconstruction mechanism that cleverly leverages the restored views to promote the representation learning and the further data reconstruction.
Sufﬁcient experimental results conﬁrm that our SMVC has obvious advantages over other top methods.
I NTRODUCTION
[Figure]
It is well known that multi-view data depicts the observed objects from different perspectives (Tsai et al., 2020; Han et al., 2020; Li et al., 2021; Wang et al., 2015).
Compared with traditional singleview data, this heterogeneous data retains multi-level semantic information (Wen et al., 2020a).
In the past few years, multi-view clustering, as a novel representation learning method, has aroused extensive research enthusiasm and has been widely used in related ﬁelds such as data analysis (Zhang et al., 2019).
However, conventional multi-view clustering methods usually assume that obtaining complete multi-view data is a matter of course, which obviously goes against practical experience.
Therefore, a number of incomplete multi-view clustering (IMC) approaches have been developed to ﬁt the increasingly common incomplete multi-view datasets, which is also the focus of our paper.
In the area of incomplete multi-view learning, there are two main technical approaches to missing multi-view data, i.e. , detour –skip unavailable views by prior missing information or completion – recover missing data according to existing information.
In general, it is more difﬁcult to get good recovery quality because the information content is indeed incomplete, so most methods choose to skip these missing data to avoid a bad impact.
Classical partial multi-view clustering (PMVC)
(Li et al., 2014) is one of the most representative IMC methods, which tries to connect different view spaces employing the samples with all views.
On this basis, IMG (incomplete multi-modality grouping) (Zhao et al., 2016) constructs a complete Laplacian graph with the common representation in the latent space to provide the global property to subspace learning.
However, both of these two grouping methods require at least one complete sample in the multi-view database.
Online multi-view clustering (OMVC)
(Shao et al., 2016) and One-pass
IMC (OPIMVC)
(Hu & Chen, 2019b) introduce a weight matrix with missing prior information to perform multi-view weighted matrix factorization, assigning low weights to missing views to reduce the negative impact caused by mean padding or zero padding.
However, such detour or neglect is only the second-best solution to the incomplete issue.
Some researchers attempt to recover missing information in various ways to perform complete multi-view clustering.
Rai et al.
(2010) proposed the kernel canonical correlation analysis with incomplete views (KCCA-IV) to restore the kernel matrix of incomplete view based on the complete kernel, which requires at least one view to be intact.
The multiple kernel k-means with incomplete kernels (MKKM-IK-MKC) (Liu et al., 2019) fuses kernel completion and clustering into one framework.
Wen et al.
(2019) proposed an IMC framework named uniﬁed embedding alignment framework (UEAF), which emphasizes the importance of recovering missing views, and introduces both forward and reverse graph regularization to facilitate view recovery and cross-view consistent representation learning.
Considering the above issues, in this paper, we aim to propose a general IMC framework, called
S tructure-aware M issing V iew C ompletion network (SMVC), which can handle arbitrary viewmissing situations and enhance the performance of representation learning through efﬁcient view completion.
Meanwhile, inspired by the successful application of deep learning, especially the transformer (Vaswani et al., 2017), we combine the characteristics of the multi-head self-attention mechanism and multi-view learning to design a transformer-style cross-view autoencoder network.
Compared with a simple linear encoder, it can extract high-level semantic features and support crossview information interaction, which is conducive to mining the complementarity of multiple views.
At the same time, we skillfully integrate multi-view fusion representation learning and missing view recovery into a uniﬁed framework.
More importantly, we propose a structure-aware module (recurrent graph constraint) to push the reconstructed data inversely to participate in the representation learning process, where they are allowed to collaborate with each other to achieve better clustering.
Finally, our contributions are summarized as follows:
• We design a general IMC framework named SMVC, which includes an encoder module that integrates cross-view information interaction and high-level semantic feature extraction, and a multi-view reconstruction and recovery module based on controlled coding.
• We propose an innovative recurrent graph embedding constraint whose core, an approximately complete graph generated from imputation data, cyclically facilitates reliable feature extraction and view recovery.
A cluster-friendly two-stage training strategy is presented in detail, and extensive experiments and intuitive visualization results demonstrate the effectiveness of our SMVC.
P RELIMINARY 2. P ROBLEM DEFINITION AND NOTATIONS
For ease of expression, we ﬁrst give a formal deﬁnition of the investigated problem.
Given the multi-view data { X ( v ) ∈ R n × d v } mv =1 with m views and n samples, our goal is to divide them into c cluster centroids.
d v is the dimension of view v and the dimension of embedding feature is d e .
In our method, a missing indicator matrix is introduced, i.e. , W , whose element W i,j =denotes the j -th view of i -th sample is available, otherwise W i,j = 0 .
¯ X ( v ) ∈ R n × d v represents the reconstructed view v including missing instances and X (cid:48)
( v ) ∈ R n × d v is the imputation view v ﬁlled with recovered data.
Z ∈ R n × m × d e is the extracted embedding tensor and its fusion representation ¯
Z ∈ R n × d e is our objective matrix.
Following Goodfellow et al.
(2016), all subscript representations of matrices or tensors conform to the recommended criteria.
2. R ELATED WORK : UEAF
In this subsection, we simply introduce a related IMC method: UEAF (Wen et al., 2019).
Like most
IMC methods, UEAF aims to learn a consensus representation ( P in this work) for all views.
To do this, Wen et al.
developed a complex framework that integrates view recovery, consensus learning,
FC View
View! View!
FC FC
FC FC
View
View! View!
Fusion
FC
FC FC
FC FC
View View
View View
View
K Q
V
K Q
V
View View
View View
View{𝑋 𝒗 } 𝒗"𝟏𝒎 𝐙 ∈ 𝑅 𝒏×𝒎×𝒅 𝒆 {𝑿( 𝒗 } 𝒗"𝟏𝒎 {𝑋′ 𝒗 } 𝒗"𝟏𝒎
FC
FC FC
FC FC
View View
View View
View
Fusion
FC
FC FC
FC FC
View View
View View
View
K Q
V
K Q
V
View View
View View
View View
View View
View
View{𝑋′ 𝒗 }
[Figure]
"𝟏𝒎
Masked cross-view encoder
Cross-view encoder
[Figure]
[Figure]
𝒆 𝒁( ∈
[Figure]
𝒆
Stage Stage
[Figure] [Figure]
[Figure]
Recurrent graph embedding 𝑳 𝒓𝒆 decoder imputation and two graph constraint terms into one model:
[Figure]
[Equation]
 
[Equation]
 
U
[Equation]
T
[Equation]
= I m
[Equation]
v =1 α v = 1 ,α v
[Equation]
∀ i, S i, : 1 =(1)
Such a complex model contains many variables, so we only explain the core parts in detail.
E ( v ) is the error matrix used to model the missing instances and M is the transition matrix from missing instances to complete views.
U ( v ) and P ( v ) are the basis matrix and low-dimensional representation of view v .
The most commendable part is the two Laplacian constraint terms, i.e. , the Laplacian matrix L ( v ) constructed by the incomplete graph is introduced to constrain the inference of missing views and another nearest neighbor graph S is utilized reversely to align all recovered incomplete views.
Υ is the optimization objective concerning the above variables.
M ETHOD 3. M OTIVATION
As mentioned in the introduction, recovering the missing data must be based on existing information.
We consider this question from the key property of multi-view data.
As we know, different views enjoy same high-level semantic information in the clustering task, i.e. , they’re different descriptions of the same abstract target.
If we can capture the shared high-level semantic information, then it becomes possible to infer the missing information backwards based on the learned patterns.
From another perspective, missing data inference can be regarded as a generation task, which is usually implemented via autoencoder networks.
Inspired by above analysis, we design a cross-view autoencoder as our main framework, whose encoder learns the high-level semantic representations and decoder attempts to recover the missing views from a fusion representation.
Another thing that should not be ignored is that the intrinsic structure of the data is crucial for unsupervised learning, which has been demonstrated by numerous studies.
The classical nearest neighbor graph constraint, widely used in various traditional machine learning methods, enables the extracted semantic representation to maintain the original topology of data, which not only facilitates the learning of clustering structure to a large extent but also drives the model to ‘guess’ the missing data in a more reasonable direction.
Nevertheless, it should be noted that it is hard for us to directly obtain a complete graph from the incomplete data unless we can provide approximately intact data.
So far, re-examining the above motivations, our idea to missing data imputation is gradually blossomed, that is, fusing approximate graph construction and missing view recovery in an uniﬁed autoencoder framework.
At the same time, it must be considered that our downstream task is unsupervised multi-view clustering, which means that view imputation serves the complete multi-view clustering.
To do this, a 2-Stage training strategy is applied in our network, i.e. , Stage 1 : missing view recovery and Stage 2 : complete multi-view clustering based on recovered data.
In Stage 1, the noisy data in missing positions must be addressed to avoid the negative inﬂuence, but in Stage 2, the new imputed data (approximately complete data), generated by the recovered views of Stage 1 and raw incomplete data, is used as the input, i.e. , X (cid:48)
( v ) =
diag ( W : ,v ) X
( v ) + diag (1 − W : ,v ) ¯ X ( v ) , so the entire model can be treated as a common multi-view clustering network without any extra measures in Stage 2.
It is worth noting that the input to Stage 2, X (cid:48) ( v ) , is ﬁxed after the training is completed in Stagebecause frequent changes of input make it difﬁcult for the model to converge.
The remainder of this section elaborates on the details of our model and the main framework is shown in Figure 2.
3. C ROSS - VIEW ENCODER
It is well known that the feature dimensions of raw data are diverse, so it is difﬁcult for a model to fuse multi-view features in the original feature space.
Traditional approaches usually utilize the autoencoders, mainly composed of Fully Connected (FC) layers, to extract view-speciﬁc features, which not only aligns the different dimensions in a common space, but also maps high-dimensional features to a relatively low-dimensional space to facilitate subsequent representation learning.
However, such high-level features only extracted from their corresponding raw views lack information interaction among views.
Speciﬁcally, different views describe the objects from different perspectives, so each view can be considered both unique and biased.
These differences ( i.e. , complementary information) naturally make multi-view data more expressive than single-view data, and how to make full use of complementary information is always one of the key problems in the ﬁeld of multi-view learning.
To do this, we design a transformer-style cross-view encoder with multi-view information aggregation.
At ﬁrst, we need a group of low-level feature extractors to project various views into a common feature space, which allows subsequent modules to process the representations of all views in parallel.
For simplicity, we select m Fully Connected layers as the low-level feature extractors: (cid:8) Φ v ( X ( v ) ; θ v ) =
(cid:99) X ( v ) ∈ R n × d e (cid:9) m v =1 , where θ v denotes the parameters of extractor Φ v .
Notably, (cid:98) X
= (cid:8)(cid:99)
X ( v ) ∈ R n × d e (cid:9) m v =1 ∈ R n × m × d e is the input tensor of the subsequent selfattention module with no information interaction.
And then, the transformer-style view aggregation module is deﬁned as follows:
Given the number of heads h and the embedding (cid:98) X i, : , : of sample i , we utilize h groups of linear layers with weights (cid:8) W q t , W k t , W v t (cid:9) h t =1 to obtain corresponding query, key, and value maps { Q t , K t , V t } ht =1 ∈ R m × d h .
An entire embedding feature is split into h segments with dimension d h = d e /h .
For head t of sample i , we compute their score matrix as follows:
[Equation]
We need to highlight that this S t ∈ R m × m is for the complete views used in Stage 2.
As for the incomplete views in missing view inference stage ( i.e. Stage 1), we deﬁne the masked attention scores (cid:98) S
t ∈ R m × m by:
[Equation]
where the zeroﬁll represents the operation to ﬁll zero value with -1e9 and (cid:12) is the Hadamard product.
This aims to ignore the missing views in the computation of cross-view attentions.
Then, we aggregate all views by:
[Equation]
where A t ∈ R m ×
d h is the new embedding features with information interaction of head t .
Similar to the multi-head transformer, the ﬁnal embedding features of m views can be calculated as A =
Concat ( A 1 , A 2 , ..., A h ) ∈ R m × d e .
Besides, we sequentially input the A into a linear layer, layer norm module, and a multilayer perceptrons to get ﬁnal embedding features Z ∈ R m × d e for each sample ( Z ∈ R n × m × d e for all samples), i.e. , cross-view encoder E : { X ( v ) } m v =1 → Z .
3. V IEW RECOVERY BASED ON CONTROLLED ENCODING
As mentioned in the previous subsection, all views exchange information in the extraction of highlevel semantic representations.
However, no appropriate constraint be imposed in the extraction of encoder, which means that the encoding process is uncontrolled.
To solve this problem, we exploit a simple but effective approach, named multi-view weighted fusion, to obtain a common representation of all views, which is expected to comprehensively characterize the sample across views.
Speciﬁcally, our multi-view weighted fusion is formulated as follows:
[Equation]
where vector ¯ Z i, : denotes the i -th sample of the fusion matrix ¯ Z ∈ R n × d e .
Obviously, ¯ Z in Eq.
(5) is designed for Stage 1, and ¯ Z can be simply calculated by ¯ Z i, :
= 1 m (cid:80) v Z i,v, : in Stage 2.
Furthermore, the common representation ¯ Z is required to contain all information of multiple views.
To the end, a symmetrical cross-view decoder module D is concatenated to ¯ Z to reconstruct all the data including missing views, i.e. , D : { ¯ Z } m ∈ R n × m × d e → { ¯ X ( v ) ∈ R n × d v } mv =1 .
But in fact, due to the lack of supervisory information to directly discriminate the recovered data, we can only leverage the available original data to impose a partial reconstruction constraint.
In other words, this recovery is a natural but necessary byproduct of the proposed autoencoder framework that aims to learn the common representation shared by the available views.
As a result, we introduce a weighted reconstruction loss L re :
[Equation]
where ¯ X i, : is the i -th sample of the reconstructed data ¯ X .
And in Stage 2, the L re degenerates into
[Equation]
3. R ECURRENT GRAPH CONSTRAINT
In recent years, researchers have been accustomed to adding graph constraints to traditional multiview learning methods, which help preserve the original intrinsic structure of data by constructing a prior adjacency matrix.
This is based on this basic manifold assumption: if two samples are close to each other in original feature space, then they are also close in the embedding space.
But in the case of incomplete data, some existing methods simply skip the missing views to construct the adjacency graph, which is obviously biased, especially on the databases with lager missing rates.
Therefore, we expect to obtain an approximately complete adjacency graph to guide the encoder for the extraction of high-level semantic features.
On the other hand, more discriminative semantic features can also facilitate the recovery of missing views and thus help to construct a more realistic graph.
Combining these two points, we innovatively propose the recurrent graph constraint:
[Equation] [Equation]
where (cid:8) G ( v ) ∈ R n × n (cid:9) m v
=1 are the approximately complete graphs generated by imputation data of last epoch, i.e. , G ( v ) | k − 1 = knn ( X (cid:48) ( v ) | k − 1 , K ) and ‘ | k ’ denotes the k -th epoch.
K is the number of nearest neighbors.
G ( v ) i,j = 1 means instance j is one of the K -neighbors of instance j in view v .
The L rg is executed only if k > 0 .
In fact, deep learning models are usually trained in mini-batch iterations to reduce memory overhead, and ours is no exception.
But in doing so, the neighbor graph contains only a small batch of samples, which means that the graph constraint is local rather than global.
In order to balance the mini-batch training approach and the intact graph constraint, we rewrite Eq.
(8) in the case of mini-batch training as follows:
[Equation]
where the size of mini-batch is b , and Z batch ∈ R b
× m × d e is the output of the cross-view encoder corresponding to each mini-batch data.
In the computation of graph loss, to maximize the use of global structural information, we preserve all the embedded features Z obtained in the last epoch and update its corresponding part after the current batch is processed.
Similarly, in Stage 2, the graph G from Stage 1 is ﬁxed rather than updated by last output, i.e. , G ( v ) | k − 1 = G ( v ) | k .
3. O VERALL LOSS FUNCTION AND CLUSTERING
To sum up, our overall loss function in Stage 1 is:
[Equation]
β is penalty parameter to balance the two losses.
And the loss function in Stage 2 is:
[Equation]
As mentioned above, we conduct complete multi-view clustering in Stage 2, and the fusion embedding feature ¯ Z ∈ R n × d e obtained in Stage 2 is regarded as our clustering indicator matrix.
For simplicity, we perform K -means (MacQueen, 1967) on the ¯ Z to obtain ﬁnal clustering results p ∈ R c .
E XPERIMENTS 4. E XPERIMENTAL SETTINGS
Databases : In order to evaluate the performance of our model, we adopt ﬁve popular databases in our comparison experiments: (1) Handwritten digit (Asuncion & Newman, 2007), a dataset widely used in various ﬁelds, contains 2000 handwritten digital images with ten classes from ’0’ to ’9’.
There are ﬁve categories of features as 5 views selected in our experiments.
The ﬁrst view represents a linear combination of pixels from the original picture with the size of 16 × 15 .
(2) Caltech
(Cai et al., 2013) is a subset of the Caltech101 database (Fei-Fei et al., 2004), and we select 1474 images covering seven categories from it.
Each image is extracted with 6 types of features, i.e, LBP, Gist,
Hog, Cenhist, Gabor, and wavelet-moments.
(3) NH face (Cao et al., 2015), as a subset of the NH database, is composed of 4660 images belonging to ﬁve persons in the movie ’
Notting Hill’ (Wu et al., 2013).
3 views in terms of Gabor, gray pixels (size of 40 × 50 ), and LBP are included in the NH face database.
(4) Animal (Fei-Fei et al., 2004; Zhang et al., 2019) is a larger dataset with up to 10158 images and 50 categories, whose features are extracted by DECAF (Krizhevsky et al., 2017) and Vgg19 (Simonyan & Zisserman, 2014).
All samples and features are adopted in our experiments.
(5) Aloi deep is a new multi-view database proposed by this paper, which is derived from the Aloi database (Geusebroek et al., 2005).
The original Aloi database containsimages of 1000 small objects.
We select 100 objects as 100 categories, each with about 108 images, for a total of 10800 images.
And we utilize three typical deep neural networks, i.e. , ResNet50 (He et al., 2016), Vgg16 (Simonyan & Zisserman, 2014), and Inception-v3 (Szegedy et al., 2016) with pre-trained weights, to extract the three-view features.
Detailed information about the ﬁve datasets is listed in Table 7.
Preprocessing of incomplete datasets : To generate the incomplete datasets to simulate the missingview case, following Wen et al.
(2020b)
, we randomly disable [10%,30%,50%,70%] of the instances of each view but keep at least one view available for each sample.
As for the Animal dataset with only two views, we randomly select [10%,30%,50%] of all samples as the paired samples with two views.
The ﬁrst view is removed for half of the remaining samples, and the second view is removed for the other half.
Comparison methods : In our experiments, eight state-of-the-art methods are selected to evaluate the performance of the proposed SMVC, of which OMVC (Shao et al., 2016), OPIMC (Hu & Chen, 2019b), MKKM-IK-MKC (Liu et al., 2019), and UEAF (Wen et al., 2019) have been described in the introduction.
The other four comparison methods are as follows: (1) BSV (Zhao et al., 2016), a simple baseline method, ﬁlls missing views with the average vector and performs K -means on each view to obtain the best result.
(2) Concat (Zhao et al., 2016) is another popular baseline method, which aligns all views with the same imputation strategy as BSV and simply concatenates them to conduct single-view clustering.
(3) MIC (Shao et al., 2015), based on non-negative matrix factorization technology, also introduces the prior missing information to help learn robust latent representations.
(4) DAIMC (Hu & Chen, 2019a) not only introduces a prior matrix to avoid the negative impact of missing instances, but also additionally introduces a regression coefﬁcient matrix to align the basis matrix of individual views in the latent space.
Evaluation :
Following Wen et al.
; Liu et al.
(2020), we still select the clustering accuracy (ACC), normalized mutual information (NMI), and purity as our three metrics to evaluate these methods.
The higher the values of the three metrics, the better the clustering performance.
Besides, all comparison methods are performed multiple times to reduce randomness and their parameters are set as suggested in their papers or codes for a fair comparison
[Table]
4. E XPERIMENTAL RESULTS AND ANALYSIS
[Table]
[Table] [Table]
As shown in Table 1-5, the values of ACC, NMI, and Purity are listed and corresponding standard deviations are given after the sign ’ ± ’.
We report the nine methods on each dataset with different incomplete rates or paired rates in these tables and mask the best or second-best results in bold or underline.
Looking closely at these data, we can easily get a few points: (1) Our approach shines brightly, beating other state-of-the-art methods in almost all metrics.
For instance, our SMVC exceeds the second-best DAIMC in the ACC metric by approximately 10, 11, and 11 percentage points on NH face dataset with three different incomplete rates, respectively.
The good performance of our method on the clustering demonstrates that the high-level semantic representation extracted by it is effective and solid.
(2) Comparing all results horizontally, we conclude that the harm of
[Table] [Table]
missing views to multi-view learning is deﬁnite and the higher missing rates lead to worse learning outcomes in most cases, which is intuitive and comprehensible.
Besides, different IMC methods have different immunity to data incompleteness, such as, UEAF and our approach are relatively insensitive to the missing views due to its powerful view recovery capability
[Table]
[Figure]
In Figure 3, we give the six visual recovery results by reshaping the pixel feature in the ﬁrst view of Handwritten database and the second view of NH face database.
The ﬁrst row is the raw missing views, and the second row denotes the recovery results of our method.
As we can see, our SMVC has an amazing recovery effect on missing views, which is very beneﬁcial for the subsequent generation of approximately complete graph.
4. A BLATION STUDY
To conﬁrm the effectiveness of each component of our SMVC, we respectively remove the crossview en-decoder module leaving only the FC layers, recurrent graph constraint, and missing indicator matrix to form new models.
Results of these methods on Handwritten and NH face databases with a 50% missing ratio are shown in Table 6.
From the table, it is clear that the full version of SMVC achieves the best performance.
Furthermore, we ﬁnd that the introduction of missing prior matrix is crucial for the recovery phase of SMVC, which well avoids the negative inﬂuence of missing data.
[Table]
4. T-SNE VISUALIZATION RESULTS
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] In Figure 4, we show the ﬁnal clustering representation of different methods on the Handwritten database with a 30% missing rate via the t-SNE (Van der Maaten & Hinton, 2008) technology.
The result of OPIMC is ignored in the ﬁgure because it obtains the clustering result directly without producing any clustering indicator matrix.
Comparing all t-SNE results, it’s easy to ﬁnd that our
SMVC enjoys the best discrimination performance than other state-of-the-art methods.
C ONCLUSION
In this paper, we propose a general IMC model with a two-stage training strategy that can handle all kinds of random missing datasets.
Distinct from most existing methods, our approach focuses on cleverly recovering missing views and performing complete multi-view clustering.
To do this, we design a transformer-style cross-view autoencoder and propose a structure-aware recurrent graph constraint that circularly promotes the restoration of incomplete views and the preservation of geometry structure within the views, which help to obtain more discriminative semantic fusion information.
Sufﬁcient experimental results conﬁrm that our SMVC has obvious advantages over other top methods.
At the same time, our model can be easily extended to other multi-view classiﬁcation or regression models, which can provide more solid data support for incomplete multi-view learning just by inputting the consistent representation into the classiﬁer or regression layer.
A A PPENDIX A. H YPERPARAMETERS SENSITIVITY STUDY
In our SMVC, there are two key hyperparameters that need to be set, i e. , β and K .
In order to study the sensitivity of our SMVC to the two parameters, we perform a mesh search in different parameter combinations and plot the results in Figure 5.
From the ﬁgure, it is not difﬁcult to choose a pair of appropriate parameters for our SMVC.
For instance, we can select parameters β
and K from the range of [10 − 3 , 10] and [5 , 15] separately for the Handwritten database and the range of [1 , 10] and [5 , 15] separately for the Caltech7 database.
A CC
K 1 e 11 e1 e - 11 e - 21 e -(a)
Handwritten 0.
A CC 0.
K 1 e 11 e1 e - 11 e - 21 e -(b)
Caltech
Input : Incomplete multi view data
( v
(cid:9) m v = view-presence information hyperparameters
Initialization : Construct (cid:8) W ( v ) (cid:9) m v =1 according to the view-presence information.
Initialize parameters of model.
Set training epochs e 1 and e for two stages.
1: while do (cid:46) Stage2:
Compute multi embedding then 4:
Compute
5: end if 6:
Compute reconstructed data decoder reconstruction loss
Impute miss data in (cid:8) X
( v
(cid:9) m v
1 with (cid:8) ¯ X ( v ) (cid:9) m v
1 to get new data (cid:8) X (cid:48) ( v ) (cid:9) m v
Generate graph {G ( v ) } mv =
(cid:8) X (cid:48) ( v ) (cid:9) m v =1 .
Compute total network parameters.
10: end while 11:
X ( v ) (cid:9) m v =1 with (cid:8) X (cid:48) ( v ) (cid:9) m v =1 .
12: while k ≤ e 2 do (cid:46) Stage13:
Compute multi-view embedding tensor Z .
Compute graph loss L rg by Eq.
Compute fusion representation ¯ Z by Eq.
Compute reconstructed data (cid:8)
¯ X ( v ) (cid:9) m v =1 by decoder and reconstruction loss L (cid:48) re by Eq.
Impute miss data in (cid:8) X
( v ) (cid:9) m v =1 with (cid:8) ¯ X
( v ) (cid:9) m v =1 to get new data (cid:8) X (cid:48)
( v ) (cid:9) m v =1 .
Compute total loss L (cid:48) by Eq.
(11) and update network parameters.
19: end while 20: Run K -means on ¯ Z to obtain ﬁnal prediction p .
Output : p
A
A LGORITHM A. C ONVERGENCE STUDY
In order to study the convergence of our SMVC, in Figure 6, we plot two ACC-Loss curves on the
Handwritten dataset and Caltech7 dataset with a 50% missing rate for Stage 2.
From the ﬁgure, thanks to good data recovery, the loss value decreases steadily while the ACC keeps a slow upward trajectory, which demonstrates that our model has a good convergence.
50 Iteration number 0.
ACC
Lo ss
ACCLoss (a) Handwritten     50 Iteration number 0.
ACC
Lo ss
ACCLoss (b) Caltech
A. D ETAILED INFORMATION OF DATABASES
[Table]
A. I MPLEMENTATION DETAILS
In this subsection, we mainly present the implementation details of our model and experimental environment.
The two transformer-style cross-view en-decoders are set as one layer with 4 heads.
The learning rate is 0 .
and we select the Adaptive Moment Estimation (Adam) as our optimizer.
For all databases, we set 50 epochs for each of the two stages.
All experiments are performed on a personal computer with an Intel 10700k CPU, RTX2080s GPU, Ubuntu 20.04, PyTorch 1.12.
[Table]
and python 3.10.4.