Empirical Studies on the Convergence of Feature Spaces in Deep Learning

1 INTRODUCTION . Blessed by the capacities of feature learning , deep neural networks ( DNNs ) ( LeCun et al. , 2015 ) have been widely used to perform learning tasks , ranging from classification , to generation ( Goodfellow et al. , 2014 ; Radford et al. , 2015 ) , in various settings ( e.g. , supervised , unsupervised , and selfsupervised learning ) . To better analyze the features learned by deep models , numerous works have studied on interpreting the features spaces of the well-trained models ( Simonyan et al. , 2013 ; White , 2016 ; Zhu et al. , 2016 ; Bau et al. , 2017 ; 2019 ; Jahanian et al. , 2020 ; Zhang & Wu , 2020 ) . Invariance beyond the use of architectures and labels . While existing studies primarily focus on the interpolation of a given model to discover mappings from the feature space to outputs of the model ( e.g. , classification ( Bau et al. , 2017 ) and generation ( Jahanian et al. , 2020 ) ) , the work is so few that compares the feature spaces learned by deep models of varying architectures ( e.g. , MLP/CNN classifiers versus Autoencoders ) for different learning paradigms ( Chen et al. , 2020 ; Khosla et al. , 2020 ; Spinner et al. , 2018 ) . More specifically , we are particularly interested in whether there exists certain “ statistical invariance ” in the feature space , no matter what type of architectures or whether label information ( e.g. , supervised vs. unsupervised vs. self-supervised ( Chen et al. , 2020 ) learning ) are used in feature learning with the same training dataset . Hypotheses . It is not difficult to imagine that the feature spaces of well-trained DNN classifiers in supervised learning setting might share some linear subspace ( Vaswani et al. , 2018 ) . When models are well fitted to the same training set , the feature vectors of training samples should be projected to the ground-truth labels after a Fully-Connected Layer ( i.e. , a linear transform ) , while such linear subspace are supposed to distribute samples in a discriminative manner . We doubt that such subspace might be not only shared by supervised learners but also with AEs which are trained to reconstruct input data without any label information in an unsupervised manner , or even shared with self-supervised DNN classifiers ( e.g. , SimCLR ( Chen et al. , 2020 ) ) which train ( 1 ) CNN feature extractor ( using contrastive loss without labels ) and ( 2 ) linear classifiers ( using discriminative loss based on labels ) separately in an ad-hoc manner . More specifically , we hypothesize that ( H.I : ) there exists certain common feature subspaces shared by well-trained deep models using the same training dataset , even though the architectures ( MLPs , CNNs , and AEs ) and the learning paradigms ( supervised , unsupervised , and self-supervised ) are significantly different . Further , as the training procedure usually initializes the DNN models from random weights and learns features from the training set step-by-step , we hypothesize that ( H.II : ) the training procedure gradually shapes the feature subspace over training iterations and asymptotically converge to the common subspace in certain statistical measure . Finally , we hypothesize that ( H.III ) the convergence to the common feature subspace would connect to the data distribution and performance of models , as such behavior indicates how well the features are learned from data . This hypothesis is motivated by the observation that when the DNN model tends to be linear the DNN feature subspace should be close to the data subspace , while the well-trained DNN models should be locally linear ( Zhang & Wu , 2020 ) or piecewise linear ( Arora et al. , 2018 ) . Contributions . To test above three hypotheses , this work makes contributions in proposing new measures to the DNN features , namely P-vectors , and conducting extensive experiments for empirical studies . We train deep models using various DNN architectures , multiple learning paradigms , and datasets , with the checkpoint restored per epoch . Then , we extract the feature vectors for either training or testing sample sets , from the model ( Please see Section 3 for details ) and discover some interesting relationships or associations as discussed below . I. P-vector and Convergence : Given the matrix of feature vectors ( # samples1× # features ) for either training or testing samples , we perform the singular value decomposition ( SVD ) to obtain left and right singular vectors , characterizing the subspaces that samples distribute and the projection of features to subspaces respectively . We observe that deep models well-trained using the same 1We follow the convenience that denotes # as the term “ the number of ” for short . dataset share similar top left-singular vectors ( referring to the principal subspace of feature vectors ) , namely P-vector in this study . For example , Figure 1 ( a ) – ( b ) . show the high cosine similarity ( close to 1 ) between the P-vectors of any two well-trained DNNs for eight different deep learning architectures/tasks . This observation well backups H.I , where the common subspace in feature spaces shared by models trained with different architectures/paradigms has been observed . With checkpoints obtained along the training procedure , we estimate the angle that compares P-vectors of the checkpoint per epoch and the well-trained one . We find such angle decrease over the number of epochs ( in an overall manner ) and would converge2 to smaller ones . For example , Figure 1 ( c ) – ( h ) . demonstrate the consistent converging trends of angles between P-vectors of the well-trained model and its checkpoints in progress of training for eight different deep learning architectures/tasks . This observation supports H.II , where convergence to the common feature subspace are expected . II . Data Distribution and Performance : To connect to the data distributions , we intend to compare feature vectors and original data vectors of samples using P-vectors . In addition to the P-vector estimated from the feature vectors of a deep model ( namely a “ model P-vector ” ) , We simply form the sample vectors ( either for training or testing ) into a # samples× # data dimension matrix and perform SVD to obtain the top left singular vector as the P-vector of samples ( namely a “ data P-vector ” ) . We estimate the angles between the model and dataP-vectors , and find a trend of convergence ( from nearly orthogonal to relatively small angles ) , where we can see the well-trained models would incorporate smaller angles than ones in the early stage of training processes . We further correlate such angles with training and testing accuracy of the models , where we observe significant negative correlations in most cases of experiments . The trends show that the model with a smaller angle between the model and data P-vectors would enjoy better performance . The evidences backup H.III . 2 RELATED WORK . In this section , we first present the preliminaries in understanding feature learning of DNNs , then discuss the most relevant works to our studies . As early as 2013 , ( Simonyan et al. , 2013 ) proposed to visualize the features learned by deep convolutional neural networks ( CNN ) and made sense of discriminative learning via deep feature extraction . For generative models , ( White , 2016 ; Zhu et al. , 2016 ) studied the interpolation of latent spaces while ( Zhu et al. , 2016 ) discovered an user-controlled way to manipulate the images generated through the surrogation of latent spaces via manifolds . Later , ( Bau et al. , 2017 ) presented the visual concepts learned in the feature spaces of discriminative models through network dissection on specific datasets while the same group of researchers also proposed GAN dissection ( Bau et al. , 2019 ) – an interactive way to manipulate the semantics and style of image synthesis . ( Richardson & Weiss , 2018 ) compared GAN and Gaussian Mixture Models ( GMMs ) to understand the capacity of distribution learning in GAN . ( Berthelot et al. , 2019 ) proposed to improve understanding and interpolation of Autoencoders using adversarial regularizer while ( Spinner et al. , 2018 ) compared AEs with its variational derivatives to interpret the latent spaces . More recently , ( Saxe et al. , 2019 ) mathematically analyzed the process of neural representation construction from the perspectives of learning dynamics of deep neural networks . ( Jahanian et al. , 2020 ) studied the “ steerability ” of GAN , where it discovered point-to-point editing paths for content/style manipulation . ( Zhang & Wu , 2020 ) uncovered the phenomena that DNN classifiers with piecewise linear activation tend to map the input data to linear subregions . Other impressive studies in this line of research include ( Nguyen et al. , 2016 ; Arvanitidis et al. , 2018 ; Sercu et al. , 2019 ) . Discussion . The most relevant studies to our work are ( Bau et al. , 2017 ; Zhang & Wu , 2020 ; Saxe et al. , 2019 ; Lee et al. , 2019 ) . For discriminative models , ( Bau et al. , 2017 ) recovered visual features learned by CNN classifiers with a priorly labeled dataset , and quantified then compared the feature learning capacities ( namely “ interpretability ” in the work ) of different DNN models through patterns matching with the ground truth . Compared to ( Bau et al. , 2017 ) , we carry out the empirical studies on a wide range of datasets without any prior information on their features and observe consistent phenomena in the distribution of samples in the feature spaces . Furthermore , while ( Zhang & Wu , 2020 ) 2In our research , we name convergence as the decreasing trend of P-vector angles from a larger one to a smaller one ( e.g. , 10◦ for supervised CNN classifiers and SimCLR , ≤ 10◦ for ConvAEs/DenoiseAEs , and 30◦ for SupCon ) over training epochs . For reference , Cosine ( 10◦ ) =0.985 is close to 1.0 studied the properties of regions where a supervised DNN classifier with piecewise linear activation behaves linearly , our work observes the common linear subspaces shared by the features learned by the networks that are trained with different architectures ( e.g. , MLP/CNN classifiers and AEs with ReLU activation ) and paradigms ( e.g. , supervised , unsupervised , and self-supervised learning ) . Furthermore , both our work and ( Saxe et al. , 2019 ) propose to compare feature/representation learning by various models through SVD , while we perform SVD to investigate the distribution of samples in feature space and ( Saxe et al. , 2019 ) uncovered the latent structures in input-and-output of neurons . To understand the latent space of generative models , some statistical/mathematical tools ( Richardson & Weiss , 2018 ; Lee et al. , 2019 ) have been introduced . Through comparisons between GAN and GMM , ( Richardson & Weiss , 2018 ) uncovered GAN ’ s superiority in feature/texture reconstruction and in the meanwhile its incompetence in distribution learning , while ( Lee et al. , 2019 ) provides some analytical insights on the structure of latent spaces . In our work , we propose P-vectors to model the distribution of samples in the feature/latent spaces of DNN classifiers and AEs . To the best of our knowledge , we make unique contributions compared to the above work .

Summary: The authors identify an interesting empirical phenomenon: across a range of network architectures and training approaches (supervised, unsupervised, auto-encoders), the feature spaces identified by these networks are similar. The authors introduce a specific way to summarize the feature space of a network as a vector (the top-left singular vector of the num_examples x num_features matrix) and show that these vectors are highly correlated across networks. In addition, the authors show that the features spaces become more similar throughout training and are predictive of the generalization performance of a neural network.