Manifold Characteristics That Predict Downstream Task Performance
Authors: Ruan Henry Van der Merwe
Gregory Newman
Etienne Barnard
Venue: Submitted to ICLR 2023 (2023)
Keywords: self-supervised learning
deep learning
representation learning

Abstract: Pretraining methods are typically compared by evaluating the accuracy of linear classifiers, transfer learning performance, or visually inspecting the representation manifold's (RM) lower-dimensional projections. We show that the differences between methods can be understood more clearly by investigating the RM directly, which allows for a more detailed comparison. To this end, we propose a framework and new metric to measure and compare different RMs. We also investigate and report on the RM characteristics for various pretraining methods. These characteristics are measured by applying sequentially larger local alterations to the input data, using white noise injections and Projected Gradient Descent (PGD) adversarial attacks, and then tracking each datapoint. We calculate the total distance moved for each datapoint and the relative change in distance between successive alterations. We show that self-supervised methods learn an RM where alterations lead to large but constant size changes, indicating a smoother RM than fully supervised methods. We then combine these measurements into one metric, the Representation Manifold Quality Metric (RMQM), where larger values indicate larger and less variable step sizes, and show that RMQM correlates positively with performance on downstream tasks.

Under review as a conference paper at ICLR 2023
MANIFOLD CHARACTERISTICS THAT PREDICT DOWNSTREAM TASK PERFORMANCE
Anonymous authors Paper under double-blind review
ABSTRACT
Pretraining methods are typically compared by evaluating the accuracy of linear classifiers, transfer learning performance, or visually inspecting the representation projections. We show that the differences be-
manifold's (RM) lower-dimension tween methods can be understood more clearly by investigating the RM directly, which allows for more detailed comparison. 'oo this end, we propose frame- work and new metric to measure and compare different RMs. We also investigate and report on the RM characteristics for various pretraining methods. These char- acteristics are measured by applying sequentially larger local alterations to the input data, using white noise injections and Projected Gradient Descent (PGD) adversarial attacks, and then tracking each datapoint. We calculate the total dis- tance moved for each datapoint and the relative change in distance between suc- cessive alterations. We show that self-supervised methods learn an RM where alterations lead to large but constant size changes, indicating a smoother RM than fully supervised methods. We then combine these measurements into metric, the Representation Manifold Quality Metric (RMQM), where larger values indi- cate larger and less variable step sizes, and show that RMQM correlates positively with performance on downstream tasks.
INTRODUCTION
Understanding why deep neural networks generalise so well remains topic of intense research, despite the practical successes that have been achieved with such networks. Less ambitiously than aiming for a complete understanding, we can search for characteristics that indicate good generali- sation. Knowledge of such characteristics then be incorporated into training methods and open more research avenues. These characteristics can also be used to evaluate and compare networks.
Arguably the most successful current theories of generalisation focus the flatness of the loss surface at the minima (Hochreiter & Schmidhuber, 1997; Dziugaite & Roy, 2017; Dherin et al., 2021) (even though the most straightforward measures of flatness known be deficient Dinh et al. (2017)). Petzka et al. (2021) expands on this argument and shows that these methods correlate strongly with model performance, and reflect the assumption that the labels are locally constant in feature space. A thorough survey by Jiang et al. (2020) shows that some recent methods are, in fact, negatively correlated with generalisation.
To our knowledge, no theory looks at the structural characteristics of the learned Representation Manifold (RM) a predictor for generalisation. We investigate whether structural characteristics in the RMS correlate with generalisation to task performance.
To illustrate the intuition behind our investigation, consider Figure 1, which represents two RMs, A and B. Assume that each RM is produced by the same architecture, trained on the same dataset; both have a flat minima but are trained with different methods. the case of A, where the manifold is smooth, the sample representations of the Green class are, average, closer to other Green class's points. Likewise, presentations samples. On the other hand, if we consider RM B, there chasms in the manifold that lead to some sample representations being closer to samples of the other class rather than samples of their own class, as illustrated in the blue patch.
the Red class will, on average, be closer to other Red class's

---

Under review as a conference paper at ICLR 2023
Figure I: An illustration to give an intuitive understanding of why the structural characteristics of a RM should be considered a predictor of generalisation.
The purpose of this paper is to justify our claim that specific RM characteristics lead to general- isation. However, do this, we must first define appropriate RM characteristies that reflect this intuition and show how to measure them.
Contribution This paper aims show with enough empirical evidence that looking at the repre- sentation manifold (RM) structure is a good research direction for explaining generalisation in deep learning structures. Following these strong empirical results, future work will require deeper the- oretical investigation into findings. Our contributions in this paper then be summarised as defining a -agoss-gg and straightforward framework to measure RM characteristics. Using this framework, we compare the RMs learned by encoders trained using supervised, self-supervised and a mixture of both methods on the MNIST and CIFAR-10 datasets. We then present a new metric that calculates the quality of a manifold for generalisation, the Representation Manifold Quality Metric (RMQM). We show that this metric correlates strongly with downstream task performance. These observations support our intuition the characteristics of an RM that lead to generalisation.
RELATED WORK
Representation learning Some of the carliest work in representation learning focused pre- training networks by generating artificial labels from images and then training the network to pre- dict these labels (Doersch et al., 2015; Zhang al.; Gidaris al., 2018). Other techniques involve contrastive learning where representations from images are directly contrasted against one another such that the network learns to encode similar images similar representations (Schroff al., 2015; Oord et al., 2018; Chen et al., 2020; He et al., 2020; Le-Khac et al., 2020).
Comparing representations from trained neural networks Yamins et al. (2014); Cadena et al. (2019) compares how similar representations are by linearly regressing over the one representation to predict the other representation. The R² coefficient then used a metric to quantify similarity. This metric is not symmetric. Symmetrical methods compare representations from different neural networks by creating a similarity matrix between the hidden representations of all layers as was done in (Laakso & Cottrell, 2000; Kriegeskorte et al., 2008; Li al., 2016; Wang et al., 2018; Kornblith et al.).
Manifold Learning The Manifold Hypothesis states that practical high dimensional datasets lie on a much lower dimensional manifold Carlsson et al. (2008); Fefferman et al. (2016); Goodfellow al. (2016). Manifold learning techniques aim learn this lower-dimensional manifold by per- forming non-linear dimensionality reduction. A typical application of these non-lineat reductions is visualising high dimensional data in wo-dimmesionnal or dinee-dimeniional settings. Popular tech- niques
et al. (2000); Van der Maaten & Hinton (2008); Mclnnes et al. (2018). These techniques have heen used various studies to compare different learned representation man- ifolds (Chen et al., 2019; van der Merwe, 2020; Li et al., 2020; Liu et al., 2022).
Comparing manifolds To evaluate the performance of Generative Adversarial Networks, Baran- nikov et al. (2021) introduces the Cross-Barcods tool that measures the differences in topologies between two manifolds, which they approximate by the sampled data points from the underlying data distributions. They then derive the Manifold Topology Divergence based the sum of the
2

---

Under review as a conference paper at ICLR 2023
lengths of segments in the Cross Barcode. Zhou et al. (2021) also evaluates generative models by quantifying representation disentanglement. They do this by measuring the topological similarity of conditional submanifolds from the latent space. Shao et al. (2018) investigates the Riemannian geometry of latent manifolds, specifically the curvature of the manifolds. They conclude that having latent coordinates that approximate geodesics is desirable property of latent manifolds. To knowledge, there has not been a study done on measuring the manifold's structural characteristics based on small local alterations to the input data, applied non-generative encoders.
Predictors of generalisation Jiang et (2020) performed a large scale study of generalisation in deep learning, and we refer the reader to this work for a well-documented review. To our knowledge, there has been no work done on using the structure of the RM as a predictor of generalisation.
3 APPROACH
Describing all the details of a high- dimeensienii representation manifold (RM) is an impossible task; we can best strive find characteristics that summarise salient properties of the RM. When mea- suring these characteristics, one will therefore have discrete view of the RM Barannikov et al. (2021), made out of the predicted representations from the input data. We propose measuring indi- vidual distance metrics for each representation of an input sample relative to representations of data close to
By staying in the neighbourhood of each representation, we can measure the surface surrounding that point using standard distance metrics, effectively walking on the local structure and measuring size of each step relative to the change in input. By inspecting all these locally measured structures together, one describes the structure for the entire RM in terms of its local stability.
The caveat is that requires representations in close proximity on the RM to do these measure- ments. However, practical RMs have many dimensions, implying that data points tend to be well separated, even if they originate from the same underlying class (Bárány & Füredi, 1988; Balestriero et al., 2021). We thus need to create these proximate representations artificially.
We do this by applying sequentially larger local alterations the input data and computing the resulting representations. By increasing the size of the alteration, we step further on the RM surface and thus measure characteristics further away but still local for the magnitude of changes that employ.
3.1 ALTERATION METHODS
Let an RM be represented by é = f(X), where f is feature extractor and X is a dataset (e.g. input images). With representing function that applies small local alterations to pixels in the image, each altered data point projected down to the RM is represented oi = f(A, (z:)). where J; X and A, the jth iteration of the alteration function.
The two alteration methods we use in our experiments are the same methods Hoffman et al. (2019) used to evaluate the robustness of a Jacobian regulariser. We chose these methods because they result in either random local alterations guided alteration, thus giving us different paths on the RM to evaluate.
White noise injection. Here we alter each input image x, by adding an alteration vector randomly, with components independently drawn from normal distribution with variance e², thus (0, 2)). In order to increase the alteration strength, we increase from zero to one 100 in equal steps, indexed by Thus, alteration j for datapoint æ; is given by Fij [x, ag delpppp where aj NO es) and [-lcip clips the image to be between zero and one.
PGD attack. Whereas white noise injections will allow us walk on the surface of an RM in random directions, altering the image in a way that deliberately aims to fool the trained function fe will allow us to walk in a direction influenced by decision boundaries we will implement an extension fast gradient sign method (FGSM) Goodfellow et al. (2015), namely projected gradient descent (PGD) (Madry et al., 2018). FGSM consists of adding vector
the RM. In this paper
3

---

Under review as a conference paper at ICLR 2023
the original image, where this vector consists of the sign of the gradient for the loss functions with respect to the imput image, scaled by a value EFGSM PGD iterates this process for several iterations. Calculating the jth alteration of 24, represented by Zi.j can be defined as
Fi.j Pij-1 CPSGM (V (Vz,,-1 D(0 Zi.j (1))
(1)
where C is the loss function for the relevant training method.
Given that the original target for these adversarial attacks order to then apply PGD attack to the triplet variants, we calculate the loss precisely as usual and then calculate the gradient with regards the anchor image. When calculating the gradient for NT- XENT methods, we compare a non-augmented image with an augmented version and then calculate the gradient with respect the unaugmented image. We apply the PGD attack for 30 iterations and save each iteration, with EFGSM 2/255.
network that classifies images. In
3.2 MANIFOLD CHARACTERISTICS
Let A, be the jth iteration of an alteration method, where each successive iteration employs a stronger alteration. Also, let Φ1.j be the projected point the RM produced by f())(z))) where r; E X.
Average distance moved. The first characteristic we measure is the total of the normalised Eu- clidean distances between the original point, 61,0, and each altered point, Φi.j. where é is the nor- malised vector. The average distance moved for image is represented as J E El Φi l12, where we can then average over each point to find the average distance moved for a given RM and alteration. Finally, the average over images is
D A)
NJ Σ
Il 01,5112
(2)
where N is the number of images considered and J is the number of alterations. D(ø, A) thus indicates how robust the RM alterations
Average distance spikes. We also measure the relative change in distance between successive alterations. These relative distance changes are measured both with respect to the original represen- tation and relative to each previous alteration. We average the magnitudes of these changes are not interested in the direction of the change to gauge how smooth an RM is. To understand how relative changes relate smoothness, recall that we only apply alterations that keep us close the given data points. Therefore, we can only have big spikes if the RM contains significant chasms or bumps (small alterations input data should result in constant distance increases if the RM is smooth).
In order to calculate these relative changes for single representation, refer to Equation (3) which calculates the relative change according to the original representation, Dnc and Equation (4) which calculates the relative change according to the distance between the previous alterations PRC- In both equations, d() is distance function. In order to get the overall metrics, we average the values over all data points.
d(oroo )i. d(cioo, d(0i.o Dij)
D( A)RC
Σ
(3)
J Σ
d d(dii-1,, di.)) d( th (()
P(di, A)nc
(4)
2=
4 COMPARING DIFFERENT TRAINING METHODS
We now measure the characteristics defined in Section for five different training methods applied two different encoders and datasets. These five methods range from purely unsupervised to fully
4

---

Under review as a conference paper at ICLR 2023
supervised. By comparing these five methods, discover common properties for related methods, thereby gaining a better understanding of the learned RMs. For further details on these training methods, please refer to Appendix C.
To ensure that each learned RM can be compared fairly to each other, all image alterations are exactly the same for each method when calculating the manifold characteristics. We report in the rest this section using the average results over both the Adam and SGD trained encoders, as the results were similar for both.
24
Oybeddin Denetoomm12
Enteaking DrTANu
14-
67
Figure 2: The normalised Euclidean distance between the original MNIST digit and the same digit altered by our white noise injection method. We perform this measurement for each embedding dimension and digit the test set. We then calculate the standard error shown with the mean results.
4.1 MEASURING DISTANCES
Figure shows the average normalised Euclidean distance to the original MNIST digits as we in- crease the amount of alteration applied each digit. Here, A is the white noise injection alteration. As the embedding dimension increases, the self-supervised methods move further from the original point than the supervised signal methods. We also notice that the Cross-Entropy encoder's average distance away from the original representation stays very low, with the Triplct-Entropy and Triplet- Supervised encoders falling in between. We suspect this is because they contain both supervised and unsupervised signals in the training process.
When the PGD attack, we see the same pattern emerging for the CIFAR-10 encoders, shown in Figure 3. Here though, we can see much more significant difference between self-supervised methods and methods containing a supervised signal: the NT-XENT and Triplet-SS measurements grow to have much larger values.
In Table we summarise the D values, which is calculated using Equation (2), for the MNIST encoders. We average over embedding dimensions and calculate the standard deviation for each method. The common trend among both alteration methods is that NT-XENT and Triplet-SS alter- ations always move farther away from the original representation than the other methods. We can also see that the total distance moved decreases from sulf-super to pure supervised learning methods.
These results indicate that the encoder is more robust to minor perturbations (as measured by the distance moved from the original image) if the training method contains strong supervised signal.
5

---

Under review as a conference paper at ICLR 2023
Orbesding Disencionw12
Erdeski (6ressione/5
ErekedSing D11
Trgee Tppic
04-
Sunde moattrattrr
Figure 3: The normalised Euclidean distance between the original CIFAR-10 image and the same image altered by sequential PGD attack iterations.
Table I: The average distance each points moves relative the original point for both the white noise injection and PGD Attack alterations for the MNIST encoders. The results are averaged over both the embedding dimension of an encoder and the optimiser used.
METHOD
NOISE
PGD
MOSS--ENTROPY
0.33=0.15
0.21=0.08
TRIPLET-ENTROPY
0.73=0.11
0.33#0.08
STILEE-SUPERVISED
0.77=0.06
0.38+0.10
TRIPLET-SS
0.93-10.13
0.66+10.17
NT-XENT
1.01=0.044
0.68++0.10
4.2 MEASURING SPIKES
Following the same steps as in Section 4.1, we now study the relative change in distances measured. The relative change in distance to the original representation, plotted against the amount of PGD attack iterations, shown in Figure 4.
We see reversal of the graphs in Section 4.1: the self-supervised methods start with high relative changes, which decrease rapidly. Methods containing supervised signal have larger spikes and error bands. All this indicates less smooth journey in the RM between alterations for the supervised methods.
The same trend is present for CIFAR-10 models when we inject white noise. Here though, the Triplet-Supervised method is unstable for several values the embedding dimension, whereas the NT-XENT and Triplet-SS again have the smallest spikes.
Table shows the overall average values for each spike metric for each method. NT-XENT and Triplet-SS have the smallest spikes for both forms of alteration, whereas Triplet-Supervised results in very non-smooth RMs.
Self-supervised methods therefore learn structures in which step in most directions, at most loca- tions, induces steps of similar size on the RM. That is, these self-supervised methods have smoother RMs than the other methods.

---

Under review as a conference paper at ICLR 2023
Enteedinng Dtrenaan-do
Entbedding Cinea 21
Endwciing Dinemstone
20
03-
Orbedding Dimewsee1322
Drocadding Rinwwice3522
Deroey
The relative change in distance the original point plotted against the number of PGD iterations for the encoders trained on the MNIST dataset.
Figure 4:
Table The average change in the distance each point moves relative to the original point, compared against the previous alteration's 's distance. Results are calculated for both the white noise injection and PGD Attack alterations for the CIFAR-10 encoders. The results are averaged over both the embedding dimension of encoder and the optimiser used.
METHOD
NOISE
PGD
0.11110.03
0.3410.04
TRIPLET-ENTROPE
0.18410.04
0.8911.35
ERIPET-SUPERVED
0.86=#085
2.97#1.61
TRIPLET-SS
0.06=0.01
0.06#0.04
NT-XENT
0.04=0.01
0.10=#001
5 REPRESENTATION MANIFOLD QUALITY METRIC
In Section 4 we showed empirically that an RM learned by self-supervised methods has structure that has the following property: When moving in any direction on the surface of RM, it will result in relatively large displacement, but these displacements or in what direction a step is taken. The opposite is true with methods containing a supervised signal: moving in the surface results in smaller displacements, but those displacements are significantly more variable in size.
average the same size matter where
In order determine which of these groups of characteristics more desirable for downstream tasks, we combine these characteristics into one metric, the Representation Manifold Quality Metric (RMQM).
With this single metric describing an RM, we can perform various downstream tasks with our en- coders and see how the performance correlates with the value of the RMQM. We define the RMQM as
(5) Here D is the average distance moved measured relative to the original representation, Dpc is the relative change in distance between each subsequent alteration and the original representation and PPC is the relative change of the distances between altered representations, as defined in Equa- tions (2) to (4).
RMQM In (1+ D DFC + Pp))
We apply the natural logarithm to scale the values, and we add one to ensure we do not have any negative values.
7

---

Under review as a conference paper at ICLR 2023
[aatased MNIST Ostimize Aeamm
Datinel HINST Cpttinttnt
2.0
2.0
0.0
Supert-Sp
Dataset Cfar20 Cptim zer Adam
Dataset Citar10 Optimizer SGD-Mooeritum
Tiplet Entropy
Fonns Frrmpy
Iriplt
Ni-ENI
2.5
1.5
0.5
3.28
25.8
Ragi esenttation Ditmonsiion
Roprissenttaiinn Stmmossion
Figure 5: RMQM for white noise injections, over every embedding dimension for each encoder.
Thus, RMQM is designed to yield large values for relatively smooth RMs with relatively large sensitivity to changes in the input. Below, we only interpret the RMQM score when A is the white noise injection alteration, since the method dependence of the PGD alterations complicates our ability to compare the various methods.
In Figure we show the RMQM score for each of our encoders, with A being white noise injec- tions. For the MNIST encoders trained using SGD and Nesterov momentum, as the embedding size increases, the RMQM for Cross- Cnoss-Entropy overtakes Triplet-SS, indicating that the RM is more similar in this setup to one produced by NT-XENT trained encoders. In the other cases, there is an overall trend for the NT-XENT and Triplet-SS encoders to have the highest RMQM, followed by Cross-Entron and then lastly, Triplet-Entropy and Triplet-Supervised.
5.1 CORRELATION BETWEEN RMQM AND DOWNSTREAM TASKS.
In order to find what RM characteristics are desirable, we measure how RMQM correlates with downstream task performance. If we find strong positive correlation, RM with smooth struc- ture and large displacements is desirable. If there is strong negative correlation, then an RM that contains chasms and bumps and small displacements desirable.
We define the task performance as the normalised test accuracy of K-Nearest Neighbour (KNN) model, with only one nearest neighbour (K 1), trained on the representations created by the encoder, as well as the normalised test accuracy when doing linear probe on the final layers of the network. We believe these are appropriate measures of task performance as most use cases today utilise representations. We also use these tasks, as these task do not change the learned RMM structure, as would be the case when fine-tuning the encoders on a new dataset with Cross-Entropy.
The MNIST encoders will be tested on the OMNIGLOT (Lake et al., 2015) and the KMNIST (Clanuwat et al., 2018) datasets, with the CIFAR- encoders being tested on the Caltech-101 (Fei- Fei et al., 2004) and CIFAR- (Krizhevsky al., 2009) datasets. We believe these datasets provide a large enough semantic gap towards the original datasets, as we do not measure for extreme gener- alisation to any dataset. We exclude the Omniglot dataset results for the linear probe performs tasks as they perform poorly across all model variations (we believe the task is too hard for probing), which skews the results heavily.
8

---

Under review as a conference paper at ICLR 2023
Downstream
Ornigglo
KMNIST
3.5
Caltech-101
CIFAR-1C0
RMOM
2.5
2.0
10-
0.2
0.4
0.6
0.8
1.0
lask Performance
Figure 6: RMQM versus the normalised KNN task performance. The dots represent each encoder's performance and RMQM score, colour coded by the downstream dataset. As one can see, there is strong positive correlation between RMQM and downstream task performance
Figure visualises RMQM versus the downstream KNN task performance, where each dot refers to differently trained model. We find that RMQM correlates positively to the KNN task with coefficient of 0.75. These results also stay true for larger values K. We also find positive correlation to the linear probing results with a coefficient of 0.78. Combining both of the tasks, there a positive correlation 0.72. This justifies our statement that RM characteristics are essential. In other words, when vector search is the downstream task performance, encoder that learned an RM with a smooth structure and large displacements will tend to perform well on downstream search tasks. To gain intuition for why this can be, please refer Appendix B, where give our intuitive explanation.
CONCLUSION
We propose framework measure the characteristies of learned representation manifolds (RM). We measure the characteristics by applying sequentially stronger local alterations to the input data and measuring how these altered representations move relative the original representation and the successive alterations. We show that self-supervised learning methods learn RMs in which motion in any direction on the surface will result in relatively large displacements. However, these displacements are relatively similar matter where or in what direction step is taken.
To identify RM characteristics related to good downstream task performance, we combine our mea- surements into single metric, the Representation Manifold Quality Metric (RMQM). RMQM is designed to yield large values for relatively smooth RMs with relatively large sensitivity to changes in the input. We then measure the downstream task performance for several tasks and find strong positive correlation with RMQM. This strong correlation indicates that the structure of a learned manifold is another strong predictor for the generalisation of neural networks. This also shows that supe-super methods lead to state-of-the. performance due to the underlying RM structure, which is sensitive alterations in the input, utilising relatively smooth manifold.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Randall Balestriero, Jerome Pesenti, and Yann LeCun. Learning in high dimension always amounts to extrapolation. arXiv preprint 0rXiv:2110.09445 2021.
Serguei Barannikov, Ilya Trofimov, Grigorii Sotnikov, Ekaterina Trimbach, Alexander Korotin, Alexander Filippov, and Evgeny Burnaev. Manifold topology divergence: a framework for com- paring data manifolds. In A. Beygelzimer, Y. Dauphin, P. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL / /oppprevee.ee.eeeeeeeee net/Eorum? 11=2)66OUJUWWWS
Imre Bärány and Zoltán Füredi. On the shape of the convex hull of random points. Probability theory and related fields, 77(2)231-244,, 1988.
Santiago A Cadena, Fabian H Sinz, Taliah Muhammad, Emmanouil Froudarakis, Erick Cobos, Edgar Y Walker, Jake Reimer, Matthias Bethge, Andreas Tolias, and Alexander S Ecker. How well do deep neural networks trained on object recognition characterize the mouse visual system? 2019.
Gunnar Carlsson, Tigran Ishkhanov, Vin De Silva, and Afra Zomorodian. On the local behavior of spaces of natural images. International journal of computer vision, 76(1):1-12, 2008
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Muchine Learning Reseurch, pp. 1597-1607. PMLR, 2020. URL http: peett::/pprooe http///oceedngggg/// mlr. w.r.pppess//e v1 chen20j.. html.
Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recognition with graph convolutional networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 5177-5186, 2019.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. ArXiv, abs/1812.00 2018.
Benoit Dherin, Micheal Munn, and David GT Barrett. The geometric occam's razor implicit in deep learning. arXiv preprint arXiv:2 11.22111110000.. 2021.
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning (ICML), volume 70 of Proceedings of Machine Learning Re- search, pp. 1019-1028. PMLR, 2017. URL http: http:///pcoceen.e../ mwr.press/v70. dinh17hh html.
Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Unsupervised visual representation learning by context prediction. In 2015 IEEE International Conference Computer Vision, ICCV, pp. 1422- 1430. IEEE Computer Society, 2015. URL https: /ooii//oiii/////// 167.
Gintare Karolina Dziugaite and Daniel M. Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Gal Eli- dan, Kristian Kersting, and Alexander T. Ihler (eds.), Proceedings of the Thirty-Thrrd Conference on Uncertainty in Artificial Intelligence, UAI. AUAI Press, 2017. URL http: auai. . wwwpdiinnggppppp..pee/ee http//appesss////e pdf.
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 2944)9933- 2016.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training amples: An incremental bayesian approach tested on 101 object categories. CVPR Workshops, 2004.
10

---

Under review as a conference paper at ICLR 2023
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In 6th International Conference on Learning Representations, ICLR. OpenReview.net, 2018. URL https: htttp::/openeeeee....oo http://wwwww..eeeeooooooo/aaaaas
lan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR, 2015. URL http:
org/ abs 1412. 6572.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition 90.
(CVPR), pp. 770-778, 2016. URL https: /ttp://.....///...11/////......
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE/CVF Conference Computer Vision and Pattern Recognition CVPR, pp. 9726-9735. Computer Vision Foundation / IEEE, 2020. URL attps://doi... org/ wttp://..110//1/1 wtt.:11119////221220 .00000..22200 2020.0222000 00975.
Sepp Hochreiter and Jürgen Schmidhuber. Flat Minima. Neural Computation. 9(1):1-42, 1997. wttpe/i.rg/110.11.11111
ISSN 0899-7667.
doi: 10.1162/neco.19977 1997...11. URL
neco. 1997. 9. 1.
Judy Hoffman, Daniel Roberts, and Sho Yaida. Robust learning with jacobian regularization. ArXiv, abs/1908.02779, 2019.
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. 2020. URL https: http:s////eeeeeeeeeeeeeee forum?id-SJgIPP JBFVH..
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization. In 3rd Interna- tional Conference on Learning Representations, (ICLR), 2015. URL http::/aaxii..rrrt abs/ 1112. 6980.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neu- ral network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning (ICML).
Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:4, 2008.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. Technical report. 2009.
Aarr Laakso and Garrison Cottrell. Content and cluster analysis: assessing representational simi- larity neural systems. Philosophical psychology, 13(1):47-76, 2000.
Brenden Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning
through probabilistic program induction. Science, 133-2226)):33211333 2015.
Phuc H. Le-Khac, Graham Healy, and Alan F. Smeaton. Contrastive representation learning: A framework and review. IEEE Access, 8:193907-193934, 2020. URL https:
10. 11w.119///0 /0/0///CCSS..... 2020.3031544
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. volume 86, pp. 2278-2324, Icce, 1998.
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In European Conference on Computer Vision (ECCV), pp. 121-137. Springer, 2020.
Yixuan Li. Jason Yosinski, Jeff Clune, Hod Lipson, and John E. Hopcroft. Convergent learning: Do different neural networks learn the same representations? In 4th International Conference on Learning Representations, ICLR, 2016. URL http::t/tt/sseeee wwwp:/rggg///....... 07543.
11

---

Under review as a conference paper at ICLR 2023
Daizong Liu, Xiaoye Qu, Yinzhen Wang. Xing Di, Kai Zou, Yu Cheng. Zichuan Xu, and Pan Zhou. Unsupervised temporal video grounding with deep semantic clustering. arXiv preprint arXiv:2201.05307, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Reprexentations, ICLR, 2018. URL https: /openrevieww net/ for IJzIBfZAb.
Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint 182.::280. 2018.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. xrviv preprint arXiv: arXiv:/807.03744 2018.
Henning Petzka, Michael Kamp, Linara Adilova, Cristian Sminchisescu, and Mario Boley. Relative flatness and generalization. Advances in Neural Information Processing Systems, 34, 2021.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face
recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pp. 815-823, 2015. URL https://don .acooooorroe org/10.......oo......... 1109/CVPR. 2015. 7298682.
Hang Shao, Abhishek Kumar, and P. Thomas Fletcher. The riemannian geometry of deep generative models. In 2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 315-323. Computer Vision Foundation IEEE Computer Society, 2018.
Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Ad- vances Neural Information Processing Systems, pp. 1849-1857, 2016.
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initial- ization and momentum in deep learning. In Proceedings of the 30th International Conference on Machine Learning (ICML), pp. 1139-1147, 2013. URL htw.tttss//prrceeedngg mml..
press/ v28 www.e/err..ee........ 1.t.eeerr11.......... html.
Joshua B Tenenbaum, Vin De Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500):2319-3222 2000.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine
learning research, 9(11), 2008.
Ruan van der Merwe. Triplet entropy loss: Improving the generalisation of short speech language identification systems. arXiv preprint arXiv:2012.03775, 2020.
Liwei Wang, Lunjia Hu, Jiayuan Gu, Zhiqiang Hu, Yue Wu, Kun He, and John E. Hoperoft. Towards understanding learning representations: To what extent do different neural networks learn the same representation. pp. 9607-9616, 2018
Kilian Q Weinberger, John Blitzer, and Lawrence Saul. Distance metric learning for large margin nearest neighbor classification. In Y. Weiss, B. Schölkopf, and Platt (eds.), Advances Neural Information Processing Systems, volume 18. MIT Press. 2006. URL https: https://proceedings www.iis..euupp..... wttppppeee/0s//////.//..p 10992/22881110666777776666666611
Paper.pdl
Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan Solomon, Darren Seibert, and James DiCarlo. centormance-optimiiee hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23)88619-88)) 8624, 2014.
Richard Zhang, Phillip Isola, and Alexei A. Efros. Colorful image colorization. In Bastian Leibe, Jiri Matas, Nieu Sebe, and Max Welling (eds.), Computer Vision ECCV 2016 14th European Conference, series = Lecture Notes in Computer Science, volume = 9907, pages = 666,666, publisher Springer, year 2016.
12

---

Under review as a conference paper at ICLR 2023
Sharon Zhou, Eric Zelikman, Fred Lu, Andrew Y. Ng. Gunnar E. Carlsson, and Stefano Ermon.
Evaluating the disentanglement deep generative models through manifold topology. In 9th
International Conference Learning Representations, ICLR. OpenReview.net, 2021. URL
https: http:////eeeeeeeeeeeeeeeeeeeeeep ew net / forum? (arummid=djw50m44FF
13

---

Under review as a conference paper at ICLR 2023
A RMQM ABLATION STUDY
To find which features of the RMQM formula contribute the most to predicting the overall down- stream task performance, we performed an ablation study where we compared different combina- tions of inputs to the RMQM formula described in eq. (5).
When the eq. (5) consists only of the average distance moved. D, there is a very low correlation of only 0.27 to downstream task performance. We then looked when only spikes are considered, DRC and PRc. and found that the correlation was now 0.72, just below the reported 0.75 when everything is combined. We also measured all the remaining combinations and found that the cor- relation ranged between 0.71 and 0.74. This indicates that even though the spikes can be used to measure performance, adding the average distance increases the performance of the RMQM score.
Finally, we optimised the weights associated with each variable we found that the following combi- nation, shown in Equation (6), maximises the correlation 0.784.
RMQM = In (1 + +
0.74D 0.11Pp()
(6)
These results show that one can not only look distance moved its own and that having small spikes contribute significantly to generalisation. Nevertheless, spikes must be considered along with distance for the best predictive capability.
These results indicate that previous work on model robustness still holds, as D contributes less than spikes. However, robustness and RMQM complement each other in predicting the generalisation of neural networks.
B INTUITIVE EXPLANATION OF RMQM CONCLUSIONS
To help better understand these perhaps unintuitive results, consider the case when we calculate the RMQM using white noise alterations. Take the MNIST encoders specific MNIST image (any digit). When applied to this digit, a random vector exists that will trans- form it into one of the Omniglot characters, given a large enough alteration. In general, the variants of this Omniglot character will differ from that digit image by similar noise vectors. Thus when we encode this Omniglot character image and its variants using sulf-super trained encoder, the KNN models can accurately identify new images because, from the perspective of the RM, these new characters correspond the noise-altered version of our original MNIST digit. On the RM, this new character and its variants are projected with a similar step size away from the original MNIST digit. Due to the similar noise vector added, these projections are also in a similar direction. These projected characters are also close because there are few chasms or bumps projection can land on, allowing a nearest neighbour search to perform well.
example, and consider
C DETAILS ON TRAINING METHODS
The first method we investigate employs encoders trained with vanilla supervised learning with Cross-Entropy loss, where we then take the second to last layer output as the representations. We also use the SimCLR method introduced in Chen et al. (2020). This method compares two augmented versions of the same image and brings their representations closer together using the Cross-Entte Entsopy (NT-XENT) Loss (Sohn, 2016). Along with this, we trained encoders using two different implementations of Triplet-L.oss. (Weinberger et al., 2006; Schroff et al., 2015). We believe these techniques represent the major families of training techniques and provide enough information how different techniques learn different RM structures. In a fu- ture paper, we propose that a full-scale investigation be performed on most training techniques found in current literature.
Normalised Temperature-scut
We mine the triplets in a supervised manner for the first implementation using the image labels. We apply the SimCLR method for the second implementation, replacing NT-XENT with Triplet loss. We refer to the former method as Triplet- Sullee--ueee going forward and the latter as Triplet-SS. We do this to the effect of the indirect supervised signal the method. Lastly, see the effect of
14

---

Under review as a conference paper at ICLR 2023
directly combing contrastive signal with supervised signal, combine Triplet-Loss with Cross- Entropy loss, as was also done in van der Merwe (2020). Here the second to last layer's outputs are fed into the Triplet-Loss function, where the triplets mined using the same labels used for the Cross-Entropy loss, which takes as input the logits produced by the last layer.
We apply these methods (1998), trained on the MNIST dataset (LeCun et al., 1998) and Resnet-18 (He et al., 2016) trained on the CIFAR- 10 dataset (Krizhevsky et al., 2009). We train our encoders with six different embed- ding sizes, ranging from 16 to 512 in powers of two. We also train with two different optimisers, namely Stochastic Gradient Descent (SGD) with Nesterov Momentum (Sutskever al., 2013) and Adam (Kingma & Ba, 2015).
altered version of the LeNet-5 architecture introduced in LeCun al.
For SGD with Nesterov momentum, we the learning rate 0.001 and momentum 0.9. Our learning rate is also 0.001 with the default PyTorch hyperparameters for Adam. We train the CIFAR- 10 encoders for 100 epochs and the MNIST encoders for seven epochs.
15


Reviews:
[{"review_id": "w8Cywzl45U0", "reviewer": "Reviewer_s4yj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a framework and new methods to measure and compare different representation manifolds (RMs) to explore various pretraining methods. The analyses show that some self-supervised methods learn an RM where alterations lead to large but constant size changes, indicating a smoother RM than fully supervised methods. These measurements are integrated to develop a new  method called the Representation Manifold Quality Metric (RMQM), which is used to explore relationship between accuracy of models and RMs on downstream tasks.", "summary_of_the_review": "The paper addresses an important problem of deep learning. However, there are various major and minor issues with the paper. Therefore, the paper is not ready for publication without fixing these issues.", "strength_and_weaknesses": "The paper addresses an important problem of deep learning.\n\nHowever, there are various major and minor problems with the paper while addressing this problem. Briefly, there are unclear definitions and proposed methods are not employed correctly following common definitions and structures of manifolds. In addition, experimental analyses should be improved.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "oY8YaqE28v", "reviewer": "Reviewer_xaAq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates properties of trained manifolds with different learning methods on simple image data. For this, the authors propose data augmentation method and define metrics that lead to quantification of the quality of the overall manifold. They show that the proposed metric, i.e., RMQM, has high correlation with performances of several downstream prediction tasks. ", "summary_of_the_review": "The authors provide an interesting aspect of training machine learning models for interpretability. However, the ideas are not sufficiently validated or proved, and they should be better presented. ", "strength_and_weaknesses": "Strength:\n- Novelty. This paper relates prediction performance of downstream tasks with the characteristic of learnt manifold, which I think is quite interesting. \n\nWeakness:\n- Presentation of the paper. Too much white spaces and small text in the figures. Captions do not sufficiently describe the figures. \n- \"trained with different methods\" in the 4th paragraph in the intro is a little confusing. I thought \"different methods\" would refer to different optimization scheme but later it turns out that they refer to different tasks. \n- Perhaps a figure of the overall pipeline of the method will help a reader understand the ideas and aims of this paper. \n- There is a big jump going from Fig 2 to Fig 3 as two nubs are turned at the same time, i.e., dataset and perturbation method. It is difficult to tell which factor is causing \n- The authors propose a metric RMQM to measure the quality (e.g., smoothness) of the learned manifold. There must be other metrics, I am sorry but I am not sure which ones though (but at least the authors have done literature survey on manifold learning and comparison methods), but there is no way to tell that RMQM is the right one. \n- Lack of theory. While the paper is proposing interesting theories, they are only empirically validated with very limited settings, and it is not certain if the same observation will extend to more complicated experiments especially that may contain exhaustive local minima. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "W4E175X4Qu", "reviewer": "Reviewer_jVym", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a score for evaluating the representational space of a model to estimate the potential generalization performance. An experiment with this method across a few datasets provides evidence to support this claim.", "summary_of_the_review": "I really enjoyed reading this paper and I think the authors are doing some great work and have great ideas. However, I would prefer that the claims are supported with more complex datasets as well as contrasted with existing baselines. I believe the authors will find some from the publications in the PGDL2020 workshop that will demonstrate the potential improvement of their method.", "strength_and_weaknesses": "Strengths: The paper is well organized. The argument to predict generalization is clear. The proposed metric is explained clearly.\n\nWeaknesses: I would have preferred to see the evaluation performed on larger datasets and not just small toy datasets. The support for the claim in the paper is solely empirical so this should be expanded on more. I also believe there should be more comparisons to existing work.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject