Title: Using persistent homology to understand dimensionality reduction in resting-state fMRI
Venue: NeurIPS 2023
Accepted: no
Keywords: persistent homology

neuroimaging

neuroscience

manifold learning

dimensionality reduction

topological data analysis
Abstract: Evaluating the success of a manifold learning method remains a challenging problem, especially for methods adapted to a specific application domain. The present work investigates shared geometric structure across different dimensionality reduction (DR) algorithms within the scope of neuroimaging applications. We examine reduced-dimension embeddings produced by a representative assay of dimension reductions for brain data (“brain representations”) through the lens of persistent homology, making statistical claims about topological differences using a recent topological boostrap method. We cluster these methods based on their induced topologies, finding feature type and number --- rather than reduction algorithm --- as the main drivers of observed topological differences.

Abstract

Evaluating the success of a manifold learning method remains a challenging problem, especially for methods adapted to a specific application domain. The present work investigates shared geometric structure across different dimensionality reduction (DR) algorithms within the scope of neuroimaging applications. We examine reduced-dimension embeddings produced by a representative assay of dimension reductions for brain data ('brain representations') through the lens of persistent homology, making statistical claims about topological differences using a recent topological boostrap method. We cluster these methods based on their induced topologies, finding feature type and number - rather than reduction algorithm as the main drivers of observed topological differences.

1 Introduction

The present work investigates shared geometric structure across different dimensionality reduction algorithms within the scope of neuroimaging applications. For most applications, a 'dimensionality reduction' is any of a large class of methods that make inferences about structures underlying some data, typically to represent this data in both a more efficient and more interpretable way. Many dimensionality reduction (DR) problems can be equivalently formulated as 'manifold learning' problems (i.e., estimating the manifold from which a dataset was sampled), and we will use the terms synonymously. Efforts to understand theoretical and empirical relationships between DR methods remain active 1 -10 .

The difficulty of relating dimensionality reductions can be compounded within specific application domains because methodologies often branch into variably specialized use cases. Nonetheless, specific use cases can also suggest more stringent criteria by which to compare dimension reduction outcomes. In functional neuroimaging, specialized dimension reduction algorithms proliferate the field, bridging disparate use cases, design philosophies, and biological motivations 11 , 12 . These DR algorithms share the goal of extracting networks of functional activity from resting-state fMRI brain data, and we will refer to them throughout as 'brain representations.' We compare brain representations in terms of the topologies they induce on a single set of shared data ("subject space") and the statistical robustness of the differences between them. We measure these topological statistics through persistent homology 13 and the related topological bootstrap 14 , 15 .

Problem Statement: Brain Representations & Subject Space

Our primary goal is to compare the structural changes in a single neuroimaging dataset under a 31 variety of brain representations. Because the structure of subject space in the original (un-reduced) 32 data is unknown and impractical to compute, it is not feasible to grade brain representations' quality 33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

by structure preservation. Instead, we group brain representations based on the similarity of the subject-space structures they induce in reduced data.

We frame brain representation as a manifold learning problem, and we frame comparisons between them as comparisons between estimated manifolds. We suppose our data lies on some manifold S ↪ → R D ("subject space"), from which we extract a finite dataset ̂ S of N samples. In this work, S consists of resting-state fMRI scans, and ̂ S is the Human Connectome Project (HCP) Young Adult dataset; we then have D ∼ 10 8 and N ∼ 10 3 . A brain representation is any mapping ̂ φ i : ̂ S → R d i with d i ≪ D ; for any brain representation, we define its corresponding induced subject space ̂ S i = ̂ φ i ( ̂ S ) . We compare brain representations ̂ φ i and ̂ φ j by comparing the persistent homology of their induced subject spaces ̂ S i and ̂ S j . To link the persistent homology and manifold learning investigations, we make a key modeling assumption: there exists a local extension φ i : S → R d i of ̂ φ i that is a submersion in some neighborhood of ̂ S ⊂ S . This assumption requires that dimensionality reductions behave consistently on unseen data near training data, and constitutes only a mild smoothness assumption on ̂ φ i . Under this modeling assumption, we may assert the existence of a manifold S i ↪ → R d i containing ̂ S i such that the following diagram commutes:

̂

S

̂

S

i

S

S

̂

φ

i

φ

i

i

While the smoothness assumption on ̂ φ i is easily met by most DR algorithms, the connection between the persistent homology of ̂ S i and the manifold S i depends heavily on properties of the manifold sampling ̂ S .

To compare brain representations φ i and φ j , we compute dissimilarity metrics for all pairs of points in ̂ S i and ̂ S j and examine the resulting Vietoris-Rips complex in each space. This approach allows flexibility in the data and dissimilarities under consideration while still allowing claims about DR-induced topological differences.

Related Work: Persistent Homology & Dimensionality Reduction

Most comparisons in the literature are primarily interested in grading the relative performance of different DR algorithms. Though we do not share these goals, many comparative approaches articulate frameworks and methods with important relationships to our own. We review a selection of comparison methods, organized in roughly increasing order of the similarity between their goals and framework to our own.

Some evaluation methods for dimensionality reduction lead with intuition, formalizing helpful heuristics into rigorous ratings. We first reference Lee and Verleysen's co-ranking matrix 4 , which measures insertions to ('intrusion events') and deletions from ('extrusion events') k -neighborhoods in the low-dimension vs. high-dimension space. While this measure is non-parametric (with respect to the data geometry) and thus extremely flexible, it is sensitive only to local structure in the data. Later, Lee and Verleysen showed that the performance of a DR method closely follows its (a) insensitivity to norm concentration and (b) plasticity (i.e., the cost function gradient vanishing for distant points) 7 , leveraging a more geometric perspective to the analysis of DR performance than was typical of the contemporary literature 9 . Extending this line of thinking, Wang et al 1 recently offered a strictly empirical investigation of different DR methods in which they consider only the attractive and repulsive forces of the loss function over varying distance scales. They show that this framework is sufficient to characterize DR performance and extrapolate robust empirical principles of 'good DR methods' without need of an underlying formalism. While this work offers striking practical insights, it does not offer an immediate path to describing degrees of divergence between DR methods.

We now consider a class of methods we characterize by their tendency to originate in formal, geometric considerations of manifold learning. Singer and Wu's vector diffusion distance 8 , an extension of diffusion embeddings 17 , uses local principal component analysis to locally estimate a connection on the tangent bundle of a data manifold, from which it constructs a lower-dimensional

embedding. While the primary goal of their work is to define a manifold learning method, rather than 80 a technique for comparing such methods, their vector diffusion distance explicitly captures a variety 81 of geometric and topological data invariants. These invariants could be used analogously to our use 82 of persistent homology to compare dimensionality reductions of the same dataset. Similarly, Tyagi et 83 al 5 propose a set of tangent space estimation criteria for manifold sampling (as a function of manifold 84 curvature and intrinsic dimension) that immediately suggest geometric routes of comparison between 85 learned data manifolds. Finally, in more direct alignment with our goals, Sun and Machard propose a 86 geometric theory of manifold learning, which comes equipped with an intrinsic metric 18 , 19 . Their 87 approach is firmly grounded in classical information geometry 20 , 21 , comparing learned models via 88 the pullback of the Fisher information metric on direct probabilistic encodings of reduced data. While 89 this perspective diverges substantially from our own, our constructions are mutually translatable, and 90 their approach could provide interesting comparison and/or validation. 91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

To the best of our knowledge, only two other studies 2 , 3 have examined dimensionality reduction through the lens of persistent homology. Both works primarily consider the recovery quality of a known manifold and propose quality metrics derived from persistent homology. Paul and Chalup 2 compare DR methods while varying manifold complexity, measuring performance by the similarity of pre- and post-DR Betti numbers as a function of sampling density. While their goals differ from ours, much of our comparison also hinges on counts of topological features; however, they did not have access to the topological bootstrap 14 , 15 we employ in our study. Rieck and Leitte 3 compute the Wasserstein distance between pre- and post-DR persistence diagrams as a metric of embedding quality. While their study is most similar to our own, there are several key differences. First, they operationalize persistent homology with a sublevel-set filtration of the local density function, whereas we use the Vietoris-Rips filtration on pairwise point dissimilarities. Second, they consider reduction of a surface embedded in D = 3 reduced to d = 2 , whereas we consider data initially embedded in D ∼ 10 8 and reduced to dimensionalities ranging from d ∼ 10 2 to d ∼ 10 5 . Most importantly, we are able to leverage the topological bootstrap 14 , 15 in our work, which was not available at the time of their publication.

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

Finally, we also contrast our analysis of persistence data with a prevalent paradigm in persistent homology applications. Many persistent homology analyses 6 , 22 -27 (including those above) operationalize the assumption that most topological information lives in a diagram's most persistent components, treating low-persistence generators as noise. Instead, we follow work in distributed persistence 28 , 29 and examine the distributional properties of our persistence diagrams to parse their topological content.

Our contributions in the present study are as follows: (1) A flexible framework for the statistical comparison of dimensionality reductions, applicable to any data and dissimilarity measure compatible with a Vietoris-Rips complex; (2) Robust statistical measurement of topological differences between dimension-reduced data; (3) Application to real neuroscience data over a diverse slice of widely used neuroimaging DR algorithms ('brain representation' or "BR").

2 Methods

2.1 Brain Data and Brain Representations

The data for this study consists of pre-processed resting-state functional MRI data from N=1003 Human Connectome Project young adult (HCP-YA) 30 subjects. Each subject's minimally preprocessed data consists of 91,282 spatial 'grayordinates' by 1200 time points, giving an embedding dimension of D ∼ 10 8 in the initial space. We then chose six different brain representations that are both common within the field and showcase the methodological variability of widely adopted techniques. The brain representations we consider can roughly be grouped by their underlying models of brain function. We characterize the first group of methods as seeking to cluster neural activity into spatially contiguous cortical 'parcels.' In the parcellation family, we have Yeo's parcellated networks 31 , Glasser's multimodal parcellation 32 , and Schaefer's local-global parcellation 33 . We also sample from a family of low-rank matrix factorization methods that parse non-contiguous networks of functional activity. Independent component analysis (ICA) 34 , an extension and application refinement of PCA, underlies perhaps the most widely used brain representation in the field 35 and thus is represented here. In addition, we consider PROFUMO

36

, which parses 'functional modes' of brain activity from hierarchical Bayesian signal models. Finally, we include the 'principal gradient'

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

(or 'gradients') 37 , a diffusion embedding method that organizes brain function through cortical geometry.

From each brain representation, one or more feature types were computed to reflect the typical use of brain representations in the neuroimaging literature. The five feature types considered in this work are as follows: (1) 'amplitude,' the average power of the time signal in a given spatial component; (2) 'network matrix (netmat),' the matrix of pairwise Pearson similarities of time courses for each pair of spatial components; (3) 'partial correlation,' the variance-normalized precision matrix; (4) 'map,' the spatial membership weights of a given spatial component in grayordinate space; and (5) 'spatial network matrix', the matrix of pairwise Pearson similarities of maps for each spatial component. The decomposition rank, feature types, and number of features for each brain representation is summarized in Table 1. Note that since subject data are encoded in terms of features, it is the feature number and not the brain representation's decomposition rank that denotes the dimension d of the target embedding space in the mapping ̂ φ : ̂ S → R d . We compare subject-space embeddings using the pairwise dissimilarities of their points, which we compute as described in the next section.

2.2 Dissimilarity Measures

For each brain representation method, decomposition rank within a given representation, and considered feature type, we compute pairwise distances between all subjects. Each feature type under consideration is structured either as a vector (maps, amplitudes) or a symmetric positive semidefinite (SPSD) matrix (network matrices). This bifurcation of data types is echoed in our choice of measures when computing the dissimilarity between a pair of subjects. In both the vector case and the SPSD data case, we ran our analysis using one dissimilarity measure intrinsic to the data type and another derived from the Pearson correlation. We use Pearson-based dissimilarities in deference to the ubiquitous use of the Pearson correlation in neuroimaging analyses.

We now define the dissimilarity measures we use on vector data. Suppose s i and s j are data vectors in R d , and let ρ ( s i , s j ) denote their Pearson correlation. Let ⟨· , ·⟩ denote the usual inner product on R d . We then define

assuming the matrix D ij = ⟨ s i , s j ⟩ is scaled to have entries in [0 , 1] . Note that we can interpret d v 2 as approximately the angular distance between the vectors s i and s j after each has been centered. We refer to d v 1 as the "inner product divergence" and d v 2 as the "Pearson divergence".

In the SPSD matrix case, we consider the geodesic distance between matrices on the Riemannian SPD cone 38 alongside a (modified) Pearson divergence. The geodesic distance d pd 1 on the symmetric positive definite cone 39 is efficiently implemented via the approximate joint diagonalizer 40 , and we modify the Pearson divergence d v 2 for the correlation matrix case by precomposing it with Fisher's z-transformation 41 (the inverse hyperbolic tangent function): we write

where m i is the vector of upper-right triangle entries of the symmetric matrix M i (diagonal excluded). 168 This precomposition is necessary for correlation matrices, as it normalizes the correlation values 169 before re-correlating them. In contrast to the vector case, there is no simple comparison to be made 170 between these two dissimilarity measures. 171

For each combination of brain representation, rank parameter, and feature type shown in Table 1, 172 we compute pairwise dissimilarity according to both of whichever two measures are relevant. The 173 subject-pairwise matrix of dissimilarities then forms the Gram matrix used to compute the persistent 174 homology, as we describe in the next section. 175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

2.3 Persistent Homology

We compute the Vietoris-Rips persistence 13 of each Gram matrix (which is obtained as described above), and we now give a very brief background on Vietoris-Rips persistence. For a thorough treatment of persistent homology, see Dey and Wang's text 13 ; for a thorough treatment of algebraic topology preliminaries, see Hatcher's text 42 .

2.3.1 Brief background

The topology of a space can be summarized by its homology groups , algebraic invariants that describe its structure. Persistent homology extends the constructions of homology to finite data, delivering a multiscale and threshold-free estimation of data topology. To compute the persistent homology of a dataset X , it must first be equipped with a simplicial structure : a simplicial complex K ( X ) is a set of subsets of X with the property that σ ′ ∈ K whenever σ ′ ⊂ σ for some σ ∈ K , and a filtration is a collection { K t ( X ) } such that K s ⊂ K t when s < t . Homology groups H k ( K t ( X )) can be computed for each simplicial complex, and their persistence PH k ( X ) is described by the evolution of these groups across the filtration. A simple example of a simplicial complex on X is a graph G ( X ) . If that graph G ( X ) is weighted, then the family { G r ( X ) } of graphs obtained from G by thresholding its edges at weight r is a filtration on X . If G ( X ) is the graph on X with edge weights given by the distance between vertices, then the filtration we just described is the Vietoris-Rips filtration on X . Given any dissimilarity matrix d X , we can assume it is the Gram matrix of some graph G ( X ) and compute its Vietoris-Rips persistence PH k ( X ) .

2.3.2 Topological bootstrap

Because it is possible (and, in fact, common) for multiple data elements to define the same homology generator, bootstrap re-sampling 43 is less straightforward in persistent homology than in many other modes of analysis. However, Reani and Bobrowski recently demonstrated a "topological bootstrap" method 14 that uses image persistence 44 to register homology generators found in co-embeddable spaces. If X,Y can both be embedded into a shared space Z , then the inclusion maps X ι X ↪ - → Z and Y ι Y ↪ - → Z induce homology maps ι ∗ X , ι ∗ Y with corresponding filtration maps ι ∗ r,X , ι ∗ r,Y (assuming compatible filtrations on each space). A pair of nontrivial elements in PH k ( X ) and PH k ( Y ) is said to match via Z if ι ∗ r,X and ι ∗ r,Y map them to the same nontrivial element of PH k ( Z ) for some filtration value r . For a matched pair, the affinity score α of the match can be computed from ratios of lengths of intervals in each filtration for which elements in PH k ( X ) and PH k ( Y ) are matched via Z . We assign α = 0 when no match is found and have α ∈ (0 , 1] otherwise.

This procedure simplifies substantially in the bootstrapping case; we then have Z = X and Y = ̂ X ⊂ X , and we need only check nontrivial elements of PH k ( X ) for matches in PH k ( ̂ X ) . In the bootstrap setting, Reani and Bobrowski measure the recurrence stability of a nontrivial generator η ∈ PH k ( X ) by its prevalence score

where ̂ η j is the match of η in the j th bootstrap. This is just the average affinity (over all bootstraps) between η and its matches. In the present study, we compute prevalence scores for each generator in PH 1 ( X ) for a given subject dissimilarity matrix X .

Our implementation 45 of the topological bootstrap is a mild extension of Garcia-Redondo et al's 214 work 15 , which efficiently integrates cycle registration with Ripser 46 and Ripser-image 47 , refines the 215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

cycle affinity measures proposed by Reani and Bobrowski, and broadens the conditions under which topological bootstrapping may be applied.

To satisfy the exchangeability criteria necessary for (any) bootstrapping, we also needed to account for family relationships between subjects in our bootstrap re-samples. Following the approach of Winkler et al. 48 , we excluded all bootstrap re-samples that placed individuals with the same mother on different sides of the inclusion/exclusion divide. We conducted cycle registration using R = 1000 bootstraps per dataset at 90% re-sampling (without replacement), and we consider k = 1 -dimensional cycle registration in this work.

2.3.3 Prevalence-weighted Wassersteinp distance

The space of persistence diagrams is a metric space 49 under the Wassersteinp distance, which previous work 3 has used to compare persistence diagrams of different low-dimensional embeddings. To include statistical information about the stability of homology generators in this comparison, we define the prevalence-weighted Wassersteinp distance

Here, d 1 and d 2 are persistence diagrams, Γ 12 is the set of bijections between d 1 and d 2 , and ρ ( x ) is the prevalence of the homology generator x given in 4. This is a simple re-weighting of the usual Wasserstein distance, modified to incorporate the prevalence score as a summary of per-cycle stability statistics.

2.3.4 The "matched Betti number" β (matched) k

We also define the "first matched Betti number" β (matched) k as the number of matched cycles (i.e., matches with nonzero affinity scores) found in each bootstrapped re-sample. Intuitively, this is a count of the number of stable generators found in each bootstrap. The Betti numbers of a persistence module are typically summarized by curves, since each value of a filtration may induce a homology with a different set of Betti numbers. However, since the topological bootstrap already uses persistence interval information to find matched cycles and compute their affinity, we may consider β (matched) k as having "collapsed" these curves via cycle registration. We consider the distribution of bootstrapped β (matched) k values as a coarse summary of the distributed persistence 28 , 29 of a given dissimilarity matrix d X .

2.4 Study Design

In Table 1, we lay out parameter and feature selections considered for each brain representation. For every representation, bootstrapped persistence is computed for all combinations of feature, parameter, and dissimilarity measure considered; this gives a total of 90 subject-pairwise dissimilarity matrices for which we compute R = 1000 topological bootstraps. We compute the prevalence-weighted Wasserstein-2 distance between all pairs of methods and the β (matched) 1 distributions for each method. This method-pairwise distance matrix then undergoes Ward hierarchical clustering 50 to determine similarity. Our code is publicly available on github.

2.4.1 Hypotheses

Comparing across feature and metric choices, we expect the SPD matrix geodesic distance to exhibit less sensitivity to concentration of measure and thus provide greater distinction between brain representations. We expect that within-feature groupings for map and amplitude will differ very little between the considered vector dissimilarity measures (equations 1 and 2). For all comparisons, we expect feature number and type to be more important drivers of differences than decomposition rank. Finally, within the PROFUMO analysis, we expect that spatial network matrices will be further from null than spatial maps, where we expect the very high dimensions of the spatial maps to suffer from concentration of measure.

Comparing across different brain representations, we expect to primarily see clustering according to (approximate) feature number and type, with secondary similarity clusters forming within each given

brain representation. We expect our analysis to align with previous results in the literature linking 262 shared variance in brain representations 51 -54 , the details of which we expand upon in the results 263 below. 264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

3 Results

3.1 Persistent homology and dimension reduction

We first note several unexpected instances of trivial (or nearly trivial) persistence structure. First, full correlation matrices generated null H 1 persistence at every decomposition rank in every brain representation. By contrast, the partial correlation matrices (which is similar by conjugation to the inverse of the full correlation matrix) have interesting persistence for nearly all feature types, decomposition ranks, and dissimilarity measures. Additionally, the inner product divergence (1) generated trivial or almost trivial homology in both maps and amplitudes, across all ranks and representations; this is not true of the Pearson divergence, which we incorrectly hypothesized would exhibit similar behavior. A complete list of all methods that exhibited trivial H 1 persistence is given in Table S1.

3.1.1 Effect of embedding dimension

Our analysis saw that topological complexity (as measured by H 1 persistence) generally decreased with the number of features considered (Fig S1). Under the geodesic distance, mean prevalence score increased with feature number; for all other dissimilarity measures, mean prevalence score was not correlated with feature number (Fig S2). Taken together, these observations suggest that embeddings in higher dimensions elicit a smaller number of nontrivial H 1 generators which are also more robust. This runs counter to the consequences we might expect from concentration of measure in high dimensions, which pushes spaces towards the discrete topology (and thus a higher number of less stable generators). As expected, we also saw that feature number was a more important driver of persistence structure than the underlying rank of the decomposition (Fig S3).

3.1.2 Persistence vs. prevalence

We see evidence further corroborating Reani and Bobrowski's observation that the most prevalent cycles are not always the most persistent ones 14 . Figure 1 shows a sample persistence diagram in H 1 (colored by generator prevalence score) and a plot of all persistence-prevalence pairs observed in this experiment. Both plots demonstrate that cycles with low persistence can still have high prevalence, suggesting that the topological "noise" may carry meaningful structure in our data. In addition, we see a substantially richer difference structure between target embeddings when using the prevalence-weighted Wasserstein2 distance instead of the classical Wasserstein2 distance (Figure S4).

Prevalence-weighted Persistence in H1

ICA200_Amps_Psim

HomologyPersistencevs.HomologyPrevalence

0.325

1.30

0.8

0.300

1.25

：(death/birth)

(sne)

0.275

0.6

prevalence

1.20

0.250

Death

Persistence

1.15

0.225

0.4

1.10

0.200

0.2

1.05

0.175

0.150

1.00

0.0

0.150 0.175 0.200 0.225 0.250 0.275 0.300 0.325

0.0

0.2

0.4

0.6

0.8

1.0

Birth (radius)

PrevalenceScore

3.2 Persistence differences of brain representations 295

Schaefer600pNMs Psim-ztrans

Schaefer300pNMsPsim-ztrans

Schaefer200pNMsPsim-ztrans

Schaefer1oo0Ampsinner

Schaefer100pNMsPsim-ztrans

ICA300pNMs Psim-ztrans

-GlasserpNMsPsim-ztrans

Yeo17pNMsPsim-ztrans

ICA200pNMsPsim-ztrans

ICA50pNMs geodesic

-ICA25pNMs Psim-ztrans

ICA50pNMsPsim-ztrans

-ICA100pNMsPsim-ztrans

ICA100pNMs geodesic

grad50MapsPsim

grad300Maps Psim

grad100Maps Psim

grad15 Maps Psim

-iCA200pNMsgeodesic

Schaefer200pNMsgeodesic

grad15Mapsinner

grad50Mapsinner

grad100 Mapsinner

grad200 Mapsinner

grad300Mapsinner

Schaefer300pNMsgeodesic

ICA300pNMsgeodesic

GlasserpNMsgeodesic

grad1 MapsPsim

Schaeferio0pNMsgeodesic

-PROFUMOMapsinner

-Schaefer300AmpsPsim

-Schaefer1000AmpsPsim

-Schaefer600AmpsPsim

-PROFUMO spNMs geodesic

-Schaefer200AmpsPsim

-Schaefer100Amps Psim

-ICA25pNMsgeodesic

-PROFUMOspNMs Psim-ztrans

-ICA15 Amps Psim

-ICA25Amps Psim

-ICA100 Amps Psim

-ICA50Amps Psim

-ICA200Amps Psim

-Glasser Amps Psim

- ICA300 Amps Psim

Schaefer300pNMsPsim-ztrans-

Psim-ztrans-

Psim-ztrans-

Yeo17pNMsPsim-ztrans-

ICA50pNMsPsim-ztrans-

PROFUMOMapsPsim

ICA100 pNMs geodesic

grad15MapsPsim

ICA200 pNMs geodesic

Schaefer200pNMsgeodesic-

Glasser pNMs geodesic

ICA200 Amps Psim-

ICA300 Amps Psim-

Yeo17 Amps Psim-

- Yeo17 Amps Psim

Schaefer100pNMsP

ICA300pNMsP

The prevalence-weighted Wasserstein distance makes its strongest distinction between amplitudes 296 and network matrix/spatial map feature types, which form the two main diagonal blocks and highest 297 dendrogram branches (Fig 2). As hypothesized, this implies that our method distinguishes more 298 strongly between feature type (and number) than between brain representation type, which forms 299 the next set of blocks and branches. This is still somewhat surprising, however, because brain 300 representations differ substantially in terms of whether they are unilateral or bilateral, binary or 301 weighted, and decomposition rank. 302

We are also surprised to see PROFUMO spatial network matrices in the amplitude block. Both 303 amplitudes 55 and spatial network matrices 12 have been shown to be highly sensitive to individual 304 differences in behavior, but these feature types are interpreted very differently. Amplitudes may be 305 linked to within-network synchronization 56 , within-network plasticity 57 , or within-network interneu306 ron function 58 , whereas spatial network matrices are indicative of between-network shared brain 307 regions that may play a role in cross-network integration 12 . Both amplitudes and spatial netmats have 308 higher test-retest reliability (i.e., within-subject stability) than the features in the other block 36 , 59 . 309 Given this context, the clustered blocks of the prevalence-weighted Wasserstein may constitute a 310 segregation of trait-sensitive (amplitude and spatial network matrix) from state-sensitive (temporal 311 network matrix) features. This observation highlights the need for an evaluation method that can 312 detect which elements of the persistence module are shared across representations, rather than only 313 being able to similar topologies of subject similarity. 314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

3.3 Computational resources

All other computations, including cycle registration, were negligible in cost compared to the computation of persistence modules for all bootstraps - roughly 270 , 000 persistence modules were computed in total. Memory demands remained relatively low ( ≤ 50 GB per homology computation). Our implementation was embarrassingly parallel on a queue-managed HPC cluster. We estimate that this experiment used approximately 80 , 000 CPU hours over the course of a month. Computation of image-persistence was the most costly individual step, with each embedded persistence module taking 1-3 hours to compute (compared to order of 10 minutes or less for other persistence computations).

4 Discussion

4.1 Conclusions

Our method reveals interesting relationships between dimensionality reductions of resting-state fMRI data. The prevalence-weighted Wasserstein distance distinguishes much more strongly between feature type than dimensionality reduction, potentially segregating trait-sensitive from state-sensitive features. Notably, this distinction holds without regard to choice of dissimilarity measure.

Without exception, full network matrices gave rise to trivial PH 1 modules. Persistence modules generated from the inner product divergence (1) were (approximately) trivial as well, in sharp contrast to those generated from the Pearson divergence (2); this suggests that amplitude and spatial map features of brain representations tend to be "mean-dominated," in the sense that per-subject deviations from group-level structures are typically small.

In addition, we saw a counterintuitive decrease in persistence "complexity" as a function of increasing embedding dimension, which highlights the difficulties of evaluating dimension reduction in highdimensional target spaces. We also examined the relationship between persistence and prevalence, finding that the two are largely uncorrelated for our data. Coupled with the stronger distinctions realized by the prevalence-weighted Wasserstein-2 distance, we believe that persistence and prevalence may be somewhat complementary as measures of cycle importance.

4.2 Limitations

Because of the high cost of parameter exploration, dimensionality reduction computation, and topological bootstrapping, only a few dimensionality reduction methods were examined in this work. An extension of this analysis to a wider array of brain representations may be warranted, especially newer methods that derive an explicitly geometric basis for functional activity (e.g., Laplacian eigenvalues 60 ).

Another important limitation of our work is the very high dimension-to-sample-size ratio ( N ≪ d ) of our data. In this regime, it is difficult to ascertain what features we see because of structure in the data and what topological features are products of the curse of dimensionality. This could be partially ameliorated by conducting our analysis over adequately constructed null data and comparing the results, which is beyond the scope of this work.

4.3 Future Directions

In addition to addressing some of the limitations noted above, we offer several directions for followup work on this study. First, we propose a consideration of the per-bootstrap Wasserstein distance between methods; a distributional picture of differences in the endogenous metric of persistence modules could yield important insights. Second, it is possible to repurpose the topological bootstrap to track the addition/deletion of homology components by different brain representation; practically, this is primarily hindered by the lack of a suitable dissimilarity metric between pairs of points under different embeddings. Finding and validating such a metric would be a valuable direction of inquiry. Finally, we wish to suggest an investigation into the theoretical properties of the prevalence-weighted Wasserstein metric.

References 361

Wang, Y., Huang, H., Rudin, C.& Shaposhnik, Y. Understanding how dimension reduction 362 tools work: an empirical approach to deciphering t-SNE, UMAP, TriMap, and PaCMAP for data 363 visualization. J. Mach. Learn. Res. 22, 201:9129-201:9201 (2022). 364

365

366

Paul, R.& Chalup, S. K. A study on validating non-linear dimensionality reduction using persistent homology. Pattern Recognition Letters 100, 160-166 (2017).

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

Rieck, B.& Leitte, H. Persistent Homology for the Evaluation of Dimensionality Reduction Schemes. Computer Graphics Forum 34, 431-440 (2015).

Lee, J. A.& Verleysen, M. Quality assessment of dimensionality reduction: Rank-based criteria. Neurocomputing 72, 1431-1443 (2009).

Tyagi, H., Vural, E.& Frossard, P. Tangent space estimation for smooth embeddings of Riemannian manifolds®. Information and Inference: A Journal of the IMA 2, 69-114 (2013).

Lotz, M. Persistent homology for low-complexity models. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 475, 20190081 (2019).

Lee, J. A.& Verleysen, M. Two key properties of dimensionality reduction methods. in 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM) 163-170 (2014). doi:10.1109/CIDM.2014.7008663.

Singer, A.& Wu, H.-T. Vector Diffusion Maps and the Connection Laplacian. Commun Pure Appl Math 65, 10.1002/cpa.21395 (2012).

Gracia, A., González, S., Robles, V.& Menasalvas, E. A methodology to compare Dimensionality Reduction algorithms in terms of loss of quality. Information Sciences 270, 1-27 (2014).

Amari, S. Information Geometry and its Applications. vol. Volume 194 (2016).

Bijsterbosch, J. et al. Challenges and future directions for representations of functional brain organization. Nature neuroscience 1-12 (2020) doi:10.1038/s41593-020-00726-z.

Bijsterbosch, J. D., Beckmann, C. F., Woolrich, M. W., Smith, S. M.& Harrison, S. J. The relationship between spatial configuration and functional connectivity of brain regions revisited. eLife (2019) doi:10.7554/eLife.44890.001.

Dey, T. K.& Wang, Y. Computational Topology for Data Analysis. (Cambridge University Press, 2022). doi:10.1017/9781009099950.

Reani, Y.& Bobrowski, O. Cycle Registration in Persistent Homology with Applications in Topological Bootstrap. Preprint at https://doi.org/10.48550/arXiv.2101.00698 (2021).

García-Redondo, I., Monod, A.& Song, A. Fast Topological Signal Identification and Persistent Cohomological Cycle Matching. Preprint at https://doi.org/10.48550/arXiv.2209.15446 (2022).

Böhm, J. N., Berens, P.& Kobak, D. Attraction-Repulsion Spectrum in Neighbor Embeddings. arXiv:2007.08902 [cs, stat] (2021).

Coifman, R. R.& Lafon, S. Diffusion maps. Applied and Computational Harmonic Analysis 21, 5-30 (2006).

Sun, K.& Marchand-Maillet, S. An Information Geometry of Statistical Manifold Learning. in Proceedings of the 31st International Conference on Machine Learning 1-9 (PMLR, 2014).

Sun, K. Local measurements of nonlinear embeddings with information geometry. in Handbook of Statistics vol. 46 257-281 (Elsevier, 2022).

Amari, S.& Nagaoka, H. Methods of Information Geometry. vol. 191 (American Mathematical Society, 2007).

Amari, S.-I., Barndorff-Nielsen, O. E., Kass, R. E., Lauritzen, S. L.& Rao, C. R. Differential 404 Geometry in Statistical Inference. Lecture Notes-Monograph Series 10, i-240 (1987). 405

Banman, A.& Ziegelmeier, L. Mind the Gap: A Study in Global Development Through Persistent 406 Homology. in Research in Computational Topology (eds. Chambers, E. W., Fasy, B. T.& Ziegelmeier, 407 L.) 125-144 (Springer International Publishing, 2018). doi:10.1007/978-3-319-89593-2_8. 408

409

410

Bhattacharya, S., Ghrist, R.& Kumar, V. Persistent Homology for Path Planning in Uncertain Environments. IEEE Transactions on Robotics 31, 578-590 (2015).

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

Petri, G. et al. Homological scaffolds of brain functional networks. Journal of the Royal Society Interface (2014) doi:10.1098/rsif.2014.0873.

Lord, L.-D. et al. Insights into Brain Architectures from the Homological Scaffolds of Functional Connectivity Networks. Frontiers in Systems Neuroscience 10, 85 (2016).

Fasy, B. T.& Qin, Y. Comparing Distance Metrics on Vectorized Persistence Summaries. 6 (2020).

Berry, E., Chen, Y.-C., Cisewski-Kehe, J.& Fasy, B. T. Functional summaries of persistence diagrams. J Appl. and Comput. Topology 4, 211-262 (2020).

Solomon, E., Wagner, A.& Bendich, P. From Geometry to Topology: Inverse Theorems for Distributed Persistence. (2021).

Bubenik, P., Hull, M., Patel, D.& Whittle, B. Persistent homology detects curvature. Inverse Problems 36, 025008 (2020).

Glasser, M. F. et al. The Human Connectome Project's neuroimaging approach. Nature Neuroscience 19, 1175-1187 (2016).

Thomas Yeo, B. T. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of Neurophysiology 106, 1125-1165 (2011).

Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 536, 171-178 (2016).

Schaefer, A. et al. Local-Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cerebral Cortex 28, 3095-3114 (2018).

Comon, P. Independent Component Analysis, a new concept? Signal Processing 36, 287-314 (1994).

Nickerson, L. D., Smith, S. M., Öngür, D.& Beckmann, C. F. Using Dual Regression to Investigate Network Shape and Amplitude in Functional Connectivity Analyses. Frontiers in Neuroscience 11, 115 (2017).

Harrison, S. J. et al. Modelling subject variability in the spatial and temporal characteristics of functional modes. NeuroImage 222, 117226 (2020).

Margulies, D. S. et al. Situating the default-mode network along a principal gradient of macroscale cortical organization. Proceedings of the National Academy of Sciences 113, 12574-12579 (2016).

Moakher, M. A Differential Geometric Approach to the Geometric Mean of Symmetric PositiveDefinite Matrices. SIAM J. Matrix Anal.& Appl. 26, 735-747 (2005).

Pennec, X., Fillard, P.& Ayache, N. A Riemannian Framework for Tensor Computing. International Journal of Computer Vision 66, 41-66 (2006).

Congedo, M., Afsari, B., Barachant, A.& Moakher, M. Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices. PLOS ONE 10, e0121423 (2015).

Fisher, R. A. Frequency Distribution of the Values of the Correlation Coefficient in Samples from an Indefinitely Large Population. Biometrika 10, 507-521 (1915).

Hatcher, A. Algebraic Topology. (Cambridge University Press, 2002).

Abu-Mostafa, Y. S., Magdon-Ismail, M.& Lin, H.-T. Learning from Data. (2012).

Cohen-Steiner, D., Edelsbrunner, H., Harer, J.& Morozov, D. Persistent Homology for Kernels, Images, and Cokernels. in Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms 1011-1020 (Society for Industrial and Applied Mathematics, 2009).

doi:10.1137/1.9781611973068.110.

tyo8/interval-matching-precomp_metric at 67d87792887574195be1376472a9c11f522acc8f. 454 GitHub https://github.com/tyo8/interval-matching-precomp_metric. 455

Bauer, U. Ripser: efficient computation of Vietoris-Rips persistence barcodes. J Appl. and 456 Comput. Topology 5, 391-423 (2021). 457

458

459

Bauer, U.& Schmahl, M. Efficient Computation of Image Persistence. Preprint at https://doi.org/10.48550/arXiv.2201.04170 (2022).

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

Winkler, A. M., Webster, M. A., Vidaurre, D., Nichols, T. E. & Smith, S. M. Multi-level block permutation. Neuroimage 123, 253-268 (2015).

Cohen-Steiner, D., Edelsbrunner, H. & Harer, J. Stability of Persistence Diagrams. Discrete Comput Geom 37, 103-120 (2007).

Ward, J. H. Hierarchical Grouping to Optimize an Objective Function. Journal of the American Statistical Association 58, 236-244 (1963).

Dadi, K. et al. Benchmarking functional connectome-based predictive models for resting-state fMRI. NeuroImage 192, 115-134 (2019).

Botvinik-Nezer, R. et al. Variability in the analysis of a single neuroimaging dataset by many teams. Nature 582, 84-88 (2020).

Bijsterbosch, J. D. et al. The relationship between spatial configuration and functional connectivity of brain regions. eLife (2018) doi:10.7554/eLife.32992.001.

R, K. et al. Comparison Between Gradients and Parcellations for Functional Connectivity Prediction of Behavior. Neuroimage 120044 (2023).

Miller, K. L. et al. Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat Neurosci 19, 1523-1536 (2016).

Lee, S. et al. Amplitudes of resting-state functional networks - investigation into their correlates and biophysical properties. Neuroimage 265, 119779 (2023).

Sydnor, V. J. et al. Intrinsic activity development unfolds along a sensorimotor-association cortical axis in youth. Nat Neurosci 26, 638-649 (2023).

Anderson, K. M. et al. Convergent molecular, cellular, and cortical neuroimaging signatures of major depressive disorder. Proceedings of the National Academy of Sciences 117, 25138-25149 (2020).

Dutt, R. K. et al. Mental health in the UK Biobank: A roadmap to self-report measures and neuroimaging correlates. Human Brain Mapping 43, 816-832 (2022).

Pang, J. C. et al. Geometric constraints on human brain function. 2022.10.04.510897 Preprint at 485 https://doi.org/10.1101/2022.10.04.510897 (2023). 486

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper is an empirical study of different dimensionality reduction (DR) methods for brain activity data. \nAuthors compare various approaches to brain representation using the MRI data from Human Connectome Project. \nIn the manuscript, \"brain representations\" are called \"dimensionality reduction\" (DR) since they present brain MRI data in a compact way.\nBy DR, the author mean ways of presenting brain activity: selecting brain segmentation (parcellations), measuring activity inside these zones, calculating cross-correlation etc. The definition is different from the one used in ML/AI community, where by DR we mean algorithms like t-SNE, UMAP, etc. \nAuthors use topological data analysis (persistent homology, topological bootstrap, prevalence score) to evaluate the quality of DR.\nLots of computational resources were (80, 000 CPU hours over the course of a month) were spent to such an evaluation.\n\nSTRENGTHS:\nNeuroimaging is an active field of research. The paper is technically correct in my opinion. Some recent tools of TDA (like prevalence, topological bootstrap) are applied.\n\nWEAKNESSES:\nFirst of all, I'm not an expert in neuroscience, so I can evaluate only the dimensionality reduction/topology part.\n\n1. In the manuscript, \"brain representations\" are called \"dimensionality reduction\" (DR) since they present brain MRI data in a compact way. By DR, the authors mean ways of presenting brain activity: selecting brain segmentation (parcellations), measuring activity inside these zones, calculating cross-correlation etc. The definition is different from the one used in ML/AI community, where by DR we mean algorithms like t-SNE, UMAP, etc. \n2. No contribution for ML/AI/DL. Typical conclusion from the study: \"As expected, we also saw that feature number was a more important driver of persistence structure than the underlying rank of the decomposition\". The conclusion gives insights about types of brain features, not DR algorithms from ML.\n3. I didn't understand some parts of the paper with neuroscience jargon (cortical parcels, grayordinates, subject space). Captions on Fig. 2 are not explained, what does mean \"Shaefer600 pNMs Psim-ztrans\", etc?\n4. Some references are missing and some relevant methods are not evaluated.\n\nOverall, the paper seems to fit more traditional neuroscience community than NeurIPS community.\nBut I can be mistaken.\n\nQUESTIONS:\n1. Why not to evaluate representations after DR via Representation Topology Divergence [1]? \n2. Why you don't use standard tools for DR, like t-SNE, UMAP?\n3. Recently, some tools for topologically-aware dimensionality reduction were proposed [2, 3].\nCan they by applied for your problem?\n4. The topological bootstrap and prevalence are rather novel tools in topological data analysis. Given significant computational budget (bootstraping and calculating persistence diagrams 1000 times) are there real benefit in using it?\n\n[1] Barannikov, S., Trofimov, I., Balabin, N., & Burnaev, E. (2021). Representation topology divergence: A method for comparing neural network representations. arXiv preprint arXiv:2201.00058.\n\n[2] Moor, M., Horn, M., Rieck, B., & Borgwardt, K. (2020, November). Topological autoencoders. In International conference on machine learning (pp. 7045-7054). PMLR.\n\n[3] Trofimov, I., Cherniavskii, D., Tulchinskii, E., Balabin, N., Burnaev, E., & Barannikov, S. (2023). Learning topology-preserving data representations. ICLR' 2023.\n\nLIMITATIONS:\nAuthors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-03"}

comments: SUMMARY:
This paper investigates shared geometric structure across different (very broadly speaking) dimension reduction algorithms for functional brain connectivity. The authors examine different connectivity representations through persistent homology via topological statistical and bootstrap.

STRENGTHS:
- The paper is written clear language. 
- The paper proposes novel metrics to evaluate graph structures.

WEAKNESSES:
- This paper is quite ambiguously written and not self-contained although the languages are clear. 
- I believe the presentation can be far improved by explaining background better, restructuring paragraphs and better description of mathematical notations. 
- For example, topological sampling is not well explained and readers would have to rely on other papers. Also, sections do not flow smoothly as mathematical notations are either not consistent or variables in equations do not connect. 
- There is no baseline experiments and the result from the analysis cannot be properly validated.

QUESTIONS:
- The authors mention that the primary goal is to compare the structural changes in a single neuroimaging dataset under a variety of brain representations, which is not so clear. Where is the "changes" coming from? 
- Where is $D \sim 10^8$ in line 40 coming from? This is explained far later in line 123 and very data specific. 
- $d$ in section 1 is dimension, then it becomes dissimilarity in section 2.2, and then it becomes persistent diagrams in section 2.3.3. 
- There is no baseline or comparisons with other methods. At least there should some naive approach that should demonstrate the benefits of the proposed analysis.

LIMITATIONS:
There is a section describing the limitation.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study shared geometric structure across different dimensionality reduction (DR) algorithms applied to neuroimaging data (fMRI data from the Human Connectome Project). In particular, they compare different DR algorithms (which they call \"brain representations\") by applying them to the same data sample and comparing the resulting Vietoris-Rips complex in each low rank data embedding using a modified topological bootstrap and cluster on the resulting estimated topologies.\n\nSTRENGTHS:\nThe authors do introduce framework for the comparison of DR methods that work with any data or dissimilarity measure amenable to Vietoris-Rips filtration and apply their method to real neuroscience data.\n\nWEAKNESSES:\nThe authors seem to have put in a good amount of work. However the paper currently lacks motivation or clear significance. Instead the paper reads like a number of different parcellations and DR methods were all applied to a public dataset after which a number of somewhat justified, somewhat arbitrary sequential analysis decisions were made to arrive at a clustering to determine similarity. Good background is given on each step, but it is not clear where the novelty lies here, or why these steps are the right ones. It is unclear exactly what deeply useful conclusions can be drawn from this analysis.\n\nThe writing and definitions could be much more clear throughout. Some are used before they are defined well, some terms with precise meaning seem misused (e.g., induced topology seem misused; \"induced on...data\"), some terms are highly redundant and misleading (e.g., \"brain representations\").\n\nThe github link is not really anonymized (there is another project at the same github link clearly from the \"Personomics Lab\").\n\nQUESTIONS:\n\"Brain representations\" is confusing, sounds like something it isn't, and results in redundant confusing phrases like \"We frame brain representation as a manifold learning problem\" when you've already said you would use the terms \"brain representation\", \"dimensionality reduction\" and \"manifold learning\" synonymously. Why not just use \"DR method\" as you do in lines 62-112?\nAfter wading through this relatively confusing exposition, I am disappointed to be left with little idea as to the impact of the findings. Evidently methods cluster according to feature number and type...why does this warrant publication in NeurIPS? What is the major new insight into DR methods enabled by this approach?\n\nLIMITATIONS:\nThe authors have a limitations section.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an approach of comparison of different standard dimensionality reduction technics (DRT) of fMRI, using topological data analysis tools. In this case, these DRT computes lower dimensional representations of the Human Connectome Project dataset.\nThen, for this specific dataset, the authors associate to each DRT (and for each feature type of each DRT) divergence matrices computed using correlations of these representations.\nFinally, these matrices, seen here as distance matrices, are compared to each other, by \n 1. Computing persistence diagrams associated to the Vietoris-Rips complex of these matrices,\n 2. Reweighting these diagrams by *prevalence scores*,\n 3. Computing wasserstein distances between these diagrams,\n 4. Cluster the DRTs using this distance matrix.\n\nSTRENGTHS:\nLooking at the fundamental differences between different dimension reduction techniques is a very interesting topic: on one hand, this can help increasing the performance of these methods by cleverly taking all of them into account; and on the other hand, this allows to have an insight on what these algorithms are retrieving from the original data. This also motivates to ignore the individual statistical performances of these algorithms. Furthermore, looking at the topology, and the *prevalence* of topological structures of the output of such method, for such a geometric dataset (brain representations), seems to be a very interesting and promising idea.\n\nWEAKNESSES:\nSome techniques used to tackle this problem are not natural or are not motivated enough; either intuitively or theoretically, especially the prevalence-weighted wasserstein distance. In particular, I have the following comments:\n\n - The bijections between persistence diagrams usually add the points of the diagonal, with an infinite mass. In particular, \n    - how is defined the prevalence on the diagonal?\n    - If the diagonal has no mass here, there are also a few problems\n        - A bijection may not exist (not the same cardinal)\n        - This distance is not symmetric, as points of $d_2$ that are not matched to points of $d_1$ are not taken into account.\n - Multiplying points in the diagram by a real correspond to do an **homothety with center $(0,0)$**, which raises a few questions\n    - To my knowledge, 0 has no particular role in a diagram, so what is the motivation for using it?\n    - For a same prevalence $\\alpha$, the rescaling depends at which scale the topological structure appears. And the same goes for the angle direction in which the points are moved. Is this behavior wanted, and why? \n\nI think that taking into account the prevalence is an interesting idea, but I'm not convinced that this is the best way to do it.\n\nMoreover, there are small typos:\n - Section 2.1\n   - Feature types details could be improved, i.e., mathematical definitions.\n - Section 3.2.1\n   - Mention who is $k$\n   - The Vietoris Rips filtration is not a graph, but a clique complex (it has simplices of arbitrary dimensions)\n   - This section could be made bigger, for non-TDA practitioners. \n - Section 2.3.2\n   - Clarity could be improved here,\n   - line 196 \"multiple data element to [represent] the same homology [class]\"\n   - line 205 : intervals are never defined\n\nQUESTIONS:\n1. Looking at the covariance-like matrices of the representations (feature-type) for each method, instead of looking directly at the topology they produce seems weird to me. \n    - How to interpret the topology of these covariance matrices-like space? \n    - How is this topological information relevant to inspect the  structures of the representations?\n 2. The different features types are living on very different mathematical spaces, \n    - How can you assert that your model is identifying brain representation structures rather than these mathematical spaces? These spaces may have significantly different scale densities, so this is not clear.\n 3. The paper does intentionally not take into account pre-representation data.\n    - Is the data fundamentally too large to be taken into account by TDA techniques?\n    - How to identify if the feature extracted by the different representations are given by noise or useful structure from the dataset?\n 4. The constants used to compute the prevalence are fixed at $R=1000$ bootstrap per resampling, at $90\\%$, with homology of degree $1$.\n    - How does these choices matter? i.e., do the results change with other parameters?\n    - What is the motivation for using these values?\n\nOverall, I think the proposed approach is currently too fuzzy and lacks proper motivation, which makes me lean towards rejection.\n\nLIMITATIONS:\nLimitations have been addressed.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA very interesting and rigorous exercise of comparison of embeddings for the purpose of evaluating manifold learning is presented. The chosen framework is root in trendy topological concepts such as persistent homology for the purposes of analyzing the data topology mixed with geometry-based measures of similarity across representations, and coupled with stochastic (topological) bootstrap to study variations over co-embeddings.\nContributions are explicitly mentioned in lns 113-117 and certainly delivered.\nBest of my lot for this year.\n\nSTRENGTHS:\n+ The idea is exceptional IMHO. I’ve known of (and used myself) some other frameworks trying to establish and understand similarities or dissimilarities of projections, but they were all much more naïve. This one offers a clearly more sophisticated approach that yields a much richer picture without substantially sacrificing interpretability of results, with the “only” price to pay of a very large computational cost.\n+ Extremely well explained despite the very complex concepts involved.\n+ The observation on lns 290-1 that cycles with low persistence may carry meaningful structure is something that I have thought myself occasionally but couldn’t put my finger on it nor how to articulate it nor know how to reveal it. This is a very nice confirmation of that intuition.\n\nWEAKNESSES:\nNot many that I can see to be honest…\n+ The study is only conducted in experimental data from the human connectome project but it is never validated in synthetic known manifolds with and without added noise. This means that some of the observations in the last part of the draft are (most likely correct but) difficult to verify. For instance, the implication that the proposed framework distinguishes more feature types and numbers than representation types.\n\nQUESTIONS:\n+ Lns 150-2: Is the separated treatment of the vectors and matrix intentional? Would a tensorial treatment lead to a more homogenous framework?\n+ Lns 298-9: In order to “shift” the focus towards the representation, perhaps one can substitute Pearson’s correlations for Chatterjee’s correlations.\n\nLIMITATIONS:\n+ Lns 92-106 literature review is perhaps missing some of the most primitive approaches; e.g. distance distortions plots, (graph/manifold) isomorphisms, … this is possibly intentional as the persistence element there is only implicit rather than explicit as in the case of the works reviewed. But if it wasn’t, well, it is perhaps convenient to at least mention some early efforts.\n+ Perhaps the number of compared embeddings (brain representations) is not as large as one would like for this type of exercise but the authors clearly state why they stay on low numbers (computational cost) and promise larger comparisons in the future. Looking forward to those!", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}