Surreal-GAN:Semi-Supervised Representation Learning via GAN for uncovering heterogeneous disease-related imaging patterns
Authors: Zhijian Yang
Junhao Wen
Christos Davatzikos
Venue: ICLR 2022 Poster (2022)
Keywords: Representation Learning
Disease-related imaging patterns
Alzheimer's disease
MRI
GAN

Abstract: A plethora of machine learning methods have been applied to imaging data, enabling the construction of clinically relevant imaging signatures of neurological and neuropsychiatric diseases. Oftentimes, such methods don't explicitly model the heterogeneity of disease effects, or approach it via nonlinear models that are not interpretable. Moreover, unsupervised methods may parse heterogeneity that is driven by nuisance confounding factors that affect brain structure or function, rather than heterogeneity relevant to a pathology of interest. On the other hand, semi-supervised clustering methods seek to derive a dichotomous subtype membership, ignoring the truth that disease heterogeneity spatially and temporally extends along a continuum.  To address the aforementioned limitations, herein, we propose a novel method, termed Surreal-GAN (Semi-SUpeRvised ReprEsentAtion Learning via GAN). Using cross-sectional imaging data, Surreal-GAN dissects underlying disease-related heterogeneity under the principle of semi-supervised clustering (cluster mappings from normal control to patient), proposes a continuously dimensional representation, and infers the disease severity of patients at individual level along each dimension. The model first learns a transformation function from normal control (CN) domain to the patient (PT) domain with latent variables controlling transformation directions. An inverse mapping function together with regularization on function continuity, pattern orthogonality and monotonicity was also imposed to make sure that the transformation function captures necessarily meaningful imaging patterns with clinical significance. We first validated the model through extensive semi-synthetic experiments, and then demonstrate its potential in capturing biologically plausible imaging patterns in Alzheimer's disease (AD).

Published as a conference paper at ICLR 2022
SUBREAL-GAN.SSM.-SPEEVVVSSS RNTTENNNTATION LEARNING VIA GAN FOR UNCOVERING HETEROGE- NEOUS DISEASE-RELATED IMAGING PATTERNS
Zhijian Yang'.2, Junhao Wen' and Christos Davatzikos'
1 Center for Biomedical Image Computing and Analytics, University of Pennsylvania
2 Graduate Group in Applied Mathematics and Computational Science, University of Pennsylvania {zhi jian. yang, junhao wen, christos davatzikos (eeeaeieeii upenn. edu
ABSTRACT
A plethora of machine learning methods have been applied to imaging data, en- abling the construction of clinically relevant imaging signatures of neurologi- cal and neuropsychiatric diseases. Oftentimes, such methods do not explicitly model the heterogeneity of disease effects, or approach it via nonlinear models that are not interpretable. Moreover, unsupervised methods may parse hetero- geneity that is driven by nuisance confounding factors that affect global brain structure function, rather than heterogeneity relevant to pathology of inter- est. On the other hand, semi-supervised clustering methods seek to derive a pathology-relas related dichotomous subtypes, ignoring the truth that disease hetero- geneity spatially and temporally extends along a continuum. To address the afore- mentioned limitations, herein, we propose novel method, termed Surreal-GAN Slemi-SUpeRviine ReprEsentAtion Learning via GAN), Using cross-sectional imaging data, Surreal-GAN dissects underlying disease-related heterogeneity un- der the principle of semi-supervised clustering [cluster mappings from the normal control (CN) patient (PT) domain], proposes a continuously dimensional repre- sentation, and infers the disease severity of patients at individual level along each dimension. The model first learns a transformation function from the CN domain to the PT domain with latent variables controlling transformation directions. An inverse mapping function together with regularization on function continuity, pat- tern orthogonality and monotonicity formation function captures necessarily meaningful imaging patterns with clinical significance. We first validated the model through semi- myni-syythetic experiments, and then demonstrated its potential in capturing biologically plausible imaging patterns in Alzheimer's disease (AD).
also imposed make sure that the trans-
INTRODUCTION
Neuroimaging, conventional machine learning, and deep learning have offered unprecedented op- portunities understand the underlying mechanism of brain disorders the past decades (Da- vatzikos (2018)) ), and pave the road towards individualized precision medicine (Bzdok & Meyer- Lindenberg (0118) A large body of case-control studies leverage mass univariate group compar- isons to derive structural or functional neuroimaging signatures (Habeck et al. (2008), HHmmee et al.) 2008 Ewers al. (201T). However, these studies suffer from under-powerss statistical infer- ences since the heterogeneous nature of neurological diseases usually violate the assumption that each group population are pathologically homogeneous.
A developing body of methodologies aim to disentangle this heterogeneity with various machine learning methods. Zhang et al.) (2016) used the Bayesian Latent Dirichlet Allocation (LDA) model to identify latent atrophy patterns from voxel-wise gray matter (GM) density maps. However, due to the nature the LDA model, the density maps need to be first discretized, and the model hy- pothesized to exclusively capture brain atrophy while ignoring the potential regional enlargement. Young et al. (2018) proposed another method, Sustain, to uncover temporal and phenotypic hetero- geneity by inferring both subtypes and stages. However, the model only handles around 10-20 large

---

Published as a conference paper at ICLR 2022
brain regions without more detailed information. Notably, both methods applied directly in the PT domain, confront main limitations in avoiding potential disease-unrelated brain variations. In con- trast, semi supervspeeeieed clustering methods (Varol al. (2016), Dong al. (2015), WWen et al.) (202I) Yang et al. (202T)) were proposed to cluster patients via the patterns or transformations between the reference group (CN) and the target patient group (PT). A recent proposed deep learningbbased semi-supervised clustering method, termed Smile-GAN (Yang et al. (202)), achieved better cluster- ing performance by learning multiple mappings from CN to PT with an inverse clustering function for both regularization and clustering. Albeit these methods demonstrated potential in clinical appli- cations, they have a major limitation. In particular, these methods model disease heterogeneity as a dichotomous process and derive a discrete output, i.e., subtype, ignoring the fact that brain disorders progress along a continuum. Moreover, variations among subtypes were contributed by both spatial and temporal differences in disease patterns, thus confounding further clinical analysis. [a) To address the affrcementioned limitations, we propose a novel method, Surreal-GAN (Semi- Supervised Representation Learning via GAN), for deriving heterogeneitt--rffinn tures. Surreal-GAN is a significant extension of Smile-GAN under the same principle of semi- supervised learning by transforming data from CN domain X to PT domain Y. Herein, the key extension is to represent complex disease-related heterogeneity with low dimensional representa- tions with each dimension indicating the severity of one relatively homogeneous imaging patterns (Fig. [p). We refer to these low dimensional representations as R- R-indices (r,, where i represent the ieh dimension) of disease pattern. The first key point of the method is modelling disease as a continuous process and learning infinite transformation directions from CN to PT, with each direc- tion capturing a specific combination of patterns and severity. The idea is realized by learning one transformation function which takes both normal data and continuous latent variable as inputs and output synthesized-P7 data whose distribution is indistinguishable from that of real PT data. The second key point is controlling the monotonicity (from light to severe with increasing index value) of disease patterns through one monotonicity loss and double sampling procedure. The third key point is boosting the separation of captured disease patterns through one orthogonality regulariza- tion. The fourth key point is introducing an inverse mapping function consisting of a decomposition part and a reconstruction part which further ensure that patterns synthesized through transforma- tion are exactly captured for inferring representations of disease patterns, the R-indices. Besides the above mentioned regularization, function continuity, transformation sparsity and inverse consistency further guide the model to capture meaningful imaging patterns.
imaging signa-
To support our claims, we first validated Surreal-Gr GAN on semi-synthetic data with known ground truth of disease patterns and severity. We compared performance of Surreal-GAN to NMF (Lee) & Seung (199)), opNMF (Sotiras et al. (2014)) Discriminant-NMF (Lee et al. (0112), Factor Analysis and LDA (Blei et al. (2001)) models, and subsequently studied the robustness of model under various conditions. Finally, we applied the model to one real dataset for Alzheimer's dis- ease (AD), and demonstrated significant correlations between the proposed imaging signatures and clinical variables.
2 METHODS
The schematic diagram of Surreal-GAN is shown in Fig. Td. The model is applied on regional volume data (volumes of different graymatter (GM) and whitematter (WM) regions) derived from structural MRI. The essential idea is to learn which transforms CN data x to different synthesized PT data y' = f(x, 2). with latent vari- able z specifying distinct combinations of patterns and severity. Here, the Latent (LAT) domain, Z = {z: : 0 < 24 < 1 A1 < i < M). is a class of vectors with dimension M (predefined number of patterns). We denote four different data distributions as x Pen (x). y ~ (y). y' ~ Peyn(yy) and z ~ Piat (z), respectively, where ~ Plat (z) is sampled from a multivariate uniform distribution U[0, 1 rather than a categorical distribution as Smile-GAN. continuous variable lay the foundation for learning continuous representations. In addition to the transformation function, an adversarial discriminator D is introduced to distinguish between real PT data y and synthesized PT data y', thereby ensuring that the synthesized PT data are indistinguishable from real PT data (i.e. minimizing the distance between real PT data distribution and synthesized PT data distribution). A continuous latent variable does not necessarily lead to desired properties of CN to PT transfor- mation (e.g. sqparation/monooonictyy of captured patterns). That is, there are a number of functions potentially achieving equality in distributions, thus making it hard guarantee that the transfor-
transformation function f X Z - Y
2

---

Published as a conference paper at ICLR 2022
(a)
(b)
P2
1
0.75
P
P4
0.75
P3
2
(c) Network Architecture
Function
(d) Schematic Diagram
CN Domain
f
PT
Domain Y
D
Function g
Discriminator D
LAT Domain
y/y
Z
Figure 1: Model Ideas and Schematic Diagram. (a) Results from Semi-supervised clustering for AD disease: model clusters PT into discrete subtypes based on both spatial and temporal variations: P1 and P4 show mild and severe stages and are mixture of two patterns; (b) Improvements from Surreal-GAN: two patterns are captured by two indices with values indicating severity; (c) General architecture of networks; (d) Schematic diagram of Surreal-GAN.
mation function learned by the model is closely related to the underlying pathology progression process. Morcover, during the training procedure, the transformation function backboned by the neural network tends to ignore the latent variable z. Even when the Latent variable is emphasized through regularization, different components of variable do not naturally lead separate disease patterns and are not positively correlated with pattern severity. Therefore, with the assumption that there is one true underlying function for real PT data y h(x, z). Surreal-GAN aims boost the transformation function f to approximate the true underlying function h, while having the latent variable, z, contributing recognizable imaging patterns rather than being random "activators" of the GAN. For this purpose, we constrain the function class of ff via different types of regularization as follows: 1) encourage sparse transformations; 2) enforce Lipschitz continuity of functions; 3) introduce one inverse function Y + Z serving to decompose and reconstruct; 4) boost orthog- onality among synthesized/capturee patterns; 5) enforce monotonicity of synthesized patterns and their positive correlations with components of The training procedure of Surreal-GAN is thus adversarial learning process constrained by these regularization.
2.1 ADVERSARIAL LOSS
The adversarial loss (Goodfellow et al. 2014)) is used for iterative training of discriminator and the transformation function f. which can be written
LGAN f) = Eyrpp log( Dyy)) + Ex (x) am Pra [log(1 - (x)(xx2))))
(1)
Ey ( [log ((y)) + Ey (x)) log(1 D(y')))
(2)

---

Published as a conference paper at ICLR 2022
The transformation function f attempts transform CN data synthesized PT data that they follow similar distributions as real PT data. The discriminator, providing a probability that y comes from the real data rather than transformation function, tries to identify the synthesized PT data and distinguish it from real PT data. Therefore, the discriminator D) attempts to maximize the adversarial loss function while the transformation function f attempts to minimize against it. The training process can be denoted as:
min max LGAN f
L (D, f) Ey-p (ogg((()))))))) Bn (n log(11 D(y')l
(3)
2.2
REGULARIZATION
2.2.1 SPARSE TRANSFORMATION
We assumed that disease process will not change brain anatomy dramatically and primarily only affect certain regions throughout most of disease stages, which means that the true underlying trans- formation only moderately changes some regions while keeping the rest mildly affected with most of z variable values. Therefore, to control sparsity and distance of transformations, we defined one change loss to be the | distance between the synthesized PT data and the original CN data:
Lehange (f) Exee-Pen ae (x) fl. z) - x
2.2.2 LIPSCHITIZ CONTINUITY AND INVERSE MAPPING
First, by enforcing the transformation function f to be K₃ -ipiscchitz continuous, we can derive that, for fixed latent variable z a and Væ 22 E X. (z a) f(222 a)]l2 S K1|læ, æ2112- Thus, by controlling the constant Ki, we constrained the transformation function to preserve the original distances among CN data instead of scattering them into the PT domain if same combination of discase patterns is imposed them.
Second, to prevent latent variable z from being ignored, we introduced an inverse mapping function from PT domain to Latent domain, g Y + Z. By Lemmal (section A.1), with function being Ky-Lipschitz continuous and d(-..) being any distance metric satisfying triangular inequality, we derive that, Vz Ø2
Piax (2))) 21 * Ø2, and E (x), d(f 21), f x2)) is lower K2 (d(g(f(@, 21)), 21) + d(g(f(a, z2)), 22))). Therefore, by minimizing the distance between sampled latent variable z and reconstructed latent variable g(f(x, 2)), we can control differences between synthesized patterns to be positively related to distances between latent variables, and thus to be non-trivial (i.c., same CN data is transformed to significantly different PT data with different variables). Therefore, we define a reconstruction loss as the distance between g(f ox((xx (x,z) z)) and z:
bounded by (d)
K2
Lrecons (fi9) Exxpor (x). (lll -212]
(5)
2.2.3 PATTERN DECOMPOSITION
Nevertheless, a simple function directly reconstructing sampled z from synthesized PT y' do reconstruct each component z; completely based on changes led by z, in transformation. We are more interested in pattern representations of real PT data than generating synthesized PT, and inverse function g will be used for inferring R-indices as introduced in section 2.4. Thus, it is very important for the inverse function to accurately capture patterns synthesized by intensively regularized transformation function f. For this purpose, we further separated the function g Y Z into 91 Y + Ig (where M is number of patterns and S is number of input ROIs) and g2 : R + I. Decomposer 91 serves to reconstruct changes synthesized by each component 2, in transformation process: q: f(x, a') x, where a' is vector with éeh component al z; and a$ = 0, Vi * j. Let Qf(2,)) = [qf. ; ................................ 4 be concatenation of all synthesized changes q, we
defined the decomposition loss as:
Liteor (f, (f.g1) = EEx-p.. (x. Illg1 (f(xx, z)) - (x(
(6)
S2 serves to further reconstruct each component of sampled z variable from g1 (f((xx))) dently, that the reconstruction loss can be rewritten as:
z)) indepen-
Liccons (f. g) Licsons (f.91.92 Expxpo (x), (xar (ll =/122
4

---

Published as a conference paper at ICLR 2022
where 1n2(2 (g: (f(x.x)) of z)) (f ((((xx2))0s0 z))o.))... 92((9 (f (M) 1):8.M)]
Function here can be also considered as the approximation of expectation of posterior distribution. In this sense, minimization of l2 distances can also be interpreted as maximization of mutual infor- mation between latent variable 2 and synthesized PT data y' [(x,2) as explained in Remark1 in section A.2 {Chen al.) (20T6). From this perspective, the transformation function is forced to best utilize information from latent variable z, while also keeping mutual information between synthesized PT data y' and original CN data by controlling transformation distances.
2.2.4 ORTHOGONALITY OF PATTERNS
Inclusion of latent variable z in the transformation function does not guarantee that different com- ponents of the latent variable, 2;, contributes to relatively different patterns in synthesized PT data. Instead, during training procedure, different components, 2,, are more likely to synthesize pattems in same regions and lead to accumulated severity. Therefore, we define another orthogonal loss to encourage separation among patterns synthesized by different components. With changes led by each component, q defined in section 2.2.3, we constructed a matrix A f(x.2) with the 4n column A f(x.x),, = //lll To encourage separation of changes led by different components, we boosted the matrix A fxx,2 to be relatively orthogonal by minimizing the following orthogonality loss:
Tionho (f) Expopen 2x) (2) [(xx2 A(x2)) ((x,))
- [[F]]
2.2.5 MONOTONICITY AND POSITIVE CORRELATION
Besides separation of synthesized patterns, positive correlation between values of each compo- nent z; (from to 1) and severity of synthesized patterns can not be acquired spontaneously either. Regarding severity of the disease pattern, assumed that, pattern is becoming more severe, absolute values of changes in regional volumes can only increase monotonically or remain constant. In another word, there is no oscillation regional volumes as disease pattern is becoming more severe, while more regions can be included gradually with absolute changes switched from 0 to positive value. To satisfy this requirement, we sampled another latent variable z' Psev (z), conditioned on previous sampled variable, such that z 2 2; V1 S < M. With these double sampled latent variables, we defined the monotonicity loss to be:
Limono (f) = Ex-pen (x) xmiue (x) -Psev [I] m 2) - x - (f x| 0)1la
Minimization of this monotonicity loss penalize the case that volume changes induced by is larger than that induced by z', while the other direction permitted. To further ensure positive correlation (small z lead to mild patterns; large lead to severe patterns), we introduced a en loss, which serves more strictly constrain the distance between synthesized PT data y' and CN data x, when latent variable z is close to O vector. By letting Pen (z) = U (0, 0.05) to be multivariate uniform distribution, the en loss defined as:
(fn( = Exxppnn (x). (xlf(x,z - x||1]]
(10)
2.3 FULL OBJECTIVE
With the adversarial loss and all other loss functions defined for regularization, we can write the full objective as:
(ID,J.9199) 92) = TGAN( D, f) + TTchhanee (f) + K. KT (f + [Trecun (f. I1. 92)
+ XLonto (f) )))()))))))))))) + pLmmooo (f) + (f)
(11)
with 7, K. Ç. A, a and 17 be parameters controlling the relative importance of different loss functions. Through training process, want to derive parametrized functions f.gi and g2 satisfying:
f, 92 arg min max (191,1)
(12)
f.g
2.4 REPRESENTAATIN INFERENCE
With the assumption that Paym (x, z)) so (y) and f satisfying all constraints, we consider learned transformation function, be good approximation of the true underlying function h, such
5

---

Published as a conference paper at ICLR 2022
that f(x.2) 2) = h(x, (z),, where Q and Q is class of permutation functions which change orders of elements in vector z. Since the order of indices in derived representation is not important simply rewrite it as f(x, z) as h(x, z) without loss of generality. For any real PT data, y h(x, r) ~ (y), we can derive that the ground truth R-indices g(f(x, r)) g(h(x,r) r)) g(y). Therefore, reconstruction function g can be directly applied to infer R-indices of disease patterns, r, of any unseen PT data.
and we always reorder them to find the best matching,
3
IMPLEMENTATION DETAILS
3.1 MODEL ARCHITECTURE
The general architecture of networks can be understood from Fig. D.. The transformation function f utilizes encoding-ded -eeooding structure. Latent variable z and CN data x are first mapped two vec- tors with dimension 34 and the element-wiss multiplication of them are then decoded to construct the synthesized PT data with dimension 139. The inverse function g first decodes real/synthesized PT data y /yy to larger vector with dimension 139*M and then maps them into a vector with di- mension M. The discriminator utilizes the encoding structure which maps y/y' to a vector with dimension 2. More details of model architectures can be found in Appendix Table 2 [ndd]
3.2 LIPSCHITZ CONTINUITY
In implementation, we performed weight clipping to ensure Lipschitz continuity of transformation function f and inverse funetion g (Arjovsky al. (0177). With 6 and G denoting the weight spaces of function f and g. the compactness of them imply the Lispchitz continuity of two func- tions, with Lipschitz constants K, and K₂ only depends on O, and O,, The compactness of θ and G was guaranteed by clapping weight spaces into two closed box, θ = [y and implementation, both cy and C were set to 0.5 and these bounds can be
Og = l-c, further relaxed. Weight clipping was claimed not be the most satisfactory method to ensure Lips- chitz continuity because of difficulty in choosing the appropriate clipping bounds. However, in our case, performed weight clipping for function and rather than discriminator D for different purpose and weight clipping contributed to good performances for deriving representations.
3.3 TRAINING DETAILS
Regarding optimization procedure, ADAM optimizer(Kingma Ba (2014) was used with a learn- ing rate (Ir) 4 10 for Discriminator and 2 10 for transformation function f and clustering function g. 31 and 33 are set to be 0.5 and 0.999 respectively. For
we set 7 = 6, 80, Ç 80, f 500, 17 6. However, performance of model different tasks are robust varying papre-paraeeters as shown in section B.2. Parameter X determines the relative importance of orthogonality loss and controls the degree of overlapping among synthesized patterns. Without prior knowledge on overlapping degree of underlying disease patterns, we set À to different values in the following experiments and used agreements among multiple trained models determine the optimal one. Moreover, for all experiments, the batch size was set to be 1/8 of the PT data sample sizes. The model was trained for at least 100000 iterations and saved until the reconstruction loss and the monotonicity loss are smaller than 0.003 and x 10-1 respectively. Higher Ir might improve convergence speed some cases. Detailed training procedure is revealed by Appendix Algorithml.
4 EXPERIMENTS
4.1 DATA PREPROCESSING
For semi-synthetic and real data experiments, we used Tl-weighted (TIw) MR imaging (MRI) and cognition from the iSAGING (Imaging-basee coordinate SysTem for AGIng and NeurodeGenerative diseases) consortium for cognitive impairment and dementia. All baseline MRIs were corrected for intensity inhomogeneities (Sled et al. (1998)) Then, 139 regions of interest (ROIs) of brain volumes were extracted using a multi-atlas label fusion method (Doshi et al. (2015)). harmonized (Pomponio et al. 2020)) remove site effect, and then utilized input features for Surreal-GAN (Fig.

---

Published as a conference paper at ICLR 2022
4.2 SEMI-SYNTHETL EXPERIMENTS
Semi-synthetic data construction To set the ground truth of disease patterns and severity a prior, we generated comi-synthetic data by imposing simulated disease-related patters real CN data, thereby retaining the variation from the real CN data. Specifically, we split 1392 cognitively normal (age < 70 years) into one real CN group with 492 subjects and one Pseudo-PT group with 900 sub- jects. For the ith Pscudo-PT subject, sampled three-dimensional vector as pattern severity from a multivariate uniform distribution, S ~ (i(0, 1 Three types of atrophy patterns were introduced to 900 Pseudo-PT subjects. Different regions were included in different patterns shown in Appendix Table 4 Five different datasets were constructed for testing the robustness of model under different conditions: (1) Basic dataset: For the &ta. Pseudo-PT subject and the jmh ROI included in pattem k, we decreased the volume Ui depending on sampled severity: eaj = Vij Vig 85k * (1,, 0.05) 0.3. Each pattern consists of changes in 14 ROIs and has four of them shared with other patterns; (2) Large overlapping dataset: Same as the Basic dataset, except that each pattern shares 8 ROIs with others; (3) Scarce regions dataset: Each pattern consists of changes in only ROIs with other parts the same as the Basic dataset; (4) (4)Within-patter) noise dataset: Regional changes incorporated in the same pattern have larger within- patterr variances with Biy = Vay Vii Sik (111, 0.2) * 0.3; (5) Mild atrophy dataset: Compared to the basic case, smaller changes were imposed by letting Mey Uij Vij S.k * (.000.5)) * 0.2.
Evaluation and Agreement Metric In our experiments, Concordance Index (c-index) was used as the evaluation metric. Specifically, for each pattern, we calculated a c-index between the inferred R- R-index and the ground truth. The average of M derived C-indices, refereed as pattem---innex,, was used for model evaluations. Moreover, to quantify the pair-wise agreement between two trained models, we proposed another Pattem-agr-index which equals pattern-c-index between R-indices de- rived by two different models.
Experiment Procedure We first validated the relationship between model agreements and their accuracy. On each dataset, we ran the model 10 times with parameter À = 0.1, 0.4, 0.6, 0.8 among 10 models was com- Further, the À value leading to the best agreement was
respectively. With each X, the average pair-wise pattern-agr-inde pared with their average pattern-c-indice considered as the best choice, and results from 10 corresponding models were reported as model performances for each task. Moreover, we fixed the optimal À value for each task and changed other parameters to to understand contributions from each regularization term (section B.3). Lastly, we compared our model with five other models. NMF and Factor Analysis (FA) are well-known for deriving lower-dimensional representation of complex dataset. The variant oPNMF (Sotiras et al. 22044) has shown promise in parsing complex brain imaging data with extra orthogonal constraint. Discriminant-NMF (DNMF) (Lee et al. (2012) was proposed to learn representations which best classify subgroups. LDA, though known for extracting topics from documents, have been recently applied to uncover heterogeneity of AD disease from neuroimaging data (Zhang et al. (2016)). Data preprocessing for these models were introduced in section B.4.
4.3 REAL DATA EXPERIMENTS
Data Selection We applied Surreal-GAN to an AD dataset. For AD, we defined the CN group (N=850) to be subjects with Mini-mental state examination (MMSE) scores above 29, and the PT group (N=2204) as subjects diagnosed as mild cognitive impairment (MCT) or AD at baseline. Experiments Procedure All CN and PT subjects were first residualized to rule out the covariate effects estimated in the CN group using linear regression. Then, adjusted features were standardized with respect CN group. Without ground truth, we selected both the optimal number of patterns, M, and À parameter by measuring agreements among repetitively trained models. For each combi- nation of M 2. 3, 4 and \= 0.1, 0.2, 0.4, 0.6, 0.8. we ran the model 10 times respectively. The M and X values leading to the highest agreement was considered optimal. Among the 10 corresponding models, the one having the highest mean pair- agreements with the other models was used to derive R-indices for all PT subjects. To visualize patterns corresponding to the ish dimension of R-indices, r,, we grouped subjects into three different subgroups: r; < 0.4, 0.4 < r, < 0.7 and r; > 0.7, with r2 < 0.4 for all + i. Voxel-wise group comparisons were performed between each subgroup and the CN group via AFNI- (R. W. (1996) and GM-tissue map (Davatzikos et al. (2002h) which encodes the volumetric changes in GM observed during the registration. Pattern representations, R-indices, of PT subjects were further compared with other clinical variables to test their significance. Details of clinical variables and statistical tests can be found in section B.9.
7

---

Published as a conference paper at ICLR 2022
(a)
Representation accuracies meesured pattern c-index
(b)
Model agreements measured by pallerm-arr-inex
0.800
0.825
0.825
0.500
Pallm---addx
0.00
Papern Paberaagg indx
0.880
0.77
0.623
0.750
0.800
0.775
0.700
0.75
0.725
lambda
(1/Basic
L2]Large overlapping
(3/Scarce region
(4]Within pattern noise
(5]Mild changes
Figure 2: Results on Semi-synthetic data (results presented as mean value + standard error). (a) Representation accuracies on different tasks (measured by Pattern-c-index); (b) Agreements among repetitively trained models on different tasks (measured by Pattern-agr-nndex).
Table Model Comparison Results
Surreal-GAN
NMF
LDA
FA
OPNMF
DNMF
0.838
0,680
0.605
0.631
0.648
0.562
Basic Case
+ 0.011
+ 0.000
+ 0.001
+ 0.000
+ 0.000
+ 0.017
Large
0.837
0.692
0.591
0.612
0.670
0.566
Overlapping
+ 0.008
0.000
0.000
0.000
+ 0.000
#.022
0.783
0.625
0.582
0.526
0.608
0.525
Scarce Regions
0.007
+ 0.000
+ 0.001
0.000
+ 0.000
+ 0.006
Within-pattern
0.818
0.675
0.626
0.619
0.651
0.550
Noise
0.013
+ 0.000
0.000
0.000
0.000
- 0.013
0.762
0.611
0.574
0.526
0.594
0.539
Mild Changes
0.023
+ 0.000
0.004
+ 0.000
0.000
+ 0.013
RESULTS
5.1 RESULTS SEMI-SYNTHTTIC DATA
We first validated that agreements among multiple trained models (Fig. 2b) are good indicators of model accuracy (Fig. 2h) for parameter selections. Besides, from Fig. 2 we proved the model's ability in capturing ground truth of patterns and severity under different cases including pattern over- lapping and large within-pattern noise. However, the model experienced declining performances when each pattern includes only scarce discase-related regions, and showed even worse perfor- mances when very mild atrophies are led by disease process. This is expected since, for the mild atrophy case, the simulated atrophy rate, 0-20% of the regional volume, is even smaller than the standard deviation of most regional volumes among the normal population. Finally, Surreal-GAN constructed (Table Ip. However,
significantly outperformed all five methods five data none of these methods optimally utilize the CN data as a reference distribution, which is one of important limitations preventing them from precisely learning disease-specific pattern representa- tions rather than capturing liscasc-unrelate confounding variations. The family of DNMF methods capture discriminant representation by minimizing and maximizing within-class and among-class variations respectively, thus did not capture heterogeneity within the PT class accurately either. De- signed for parsing disease heterogeneity, semi-supervised methods focused on clustering patients into hard categorical subtypes (Varol et al. (2016), Dong et al. (2015), Wen et al. (2021). Yang et al. (2021). With different end points, these methods (including Smile-GANi can not be directly compared with Lurrea--GAN To the best of our knowledge, approach for learning disease-related representations, using a healthy reference group as a means for comparison. Therefore, to better understand the model performance, leveraged the simulated
model is the first semi-supervised
8

---

Published as a conference paper at ICLR 2022
0.4 < 1
< 02 < 0.4
n> 0.7, < 0.4
AD Disease
T-stats 8
< 0.4, 12 0.4
< 4.4 <.77
< 0.
< 0.7
Voxel-wise statistical comparison results between selected subgroups and CN. (FDR method with p-value threshold of 0.05 was used for multiple comparison). each direction guided by arrows, we increased values of one R-index and keep the other relatively fixed. Therefore, chose out groups subjects who have much stronger expression of one pattern for visualization.
Figure 3:
ground truth information (not available in real experiments), derived "practical upper bounds" for compared unsupervised models, and also computed the gap between Surreal-GAN and supervised methods. (section B.5 and B.6)
5.2 RESULTS ON REAL DATA
On the AD dataset, models show highest agreement when the number of patterns M 2. From voxel-based comparison results (Fig. 3). we can clearly visualize two patterns related to AD, Fach dimension of R-indices, r;, shows clear positive correlation with the corresponding pattern ity. Patternl, correlated with rı, shows diffuse cortical atrophies (c.g., bilateral insula, orbitofrontal and frontal poles), while pattern2 reveals focal atrophy in medial temporal lobes. Reproducibility of these two patterns were validated in section B.7. Furthermore, the 2D R-indices were effective and interpretable signatures for disease diagnosis and prognosis, achieving similar performance AD classification compared to original 139 ROIs (section B.8).
Continuity and monotonicity of inferred R-indices further enable us to easily test correlation among them and other clinical variables (Table 7). Both rı and are highly correlated with CSF-Abeta value and memory dysfunction mcasured by ADNI-MEM. However, F1 shows stronger correlations with white matter lesion volumes, presence of hypertension, executive function and language func- tioning, while r2 is more correlated with CSF-Tau, CSF-pTau and presence APOE-EA E- alleles.
CONCLUSION
In this study, we have proposed a novel method, Surreal-GAN, for learning representations of under- lying disease-related imaging patterns. This model has overcome limitations in previous published semi-supervised clustering methods and shown great performance on semi-synthetic data sets. On AD dataset, the model parse heterogeneous PT data into two concise and clinically informative in- dices with preserved discriminant power, further proving its huge potential in uncovering disease heterogeneity from MRI data.

---

Published as a conference paper at ICLR 2022
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan, 2017.
David Blei, Andrew Ng, and Michael Jordan. Latent dirichlet allocation. volume 3, pp. 601-608, 01 2001.
Danilo Bzdok and Andreas Meyer-Lindenberg. Machine learning for precision psychiatry: Oppor- tunities and challenges. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 3, 02 2018. doi: 10. http:/..pp....1..1...oo
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in Neural Information Processing Systems, 29, 06 2016.
Christos Davatzikos. Machine learning in neuroimaging: Progress and challenges. Neurolmage, 197, 10 2018. doi: 10. 1001/f..euroimage..2. 8. 10.003.
Christos Davatzikos, Ahmet Genc, Dongrong Xu, and Susan Resnick. Voxel-based morphometry using the ravens maps: Methods and validation using simulated longitudinal atrophy. Neurolm- age, 14:1361-1369, 2002. doi: 10.1006/nimg2200.009
Aoyan Dong, Nicolas Honnorat, Bilwaj Gaonkar, and Christos Davatzikos. Chimera: Clustering of heterogeneous disease effects via distribution matching of imaging patterns. IEEE transactions on medical imaging, 35, 10 2015. doi: 10. 10..11999...011228777777
Jimit Doshi, Guray Erus, Yangming Ou, Susan Resnick, Ruben Gur, Raquel Gur, Theodore Sat- terthwaite, Susan Furth, and Christos Davatzikos. Muse: Multi-atlas region segmentation uti- lizing ensembles of registration algorithms and parameters, and locally optimal atlas selection. Neurolmage, 127, 12 2015. doi: 10..016j.eerrrmgge..1111..1.1
Michael Ewers, Reisa Sperling, William Klunk, Michael Weiner, and Harald Hampel. Neuroimag- ing markers for the prediction and early diagnosis alzheimer's 's disease dementia. Trends in neurosciences, 34:430-42, 06 2011, doi: 10.1016/j.timm wttp:..iin........... 1.05.005
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Y. Bengio. Generative adversarial networks. Advances in Neurul Informa- fion Processing Systems, 3, 06 2014.
Christian Habeck, Norman Foster, Robert Perneczky, Alexander Kurz, Panagiotis Alexopoulos, Robert Koeppe, Alexander Drzezga, and Yaakov Stemn Multivariate and univariate neuroimag- ing biomarkers of alzheimer's disease. Neurolmage, 40:1503-15, 06 2008. doi: 10.1016/j.
neuroimage. 200..01...000...00 00..60.0066.
Harald Hampel, Katharina Bürger, Stefan Teipel, Arun Bokde, Henrik Zetterberg, and Kaj Blennow. Core candidate neurochemical and imaging biomarkers of alzheimer's disease. Alzheimer's & dementia : the journal of the Alzheimer's Association, 4:38-48, 01 2008. doi: 10.1016/j..alz.
2007.08.006.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference Learning Representations, 12 2014.
Daniel Lee and H. Seung. Learning the parts of objects by non- non-negative matrix factorization. Nature, 001:788-91, 11 1999. doi: 10. 1038//4/4455.
Soo-Young Lee, Hyun-Ah Song, and Shun-ichi Amari. A new discriminant nmf algorithm and its application to the extraction of subtle emotional differences in speech. Cognitive Neurodynamics, 6, 12 2012. doi: 11111007/1111--12-999999
Raymond Pomponio, Guray Erus, Mohamad Habes, Jimit Doshi, Dhivya Srinivasan, Elizabeth Mamourian, Vishnu Bashyam. Ilya M. Nasrallah, Theodore D. Satterthwaite, Yong Fan, Lenore J. Launer, Colin L. Masters, Paul Maruff, Chuanjun Zhuo, Henry Völzke, Sterling C. Johnson, Jurgen Fripp, Nikolaos Koutsouleris, Daniel H. Wolf, Raquel Gur, Ruben Gur, John Morris,
10

---

Published as a conference paper at ICLR 2022
Marilyn S. Albert, Hans J. Grabe, Susan M. Resnick, R. Nick Bryan, David A. Wolk, Rus- sell T. Shinohara, Haochang Shou, and Christos Davatzikos. Harmonization of large mri datasets for the analysis of brain imaging patterns throughout the lifespan. Neurolmage, 208: 188:116450, 2020. ISSN 1053-81 19. doi: http://doo.ogy/0000feejeeummmee
http:///0/.////.uooaeeee//011110 16450. URL http:
7 7/www. sc encedi rect. com http:////...cceeeeaaaaaaaraa///a
Cox R.W. Afni: software for analysis and visualization of functional magnetic resonance neuroim-
ages. Comput Biomed Res, 29: 19:113277, 06 1996. doi: 10.1006/b 1996.0014.
J Sled, Alex Zijdenbos, and Alan Evans. Sled ig. zijdenbos ap, evans aca non-parametric method for automatic correction of intensity nonuniformity in mri data. ieee trans med imaging 17:87-97. IEEE transactions medical imaging, 17:87-97, 03 1998. doi: 10.1109/42.666699.
Aristeidis Sotiras, Susan Resnick, and Christos Davatzikos. Finding imaging patterns of structural covariance via non-negative matrix factorization. Neurolmage, 108, 12 2014. doi: 10. neuroimage.2014. 2.4.11m
Erdem Varol, Aristeidis Sotiras, and Christos Davatzikos. Hydra: Revealing heterogeneity of imag- ing and genetic patterns through multiple max-margin discriminative analysis framework. Neu- rolmage, 145, 02 2016. doi: 10.
Junhao Wen, Erdem Varol, Aristeidis Sotiras, Zhijian Yang, Ganesh B. Chand, Guray Erus, Haochang Shou, Ahmed Abdulkadir, Gyujoon Hwang, Dominic B. Dwyer, Alessandro Pigoni, Paola Dazzan, Rene S. Kahn, Hugo G. Schnack, Marcus V. Zanetti, Eva Meisenzahl, Ger- aldo F. Busatto, Benedicto Crespo-Facorro, Romero-Garcia Rafael, Christos Pantelis, Stephen J. Wood, Chuanjun Zhuo, Russell T. Shinohara, Yong Fan, Ruben C. Gur, Raquel E. Gur, Theodore D. Satterthwaite, Nikolaos Koutsouleris, Daniel H. Wolf, and Christos Davatzikos. Multi-scale semi- sumi-super clustering of brain images: deriving disease subtypes. Medical Image Analysis. pp. 102304, 2021. ISSN 1361-8415. doi: https://doo.rrg/00.100/6//
2021.102300 102304. URL https: /ww. scw.ncedi rectt.edddeer...e.e com//science/artiicee/pop 013618415210332
Zhijian Yang. lya Nasrallah, Haochang Shou, Junhao Wen, Jimit Doshi, Mohamad Habes, Gu- ray Erus, Ahmed Abdulkadir, Susan Resnick, Marilyn Albert, Paul Maruff, Jurgen Fripp, John Morris, David Wolk, Christos Davatzikos, Yong Fan, Vishnu Bashyam, Elizabeth Mamouiran, Randa Melhem, and Balebail Raj. A deep learning framework identifies dimensional represen- tations of alzheimer's s disease from brain structure. Nature Communications, 12, 12 2021. doi:
Alexandra Young, Razvan Marinescu, Neil Oxtoby, Martina Bocchetta, Keir Yong, Nicholas Firth, David Cash, David Thomas, Katrina Moore, Manuel Jorge Cardoso, John Swieten, Barbara Bor- roni, Daniela Galimberti, Mario Masellis, Maria Tartaglia, James Rowe, Caroline Graff, Fabrizio Tagliavini, Giovanni Frisoni, and Daniel Alexander. Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with subtype and stage inference. Nature Communica- tions, 9, 10 2018. doi: 10.1038/s41467-01 000--77--------000
Xiuming Zhang, Elizabeth Mormino, Nanbo Sun, Reisa Sperling, Mert Sabuncu, and B.T. Thomas Yeo. Bayesian model reveals latent atrophy factors with dissociable cognitive trajectories in alzheimer's 's disease. Proceedings of the National Academy of Sciences, 113:56535-E6544 10 2016. doi: 10.1073/pass16110731133
11

---

Published as a conference paper at ICLR 2022
A APPENDIX: METHODS
A.1 LEMMA1
Let f X x Y and g Y X be the transformation and inverse mapping function spectively. If g is K-Lipschitz continuous and d(:.)) is any distance metric satisfying triangu- lar inequality, then for Vz 22 Z, and Vo X, d(f(x, 21), f(x, is lower bounded by d(z). * (d(g(f(z, z1)), x1) + d(g(f(z x2))
Proof: By triangular inequality, can derive that:
1
x2)) 2 =
(13)
K
1
2 (d(z), x2)
- -(g((ddd())))))))//)) dd((fa.)))))
(14)
K
d(z1,2)) 22)
1
(15)
K
A.2 REMARKI
By minimizing the reconstruction loss Liecons defined in section 2.2.2 and 2.2.3, maximizing
a lower bound the mutual information between latent variable z and synthesized PT data y' f(x, 2). Considering Qz)yy') to be approximation of distribution P(z) 2)) and assuming that it follows a Gaussian distribution N(g(y'),)) mutual information denoted by I and entropy denoted by II, we can derive that:
1 (z, fxx z))
(z) (z z))
(16)
By' ~p (v') [ wplatt)) (oogiiylloog [log P(zlly)]] (+))
(17)
o (x)) D (P( ([)()(((()))))))))))))))))))))) =pp ((xls((((())))))] H (z)
(18)
By' (x)) p [log (x'|l'))]
(19)
Eg-ppa 8) (f 2x) log (zlly')
(20)
Egeepac (z) x (x) [oo (z) [xx z))]
(21)
In Σ
22 f(x,, 21)= s(x;,2:) + constant)
(22)
(21) and (22) follows the Lemma 5.1 in Info-GAN (Chen et al. (016)) and law of the unconscious statistician (Lotus) Theorem respectively. Therefore, we derive that the mutual information between the latent variable z and synthesized PT data y' is bounded below by _ E, 1 ((2 (x,, 2, = (z, - fx 2)) constant). With further assumption that Σ an identity matrix, maximization of this lower bound is equivalent to minimizing the reconstruction loss Lyccons-
A.3 NETWORK ARCIITECTURE
Table 2 and Table display network architectures of transformation function f. discriminator D and reconstruction function g (consisting of and g2).
Table 2: Architecture of transformation function f
Layer
Input Size
Bias Term
leaky relu a
Output Size
Mhnearl+Leaky-Ree
139*1
No
0.2
69*1
Encoder from x
Linear2+Leaky-Rell
69*
No
0.2
34
Decoder from z
Lingarl+Sggmoid
M*1
Yes
NA
34
Linear1+Leaky-RRel
34#1
No
0.2
69*1
Decoder to y'
Linear2+Leaky-eel
69*1
No
0.2
139*1
12

---

Published as a conference paper at ICLR 2022
Table 3: Architecture of discriminator D, Decomposer 91 and Reconstruction 92
Layer
Input Size
Bias Term
leaky relu
Output Size
Linearl+I.a +L.akyy-eeeekeee
139*1
Yes
0.2
69*1
Discriminator
Linear2+ Linear2+Leaky-Reil
69*1
Yes
0.2
34*1 1
Linear3+Softmax
34*1
Yes
NA
2*1
91
Leaky- Likky-Rel++Linerr
139º1
Yes
0.2
(139*
Linear +LeallL-eeaaRRe
139
Yes
0.2
69*1
g2
Linear2+Leaky-Rell
69*1
Yes
0.2
34*1
Linear34Sottmax
34*1
Yes
NA
M*
A.4 ALGORITHM
Detailed training procedure of Surreal-GAN is disclosed by Algorithm 1
Algorithm 1: Surreal-GAt GAN training procedure. le represents cross entropy loss and e; represents a one hot vector with 1 at ith. component. (f(x,=)) (2(0) and Az,,,) are defined in section 2.2.3 and 2.2.4.
while not meeting stopping criteria or reaching max-epoch do for all batches {2,
do
sample latent vectors { from multivariate uniform distribution with U00,1 1|M
Update weights of discriminator D: Use ADAM to update dD with gradient: th Vep Lisal( (D( 24), eu)))]
sample m latent vectors {= conditioned on previously sampled {2) with each z ~ UZz 1
sample another m latent vectors {2(") J with each 2.th ~ (0,00.05)
Update weights of transformation function f: Use ADAM update 0, with gradient: Ver m (D((((z))
f 2;) x;|l1)+
K| k|19ı (f,,,, - )mmm) + < ga(m (f(a...))) z1122 12 + A|| f(m...)) A ((m...))
T[[ 14/l max(] (z;; - æ, [f 2/) æ, 0)]12 M /llf 2(th mxlli]
Update weights of function 91: Use ADAM to update 9g, with gradient: Vog 1 (f x))
Update weights of function 92: Use ADAM to update 9,, with gradient:
V m Em Em (f
2:112
end end
B APPENDIX: EXPERIMENTS
B.1 REGIONS WITH SIMULATED ATROPHY IN SEMI-SYNTHETIC DATASETS
Regions of interest (ROI) included in three patterns (P1-P3) in different semi-synthetic datasets are revealed by Table Small overlapping case indicates the included ROIs in three different dataset described in section 4.2: (1) Basic dataset: (4)Within-patte dataset. Each ROI a combination left and right parts of the region, so each check sign represents inclusion of 2 out of 139 regions.
noise dataset and (5) (5)Mild atrophy
13

---

Published as a conference paper at ICLR 2022
Table ROIs included in synthetic patterns (OP:opercular part: TP: triangular part)
ROI
Small overlapping
Large overlapping
Scarce Region
P1
P2
P2
P3
P1
P3
Amygdala
V
Hippocampus
Angular gyrus
/
V
entorhinal area
v
frontal opereulum
/
d
inferior temporal gyrus
lateral orbital gyrus
medial frontal cortex
V
middle frontal gyrus
V
V
middle occipital gyrus
V
OP of the inferior frontal gyrus
parahippocampal gyrus
posterior insula
d
parictal operculum
supramarginal gyrus
superior parietal lobule
V
temporal pole
TP of the inferior frontal gyrus
B.2 SYPEE-PARAMETER SELECTION
We trained the model on the basic case and the mild changes case (the easy and hard case) with varying hyper-parametto to test the model robustness to hyper-parameter selections. Specifically, we changed each parameter from 50% to 150% of the preset value (introduced in section 3.3) while keeping other parameters fixed. With each set of parameters, we trained the model 10 times and reported average pattem--cinnex and average pattern-agr-inder as described in section 4.2. From Fig. 4. we can see that model performances are mostly robust to different choices. For the mild changes case, models, with different values, do show higher variations in performances. However, pattern-agr-inde still have good indication the best choice, and this metric can always be used for parameter selections different datasets.
B.3 CONTRIBUTION FROM REGULARIZATION
Similar to section B.2, we set different parameters to 0 to understand contributions from correspond- ing regularization terms. Specifically, we set
(no decomposition loss), = (no orthogonal- ity loss), 0 (no monotonicity loss), 0 (no en loss), /1/Ç = O (no constraint on monotonicity and positive correlation), x/ x/x/p/S = (no extra regularization compared to Smile-GAN frame- work) respectively with other parameters fixed. With different sets of parameter values, we trained the model 10 times and report average pattern-c-indice importance of different regularization terms and visualize how additional regularization terms lead to improved representation performances compared to pseudo comparable "Smile-GAN" (Using same regularization Smile-GAN but with continuous latent variable).
From Fig/4b, we can clearly understand the
14

---

Published as a conference paper at ICLR 2022
Representation Accuracies
Model Agreements
(a) 0,075
(b) 0.950
0.850
Pattom- aor--dord
0,900
0.800
0.775
0.850
0.790
0.825
0.725
0.000
0.700
0.5
0.75
1.25
1.5
0.5
0.75
1225
1,5
ralio to preset hyperparameters
ranio preset hyperparameters
Basic Case
Mild Changes
(c)
Ablation study on regularizations
0.85
all regularizations
no ortho
0.80
Pattern c-index
mono
no cn/mono
Smile GAN
0.60
Basic Case
Mild Changes Case
Figure 4: Regularization and hyper-parameter (a) Representation accuracy on different tasks with variations in syper-parameters (measured by Pattem-c-index); (b) Agreements among repetitively trained models on different tasks with variations in hyper-parameters (measured by Pattern-agr- index) (results presented mean value standard error for (a.b)); (c) Representation accuracy with regularization terms removed. (*Smile-GAN here refers to a "pseudo Smile-GAN" with the same regularization terms included in Smile-GAN model. Latent variable z is sampled from same continuous distribution as in Surreal-GAN.)
Table 5: "Practical upper bounds" of compared models
Surreal-GAN
NMF
LDA
FA
OpNMF
DNMF
0.838
0,834
0.769
0.750
0.751
0.631
Basic Case
+ 0.011
+ 0.000
+ 0.005
+ 0.000
+ 0.000
+ 0.017
Large
0.837
0.844
0.777
0.721
0.715
0.641
Overlapping
+ 0.008
0.000
+ 0.001
+ 0.005
+ 0.000
- 0.017
0.783
0.760
0.640
0.618
0.666
0.548
Scarce Regions
+ 0.007
+ 0.000
+ 0.015
0.001
+ 0.000
+ 0.006
Within-pattern
0.818
0.831
0.775
0.764
0.730
0.644
Noise
+ 0.013
+ 0.000
0.003
0.000
+ 0.015
- 0.031
0.762
0.756
0.657
0.613
0.641
0.571
Mild Changes
0.023
+ 0.000
0.028
+ 0.000
+ 0.000
+ 0.015
B.4 DATA PREPROCESSING FOR COMPARED MODELS
Z-scores of ROIs with respect to the CN group directly used input features of EA model. However, for NMF, OpNMF and LDA model, we need to borrow prior knowledge that atrophy (volume decreasing) common in dementia-related brain diseases, and thus, reversed the sign
15

---

Published as a conference paper at ICLR 2022
Table 6: Comparisons with supervised regression models
Task
Surreal-GAN
LR
SVR
Train
Test
Train
Test
Train
Test
0.827
0.830
0.896
0.871
0.926
0.863
Basic Case
+0.005
10.007
#0.001
=0.003
10.001
10.006
0.828
0.827
0.901
0.876
0.927
0.869
Large Overlapping
+0.008
#0.007
#0.001
=0.033
#0.001
#0.003
0.764
0.761
0.853
0.816
0.932
0.804
Sparse Regions
+0.010
+0.014
+0.001
0.005
+0.002
+0,009
0.813
0.812
0.890
0.862
0.926
0.857
Within Pattern Noise
+0.007
#0,009
#0.001
-.003
10.001
#0,009
0.741
0.743
0.853
0.817
0.929
0.811
Mild Changes
+0.010
#0.017
#0.001
=0003
#0.009
#0.005
of z-scores, kept positive z-scores, and set negative values to 0. For LDA model, further dis- cretized the value by first multiplying z-scores by 10 and then mapping them to the closest integer described in Zhang et al. (2016). For the DNMF model, both CN and PT data and their cor- responding diagnosis labels are provided as input, with ROI data first standardized to lie in range 0-1.
B.5 "PRACTICAL UPPER BOUNDS" OF UNSUPERVISED METHODS
As discussed in section 5.1, unsupervised methods were not designed to best utilize the CN data reference distribution. Thus, derived "practical upper bounds" for compared models different tasks, with the assumption that these methods have some components capturing the ground truth while having other components representing non-disease related variations. On each semi- synthetic test, we ran each method with number of components, M, ranging from 3-10. For M > 3, intensively searched for the combination of 3 components that best match the ground truth and calculated the pattern c-index. The best result among M 3 - 10 is considered as an "practical upper bound" for the model. As shown in Table 5 upper bounds of NMF methods are comparable to results from Murreal-GAN while other methods still show much worse performances. Note that these are "practical upper bounds" derived with known ground truth. real cases, we are not interested in set patterns that includes the true disease effects but want to find disease-related patterns only. However, it will be very hard to know what the best number of components and truly disease-related with these compared models. In contrast, Surreal-GAN can directly and accurately output disease-specific representations without indices capturing non- disease related variations, which is one of the key advantages of the proposed method.
which components
B.6 COMPARISONS WITH SUPERVISED METHODS
We further compared the model with linear regression (LR) support vector regression (SVR) with RBF kernel to understand the gap between model and the most optimal model performances in semi-synthetic tests. A five fold cross validation were run with three different models. Average pattern-c-indices both train and test sets reported. We can first observe that Surreal-GAN, as an semi-supervised methods, shows almost equal performances on train and test set, proving good ability in generalization. On five different tasks (table6), there 0.04-0.07 pattern-c-indices gap between Surreal-GAN and supervised model performances. This gap is considered reasonable but there is still space for model improvements.
B.7 GENERALIZATION AND REPRODUCIBILITY ON REAL DATASET
We randomly half split the real data (described in section 4.3) train and test set, with both CN and PT data equally distributed in two sets. Following section 4.3, we first trained the model on the training set, and then applied the model derive 2- dimensional R-indices for all CN and PT data in the test Voxel-wise group comparisons were
experiment procedure described in
16

---

Published as a conference paper at ICLR 2022
0.4 < 1 < <0.7,12 <..4
r 0.7,12 < 0.4
T-stats
8
0.4 12 <.77 I1 0.4
< 0.7
TXAO
Figure 5: Voxel-wise statistical comparison results on test set. (FDR method with p-value threshold of 0.05 was used for multiple comparison).
conducted between subgroups and CN in the test set visualization. From Fig. 5
we can observe
that R-indices are correlated with almost same patterns shown in Fig. though much smaller sample size led to relatively lower statistical significance.
B.8 Low DIMENSIONAL REPRESENTATION FOR AD DIAGNOSIS
We used R-indices of test data introduced in section B.7, and compared their ability to classify CN and PT with that of 139 ROI volumes. Support vector machine (SVM) was selected as the classification model. 20% leave-out experiments were performed 100 times with derived R-indices and 139 ROI as input features respectively. In each run, we randomly sampled 80% of data as training set and applied trained models on the rest 20% to derive the AUC metric. Grid search were used for parameter selection. With 2D R-indices as features, SVM model was able to reach an average AUC, 0.895 + 0.011, which is only mildly lower than AUC derived with 139 ROIs as features, 0.929 + 0.015. Therefore, we can conclude that, 2D R-indices, kept almost all discriminant information among PT data, while parsing the heterogeneity in clear, concise and interpretable way.
ROI1
RO12
ROI3
Surreal-GAN Model
RO1138
RO1139
Figure 6: Procedure for deriving Surreal-GAN input features (ROI volumes): MRI images segmented using the MUSE method (Doshi et al. (2015)). Volumes of segmented regions (ROI volumes) were then extracted input features for Surreal-GAM model.
17

---

Published as a conference paper at ICLR 2022
B.9 ASSOCIATIONS WITH CLINICAL VARIABLES
Each dimension of R-indices, rú, is compared with different clinical variables for AD disease. For each clinical variable, we only selected out samples who have it available for statistical analyses. ADNI composite scores, including ADNI-EF, ADNI-MEM and ADNI-LAN, measure subjects' functions in executive functions, memory and language respectively. They are only available among subjects from Alzheimer's s Disease Neuroimaging Initiative (ADNI) study. Digit Symbol Substitu- tion Test (DSST) measures range of cognition operations including motor speed, attention, and vi- suoperceptual. For all mentioned clinical scores, higher value indicates a better performance. CSF- Abeta, Cau and CSF-PTau are three very important hallmarks of AD disease corresponding to amyloid and tau hypotheses. Since measures of these three values have great variance among differ- ent studies, only ADNI samples with CSF measures were included for analysis. For dichotomous variables (Apoe-E4 E4 Alleles Carrier & Hypertension), point biserial correlation with corresponding P value was derived between the variable and each pattern dimension, r,. Pearson correlations were derived for other continuous variables.
Table 7: Correlation between pattern severity and clinical variables for AD disease
12
White matter lesion volumes
0.415 (p<0.001))
0.184 (p=0..01)
DSST
-0.055 (p=0.603)
-0.212 (p=0.02)
Apoe-E4 Alleles Carrier
0,003 (p-0.965)
0.212 (p<00.00))
Hypertension
0.181 (p<0.001)
-0.006 (p=0.890)
CSF-Abeta
-0.238 (p<0..001)
-0.302 (p<0.001)
CSF-Tau
0.111 (p=0.002)
0.270 (p=00001))
CSF-PTau
0.059 (p=0.103)
0.152 (p<00.01)
ADNI-EF
-0.409 (p<0.0011)
-0.240 (p<0.001)
ADNI-MEM
-0.453 (p<0.001)
-0.466 (p<00.001)
ADNI-LAN
-0.422 (p<0.001)
-0.281 (p<0.001)
18


Reviews:
[{"review_id": "t2JGGAGhBM", "reviewer": "Reviewer_jVuo", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n(1) The proposed method models disease as a continuous process and learning infinite transformation directions from CN to PT.\n\n(2) Several key components(function continuity, transformation sparsity, and inverse consistency) can guide the model to capture meaningful imaging patterns).\n\n(3) The experimental results have shown the effectiveness of the proposed model.\n\n\n\nWeaknesses:\n\n(1) The overall objective function includes several hyper-parameters, and the authors set fixed values for some parameters. Thus, how to set them? The effects of different settings should be discussed. Besides, when applying to other datasets, whether we can use the same parameters?\n\n(2) As a significant extension of Smile-GAN, the authors should compare the proposed model with Smile-GAN.\n\n(3) In Table 1, the compared methods are out of date, thus more state-of-the-art methods should be included. \n\n(4) It is interesting to validate AD diagnosis performance when using the proposed representation learning method.\n\n(5) The authors state that there are four novelties in this study, however, some of them only introduce the existing technologies into the proposed framework.\n", "summary_of_the_paper": "This paper proposes a Surreal-GAN method for learning representations of underlying disease-related imaging patterns. This model has overcome limitations in previously published semi-supervised clustering methods and shown great performance on semi-synthetic data sets.", "summary_of_the_review": "This paper proposes a Surreal-GAN method for learning representations of underlying disease-related imaging patterns. However, the comparison experiments are not sufficient, and the novelty section needs to be improved. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "hI3xjMMj8bK", "reviewer": "Reviewer_8j3t", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well clearly organized, and the general idea of the method is very interesting. However, there are a few things I would like to address.\n1. It is not easy to understand the actual model input and how it is generated from the structural MR images. An example image of the preformed atlas segmentation would be beneficial for understanding.\n2. Figure 1 is difficult to understand. Is Figure 1b the result of the experiments or an illustration of what to expect from the presented method?\n3. In the introduction and later in the paper, the authors refer to the low dimensional representation as R indices r_i and refer to Figure 1b. However, Figure 1b only shows z_1 and z_2. It would be great if the authors could comment on the relation between R indices and the latent representation z.\n4. For the model architecture, the latent variable and the input are mapped into a 34-dimensional space. According to the paper, the dot product is applied to both vectors. However, the input to the decoder is 34x1. Is an element-wise multiplication used instead of the dot product?\n5. It is not clear how the evaluation of the model f for each z_i is done separately for the pattern decomposition. It would be great if the authors could comment on this.\n6. It is not clear what the number of features S relates to in the paper. \n7. For the evaluation, all necessary information is provided. However, the selection method of the hyperparameter for the loss function is not given. Furthermore, it is not evaluated how the selection of the hyperparameter influences the outcome of the model. This is only done for lambda and M.\n8. The y axis label of Figure 2b is cut off.\n9. The authors mentioned that this method is an extension of the Smile-GAN. Is there a specific reason why the Smile-GAN is not used for comparison?\n", "summary_of_the_paper": "In this paper, the authors present a method for learning a representation of disease-related image patterns from regional volume information generated from structural MRI images. The method is based on a GAN approach, and different regularization methods on the latent representation are presented to improve the learning of disease-related representations. There are four contributions presented in the paper:\n1. Modelling a disease as a continuous process\n2. Controlling the monotonicity of the disease pattern\n3. Separation of captured disease pattern\n4. Inverse Mapping to ensure that disease patterns synthesized are captured\n\nThe evaluation of the method is performed on semi-synthetic data sets and an actual Alzheimer’s disease data set. The results show that the method can identify two clinically informative patterns.\n", "summary_of_the_review": "The presented methods present some interesting ideas for modelling disease-related representation. However, the description of the presented method needs some improvements. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "W9NM9VMb9e7", "reviewer": "Reviewer_QqEE", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong Points\n\n1.\tPerhaps the biggest shining point is the astonishing number of carefully crafted regularizations – a total of six regularization terms in a single loss function. They are delineated in significant detail from section 2.2.1 to 2.2.5. I appreciate the amount of thinking and hard work that probably goes behind the scene, and I believe the author did a good job persuading me why each of these regularizations are helpful and what they respectively achieve. I particularly liked how the authors pointed out some potential trivial solutions the model can cheat with if proper regularization were not in place.\n2.\tAs for the results, I would consider the voxel-wise statistical comparisons on the real data a rather definitive result demonstrating that Surreal GAN was able to isolate two major sources/locations of atrophy in Alzheimer’s disease. The two distinctive patterns, one showing up across multiple cortical regions and the other localized to the medial temporal lobe would probably be appreciated by many neurobiologists. Meanwhile, the statistical power is quite strong, given that it has been corrected for multiple comparison using the FDR method.\n\n\nWeak Points\n\n1.\tAs many as six regularization terms, while I stated as a “strong point”, can also been viewed as a sign of over-engineering. It will be better rationalized if the authors can conduct an ablation study on these five regularizations. With that being said, I acknowledge it might be an unrealistic commitment depending on how long each experiment would take, so this is a soft suggestion rather than an explicit request.\n2.\tIt appears that the authors are utilizing the gradient clipping method to ensure Lipschitz continuity (section 3.2). However, that method is relatively deprecated, as emphasized in the original Wasserstein GAN paper, “weight clipping is a clearly terrible way to enforce a Lipschitz constraint”. While the authors are aware of the case and are conscious about their different use case, nevertheless, alternative ways such as gradient penalty (Improved Training of Wasserstein GANs by Gulrajani et al 2017, aka., WGAN-GP) may be worth considering.\n3.\tThe baseline that the authors compared against are not very competitive. It might be beneficial if a supervised method (is it possible?) can be included, perhaps to set a “practical upper-bound” so that we can see how far the gap there is between that and the proposed method.\n\n\nQuestions to authors\n\n1.\tSection 3.3. When the authors mentioned that “the model was trained for at least 100000 epochs and saved until the reconstruction loss is smaller than 0.003 and the monotonicity loss is smaller than 6E-4”, I was surprised by the huge number of epochs. My primary concerns are: 1) does that mean the initial learning rate is a bit too low and/or a bad learning rate scheduling is used? 2) how do you avoid overfitting, besides using the suite of regularizations in the loss function?\n2.\tSection 2.3. The full loss formula is indexed twice (14 and 15). Is it a mistake?", "summary_of_the_paper": "In this paper, the authors pointed out the lack of a good method to predict disease progression in a continuum based on representations explicitly modeling specific disease effects. To tackle this problem, they proposed a novel method called Surreal-GAN, a tailored version of generative adversarial networks, to learn separate representations of neurological and neuropsychiatric diseases in an unsupervised manner. The method frames the diseased brains as multiple disease-related features at various severity imposed on normal brains, and by learning these features and severities, it can generate a representation with improved reliability and explanability.", "summary_of_the_review": "Overall, I would recommend this paper to be accepted. Personally I find the problem the authors are tackling legitimate and usually neglected, and I am persuaded that the proposed solution has some generalizability.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "EhBgoMuAvl5", "reviewer": "Reviewer_Rbz3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The model proposed provides an interesting idea of using GAN to create an inverse function to detect the pattern and severity of disease from real data. The regularizations suggest many ways it could be used for other types of data that has a continuous output space. The reasoning behind the regularizations is overall reasonable. There could be more explanation, however, on why double decomposition is a good idea for the model.\nRegarding the experiment with simulated data, it is hard to understand why the number of patterns were chosen to be 3, unlike 2 which was suggested to be optimal from the real data. Also, since the regularizations are the major extensions of the model from Smile-GAN, it would be more comprehensible for the reader if the paper had cases where some parameters were set to 0 (where some regularizations were not used). If the regularizations indeed created the effect the authors propose them to, removing them one by one would show the significance of each loss.\nThe paper also compares between other methods, without a reasonable explanation on why they were selected for comparison. Other references on cases where these approaches were used for the same goal would be helpful.\nSome things that could be considered would be using W-GAN’s loss. It has been proposed to solve some issues of GAN by making the model architecture more stable and easier to test (Arjovsky, Chintala, Bottou, 2017), which is useful in generalization of the model or training using different datasets. Additionally, there are words that gradient clipping is a better way than weight clipping to ensure Lipschitz continuity as it still can suffer problems from having a wrong weight clipping window.\nOne trivial thing is Figure 2.b axis description is cut.", "summary_of_the_paper": "Surreal-GAN aims to create fake pathological data with a latent variable and a healthy input and includes an inverse function that predicts the latent variable from a fake/real pathological data. However, unlike its predecessor model Smile-GAN, this adds 5 regularizations (L1 loss between generated output and input, L2 loss between the latent variable and the reverse generated latent variable, decomposing the latent variable, making the latent variables orthogonal to each other, and adding positive correlation to disease pattern and the latent variable) to make the latent variable more interpretable and continuous. This would increase the interpretability since instead of clustering the real data it could be used as understanding how severe the patient’s disease progress is. The results show that the model has a higher c-index than NMF, LDA and FA model predictions. It also shows that the best number of patterns for the model to predict is two, and it indeed has positive correlation to the severity of the disease.", "summary_of_the_review": "Overall, the paper is convincing and has reasonable suggestions in creating a correlated continuous latent space for pathological data. However, there could be more comparisons important in proving their case. Also, there could be more considerations on new ways the model could be developed, given how GAN has been evolving over the years.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors present a GAN for learning a continuous representation of disease-related image patterns from regional volume information generated from structural MRI images.
The reviewers find the problem relevant and appreciate the proposed solution. They find the paper well-written and find the empirical results on Alzheimer brain MRIs relevant for the neuroscience community.
 
The overall objective function includes several hyper-parameters. As pointed out as the main weak point by multiple reviewers this may hint at overengineering/overfitting to a data set. However, the reviewers also mention that the regularizers are all sufficiently well-motivated in the paper and the author response.

Reviewers highlight comparisons on the real data as a strong result demonstrating that Surreal GAN was able to isolate two major sources/locations of atrophy in Alzheimer’s disease. Overall, the reviews are positive in majority.
Decision: Accept (Poster)