Topic Models with Survival Supervision: Archetypal Analysis and Neural Approaches
Authors: George H. Chen
Linhong Li
Ren Zuo
Amanda Coston
Jeremy C. Weiss
Venue:  (2020)

Abstract: We introduce two approaches to topic modeling supervised by survival analysis. Both approaches predict time-to-event outcomes while simultaneously learning topics over features that help prediction. The high-level idea is to represent each data point as a distribution over topics using some underlying topic model. Then each data point's distribution over topics is fed as input to a survival model. The topic and survival models are jointly learned. The two approaches we propose differ in the generality of topic models they can learn. The first approach finds topics via archetypal analysis, a nonnegative matrix factorization method that optimizes over a wide class of topic models encompassing latent Dirichlet allocation (LDA), correlated topic models, and topic models based on the ``anchor word'' assumption; the resulting survival-supervised variant solves an alternating minimization problem. Our second approach builds on recent work that approximates LDA in a neural net framework. We add a survival loss layer to this neural net to form an approximation to survival-supervised LDA. Both of our approaches can be combined with a variety of survival models. We demonstrate our approach on two survival datasets, showing that survival-supervised topic models can achieve competitive time-to-event prediction accuracy while outputting clinically interpretable topics.

know they live at Under review as a conference paper at ICLR 2020
TOPIC MODELS WITH SURVIVAL SUPERVISION: ARCHETYPAL ANALYSIS AND NEURAL APPROACHESS
Anonymous authors Paper under double-blind review
ABSTRACT
We introduce two approaches to topic modeling supervised by survival analysis. Both approaches predict time-to-event outcomes while simultaneously learning topics over features that help prediction. The high-level idea is to represent each data point as a distribution over topies using some underlying topic model. Then each data point's distribution over topics is fed as input to a survival model. The topic and survival models are jointly learned. The two approaches we propose dif- fer in the generality of topie models they can learn. The first approach finds topics via archetypal analysis, a nonnegative matrix factorization method that optimizes over wide class of topic models encompassing latent Dirichlet allocation (LDA), correlated topic models, and topic models based on the "anchor word" assump- tion; the resulting survival-supervised variant solves alternating minimization problem. Our second approach builds on recent work that approximates LDA in a neural net framework. We add a survival loss layer this neural net to form an ap- proximation to survival-supervised LDA, Both of our approaches can be combined with variety of survival models. We demonstrate our approach on two survival datasets, showing that aurvial--supervise topic models can achieve competitive time-to-event prediction accuracy while outputting clinically interpretable topics.
INTRODUCTION
Predicting time-to-event outcomes arises in a variety of applications. For example, in healthcare, we may be interested in predicting how much time patient has to live. In criminology, we may be interested in predicting when a convicted criminal might reoffend. e-commerce and on streaming platforms, companies with subscription services like Amazon and Netflix may be interested in pre- dicting when users might cancel their subscriptions. In many such applications, we can now collect an enormous number of measurements per person/subject. However, how all of these measurements relate is typically unknown. In this paper, we aim to address the twin objectives of learning how measurements relate in the form of topic model, and learning how topies can assist in predicting time-to-event outcomes via a survival analysis model.
For ease of exposition, phrase the problem consider in the classical survival analysis context of predicting time until death. We assume that we have access to a training dataset of n subjects. For each subject, we know how many times each of d "words" appears, where the dictionary of words is pre-specified. As an example, in a clinical context, one word might correspond to "low blood pressure reading"; for given subject, we can count how many such readings the subject has had recorded in the past. We denote X to be the number of times word u E {1...., d) appears for subject i E {1....,n} n). Viewing X an n-by-d matrix, the i-th row of X can be thought of as the feature vector for the i-th subject. As for the training label for the i-th subject, we have two recordings: event indicator 5, E 10 specifies whether the i-th subject died, and observed time Y, R4 is the i-th subject's "survival time" (time until death) if 6; 1 or the "censoring time" 8, = 0. The censoring time gives a lower bound on the survival time for the i-th subject. For example, when stop collecting data, some subjects will still be alive, least as long as when we stopped collecting training data.
Our goal is to discover topics for the d words that help predict survival times of unseen test subjects. Note that an unsupervised topic model like latent Dirichlet allocation (LDA) (Blei et al., 2013) would not any of the training labels (the event indicators 5, 's and observed times Y,s), learning

---

Under review as a conference paper at ICLR 2020
topics using only the word counts matrix X. Meanwhile, in survival analysis, standard approach would involve learning a survival model using all the patients' feature vectors and labels but the model would not learn thematic structure in the different features, e.g., topics. Jointly learning both a topic model and a survival model was first done by Dawson & Kendziorski (2012), who combined LDA with Cox proportional hazards model (Cox) 1972). Using LDA with topics, Dawson and Kendziorski represent the i-th subject as a probability vector W; E [0, 1J" specifying the subject's ss membership in each of the topics; then Wj's are treated as the input covariates the Cox model. Dawson and Kendziorski called this joint model SURVLDA and derived variational EM algorithm to estimate its parameters.
In this paper, we build on SURVLDA by proposing two new survival -urivvl-supervviedd topic modeling approaches, both of which allow for either the topic or the survival model to be replaced. Our contributions are as follows:
(Section We show how to take discriminative approach to jointly learning topic and sur- vival models, where topics are estimated via archetypal analysis (Cutler Breiman] 1994; Javadi & Montanarii 2019). Archetypal analysis represents each subject as convex combination of "archetypes" which are optimized to be diverse yet still be close to the convex hull of the subjects' feature vectors. Applied to topic modeling. the archetypes specifying a particular topic's word distribution. Archetypal analysis does not assume paramet- ric model and can learn wide class of topic models. We describe how combine archetypal analysis with any survival analysis model for which we can take a specific partial derivative.
the topics, with each archetype
(Section 4 We approximate Dawson and Kendziorski's SURVLDA model in neural net frame- work, which allows for different choices of topic and survival models to be combined. This approach requires that the topic and survival models already have neural net approximations or formulations. For example, LDA and some variants of it can already be approximated using varia- tional autoencoders (Srivastava & Sutton| 2017 Card al. 2018). In particular, Card et al. (2018) show how to approximate supervised LDA (McAuliffe & Blei 2008) in a neural net framework that they call SCHOLAR; they specifically consider classification as the supervised task although they mention that their framework could be used to predict other real-valued outputs. We specifi- cally combine their approach with that of Katzman et al.) (2018) to handle survival supervision.
(Section 5 We apply our two proposed approaches to two survival analysis datasets (predicting how long pancreatitis patients stay in an intensive care unit, and time until death for breast cancer subjects), comparing against a number of classical and recently developed deep survival analysis baselines.
topic models have time-to-event prediction accuracy that is com- petitive with top- perforrfonn existing baselines while producing clinically interpretable topics.
BACKGROUND
We begin with some background on archetypal analysis. topic modeling, and survival analysis. Along the way, we introduce notation that recurs throughout the paper. As a reminder, we as- sume that we have access to training data (X1, Yú, 51), (X2, Y2, d.).... (Xa, Yn, 6n), where the i-th training subject has feature vector X, Rdd observed time Y; E R4. and event indicator 8, E 10. Throughout this paper, generally take X1,, (for E (1,2...., dd) to be the number of times word d appears, for some user-specified dictionary of d words. We let X denote the fraction of times a word appears for a specific subject, meaning that
X
X
Ed D0=1
X
Note that X is an n--y-d matrix, and we use X, to denote the i-th row X. We use this indexing notation for other matrices well.
2.1 ARCHETYPAL ANALYSIS AND TOPIC MODELING
Archetypal analysis (Cutler& & Breiman 1994 Javadi & Montanari 2019 posits that each training
vector X; be well-approximated by convex combination of different unknown "archetypes"
2

---

Under review as a conference paper at ICLR 2020
H1, H2. H, R
X, Σ Wiglt, 9=1
(2.1)
for some weights W.,1- w..,...., Wr E [0, 1] that sum to i.c., the vector W; == (Wa...... W, resides in the probability simplex A == {w E 1], -g wy - 13. By stacking the archetypes
Hi....,, H, as rows to form the matrix H,, equation 2.1 can be expressed as X ~ WH. Archetypal analysis aims to estimate W and H given X.
If the archetypes HI...., Hr are constrained to be in the probability simplex Ad, then we get topic model, and each archetype corresponds to word distribution. For example, if rows of W are generated i.i.d. from a Dirichlet distribution, and rows of H are generated i.i.d. from another Dirichlet distribution, then we get L.D (Blei et al. 2013). As slight modification of this setup, if the rows of W are instead generated from a logistic normal distribution that allows correlation between topics, we get the correlated topic model (Lafferty & Blei 2006). For an example that is not generative, if the archetypes there exists a word w that only appears in archetype g (i.e., Hg > O and Hn for all + g) then we have topic model satisfying the separability "anchor word" assumption (Donoho & Stodden] 2004] Arora et al. 2012a b. 2013). Archetypal analysis optimizes over matrices W and H that include all of the aforementioned topic models above as special cases. In fact, archetypal analysis does not require that archetypes be on probability simplex or that they be nonnegative; the input matrix X need not consist of word frrqueeciess and could be positive or negative vealevalued measurements. Crucially, the error in approximation (211 should be small; precise details including identifiability and degeneracy issues can be found in Section 3 of Davadi & Montanari (2019).
on a probability simplex, and for each archetype g E {1, (1.....r]) r),
To estimate weights W and archetypes H, Javadi and Montanari proposed the following approach. First, for a point u E Rd and a matrix V R.mxd we define the distance from u to the convex hull of the rows of V as
(u, V) == min "EA"
||u -u-VV' V VTwll2-
The vector w E A that achieves the minimum consists of the convex combination weights that best combine rows of V approximate the point u. Then Javadi and Montanari (approximately) minimize the nonconvex loss
Larch (arch(WWr (W, H;A)
Σ |X. HTT.WWW
D² D2
(2.2)
subject to the constraint that W; E A for 1,..., n; constant À 2 O is a user-specified regulariza- tion parameter. Minimizing term (error of approximating input data X,'s convex combination of archetypes) encourages the archetypes to be far apart and have a convex hull that contains the input data. However, this term does not prevent the archetypes from taking on extreme values; for example, if the archetypes already have convex hull that contains the X; 's (so that = 0), we can move the archetypes even farther apart and still have their convex hull contain the X ''s (so we still have 0). We prevent this behavior by minimizing term D, which encourages each archetype to be close to the convex hull of the input data.
To learn a topic model, we enforce that the archetypes correspond to distributions over words by requiring each row of to be in probability simplex A The resulting optimization problem
WW. M)
argmin
Trach (W, A):
(2.3)
" HERrKa
WIEA" forall HyEA* forall
local minimum can found by alternating between minimizing W with H fixed, and vice versa.
2.2 SURVIVAL ANALYSIS
Archetypal analysis and topic models are unsupervised methods. To predict time-to-event outcomes, we turn toward survival analysis models. Suppose we take the i-th subject's feature vector to be W: E R" instead of X,. As this notation suggests, when we combine topic and survival models,
3

---

Under review as a conference paper at ICLR 2020
W: corresponds to the i-th subject's archetype/topic combination weights; this strategy for combin- ing topic and survival models was also done by Dawson & Kendziorski (2012), which in turn is based on the supervised LDA formulation (McAuliffe & Blei, 2008). We treat the training data as (Wi, V1, (1),,,..... (Wn, Yn, 6n), disregarding the X, and X, values from earlier.
We aim to reason about the survival time of unseen test feature vector Wo E R". Specifically, let random variable To denote the survival time corresponding to feature vector Wo (treated as a random variable). Then our goal is to produce an estimate ô of the (conditional) survival function S(t)w) == PTo > i Wo - w) for & 2 and w E IR".
Importantly, for given feature vector E Rd note that ((-(w) is a function. If we have estimate S(-ww) for S(-Jw), we can compute single number for the predicted survival time for feature vector w. The basic idea is to find a time / such that ((()w) s 1/2; such a time corresponds to median survival time. Details for computing this median survival time estimate is in Appendix [A]
Different survival models place different assumptions on S, where we typically assume that the training and test data points are i.i.d. samples from the same underlying distribution. The technical challenge is that in general, we do not see the survival times for all of the training subjects: the observed times Y;'s are equal to survival times only for subjects who have 6, = all other Y; values are censoring times. Different censoring models are used. A standard approach is to assume that the i-th training subject has survival time T; and censoring time C; that are conditionally independent given feature vector W;, and if the survival time occurs before censoring (7; S C.), then Y; = T; and 6, = I; otherwise Y, = C; and 6; = O. This setup referred to as random censoring. Details can be found in a survival analysis textbook (c.g.. Kalbfleisch & Prentice 2002).
As a concrete example of how survival function S can be computed via minimizing a loss function, next present the classical Cox proportional hazards model (Cox 1972). As we discuss shortly, this is just one example of a survival model that can be combined with our proposed survival- supervised archetyptal analysis neural topic modeling approaches.
Example 1 (Cox proportional hazards) Recall that survival function S 1 minus the CDF of the distribution of survival time To given feature vector Wo = w. We denote the CDF of this distribution as (( and assume has a probability density function f(|lw) at F(tlw). Then the Cox model constrains S through the so-called hazard function h of S, given by
à
at [1 F(t]w)]
f([[w)
h(4/w)) at
log S((lw)
(2.4)
(t((w)
S(tw)
which is the instantaneous rate of death at time divided by the probability of surviving up to time t, all conditioned on the feature vector being w. Specifically, the Cox model assumes that hazard function h factors as
h(t[w) ho(t)e
where the two parameters are the baseline hazard function ho R4 and the vector of regres- sion coefficients 3 E IK." Under random censoring (and actually more general censoring models), we can estimate 39 without knowing bo via maximizing a profile likelihood, which is equivalent to minimizing the loss
Loor E
Σ WT-3WW
Σ exp(3T W.)]
(2.5)
j=1xI.
Given an estimate of B, we can deterministically compute a nonparametric estimate for baseline hazard function ho; this estimation procedure standard and can be found in Section 7.8 of Cox Oakes 1984). Once we have estimates by and B for ho and B, then for any test feature vector W E IR", we can estimate this test feature vector's 's corresponding hazard function via
Tr(tlw) (t) w.
Using the first equality of equation 2.40, note that S(t[w) = exp (-1o h(T[w)dr)) We can plug estimate h for h into this equation get an estimate of S:
S(((w) = exp (- h((llwddr).
(2.6)
In practice, the integral is computed via sunmation.
4

---

Under review as a conference paper at ICLR 2020
Other survival losses are possible aside from Loxx W. As second example, provide the survival loss function for the Weibull accelerated failure time (AFT) model in Appendix B The critical requirement of our proposed methods to follow is that the survival loss used is differentiable with respect to W. For example, the elastic-net-regularized Cox proportional hazards model by Park & Hastie (2007) also satisfies this condition.
SURVIVAL-SUPERRVSSD ARCHETYPAL ANALYSIS
Survival supervision readily be incorporated into the archetypal analysis optimization prob- lem (2.3 by adding a survival loss (wu 0) to the objective function, where 0 here the col- lection of all parameters specifying the survival model. For example, could have (aurv(W, 8)
Loox (5)W) as defined in equation (2.5) with parameters 6 s Specifically, letting & denote the set of possible values that parameter O can take on, and v >> denote user-specified importance
weight of the survival loss, we now instead solve
(W, (W, TT,
argmin WER"Xr HER lce W,EA" for i, H forall
Larch (W, A1 + 17 (W, 0),
(3.1)
where Larch is given in equation (.2 Javadi & Montanari 2019) solve the unsupervised archety-
pal analysis optimization problem (2.3) using the Proximal Alteraaiing Linearized Minimization
(PALM) algorithm by Bolte et al. (014).. We augment this algorithm handle survival supervi-
sion, resulting in an algorithm we call SURVIVAL-ARCIETY
We first state what SURVIVAL- archetypes before explaining our algorithmic modifications to the unsupervised variant.
3.1 THE SURVIVAL-ARCIITTYER ALGORITIM
In what follows, we let (V)) denote the version of V where each of its rows has been projected onto the set U. Formally, i-th row of llx(()
(V) ) given by
[ (V)), = min ||u Vill2- wet
For example, if V e n consists of nonnegative entries where each row's sum is strictly greater than 0, then [[la-(V]] = v. -/ Vij. Next, we denote the convex hull of the rows matrix V E R by
m
conv(V)
W. E A
Then the SURVIVAL-ARCHETVPER algorithm repeats the following steps until convergence:
Update archetypes: with step size parameter 71 = 2 where || || denotes the Frobenius norm, set
# H W
(WH - -X),
11
X
H - D X+
(H
conv (H)))
71
2. Update convex combination weights: with step size parameter 72 found using backtracking line search (Parikh & Boyd
2014 Section 4.3), set
1 (WH-X)L + T (W, 0) 1)) aW
W - Ila-
W
72
3. Update survival model:
argminggge Lourr Ō).
This step amounts to fitting the survival model with rows of W treated as the feature vectors and can just use the model's existing fitting code black box.
For the Weihull AFT loss given in Appendix B that has parameters 3 R E R. and or > 0, would set T (W, 0) TAFT (8, (8, or W)) W defined in equation B.I with parameters (3, a).
5

---

Under review as a conference paper at ICLR 2020
Initialization. Following Javadi & Montanari (2019), we use the successive projections algorithm by Araújo et al. (2001) to initialize archetypes H. We can initialize each row W by setting
W; <- argmin |X, - HTwl] wll2-
wEAr
Lastly, how the survival model parameters 0 is initialized depends on the survival model. For exam- ple, for the Cox proportional hazards model, we can initialize 3 to be the all zeros vector. For the Weibull AFT model, we can initialize and / to be all zeros, and to be 1.
that we need to able to compute the gradient @ww (W.0) As illustrative examples, we show what this gradient is equal to for the Cox aw and Weibull AFT models in Appendix C
key technical requirement SURVIVAL-ARCHETY
3.2 RELATING TO THE UNSUPERVISED ARCHETYPAL ANALYSIS PALM ALGORITHM
The original PALM algorithm for unsupervised archetypal analysis can be recovered by setting and removing step 3. Moreover, the step size in step 2 need not be found using backtracking line search. In particular, when 7 = 0, step 2 takes a proximal gradient step, where the gradient is
(WH X)H
which has Lipschitz modulus H |[;;
hence, can step size parameter 7/2 2 || (oltte et al. 2014 Remark 7(i)). When > 0, the Lipschitz modulus can vary by the survival model used and in general does not have closed-form expression, so we use a line search. Lastly, we are not constraining the archetypes to correspond to word distributions, then the projection onto A in step can be removed.
NEURAL SURVIVAL-SUPERVIII TOPIC MODELS
Our proposed approach to neural survival-supervised topic modeling builds on the SCHOLAR framework by [Card et al. (2018). Card et al. do not explicitly consider survival analysis in their setup although they mention that predicting different kinds of real-valued outputs can be incorpo- rated by using different label networks. We use their same setup and have the final label network perform survival analysis via the same approach as Katzman et al. (2018); note that Katzman et al. specifically consider the Cox proportional hazards model but their neural net approach works with some other survival models as well such as the Weibull AFT model. We first give an overview of SCHOLAR and then explain how implement the final survival analysis label network.
For ease of exposition, we present the SCHOLAR framework without what Card et al. refer to as "covariates" (auxiliary information known about subjects in addition to the word count matrix X). The SCHOLAR framework specifies a generative model for the data, including how each individual word in each subject is generated. In particular, recall that X1,, denotes the number of times the word u E 21,2.... d} appears for the i-th subject. Let n, denote the number of words for the i-th subject, i.e., n; Ed Xt..- We now define the random variable thy f1,2..... d} to be what the ε-th word for the i-th subject is (for = 1,2,.... ......., n and 1, 2,...,n.). Then the generative process for SCHOLAR with topies is as follows, stated for the i-th subject:
1. Generate the i-th subject's topic distribution:
(a) Sample W2 from a logistic normal distribution with mean vector μ E RR" and covari- ance matrix Σ E Brxx.
(b) Set the topic weights vector for the i-th subject to be W; softmax (W)..
2. Generate the i-th subject's words:
(a) Set word parameter 7 Sword (W.), where sword is a generator network.
(b) For word & 2, ,4:
Sample s/a,e ~ Multinomial(softmaxy
3. Generate the i-th subject's output label:
Sample Y, from distribution parameterized by label network ftabel (W.).

---

Under review as a conference paper at ICLR 2020
There are wide variety of choices for the parameters μa, ت, fword: and flabel For example, to approximate supervised LDA (McAuliffe & Blei 2008) where the topic distributions are sampled from a symmetric Dirichlet distribution with parameter > 0 and the output label is continuous and has unit variance, we set to be the all zeros vector, Σ = diag((r - 1)/ (arr)) fword (w) = wc" H w for a parameter vector
where H RRnd has Dirichlet prior per row, [wa and set Y; to be generated from a Gaussian with mean flabel(w) and variance 1. Card et al. (2018)
IR",
also explain how to approximate the correlated topic model by Lafferty & Blei (2006). To estimate the model parameters, Card et al. use a sampling -baed variational autoencoder framework (Kingma & Welling 2014 Rezende et al. 2014).
Survival supervision. To incorporate survival analysis, we follow the same approach as Katzman jt al. (2018) and change step 3 of the generative process above be deterministic and instead output the variable E. (W,) == 3 W; for parameter vector 3 E R.. In particular, we do not actually model how observed times Y,'s are generated; modeling E, 's sufficient. Then we can minimize the Cox proportional hazards loss:
Cox (3)) = Σ de - log Σ
exp(
j=l Y,
where 2; = log -.
Regularization on 33 can easily be added (e.g., lasso, elastic net). Other losses are also possible. The Weibull AFT loss given in Appendix B uses the same label network as the Cox example above, namely Suaee = 3 W,, For both the Cox and Weibull AFT examples, the label network could instead be a multilayer perceptron or more complex neural net rather than a simple inner product. We refer to SCHOLAR with survival loss as SURVIVAL-SCHOLAR
EXPERIMENTS
We apply SURVIVAL- and SURVIVAL-SCHOLAE to two survival analysis datasets fo- cusing two discases: pancreatitis and breast cancer. For pancreatitis, we use the MIMIC III electronic health records dataset (Johnson et al. 2016), looking only at the pancreatitis patients ad- mitted to the intensive care unit (ICU) and who did not die while in the ICU; this amounted to 371 patients where we extracted 2557 features (preprocessing details are in Appendix D.I.. We predict how long each patient will stay in the ICU. For breast cancer, we use the METABRIC dataset (Cur-) tis et al. 2012), which consists of 1981 patients. We use the same 79 one-hot encoded features as Lee al. (2018) to predict time until death per subject. Some features are continuous and need to be discretized for use with our topic models (resulting in 100 total features; see Appendix D.2 for details). For both datasets, we randomly divide the dataset into a 75%% train-test split.
We benchmark our approaches against a total of 10 baselines: 7 classical methods lasso--egularized Cox proportional hazards with and without PCA preprocessing, Weibull AFT with and without PCA, k-nearest neighbor survival analysis (Beran 1981 Lowsky al. 2013) with and without PCA, and random survival forests (Ishwaran et al. (008)), 2 deep learning methods (DEEPSURV (Katzman et al. 2018)) and DEEPHIT (Lee et 2018)) and Dawson and Kendziorski's SURVLDA (Dawson & Kendziorski 2012). For lasso-regularizer Cox, our hyperparameter sweep does include an ap- proximation the standard unregularized Cox model. For simplicity, for archetypal analysis and neural approaches, we use the standard Cox model as the survival model. For all methods, if the method does not already have hyperparameter selection procedure, we use 5-fold cross-validation on the training data to select hyperparameters prior to training on the complete training data us- ing the best parameters found; hyperparameter search grids are in Appendix D For the pancreatitis dataset, due to the number of subjects being small, we use repeated 5-fold cross- volsddvali with 10 repeats. Repeated k-fold cross validation has been found to useful in such small dataset regimes (Braga-Neto & Dougherty] [0044). For both cross- cross-validttion and for evaluating test set accuracy, we use the standard survival analysis metric of concorlance index (Harrell Jr et al., 1982), which the fraction of pairs of validation/test subjects correctly ordered by the prediction algorithm in terms of which subject has a longer survival time (amongst pairs that can be ordered).
Test set concordances are reported in Table [ On the pancreatitis dataset, SURVLDA followed by SURVIVAL-ARCHETVV
outperform all the other methods, and the two deep learning baselines (DEEPSURV and DEEPHIT) perform worse than standard Cox proportional hazards well as many of the other classical baselines. Meanwhile, on the breast cancer dataset, DEEPSURV achieves the best performance although Weibull AFT, k-nearest neighbors with PCA preprocessing, DEEPHIT,
7

---

Under review as a conference paper at ICLR 2020
Dataset
Model
Pancreatitis
Breast Cancer
Lasso Cox
0.56
0.63
Lasso Cox PCA
0.53
0.64
Weibull AFT
0.50
0.66
Weibull AFT + PCA
0.60
0.65
k-nearest neighbors
0.59
0.58
k-nearest neighbors PCA
0.53
0.66
Random Survival Forest
0,56
0.60
DERFSURV
0.55
0.67
DEEPHIT
0.53
0.66
SURVLDA
0.64
0.64
SURVIVAL-ARCHETYPSS
0.63
0.63
SURVIVAL-SCHOLAR
0.59
0.66
Table 1: Test set concordance indices for various methods on the pancreatitis and metabric datasets.
Per dataset, we use bold for the highest and italics for the second highest concordance indices.
and SURVIVAL-SCHOLAR all do nearly as well. Also, note that despite SURVIVAL-SCHOLAN being neural approximation of SURVLDA, the two methods' accuracies different; this phenomenon has also been reported for SCHOLAR and the various topic models it approximates (Card et al. 2018). Overall, there is no single best survival estimator. The three sureivel-supervieed topic models also jointly estimate topics, and per topic, tells us whether presence of that topic leads to greater or lower probability of survival. As we discuss next, despite SURVIVAL- -ALLLL having only a concordance index of 0.59 the pancreatitis dataset, it still manages to produce clinically interpretable topics predictive of whether panereatitis patients will stay longer in the ICU.
We now give brief summary of learned topics. Note that for both datasets, the vast majority of words we used require clinical expertise to interpret. For ease of exposition, we defer examples of actual topics learned to Appendix F. where per topic, we list its top 20 most probable words along with the topic's Cox 3 coefficient higher coefficient corresponds to predicting a shorter ICU length of stay in the pancreatitis dataset and shorter time until death in the breast cancer dataset. topic with coefficient 0 gets ignored for prediction.
identified one archetype with nonzero Cox 3 coefficient (4.5) corresponding to a healthy group with lower-risk interventions (e.g., smaller-bore IV, normal MCV and HCT, top words do not have data elements related to severe illness). All other archetypes have B coefficient and correspond to sicker patient characteristics (e.g., atypical lab tests and toxicology panels). SURVIVAL-SCHOL for laboratory tests, one for patient presentation characteristics, and one for procedures, precautions, monitoring, and vitals (this last topic has the smallest coefficient associated with longer ICU length of stay). SURV LDA also produced interpretable topics such as critical illness, normal health state, and acid base disorders and liver involvement.
Pancreatitis. SURVIVAL-ARCHETYER
separated clinical events into 3 meaningful topics: one
Breast cancer. SURVIVAL-SCHOLE found topics that distinguish elderly, advanced cancers (B co- efficient 0.71) from ones with early and younger hormone positive characteristics (3 coefficients -0.74 and -9.79. SURVLDA also produced topics with identifiable characteristics; however more topics were found (7 topics) with two overlapping topies indicative of elderly stage 2 breast cancer, and three other overlapping topics (all indicative of hormone positive, cellular, and proliferative fea- tures). SEEVIVAL-ARCHETE reflected in the topics it learns: the two topics with nonzero 3 coefficients have opposite 3 coefficient signs yet have mostly the same top words, suggesting too much topic overlap.
has noticeably lower prediction accuracy on this dataset, which is
CONCLUSIONS
Many methodological advances have been made in survival analysis especially with the help of deep learning. The advances have largely focused on prediction accuracy and less on interpreting time- to-event predictions in the application domains of interest. This interpretation can be challenging when the number of features is large and how features relate is not obvious. In this paper, we show that survival-supervised topic modeling can address this challenge: the topics learned reveal feature co-occurrences and have relative weights indicating their impact on predicting longer or shorter survival times. These topics can be used by practitioners to check if the models agree with existing domain knowledge and to help with model debugging. These survival-supervieed topic models are flexible and can be used with a variety of topic and survival models.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Mário César Ugulino Araújo, Teresa Cristina Bezerra Saldanha, Roberto Kawakami Harrop Galvao, Takashi Yoneyama, Henrique Caldas Chame, and Valeria Visani. The successive projections algorithm for variable selection in spectroscopic multicomponent analysis. Chemometrics and Intelligent Laboratory Systems, 57(2):65-73, 2001.
Sanjeev Arora, Rong Ge, Ravi Kannan, and Ankur Moitra. Computing nonnegative matrix factor- ization - provably. In Symposium on Theory of Computing, 2012a.
Sanjeev Arora, Rong Ge, and Ankur Moitra. Learning topic models going beyond SVD, In Foundations of Computer Science, 2012b.
Sanjeev Arora, Rong Ge, Yoni Halper, David Mimno, Ankur Moitra, David Sontag, Yichen Wu, and Michael Zhu. A practical algorithm for topic modeling with provable guarantees. In Interna- lional Conference on Machine Learning. 2013.
Rudolf Beran. Nonparametric regression with randomly censored survival data. Technical report, University of California, Berkeley, 1981.
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 2013.
Jérôme Bolte, Shoham Sabach, and Marc Teboulle. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming, 146(1-2):459 2014.
Ulisses M. Braga-Neto and Edward R. Dougherty. cross-validation valid for sample microarray classification? Bioinformatics, 20(3):374-380, February 2004. ISSN 1367-
4803. doi: 10. 10.1093hiionffrrmaiichhh.lg sloiiforratics//beeee
URL http://dx. doi. wrg/r0..10000 wwwpdd/..........o.ooooo
Dallas Card, Chenhao Tan, and Noah A. Smith. Neural models for documents with metadata. Pro- ceedings of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2031- 2040, 2018
David R. Cox. Regression models and life-tables. Journal of the Royal Statistical Society: Series B, 34(2):187-202, 1972.
David Cox and David Oakes. Analysis of Survival Data. CRC Press, 1984.
Christina Curtis, Sohrab P. Shah, Suet-Feung Chin, Gulisa Turashvili, Oscar M. Rueda, Mark J. Dunning, Doug Speed, Andy Lynch, Shamith Samarajiwa, and Yinyin Yuan. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486 (7403):346, 2012.
Adele Cutler and Leo Breiman. Archetypal analysis. Technometrics, 36(4):338- 1994.
John A. Dawson and Christina Kendziorski. Survival-supervised latent Dirichlet allocation models for genomic analysis of time-to-event outcomes, 2012. arXiv: 1202.5999v [stat.ME].
David Donoho and Victoria Stodden. When does non-negative matrix factorization give a correct decomposition into parts? In Advances neural information processing systems, pp. 1141-1148, 2004.
Frank Harrell Jr, Robert M. Califf, and David B. Pryor. Evaluating the yield of medical tests. Journal of the American Medical Association, 247(19):553322446 1982.
Hemant Ishwaran, Udaya B. Kogalur, Eugene Blackstone, Michael Lauer. Random sur- vival forests. The Annals of Applied Statistics. 2(3):841-860. 2008.
Hamid Javadi and Andrea Montanari. Nonnegative matrix factorization via archetypal analysis. Journal of the American Statistical Association, pp. 1-22, 2019.

---

Under review as a conference paper at ICLR 2020
Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. MIMIC-III, freely accessible critical care database. Scientific Data, 3, 2016.
John D. Kalbfleisch and Ross L. Prentice. The Statistical Analysis of Failure Time Data. John Wiley & Sons, 2nd ed. edition, 2002.
Jared L. Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yuval Kluger. DeepSurv: Personalized treatment recommender system using Cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(1):24, 2018.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014.
John D. Lafferty and David M. Blei. Correlated topic models. In Advances in Neural Information Processing Systems, pp. 147-154, 2006.
Changhee Lee, William R. Zame, Jinsung Yoon, and Mihaela der Schaar. DeepHit: A deep learning approach to survival analysis with competing risks. In AAAI Conference on Artificial Intelligence, 2018.
Zachary C. Lipton, David C. Kale, and Randall Wetzel. Modeling missing data in clinical time series with RNNs. Machine Leurning for Healthcare, 2016.
David J. Lowsky, Yichuan Ding, Donald K.K. Lee, Charles E. McCulloch, Lainie F. Ross, Richard Thistlethwaite, and Stefanos A. Zenios. A K K-neerest neighbors survival probability prediction method. Statistics in Medicine, 32 32(12):2062-2069, 2013.
Jon D. McAuliffe and David M. Blei. Supervised topic models. In Advances in Neural Information Processing Systems, pp. 121-128, 2008
Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and Trends in Optimization, (3):127-239, 2014.
Mee Young Park and Trevor Hastie. Ll-regularization path algorithm for generalized linear models. Journal of the Royal Statistical Society, Series B, 2007.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International Conference on Machine Learn- ing, 2014.
Akash Srivastava and Charles Sutton. Autoencoding variational inference for topic models. In International Conference Learning Representations, 2017.
A ESTIMATING POINT ESTIMATES FOR SURVIVAL TIMES
We now explain how an estimate S(-lw) of the survival function 5(-lw) can be used to construct an estimated survival time feature vector w. Recall that random variable Wo denotes a feature vector with survival time To. another random variable. Note that the survival function S can be written as
S(tlw) P < Wo w).
which is minus the CDF To conditioned on Wo w: we denote this CDF as F(-lw).
Once we have an estimate for S, to predict a single value for the survival time corresponding to test feature vector w, the most common approach is to look at where the survival function ((-lw) crosses 1/2 (this is also where CDF F(-(w) crosses 1/2, which is a median of the distribution). Specifically, for feature vector w, we find a median survival time estimate To(w) to be a time 7 such that (w) x 1/2. For example, we can compute (w(w) using the equation
To(w) =
1 2 inf {t: S(t)w) < 2 sup {t: S(tlw)
! |
1 2
(A.1)
where in practice the infimum and supremum taken over the observed times in the training data.
10

---

Under review as a conference paper at ICLR 2020
B THE SURVIVAL LOSS FUNCTION OF THE WEIBULL ACCELERATED FAILURE TIME (AFT) MODEL
As second example of survival model that can be combined with either of proposed survival- supervised topic modeling approaches, we present the Weibull AFT model.
Example 2 (Weibull AFT) The Weibull AFT model assumes each subject's (possibly unobserved) survival time T; satisfies
log = + B W dE;:
where P R, E R2, and > model parameters, and noise variable has probability density function fz(2)= (xp(=-expzz))Bf E R. Under random censoring, maximum likelihood estimation for μ., B, and a amounts to minimizing the loss
(6, log fe(z)) ò, (1 6,) log Sz(z))
(B.1)
where
log Y, - - B W.
Sc(t)= - J." fe fe(w)du=e-d
Note that the AFT model is actually a proportional hazards model with hazard function
h(t[w) exp( exp(-w/ojtt/o-1 exp(3 w).
Hence, after minimizing the loss function LAFT(B, p. ( to estimate B, μs, and o, we can plug in their estimates into the hazard formula above and compute an estimate for survival function S using equation (2.
To the Weibull AFT survival model with survival-supervised neural topic modeling approach in Section 4 we would minimize loss [B.I. where we redefine
log Y. -
As reminder, E, here is the output of the label network for subject In the simplest case, would set E; = 8 W.. However, we could replace this inner product with a neural net.
C Cox AND WEIBULL AFT SURVIVAL Loss GRADIENTS
For archetypal analysis, Cox and Weibull AFT survival models could both be readily used by using the gradients given below.
Example 3 (Cox survival loss gradient) When Leur((W,6 0) LCor(BlW) as given in equa-
tion 2.5 with A 3, we have
8, exp(( T We)
Σ
Lj=lss YSY, exp(3T W,) 5z
1= KL Y.SY.
Example 4 (Weibull AFT survival loss gradient) When Leunv (W, = LAFT( u, alW) as given in equation (.7 with 0 = (B, p1, G). we have
o W. 1 3 exp
Ye T We 11 1 55
aW t.a
D DATASET PREPROCESSING
D.1 PANCREATITTS
The pancreatitis dataset we evaluated on created from MIMIC III. a critical care health records database containing 52 thousand individuals and their hospital encounters involving admission to
11

---

Under review as a conference paper at ICLR 2020
the ICU Beth Israel Deaconess center between 2001 and 2012 (Johnson al. 2016). Experi- ments were conducted using subset of MIMIC III version 1.4 dataset consisting of patients having pancreatitis requiring admission to the ICU. Patients were included in the study if they have ICU admission with a primary billing code of pancreatitis, resulting in cohort of 371 individuals. For patients who are admitted the ICU multiple times, only consider their first visit to the ICU. This subset of the data has no right-censoring.
Features extracted include demographics, medications, billing codes, procedures, laboratory mea- surements, events recorded into charts, and vitals. Features were extracted from the relational database into a 4-column format for patient id, time, event, and event value. To prevent erroneous merging of different events into single event, and to provide more informative events, event strings are concatenations of the event descriptor prefixed with the table from which they are derived and additional relevant information such measurement type, measurement units, etc. Because events recorded charts are sometimes automated and sometimes manually entered, physician-developee mapping and lower-casing all fields were used to resolve duplicate entries. As we aim to predict the patient length of stay in ICU, we extract clinical events from the subjects' electronic health records strictly before ICU admission. Our definitions of clinical events mean that subject can have multi- ple instances of one event; for example, one patient might have multiple results for particular lab test file.
Single-occurrence categorical events (e.g., gender) are one-hot encoded, with one category removed the reference category. Multiple-occurrence categorical events (e.g., urine color) encoded by counting each categorys occurrences in a single subjects records. Numeric clinical events are treated categorical by mapping observed values to equally spaced ranges by quantile (5 bins of roughly equal number of subjects per bin). Missing records are not imputed as missing certain events can have clinical significance. Therefore, for features with incomplete records, an additional feature is added solely to indicate whether missingness is observed for each subject; this approach to handling missing data motivated by the work ofppon et al. (2016). After preprocessing. the total number of features used for prediction is 2557.
D.2 BREAST CANCER
The Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset contains genetic and clinical information for 1981 subjects with primary breast tumors Curtis et al. (2012). We use the preprocessed version by Lee et al. (2018), who one-hot encoded 21 clinical features to obtain 79 features Of the 1981 subjects, 55% (1093) are right-censored. After one-hot encoding the categorical features, a total of 79 features topic modeling. As the survival-supervieed topic models we use do not handle continuous features, discretize each continuous feature into 5 equal-sized bins based on feature value quantiles, the same way we did for the pancreatitis dataset. This results in a total of 100 features.
used for prediction in experiments that do not involve
E HYPERPARAMETER SEARCH GRIDS
We perform grid search for all models except for DEEPSURV and DEEPHIT, which use their own hyperparameter selection. We specify the search grid per method below.
Lasso-regularized Cox: lasso regularization parameter: {0.0001, 0.025, 0.05, 0.075, 0.1, 0.15, 0.25, 0.5)
parameter: (0.0001, 0.025, 0.05, 0.075, 0.1, 0.15, 0.25, 0.5]: number of PCA components: {3,5, 7, 10, 25}
Lasso-regularized
Cox
PCA:
lasso regularization
Weibull AFT: no hyperparameter tuned
Weibull AFT + PCA: number of PCA components: (3,5, 10,
k-nearest neighbor survival analysis:
number of nearest neighbors k:
(4, 8, 16, 32, 64, 128, 256,512 512)
k-nearest
neighbor survival analysis (4, 8, 16, 64, 128, 256, 512}; number of PCA components: {3, 7,,,,7,10,2
PCA: number of nearest neighbors k:
12

---

Under review as a conference paper at ICLR 2020
Random survival forest: number of trees: {16, 32, 64, 128, 256,
SURVLDA: number of topics 7: {3,5, 7, 10); Dirichlet a: {0.1,
SURVIVAL-ARCHETYPES number of archetypes {3,5, 7, 10); archetypal analysis regularization parameter {10, 100, 1000}; survival loss weight ry: {10*, 105, 1,,10,,,00 10"}
SURVIVAL-SCIIOLARR number of topics: {10', 10",,10",,00 109, 10%}
{3,5, 7, 103;
survival
loss weight w
F EXAMPLES OF TOPICS LEARNED
Note that for each survival-supervised topic model learned, what matters terms of how topics predict survival time is the relative values of the Cox 3 coefficients for that specifie model. A coefficient of O corresponds to topic that gets ignored by prediction. Below, we present the topics with nonzero coefficient learned using all training data (with hyperparameters chosen via cross- validation) for SURVIVAL-ARCIIE AARHEEEPEESS and coefficient and list the top 20 most probable words for that topic.
Per topic, we indicate the B
F.1 PANCREATITIS
SURVIVAL-ARCHETYPER finds 10 topics but only one that has nonzero Cox B coefficient:
Topic T- Cox 3 coetheient 1.51308799
Wond
Probahility within topic
20. w0..ggge..a. inserticr, dateritl
0.00254
leb:blood: chemmmpppdhettttteeceeeee chwmistryyyycc............yyy sodiumm:::iiiii1 (133, 0- 142. count
0.00154494
1eb :bbobdood: wwt.://atr..ree...ee.......
"gl (2, 122.0-12.00 .count
0,00153877
Lab:
(90. 01-ouunt
0.00152838
labibl ood
136. 5-51, _1_count
0.00147893
lab:blaod: cha wtt.://ww.....a............aaaia minaiaa missing Flag
0.00147567
0.00147308
fa..:/odddccaatttttyimmmmooooood miieing.faag
0:00147301
0.00147280
chart::ovpal phartt:coopallmmm/hhhhh...... sminginffflag
0.00147280
chart:al:: misaing Cla
0:00147145
chart:ost::n//1:: minsing.faas
0.00147145
leb:bleodd hrg.blooihhemtttoooooyynnenaeaae red cells::: missing Elag
0.00147084
lab:blood: l htg.aaooottteiaaalolllggggggagoo
missing._las
0.00147082
misssng..las
0:00147058
lab:bl
Id], mee.sll__meuuuuue measured: mmlasing.ffaa
0.00146990
chart: chart:troopcrss-- 1:2 mwssiing...la....mmm
0.00146951
soddc whw.phattphpppttteeeccoo via ivinmol::: missing Elag
0:00146929
chart:ttak..bppst
missing.flag
0.00146923
charL:Lank.a.pes: :nact: "mli: miseing.flag
0.00146928
SURVIVAL-SCHOLAR finds three topics, all with nonzero Cox & coefficients:
Topic
9 coeflicient 0.30225980
Word
Probability within topic
bab:blood: wtw.eetehisssssseeeeess.. 92 (93, 12 3) _ounnt
0,28912985
lab: bi blo:bbood:oom hal.occchhemissss.raaooo..eggree (13.0 15.0) .count
0,28761890
Jab:b strrrrch mitrorr
02.2.1
0,287061 142
abbb:llooddhemtttoooyy:oee q4 (15. ()
0,28468764
q3 (30. 3-3333 count
0,28418380
otg.ar/cccheeaaaaorrr..aa.eaaaaa (214.
0,28227943
Labpgloooodfieeatooooyyyeeeeeee: (137.011110000 39.0) 39_0)1_oount _oount
Ocount
0,28181386
lab:bl
1.281
.coutt
0,28062224
lab:blood thw.s//arryyooootaaasaaa.tt. q3 (3.00-0444 44. .ccunt
(4. 3-4
0,27537790
lab:bleod:: www.::hhhssssseessee.eo....
0,27385600
lebobblood: lebbrbboooddtcheesisseyyyyasssse wtt.ssssssay....aapsssiiattaaaaa 44 (2.0 2.2)
0,27342620
labs'ol ood: hheaattlooggyoohhhh::1000000000
0.27324 143
Labsbloon hoooboooohhhhmttooooo het.ooddhhheemttoooooo:::::::::. 7-1...........111 count
0,2720 820
(23.7-30. 411-330044--om0 -ouunt
0.27129397
-coutt count
0,26823607
Labsbloodd htg.:pochhhhatttlooyyy..eeeeeeea
0,26770705
lab:' bl lod hooohtohttaallyyymma::000222233 -2nooot
0.26591137
Lab:blood:
_ouut
0,26530087
lab:bload hww.te htt.oe/eeeeolllyyyaaaaooeeeeeeee (3..3443333))))1
0,26439255
13

---

Under review as a conference paper at ICLR 2020
Topic 2 Cox B coefficient -0.7 7300321
Word
Probability within topic
chart sadmit stgi "minsing flag
0.17567450
chart:head Bedinar:: missing Elag
0.35496010
Leb:blood: oe misiins missing.Flag
0.35346085
0.35066SBS
"miasing.
0.35065 705
whbatttroadialllllcccsaadyd.yy.. ff
0.74568897
presciiber.ataavatttttta... : missing Elag
0.34543383
wes.cc/hac..eerrrro..oooooava... minsing.flag
preeccleeesjeeee..oeeeeee....... mieeing..leg
0.34357762
preser/[eed:deex]cceee00000
0.343 2600
chartschloride (ttp
0.34123975
minsing.flag
0.34082162
chart:iav42
hts::n/www.vettocsaeeerrreee missing fl4g
0.3408 1900
0.34070030
0.3388838
chart:patn leve acceptablee:aa:
flag
0.13872578
chast::pw..hhhh miniggg
chert:resp.aless mis.igg.
0,33690770
chart:alt:io/ www:::::mmmiinnnn missing.Elag
0.336X5216
Topic 3 Cox 8 coefficient 1.06347895
Wont
Prohability within topic
arterial io
0,27548000
03.00....00....00. 93.0))
0.26509485
0,25576746
0,25267342
chert
hi high:
80.0)
0,245 14755
chart: hhw.hha.ddd..dddaaa.eee minsing_Elag
0.23182970
wuilti.luenn.ineettt..e wtw.siioo..nttdd.oo.. deter::l
0.22956896
lab:blood:b
(0.333.0).oom
0.2225 (068
chai 1ag.....lag
0.22050865
lab:bl
wod.:hhee//serrraattt--u-u. ant lbody:: "missing flag
0.21709190
char::nazes misiiagg...iiimmm..........m flag
0,21417704
0.21182594
charts/eepp.lammm..iihh....-gppp "ww.:::::mmmmmmmmm missing.flag
0.21160777
0,20904100
0,20473634
missing flag
0.20447205
0,20438956
: aal. middiiiiiii missing.fleg
0.20393686
0.21348457
htt.: min.gggnnnnn.::.iiiooooo.... "missing_flag
0,20335248
F.2 BREAST CANCER
SURVIVAL-ARCHETVPES finds 3 topics; only have nonzero Cox f coefficients:
Topic
coefficient 102.32197794
Word
Probubility within topic
TBR2_IRC_ataturr
0.04181057
1lpr2.xxpr.1
0,03842817
ogg cal.
0.037643477
inf met.eettns
0.03627912
2R. 1HC, stetus.1
0,03445554
EX.ExpE
0.03336668
Genefu
0.03303586
H7R/_8NP6_state_.
5.031
clustclust..m memb.
0.02787247
0.02684 69
cellularity
0.0238882U
FR_Expu.1
0.02362706
PS_Rxpz
0.02183748
wymph.nodess.om
0.02049815 15
site.2
5.020155g9
cellulerity.2
0.01693353
grade.1
0.0 1.01699655
group
0.01 588346
Pam50 2
506506007
0-4.0
0.0 0.01466086
14

---

Under review as a conference paper at ICLR 2020
Topic 3- Cox 1 coefmieiet 44.67 700060
Word
Probability within topic
status.1
0.04247266
Her2 ExpE.
0,04042959
2R.IIC..Latus
0,03630399
L2_Bxpr
0,03587382
bistological.1
0.03482885
HER2_SNP Stt.ttt.e.ee22222
0.03436751
inf men status
0,03308938
Gonefu
intuclust Jeemb.
782775788
lymoh_nodee.postt 11
0,027601 966
PR_Expz
0.02661266
To laliitt
0.02389579
grade.2
0.02212554
Pem50 Subtype.2
0,01950571
group
0.0 D01930659
FR_Exp
0,01884195
grade.1
0.01851070
cellulority.2
0.01631843
1zeatment.5
0.01531958
stage:2.0-4.0
0.01528393
SURVIVAL-SCHOLAR finds 5 topics all with nonzero Cox 3 coefficients:
Topic I-Cox coefliient U. 0.71125180
Word
Probability within topie
35-65.
grade. .2
0,26243757
Treatment.3
0,23528770
PR PRExxpz.
0.22858755
BR_.HiCaatatu
0.22115701
ago_st. [ss.:tt.di.gnoosi:::: 72. 44-24-9..14
0.1997155 1548
group.2
081882179
ER_Expx.1
18029946
group.
0.16400120
lymph_nodes -_pos wtw.oodeeeeepsttteeee..ooooooeee 1e.:
0,14714028
histological.1
0.14465626
clust memb.
0. 13438721
Her2_cxpr
0.12702571
size:32.0 162.0
0.12109776
0.11865168
sige:25.0 32.0
0.10744651
fu.
0. 10639679
Pam50 gubtype
002038068
stage: 19035222222222222
0.09872285
N11:0..05-2 05..05
0.09574302
Topic coefficient
0.21598756
Word
Probubility within topic
$tage:0.0-1.0
0,40880377
Trestment
0.31841293
loggggg
0.31109884
HzR2, Stt_p2tt66,tttte1
0,28868553
int clust memb.
0,28224890
Pa.Expz
0.26736894
aLage:1. 190392220 2.0
0.25793970
Treatment.4
0.252K0258
Exp2.
Mistoooicoll
0.24185723
Her2.Expr.
0,23500897
int Lembb.
0,22067617
FR_Fxpr,1
0.21519190
Genefu
0.21518746
group_1
2.21
grade.2
0,21007578
site.2
0.20887 140
age_st_df wttp::diagnnaaaa222244..
0.20778224
H282 TFC.status
size:32 32-1220
0.20289177
15

---

Under review as a conference paper at ICLR 2020
Topic 3 Cox B coefiiciett Word
00.66332007
Probability within topic
TRC: status.1
0.17488750
HzR2 S&.6 2
0.30100745
Lnt.cluste htt.cccsstmeeeee menb.
0,25848496
H2R2.9KE 6,state.2
0,24172206
TreaLment
0.22806 108
ER_Expr
0.22049715
inf men status.1
0.21472536
.
0.17921847
histological."
0.1 7481772770
OPI:3.04-4.06
0.17427845
D. 16505045
PX_ExpZ
0.16.91334
inf men status
0.15946833
Her2_8xpr.
0.1 16669994
group
0.15509687
1nt clust memb.
D. 13600853
grade
0.15454559
cellulecity.1
0.15221779
size.2
0.15143663
Topic Word
0.74412241
Probability within topic
Her2 Expr.
0.24829830
hytthh.nodes..cssttieeee.
0,24766846
[DR2 6.stalee pei lalarity
0.21669297
D. 0.20731404
histological. histologiicol..
D. 1909915770
1HC stetus.
D.18253753
TreaLment. 3
0.1 7639663
Fam30.8ubtype
0.17552045
FR.Expe.1
0.16891813
Ganefu
0.15879555
EA Expr
0.15030436
histological.
0.12452137
HERS_TRC_atatus
0.12290693
6122:0.0-15.0
0.12223840
site.2
1.12201823
int.clust.menbl
D.1 1619608
sice.1
1273366
NPT:3.04-4.03
0.10929855
Topic 5 Cox B coefficient 0.78542084
Wonl
Probabiity within topic
Expz
0.35541302
Her2.Expr.
0.31779182
[DR2 DDDR2.IIC.. IiC_eta
0.30835040
R3_Rxpr
0.30589350
0.298501337
HRR2 WNP..NPG.... statte2
0.27557 23
grade.1
0,26845238
group
0.2099454
age_st, ag 01.-888.0--897
0.24533404
qrade
0.24168900
ER.THC.atstuan
stage: 191-592222299
0.21476460
NPI:1. 0 3.34
0,21265814
NPI:4.03-4.0
0,20501652
0.198 19605
hiatolloiicol.5
0.19405621
Pem50 Subtype.2
0.18957051
group.3
0.16390443
histological.9
D.14986861
fun
0.14619182
16


Reviews:
[{"review_id": "r1x8ZfACFB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers the problem of interpreting the predictions for survival analysis using topic models. The classical survival analysis problem assumes each datapoint is a subject (X,Y,\\delta) where X is a feature vector and Y is a life time or a censoring time depending on whether the subject is dead (when delta=1) or alive (when delta=0). The usual objective here is to predict survival times. The authors assume the features in X are interpretable readings (eg \"low blood pressure\") and indicate the number of times that reading was observed. Under this setting, such features can also be seen as words with datapoints being (BoW) documents. The goal then becomes that of finding topics that help predict life times on unseen subjects.\n\nMy main concerns with this paper are the following:\n- setting is very niche and might not be a great fit for ICLR.\n- novelty is limited since most pieces were already present in previous work.\n- experimental section only uses two datasets with little improvements.  Such small performance gap hardly convinces that the improvement by the proposed method is statistically meaningful.\n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "r1eNf14FKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper addresses the problem of survival analysis (predicting time until, e.g., death) using topic modeling. The point of introducing topic modeling here is to gain better insight into what helps predict survival times of unseen test subjects. This contrasts with much of the earlier work in survival analysis, which is mainly or only concerned with getting the best prediction without concern with the “thematic structure” of features. Such a thematic structure could be useful to clinicians, but analysis of the learned topic structure by clinical experts is apparently left for future work. Empirical results on pancreatitis and metabric datasets show that the methods of the paper are comparable to several established baselines.\n\nI think the goals of the paper are well motivated, as time-to-death is not the only measure of interest and clinicians and other domain experts may want to get better insight into why a given model makes certain predictions. The two approaches presented in the paper are mostly on par with (but do not outperform) the best baselines considered in the paper, which would be fine in the approaches were particularly novel and/or the learned “thematic structures” would have demonstrated uses. Unfortunately, I feel the latter conditions are not met, as I have the following concerns:\n\n(1) I find the paper to be relatively incremental in terms of ML methods, and I consider it to be more of an applied ML paper. The paper builds largely upon survLDA (Dawson and Kendziorsky, 2012), using Car et al. (2018)’s way of approximating LDA in a neural framework. Much of the exposition of Survival-Scholar (i.e., section 4) is based on Scholar (Card et al.). The paper presents another method based on archetypal analysis, which adds a supervised term to the loss. The archetypal approach of the paper merely consists of an “algorithmic modification” and otherwise borrows extensively from Javadi and Montanari (2019) and Bolte et al. (2014). In both cases (survLDA extension and archetypal analysis), most of the equations/algorithms can be traced back to these previous papers.\n\n(2) The main point of the paper is to go beyond simply predicting time-to-event outcomes, and instead gaining some insight into the features helping predict time of death. The authors rightly “note that for both datasets, the vast majority of words we used require clinical expertise to interpret”, but it doesn’t seem they have given the learned topics to clinicians to judge whether they make sense or are helpful in any ways. Instead, appendix F provides lots of tables that are difficult to interpret for non-domain experts with little medical background. \n\n(3) I have some more minor concerns (or questions) about the experimental results. See detailed comments.\n\nOverall, I found the paper to be clear and well written, but algorithmic and empirical contributions are I think rather small (small or no empirical gains, incremental ideas).\n\nDetailed comments/questions:\n\n- Table 1: I don’t think it makes sense to label a system as either first or second. With differences of less than 1% and the small size the test set, most of the top systems are probably within the same confidence intervals, which I think the authors should compute and add to the tables. Many of the earlier works cited in the paper (e.g., DeepSurv, DeepHit) provide such confidence intervals, which are crucial in the case of the submission as its dataset is relatively small compared to e.g. SEER used with DeepHit.\n\n- “For all methods, if the method does not already have a hyperparameter selection procedure, we use 5-fold cross-validation on the training data to select hyperparameters prior [..]”. Shouldn’t cross-validation for hyperparameter selection be used with all the methods of the paper, as the paper doesn’t define a validation set? I’m a bit concerned results may not be fully comparable if the selection procedure is not consistent across algorithms (e.g., 5-fold vs other methods). I presume “already have a hyperparameter selection procedure” refers to a procedure applied to the MIMIC dataset (as original hyperparameters borrowed from previous work might not work well). If so, I think the paper should say this explicitly.\n\n- Experimental results consist of single table that leaves much to be desired in terms of understanding why a method works well or not. For example, what about indicating the number of model parameters (for the parametric methods) and learned hyperparameters. With such a small dataset, cross-validation could still be noisy and prone to selecting a bad set of hyperparameters (e.g., significant differences in terms of number of parameters). An indication of how sensitive to hyperparameters the methods are would be useful too.  \n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "Skx9cEOatH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores the problem of simultaneous learning for topic modeling and survival prediction.  Their contributions are: (1) integrating two different topic modeling approaches with survival models for joint learning and (2) showing results on two medical datasets with some brief analysis of what topics are recovered.  This is an interesting task to be using with topic modeling.  I appreciate that the approach could be used in medical systems where interpretability of models is very important.\n\nI think the high-level idea of the algorithms is fine, but the main drawback is that the experimental section needs to be expanded on, perhaps with larger datasets and more analysis.  Unless more experiments are provided, I might lean towards rejection.  I think it's ok that some of the empirical results are inconclusive, but it should lead to more quantitative error analysis.  I definitely think the smaller size of the datasets could be one factor in the negative results, which is why I think the authors should try experimenting with a larger dataset.\n\nMore specific comments and suggestions:\n- Each dataset is small (371 and 1981 data points respectively).  This may be too small to effectively train some of these neural models, which I believe may be one reason why they underperform, especially on the pancreatitis data.  In order for the experiments to be more conclusive, maybe you can switch to a different dataset.\n- Related to the size of the data, I suspect that many of the differences in the table are statistically insignificant.  Can the authors please specify which of the results are significantly better than the others?\n- While I liked the examples in Section F of the appendix, they are difficult to compare between models, and not all of the models are included.  More quantitative analysis is needed to really understand how the models’ learned topics differ in terms of coherency, cohesiveness, interpretability, etc.  Providing really detailed quantitative comparisons might strengthen the analysis section of the paper.\n- I liked the preciseness of the descriptions of different algorithms in the background section.  They were all explained nicely and easy to understand.  However, the background section is a bit too detailed at times.  Authors may consider condensing a bit more.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes two approaches to topic modeling supervised by survival analysis. The reviewers find some problems in novelty,  algorithm and experiments, which is not ready for publish.
Decision: Reject