Censor Dependent Variational Inference
Authors: Chuanhui Liu
Xiao Wang
Venue: ICML 2025 poster (2025)
Keywords: survival analysis
variational inference
variational autoencoders
time-to-event modeling
Primary area: probabilistic_methods->variational_inference
TL;DR: Variational inference for survival models transcends the standard paradim, requiring adaptions to handle censored data.

Abstract: This paper provides a comprehensive analysis of variational inference in latent variable models for survival analysis, emphasizing the distinctive challenges associated with applying variational methods to survival data. We identify a critical weakness in the existing methodology, demonstrating how a poorly designed variational distribution may hinder the objective of survival analysis tasks—modeling time-to-event distributions. We prove that the optimal variational distribution, which perfectly bounds the log-likelihood, may depend on the censoring mechanism. To address this issue, we propose censor-dependent variational inference (CDVI), tailored for latent variable models in survival analysis. More practically, we introduce CD-CVAE, a V-structure Variational Autoencoder (VAE) designed for the scalable implementation of CDVI. Further discussion extends some existing theories and training techniques to survival analysis. Extensive experiments validate our analysis and demonstrate significant improvements in the estimation of individual survival distributions.

Censor Dependent Variational Inference
Chuanhui Liu
Xiao Wang'
Abstract
such data effectively.
This paper provides a comprchensive analysis of variational inference in latent variable models for survival analysis, emphasizing the distinctive challenges associated with applying variational methods to survival data. We identify a critical weakness in the existing methodology, demon- strating how poorly designed variational distri- bution may hinder the objective of survival anal- ysis tasks-mo time-to-event distributions. We prove that the optimal variational distribu- tion, which perfectly bounds the log-likelihood, may depend on the censoring mechanism. To address this issue, we propose censor-dependent variational inference (CDVI), tailored for latent variable models in survival analysis. More prac- tically, we introduce CD-CVAE, a V-structure Variational Autoencoder (VAE) designed for the scalable implementation of CDVI. Further discus- sion extends some existing theories and training techniques to survival analysis. Extensive experi- ments validate our analysis and demonstrate sig- nificant improvements in the estimation of indi- vidual survival distributions. Codes can be found at
Deep learning has emerged as a powerful paradigm to ad- vance survival analysis (Wiegrebe et al., 2024). Recent studies focus on modeling the individual time-to-event dis- tributions using expressive latent variable survival models (LVSMs). For example, Ranganath et al. (2016) assumed that the prior of Z belongs to the class of deep exponential family distributions (Brown, 1986). Deep survival machine (Nagpal al., 2021a) considered the finite discrete latent space, modeling the logarithm of time-to-event as the finite Gumbel or normal mixtures. Xiu et al. (2020) modeled a discrete time-to-event as a softmax-activatee neural network incorporating the Nelson-Aalen estimator (Aalen, 1978). Apellániz et al. (2024) followed a similar setup, develop- ing variational autoencoders (Kingma Welling, 2014; Rezende et al., 2014) (VAE) for continuous time-to-event Importantly, these models outperform Accelerated Failure Time (AFT) (Miller, 1976) and Cox Proportional Hazards (CoxPH) (Cox, 1972) models across multiple metrics, en- abling downstream tasks through latent features (Manduchi et al., 2022).
A unique aspect of LVSM optimization its reliance on variational inference (VI) (Jordan et al., 1999) to approxi- mate intractable likelihoods arising from flexible parametric generative models. As a result, the performance of latent variable models depends heavily on the optimality of VI and associated design choices (Cremer et al., 2018).
1. Introduction
In our opinion, the role of VI in modeling time-to-event distributions remains inadequately explored. It is unclear what form the optimal variational distribution should take under censoring or which posterior it is meant to approxi- mate. Existing designs are largely heuristic, with limited theoretical grounding in the presence of censoring.
Survival analysis, a fundamental topic in statistics, finds wide-ranging applications across healthcare, insurance, qual- ity management, and finance (Harrell et al., 2001; Nelson, 2005; Frees, 2009). It focuses on modeling the relation- ship between time-to-event outcomes and individual demo- graphic covariates, where the event of interest could be death, disease progression, or similar occurrences. A key challenge in survival analysis arises from censored obser- vations, which provide only partial information about the survival time, necessitating specialized methods handle
To address these gaps, this paper advances the understand- ing of VI in survival analysis by providing a theoretical analysis of its optimality under censoring and introducing principled variational framework for LVSM. At high level, our main contributions are
Department of Statistics, Purdue University, USA. Correspon- dence to: Xiao Wang <wangxiao@ purduc.edu>>.
1. In Section 3, we motivate and introduce the core ideas of censor-dependent variational inference (CDVT). We estab- lish conditions under which vanilla VI for survival models fails achieve inference optimality under non-informative
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Censor Dependent Variational Inference
censoring and expose key limitations of existing methods. We derive the optimal variational posterior for the log- likelihood of survival data. We further demonstrate that censor-dependent variational family is required to capture the structure of the true posterior, as illustrated in Figure 2.
rameters governing the distribution of U, C, respectively.
The goal of time-to-event modeling is to estimate the event- time distribution parameterized by 6. Under such censoring assumptions, the joint density' of y. 8 conditioned on æ can factorize as follows
2. In Section 4, we propose CD-CVAEs, amortized CDVI implementations of VAE-based survival models, free from the restrictive proportional hazards assumption and impos- ing minimal distributional assumptions, which rely solely on decoder architecture. We provide formal guarantees for our proposed tighter- bound sampling- based variants via aug- mented VI theory and offer practical insights into model implementations, such as training strategies and stable com- putation techniques. Section presents empirical valida- tion and findings through extensive experiments on various datasets. Notably, methods achieve 5% higher C-index than state-of-the-ard models on WHAS datasets.
pe, 8(x) xr pu, (u.,(ll))
(2)
Taking the log, the right-hand side of (2) gives rise to the full-parametric objective L()), which is given by
L( :- ô dog fo( (1 8) log (y(y))))
(3)
where fo 2z), So(v)zz) represents the density, survival functions of U evaluated at observed time respectively. While (3) is called the log-likelihood parameterized by (Kalbfleisch & Prentice, 2002), we note that the right-hand side of (2) is not proper density, as lacks normalizing constant with respect to r).
2. Preliminaries
Notations: Random variables (r.v.) are denoted by capital letters, e-g. X, Z, Y, U, C, and their distribution functions have matching subscripts. X denotes the sample space of X. P(),, P(),, S()), h(-) respectively denote general probability function, a cumulative distribution function, a density function, a survival (tail) function, and hazard func- tion. Subscripts in Greek letters 0, & denote the unknown parameters. E.g. (-ye()) refers to the survival function Y parameterized by 9. Additional letters f. denote different P- A proportional relationship over is denoted as xx. Estimates of functions or random variables are indicated with caret or dot symbol above, c.g., S(.) is an estimate of S(-). log denotes natural logarithms. Bold symbol x denotes vectors.
Diseparation
Z
(a) generative graph of LVSM
(b) latent structures
Figure 1: Directed acyelic graphs of LVSM. The shaded nodes æ;, y, ā are observed. (a) The dashed box shows a general generative graph of U. (b) Specific Latent structures:
density functions, e.g.. so(-) (.)
D-separation assumes X 1. U U|Z V-structure assumes X- independent latent Z. Best viewed in color.
2.1. Log-likelihood for Right-censored Data
2.2. Latent Variable Survival Model (LVSM)
Datasets: In a single-event right-censoring setting, we con- sider a non-longitudinal survival dataset consisting of n triplets {æ, yr, J In particular, the event indicator 5. = 1 signifies that y: is the observed time of the event of interest (time-to-ever to-eet,,- while 6, signifies that yı is right-censored and the true time-to-event of subject i with individual features/covariates exceeds the observed value.
LVSMs construct fe(uzz) from (3) within a latent structure using a continuous latent variable Z, enabling more flexi- ble and expressive characterization than traditional methods. As shown in Fig. general formulation given by
=fe fo folu/z, 2) (2)2))2,
fo(u|æ:) = Jsez
(4)
In this work, we assume the dataset consists of i.i.d. ran- dom variables {X.Y 7), where observed survival time Y continuous. Notably, we consider (Y; I) as the surjective transformation of two continuous latent variables under the independent censoring assumption U 1 C X::
where Pa( (z)zz) is the conditional prior of Z and fa(u,z is often called the decoder or the emission distribution.
2)
Such latent structure enhances the expressiveness of survival models fe, moving beyond the constraints of proportional hazards assumptions. For example, an AFT model can be seen as LVSM constrained by linear latent, e.g., Z| a + B X, in D-separation structure illustrated in Fig 1b.
Y (U,(U I= 1U C)
(1)
where U is the uncensored time-to-event and C' is the censoring time. Here, we denote by 0, n the unknown pa-
NiddonN derivative of the distribution P(Y, I(X) w.r.t. the product the Lebesgue and counting measure.
2

---

Censor Dependent Variational Inference
While LVSM is more flexible, the M-estimation of 0, i.e., Inte arg max T(()) is challenging due its computa- tional cost. Specifically, fo in (3) may lack closed-form integral, rendering it even harder to approximate So reliably. As a solution, VI is one common techniques used in LVSM.
Existing approaches of variational conditional posterior fail to extend to survival analysis. In (semi-Jsupervised setting (Kingma et al., 2014; Sohn et al., 2015), VI aims to approx- imate the intractable posterior pe (z) 22,))) y) with (2(2 (z. y). However, these methods struggle to account for censoring in the observed labels, i.c., the time-to-event U/ in our case.
2.3. Vanilla Variational Inference for LVSM
On the other hand, existing methods of LVSMs often lack rigorous inference optimality analysis, and their design can appear nounter-intuiinee example, Nagpal et al. (2021a) adopted lazy strategy in obtaining 96 by manually setting equal to the tractable (=(z]r). Similarly, Apellániz et al. (2024) limited qa to depend on X only, making it completely ignore the infor- mation of y.
Here, we review a naive framework of VI, referred to as the vanilla VI, as seen in Ranganath et al. (2016); Xiu et al. (2020); Nagpal et al. (2021b); Apellániz et al. (2024). Specifically, a variational distribution posed to obtain unbiased estimators of fo So((l))) By Jensen's 's inequality, log (o(y)z)) log (yll))) in (3) can therefore be lower bounded by
from a Bayesian perspective. For
( v) is pro-
So(y)z).
log. fy(y)x) 2 Eyg log fo(v)x,2) 2) KL (5)
3. Theories
log So(ylz) 2 B. log So(y| x 2) KLLgolIps [x)] (6)
This section studies inference optimality in LVSMs, where optimality of ø is defined tightly bounding the log- likelihood (3), parameterized by 9, assuming at least one censored and one uncensored observation.
By substituting the right-hand sides of (5) and (6) in (3),
it yields lower bound ELBO(O, Φ) on the log- ligeliod (Xiu et al., 2020; Nagpal et al., 2021a), which is given by
3.1. Problems in vanilla VI
ELBO((, Φ) == o.g log fo(v) z) + (1-6))E, log Salyk z) - KL[golIpa(C )))
3.1.1. AN EDGE CASE STUDY
As a preliminary step toward understanding the problems of vanilla VI, we start by analyzing the equality (tightness) conditions of log- likeeiilihood components in (5) and (6) as functions of x., u.
As a side note, Ranganath et al. (2016); Apellániz et al. (2024) further decompose the KL[gellPod (po [z|=]] (KLD) with- out assuming that po(2|(an |az) is tractable, where intractable log (xx) is moved into (0) by rearrangement.
KLD = log + KL [g=))))(())] Egg log po(1))) (8)
Lemma 3.1 (Equality conditions of (5) and (6)).
Given any parameter O, the point-wise equality in (5) holds
Consequently, (7) enables tractable and efficient computa- tion of both the expectation and the KI. divergence, improv- ing scalability for large datasets. Often, optimizing (7) can be done by amortized black-box VI algorithms (Ranganath et al., 2014) via the reparameterization trick (Kingma & Welling, 2014; Rezende et al., 2014).
for any {X- = x, U = u}. if and only if one of the following conditions holds:
(a) go u) = so(u, 2|m² [o(())) where Solu,
:= folulx, 2) (x z);
(b) 3 map e₁ folu, 2 ) / (2) |x, u) = ci (x,, u):
(c) (L[gs(z)a x u) /lllo (= x u)] 0.
2.4. Inference Optimality for Censored Data
Likewise, (6) holds for any {X x, U = u}. if and only if the following equivalent conditions hold:
The key distinction in optimizing (7), rather than directly maximizing L()), lies in its pursuit of two distinct objec- tives simultaneously: 1) the M-estimation of 0 and 2) the variational bound of L(0) defined in (3). The second ob- jective aims to minimize the inference gap (Cremer et al., 2018), i.e., the bias, of (()):
(a') ) go (2 [2| u) = =Sp(u, // 2/zx)/S((ulaz where we abuse So x/xx = -- fo(s, 2 ls)ds
(b') 3 map C22 So 2 / (z
ar u) C2 (a, u).
Proof: the conditions for (5) follow the standard VI ar- gument, and the conditions for (6) are derived under the additional assumption of Fubini's Theorem.
B(0, Φ) == L(0)- - ELLOOALBBBOOO o).
(9)
Since the optimum (0',Φ*) )²) == arg max BLBO(O,, Φ) bal- ances the best of these two results, the optimality of VI
Lemma 3.1 raises natural question about the variational inference optimality in survival analysis:
crucial for the estimation accuracy of 8", Improper varia- tional approximations introduce bias and degrade the relia-
Given any 8 and (z, u), what kind of (z)2) u) would satisfy both tightness conditions? ?
bility of estimates of 0.
3

---

Censor Dependent Variational Inference
In other words, we ask: is there an optimal variational solution that have zero inference gap of log-likelihood given both data {x, y, O} and {x, y, 1J?
structured, e-g., the latent represents an unseen individual- independent treatment. Claim (4) states that optimal ( is collapsed to the prior p(z), which leads to a notorious issue called posterior collapse (Bowman et al., 2016; Lucas et al., 2019).
For notation clarity, let P1 (f) denote the set of & where (5) holds equal, (0) denote the one for (6). Thus, we are interested in its union (0) := 1 (0) n $2(0), which is the ideal parameter set for optimal 9s that eliminates the bias of (7) and has a zero inference gap. On top of that, we define Qu = {0 u (0) Ø) } to denote the support set of 0 allowing optimal vanilla VI.
1. ON CENSORING ASSUMPTIONS
We next extend the optimality analysis by considering when go (1 a u) must simultaneously satisfy both inequalities in (5) and (6). As previously discussed, this situation arises when both data points { y, 0} and {x, y, 1 are present in the datasets. In the infinite data limit, it becomes essential to examine the underlying sampling space, which defines the supports of (5) and (6). For clarity, we denote the event space DE and the censored space De as follows,
Perhaps surprisingly, Proposition 3.1 below highlights the degradation of optimal vanilla VI solution, showing that these conditions in Lemma 3.1 are fundamentally different.
Proposition 3.1 (Degradation for optimal qo [x, u)). Assuming that 1) optimal VI is feasible: Ou F Ø, and 2) Solulæ, z) is location-scale density with location po( and scale a. Then, given any x, u,
DE = {(xx v) (z. y. 1) X x y I], Dc == {(x y) (a, y, 0) X XI
(10)
(1) Latent non-identifiability; 70 E Og. hazard function hg(ul 10|z,00 .) is independent of z; (2) Location degradation: V0 E Gu, location parameter po(a: z) is independent of
Remark 3.1 below delineates the conditions under which the issues in Proposition 3.1 persist in the infinite data limit.
Remark 3.1. For any (x, y) E DE Dc, Proposition 3.1 is applicable to the optimal (0(2|x y).
(3) Lazy posterior: 40 E Ou, Vg E (f). the varia- tional distribution KI-[ga (0/gg (z|ar, u) ()][p((z]x)]] 0;
(4) Surely posterior collapse: If z 1 x, AO E Ou.d & E
Specifically, if DE O Dc # Ø, such issues of vanilla VI are unavoidable. On the other hand, DE n Dc = Ø, it is theoretically possible for vanilla VI achieve a zero inference gap, satisfying the conditions of (5) on DE and (6) on Dc. In other words, the type of censoring and, more importantly, its effect on the partition of the sample space are crucial to the vanilla VI optimality.
Du (0), KL[ga(z|ar u) [[=) 0.
Proofs are deferred to Appendix 1. In a nutshell, claims (I) and (2) reveal the critical limitation that optimal VI can only be achieved on extremely limited support of 8. Specifically, claim (1) asserts that ha(u z). or equiv- alently fo(u|ær, 2), is independent of z, disregarding the latent information from prior (z|2))). Remarkably, such behavior of fo fu(u]z, z), known latent non- non-identiffabiliy (Wang et al., 3021), is first identified in the context of sur- vival analysis. Under additional location-scall distribution assumption, claim (2) asserts that its mean (t po(x (x, 2) reduces to an univariate function that is independent of z, restricting the expressiveness of LVSM. This observation may help explain why the aforementioned applications often adopt D-separated latent structure, where fo (u) 2) is fully dependent on in an effort to mitigate or avoid such ex- pressiveness issues.
Here, we note that the widely adopted non-informative censoring assumption (Lagakos, 1979) lacks specificity on vanilla VI optimality, since its influence on the partitioning of the sample space is not explicitly characterized. For ex- ample, under certain types of non-informative censoring, such as Type-I censoring (Lawless, 2003), it is possible that the event and censoring spaces become disjoint. That said, evident in benchmark datasets (See Table 3), observa- tional studies rarely have disjoint spaces; vanilla VI can be suboptimal in these benchmark datasets.
3.2. Censor-dependent Variation Inference (CDVI)
Moreover, claims (3) and (4) demonstrate the negligibility of the optimal solution 4o- The reason is simple-sing both fe(u(z, (z, 2) and Sy(u/æ, =) are independent of x, their posterior equals nothing but their common prior. claim (3) proves that the optimal qs collapses to the conditional prior p=(z)ar), ignoring the information of u. To this extent, the optimal 96 becomes as lazy as the one proposed in Nagpal et al. (2021a). It also explains the rationale in Apellániz et al.
Having identified the limitations of vanilla VI, we are now ready to establish less restrictive VI framework for LVSM.
3.2.1 OPTIMAL VI FOR JOINT DENSITY
First, revisit the setup and establish the optimal varia- tional distribution for the joint density in (2).
(2024), where the proposed (z) is not dependent on u.
Theorem 3.2.1 (Point-wise optimal VI). Given x. y, 5 and parameter O, go x., y, 5) is optimal if
Such negligibility can be more detrimental if the latent is V-
4

---

Censor Dependent Variational Inference
and only for almost every 2,
0.12
Pa SZ S A2/2,,,,, vol vol(Az)
True Posterior piz x. x.y) (6-marginalizerr (aargginaeeeeiii
ao- (2|0,,,,))
lim (2z)
le (events) (events)
Moreover, if DE Do X u, the optimal 96- is independent parameters of the censoring distribution 17 and for almost every 2,
2,
Ti. (cerssors)
0.06
(a) 9 (z y, 1) = 20; (z (2| u uy)n=y where o1 E (0). (b) 90- 2. 0) = gog 22 u) uyu where ob E (2)
Mu (censors)
-0.04
The proof deferred in Appendix B.2. Thm 3.2.1 states that the optimal variational distribution do for joint density (2) is equal to the posterior density of P(Z]X, Y, 6). ). In particular, if there is no overlap of sample spaces due to censoring, the
00002
Vanilia constrained shared
CDVI: misture of 6 6-dependent. øiz xx v.
optimal becomes independent of and thus becomes the one for log-likelihood satisfying Lemma 3.1.
Figure 2: Comparison on simulated datasets SD4 (see Table
Thm 3.2.1 presents an alternative perspective on the source of the previously identified limitation, attributing it to the design ds in vanilla VI.
2). The learned posterior is marginalized over censoring status 6, conditioned on 1, 0. The closed-fore true posterior is given in Appendix C.2. Best viewed in Color.
Remark 3.2 (Vanilla VI propose a marginal (6).
Assuming that there is no partition Dg De = X x U,
the marginalized q (z|æ, y) equals G0z (=| X = x, U = y)
Specifically, it defines the induced likelihood estimators :
for any i = 1, if and only if P(8 = 2 iY = y) = 1.
fz))) = fe = 2|x)/// /s. (z|æ, 8), $1(z) =- So (y. =/2>// / (= | v)-
Remark 3.2 states that the inability to obtain equality both (5) and (6) arises from defining 96 as marginal distribution, whereas the true posterior it aims to approximate is con- ditional one. As illustrated in Fig.2, (( xr, y) proposed by vanilla VI is 6-marginalized and lacks the censor-dependent structure. Consequently, the optimal solution of vanilla VI would approximate the true posterior if and only if there is no event censoring data. From this point, further limita- tions do described in Section 2.3, such employing lazy strategy or making independent of y, irrational.
(12)
The subscript 1 is purposefully added to align the notation in sampling variants defined in (14). Plugging in (12) in (3) and taking expectation, we obtain the Censor-dependent ELBO (ELBO-C):
ELBO-C :- 8[En, log folylær z) KL pl [1))]]
+(1 - (E [log (y/x i x)] KL[g0z lipe [x]
(13)
3.2.2. CENSOR- DEPENDENT ELBO
Compared to vanilla ELBO (7), we demonstrate its suitabil- ity for survival analysis below, along with how addresses the previously identified issues. Let p = {(11, Φ2)
(0(2,,,80 8) in Thm 3.2.1 is called the censor-dependent due to the explicit dependency on the indicator 6. To derive the corresponding ELBO, we now introduce an equivalent parameterization.
Φ1 E 4 (0), d2 E (0)} denote the set of optimal param- eters of CDVI.
Theorem 3.2.2 (Informal; CDVI optimality). If ( * Ø, V(g1, (22) E ( Ton (z|ar, y. 0) Xx he (y(y, z 2) .2 (2|0, y, 1) and qoi do not have issues in proposition 3.1 on larger support of
Definition 3.2 (Alternative Parameterization of qa)-
The censor-dependent variational distribution Qe1.02 can be parameterized as
401.02 (z| 8) == qoi (z|æx y) 402 (z 2)
(11)
The complete formal version included in Appendix A.2 and its proof can be found in Appendix B.3. In short, Thm 3.2.2 highlights that the cepeoddd structure in Lo eliminates the problematic constraint Φi = Φ2 in vanilla VI.
It is important note that, under this parameterization, the optimal &1 and depend the value of 0 and are not mutually independent. See Thm.3.2.2 for more discussion. Consequently, in practice, either Φ1 or Φ2 is a partition of network parameters, which are not trained separately on all event or censored observations. This choice parameteri- zation improves clarity in the following formulations.
To conclude, we have shown that the vanilla VI framework described in Ranganath et al. (2016); Xiu et al. (2020); Nag- pal et al. (2021a); Apellániz et al. (2024) is insufficient and arguably inappropriate for LVSM. Without hindering the
5

---

Censor Dependent Variational Inference
M-estimation of 0 and the expressiveness of latent survival models, have shown the importance of the censoring mechanism and have introduced the censor-dependent struc- ture for optimal VI in LVSM.
y
x
z
dense net
Ps (m, z)
p
N
q
4. Methods
N M(p diag(
In this section, we discuss the implementations of amor- tized CDVI in VAE-based LVSMs, sharing insights into its optimization and CDVI augmentation techniques.
(a) The encoder
(b) The decoder
4.1. Censor-dependent Conditional VAE
Figure 4: Generative graph of CD-CVAE.
standard normal distribution, we have
aELBO-C Oo
Eg
Σ 2:0
h(Ts)
Z
where ú (y p p,( 2)) a is the standardized y and h() is standard normal hazard function.
Ui
θ
U
Cj
The proof of Prop.4.2 is given in Appendix B.4.
Vi
Ui
4.3. Augmented CDVI and the Implementations
i 1, 2, ...y!
1,2,
Next, introduce two variants of proposed model, namely IS and DVI, incorporating established augmented VI techniques. We formulate the corresponding log ligliiklihood estimators so that the corresponding ELBO can be easily obtained as forms of their expectations.
(a) Vanilla CVAE
(b) Censor-dependent CVAE
Figure 3: Implementations Vanilla VI and CDVI.
First, we propose the Censor-dependent Conditional VAE (CD-CVAB) that estimates parameters 0, Φ as weights of neural networks. As illustrated in Fig.3, proposed CDVI implementation uniquely incorporates both and the event indicator as input of the encoder.
Definition 4.3.1 (The Importance Sampling Variant (IS)). Following Definition 3.2, the unbiased Monte Carlo estima- tors of likelihood fo (z)) So(y/xz) are defined
1 m
1 k Σ S1(2)),
Fig.4 illustrates the structure of the encoder and decoder. Our proposed decoder adopts a novel latent V-structure and employs both Gaussian and Gumbel-minimun (Gumbel, 1958) distribution families for e, which can be interpreted as infinite LogNormal Weibull mixture survival re- gression on positive survival time. We note that the decoder parameter 0 is decomposed as o) in Fbg.4b.
fm
f((z), S.
(14)
where z; and žj are independent samples from qor
suming 8 = 1,0, respectively, and m and l denotes the corresponding sample size.
Similarly, (14) defines a general i. == log(Se 51-8) for L(0). Computing its expectation allows us to generalize (13) to ELBO-C and (9) to B(m, k) defined below.
4.2. Training Strategy of Decoder Variance
Censored labels limit the flexibility of CD-CVAE's training procedure, restricting the use of advanced strategies avail- able to standard CVAE models. As emphasized in Prop. 4.2, a dual-step algorithm that updates a separately, as seen in Rybkin al. (2021) and Liu Wang (2025), not applica- ble, since cannot be estimated in closed form.
For completeness, we establish results about the properties of i providing deeper insights into augmented CDVI in both the finite-sample setting and the asymptotic regime as m, k + co. See detailed proofs in Appendix B.5 to B.7 Theorem 4.3.1 (Monotonicity of Inference Gaps). Given any 0, 6, for any m E N k E N+, B(1,1) > B(m, k) = L()) ELBOO-Cm,
Proposition 4.2 (No closed form update of (). Given the dataset (æ,, Hk, 8,) and decoder mean Ç, encoder parameters 01.02, the optimum of a by @FLBO-C(O.S.0) has no closed-form solution. particular, if follows a
2 max ((m,, 16 1), Bmm + 1,))
do
> B(m 11 2 lim B(m', K²) mm 36
6

---

Censor Dependent Variational Inference
The dependency of (mm, k) on model parameters 0, Φ is omitted; (1,1)) 1) is equal the gap of (13) in Thm 4.3.1.
S4 = exp(
(18)
where we define 0g and B2 the corresponding sample variances of (z+()))) and (Si(zi)) e.g., dr == M- - fm)
Thm 4.3.1 generalizes the well-known property of Burda et al. (2015) to labeled datasets with censoring, We prove that the generalized inference gap Bmm, k) is monotonic in both size m and k. In other words, ELBO-Cm,t yields a smaller inference gap for any m > k 1 given a fixed 0, Φ1. Φ2. which vanishes as m, k + 0o. That said, Thm 4.3.1 holds for any choice of Φ1, 222, including the case Φ1 = 22 imposed by vanilla VI, as in Xiu et al. (2020).
Inspired from Nowozin (2018), we prove in Appendix A.4 that the Delta method (Teh et al., 2006) variant Lim enjoys less asymptotic bias of L(f) compared to (14), if m, k are sufficiently large. For practical implementation, the Log- SumExp (Log-Softmax) trick used compute the bias term stably is detailed in Appendix C.5.
Theorem 4.3.2 (Seff-normalized Importance Sampling). Let (m))) (2(k) be the augmented variational distribu- tion, and Pi (m)) (2kk) be the augmented posterior distri- bution, defined as follows:
5. Experiments
The additional details of experiments are in Appendix C.
(m) = f. II.h to (z.)z.)) z. v). Q (m) = fmmm) (2) = S1 IL; aos (ox (2) v), (k) = J2 (
5.1. Evaluation Metries
Concordance index (Harrell et al., 1982; Uno et al., 2011) measures the effectiveness of a discriminative model in rank- ing survival times correctly. Specifically, it assesses whether the model assigns a shorter predicted time to the event, or more generally, lower survival probability S(()ax)) at test time t, for subject with features H; who experienced the event at time ye, compared to a subject with features 2, who survived longer. Due to censoring, only comparable pairs Yi I yj, 6, = are considered. Formally, the C-index is defined as:
(i) 2z J1 (m), (kk xzm Jz(m).
(15)
Then, given any x. Us 8,
- Eq,) [mm
(16)
Thm 4.3.2 extended and corrected the results from Domke & Sheldon (2018), formulating the inference gaps of aug- mented CDVI as the KL divergence. This result generalizes the established connection of norf-norma importance sampling (SNIS) to CDVI. For example, fi/ fm acts form of seff-normalization. However, we note that it does not enable a direct comparison between B(m, k) and B1,, 1), since the expectation is taken over Q1 and Q2- Detailed discussion can be found in Appendix B.6.
C(() = P(5(())) < $(1)2z)) De S yリ, 5, 1).
We evaluate the trained models by calculating the average C-index over ten quantiles, ranging from 10th 100 quantile in increments of 10, of event test times.
Brier score (Brier, 1950; Graf et al., 1999) is a squared pre- diction error reweighted by Inverse Probability of Censoring Weighting (IPCW), designed to assess both calibration and discrimination (Haider et al., 2020).
Theorem 4.3.3 (Informal; Consistency of estimators). Under some moment assumptions, for m - 0o, k -> oc, the variance of Lmm goes to zero, and thus Link & is a biased yet consistent estimator of L(B), i.e., for any { >>0 O,
"1 st, =1) (0- S(t(a,)))
n
Bis(t)
lim Pa((i - L(0)= > 5) = 0. m,K-000
Sc(ve)
S(t(a))),))
+ Lyy > 1) Sc(1)
A formal version is provided in Appendix A.3. Despite that Thm 4.3.1 has shown a vanishing bias of f.m,k. Thm 4.3.3 quantifies the asymptotic behavior of its variance, thereby establishing its consistency. This result is extended from Nowozin (2018), enhancing CDVI under ideal assumptions with theoretical guarantees. Thm 4.3.3 also enables the following tff for a smaller asymptotic bias.
where Sc(.) is the estimated survival distribution of the censoring random variable C. We evaluate the Brier score at the 75th quantile of event time on the test dataset.
dime-dependent C-index (Antolini et al., 2005) considers a more limited yet practical set of comparable pairs compared to the conventional C-iinde,, where selected subjects who developed the event earlier can't survive longer than the event horizon t. Formally, it is defined
Definition 4.3.2 (The Delta Method Variant (DVI)). Following Definition 3.2, A biased variant of Definition 4.3.1 is defined as
jm = ===xpfáz//22n) f)...
(17)
(1)(1)== = P(Š a.)) S S(t)ar)) lys S Uj. 6, = 1,9; S St).
7

---

Censor Dependent Variational Inference
Following conventions, we set the event horizon at the 75th quantile of the event time, and we compute it using IPCW and truncations, aiming to obtain unbiased estimate of 24 < uj by giving more weight to test samples with similar features that are not censored.
In extreme censoring scenarios with Y manually set to U or C, CD-CVAE performs identically CVAE, consistent with Remark 3.2. Interestingly, all models perform con- siderably better in the all-event scenario compared to the all-censoring scenario, and neither IS nor DVI yields sig- nificant performance improvements. Although learning a data-independent distribution of C should be simpler, such large discrepancy between these two extreme cases may imply that the amortization effect (Cremer et al., 2018) can dominate the inference gap. This observation highlights potential opportunities for practical inference improvements of amortized CDVI.
5.2. Inference Optimality on Simulated Datasets
Table 1: Summary table for simulated datasets (SDI-SD6). Sample size for each dataset is 10, 000. Event/Censored time refers to sample statistics of Y. The generated samples of U is independently sampled across each datasets. The starting point of Gibbs sampling is fixed at z O).
During the experiment, we also found that CD-CVAE mod- els could converge to various local optima with nearly the same inference gap, while having different ratios of E-KL and C-KL. This observation implies a unique trade-off in the amortization CDVI, i.e., the censor/event KL trade-off.
Summary
SDI
SD2
SD3
SD4
SDS
SD6
Censor rite
0%
5%
20%
30%
50%
100%
Population mean pee
16.00
8.50
5.50
0.00
16.00
Censored time mean
2.99
-0.11
-1.51
-4.49
16.18
Event time median
1.43
1.29
0.41
-0.30
-3.13
Event min
-12.64
-15885
-22.47
-22.59
-24.28
Event time max
21.27
21.60
17.49
18.45
14.29
5.3. Time-to-event Modeling on Benchmark Datasets
Firstly, we investigate whether amortized CDVI can prac- tically reduce the inference gaps compared to the vanilla VI. Table I provides details of 6 simulated datasets. In the simulation process, we use Gibbs sampling, where the true posterior is known and predefined. Both P(Z Y, = 1) and XY, 0) are set to normal distributions, which enable the closed-form computation of the inference gaps. We vary the mean of censoring time Pdc to generate datasets with different censoring rates, in which follows an inde- pendent normal distribution. The values of the censoring rate are rounded, with an error of 1%
Table Summary table for benchmark clinical datasets. ō|ō refers to the average event/censored survival times after applying log transformation.
Dataset
Size
Censored
Dim(X)
Ø = 1
Ø =
0
SUPPORT
9104
2904
14
6.17
6.97
FLCHAIN
6524
4662
8.20
8.37
NWTCO
4028
3457
6
7.73
7.86
METABRIC
1980
854
7.99
8.14
WHAS
1638
948
6.95
7.17
GBSG
1546
965
3.80
4.18
PBC
418
257
17
4.16
4.32
Table 2: Variational inferences on simulated datasets. E- KL/C-KL: the average KL. divergence between the encoder and true posterior of all events/censoring observations in the dataset. Lower is better. We set m
Lastly, we present comprehensive evaluation of time-to- event modeling performance, comparing CD-CVAE with state-of-the-art models. Table 3 summarizes the real-world datasets. These models include Cox-PH (Cox, 1972), Deep- Surv (Katzman et al., 2018), Deep Survival Machine (DSM) (Nagpal et al., 2021a), Random Survival Forest (RSF) (Ish- waran et al., 2008), and Deep Cox Mixture (DCM) (Nagpal et al., 2021b). All models were implemented using the Python package by Nagpal et al. (2022), and our implemen- tation follows its API for ease of reproducibility.
10.
CD-CVAE C-KL
CVAE E-KL
CD-CVAE" C-KL
CD-CVAE E-KL
Data
E-KL
C-KL
E-KL
C-KL
SD1
1.65
1.65
1.53
1.56
SD2
1.66
1.93
1.75
2.70
1.64
2.17
1.66
2.55
SD3
2,38
3.13
2.79
3.18
2.23
3.21
2.17
3.13
SD4
2.88
3.89
3.45
4.04
2.64
4.09
2.29
3.60
SD5
4.45
5.55
5.42
5.86
4.11
5.56
3.89
5.51
SD6
.0871
.0871
.0862
.0848
Table 4 and 5 highlight CD-CVAE's performance on best C, Ctd. and the Brier score metrics. See Appendix D.2 for full evaluations of the variant models with more baselines (Apellániz et al., 2024) and averaged metrics.
As shown in Table 2, our proposed CD-CVAE significantly reduces the average KL divergence between the learned posterior and the true posterior in both event and censoring subsets, resulting in a smaller inference gap, which is a weighted sum of these two metrics. Leveraging VI improve- ment techniques (IS, DVD,, CD-CVAE further reduces the inference gap across various settings of censoring. In Fig. we plot the marginalized learned go from CVAE and CD- CVAE against the true posterior on SD4 For completeness, time-to-event modeling performance on SD3-SD5 datasets provided in Appendix D.I.
While the variants of CD-CVAE do not exhibit substantial improvements in time-to-event modeling performance, proposed models generally outperform most of the state- of-the-art models. RSF and DCM are notably competi- tive in computation efficiency and hyper-parameter tuning. Nonetheless, Cox-PH with a l2 regularization performs com- parably on many benchmark datasets.
8

---

Censor Dependent Variational Inference
Table 4: Comparisons of CD-CVAE on benchmark datasets. The best model is selected by cross-validated C-index. The experiments are repeated five times using the same random seeds, with a vrain-validation-tee split ratio of 0.6, 0.2, 0.2. The highest metrics on the test dataset reported. Higher better: random guessing has a value of 0.5 and means all comparable pairs are perfected ranked.
SUPPORT
FLCHAIN
NWTCO
METABRIC
WHAS
GBSG
PBC
Model
C
Ctd
C
Ctd
C
Ctd
C
Ctd
C
Ctdd
C
Ctdd
C
Ctd
CoxPH
0.666
0.668
0.789
0.789
0.689
0.703
0.641
0.644
0.781
0.782
0.682
0.689
0.848
0.848
DeepSurv
0.648
0.649
0.780
0.805
0.674
0.741
0.664
0.676
0.786
0.762
0.609
0.618
0.855
0.852
DSM
0.666
0.674
0.801
0.802
0.706
0.694
0.666
0.669
0.811
0.805
0.615
0.663
0.862
0.869
RSF
0.683
0.655
0.768
0.793
0.677
0.726
0.686
0.684
0.808
0.811
0.706
0.731
0.857
0.867
DCM
0.682
0,676
0.788
0,803
0.680
0.736
0.689
0.691
0.803
0.811
0.625
0.637
0,866
0.865
CD-CVAE
0.685
0.678
0.811
0.804
0.708
0.751
0.681
0.675
0.868
0.812
0.706
0.702
0.863
0.865
Table 5: Comparisons of CD-CVAE in Brier Scores. The best model is selected based on the cross-validated Brier score. Experiments are repeated five times with the same random seeds, reporting the lowest test metric. Lower is better.
categories: (1) expanding the set of values for which opti- mal VI is attainable, i.e., increasing the support of 0 where min; B(0, x) = 0, for example by using more expressive variational families (Ranganath al., 2014; Kingma et al., 2016; Cremer et al., 201 (018); (2) modifying the variational bound to reduce min B(f, 6) for general 6, for example by going beyond Kl--divergence and using the x² a-, or more generally, f-divergence (Dieng et al., 2017: Li & Turner, 2016; Wan et al., 2020). (3) adopting training techniques to improve latent disentanglement (Higgins et al., 2017), improve numeric stability (Rybkin et al., 2021), to avoid posterior collapse (Fu et al., 2019), or to better incorporate label information (Joy et al., 2021). These works require nontrivial adaptation for censoring in survival analysis. As contributions, our criticism on vanilla VI, the extensions of IS and DVI on CDVI, and the discussion on decoder variance fall under each type, respectively.
Model
SUPPORT
FLCHAIN
NWTCO
MTBC
WHAS
GBSG
PBC
CoxPH
0.216
0.121
0.087
0.214
0.174
0.222
0.125
DeepSurv
0.212
0.115
0.078
0.230
0.198
0.242
0.131
DSM
0.235
0.078
0.175
0.242
0.128
RSF
0.224
0.120
0.077
0.218
0.162
0.217
0.119
DCM
0.217
0,075
0.216
0.171
0.229
CD-CVAE
0.218
0.110
0.075
0.203
0.168
0.218
0.124
6. Related Work
Deep Learning in Survival analysis. Machine learning and deep learning techniques for survival analysis are not limited to LVSM. Faraggi & Simon (1995) introduced the first neural- -wwtwonkwwww-eeeee-ee based Cox regression model, allowing nonlinear relationships between covariates. A modern yet similar one is DeepSurv (Katzman et al., 2018). Deep Cox Mixture (Nagpal et al., 202 extends this idea to finite mix- ture models, but all these Coxian models rely on the propor- tional hazards (PH) assumption, which results in separated survival functions (Antolini et al., 2005) and may be unre- alistic. A famous nonparametric tree ensemble approach, Random Survival Forest (Ishwaran et al., 2008), builds mul- tiple decision trees to model the cumulative hazard function, leveraging the Nelson-Aale estimator (Aalen, 1978). That said, hazard function estimation for discrete time-to-event can also be framed as a series of binary classification prob- lems, which can be solved by blxck-box methods via various network architectures. DeepHit (Lee et al., 2018) uses a sim- ple shared network to model competing risks, while RNN- (Giunchiglia et al., 2018) and Transformer-based (Hu et al., 2021) structures capture sequential relationships in time- specific predictions. These methods often require additional techniques to mitigate overfitting.
Variational Inference for Other Tasks. Variational meth- ods in survival analysis are not limited to time-to-event modeling. One unsupervised task is identifying potential sub- nub-populations, providing valuable insights for treatment recommendations and clinical decision decision-making (Chapfuwa et al., 2020; Franco et al., 2021; Manduchi et al., 2022; Cui et al., 2024; Jiang et al., 2024). These clustering models, if used as intermediate step of time-to-event modeling, can be seen as restricted I.SM, often in D-separation latent structure. While a restrictive approach can help prevent over- fitting, our criticism remains valid: the objective of VI in unsupervised tasks can be misaligned with M-estimation of the time-to-event distribution, undermining the performance of survival time prediction.
7. Conclusion
To the best of our knowledge, this paper has represented the first comprehensive study of variational methods for latent variable survival models. It delivers an in-depth analysis of variational inference optimality and offers valuable practical insights. The superiority of our proposed models validates a pioneering paradigm.
Inference Optimality without Censoring. Improving VI in latent variable models has been extensively studied for both labeled and unlabeled datasets. We summarize existing strategies for improving the estimate H* into three broad

---

Censor Dependent Variational Inference
Impact Statement
Cremer, C., Li, X., and Duvenaud, D. Inference subopti- mality in variational autoencoders. In International Con- ference Machine Learning, pp. 1078-1086. PMLR, 2018.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which feel must be specifically highlighted here.
Cui, C.. Tang, Y., and Zhang, W. Deep contrastive survival analysis with dual-view clustering. Electron- ics, 13(24), 2024. ISSN 2079-9292. doi: 10.3390/
Acknowledgments
This work was supported in part by NSF Grant No. SES- 2316428, awarded to Xiao Wang.
Dieng, A. B., Tran, D., Ranganath, R., Paisley, J., and Blei, D. M. The x-divergence for approximate inference. In Neural Information Processing Systems, 2017.
References
Aalen, O. Nonparametric inference for a family of counting processes. The Annals of Statistics, pp. 701-726, 1978.
Domke, J. and Sheldon, D. R. Importance weighting and variational inference. Advances in Neural Information Processing Systems, volume 31, 2018.
Angelova, J. A. On moments of sample mean and variance. Int. J. Pure Appl. Math, 79(1):67-85 85, 2012.
Faraggi, D. and Simon, R. A neural network model for survival data. Statistics in medicine, 14(1):73-82, 1995.
Antolini, I.., Boracchi, P. and Biganzoli, E. A time- dependent discrimination index for survival data. Statis- tics in medicine, 24(24):3927-3944 2005.
Franco, E. F., Rana, P., Cruz, A., Calderon, V. V., Azevedo, V., Ramos, R. T., and Ghosh, P. Performance compar- ison of deep learning autoencoders for cancer subtype detection using multi-omics data. Cancers, 13(9):2013, 2021.
Apellániz, P. A., Parras, J., and Zazo, S. Leveraging the variational bayes autoencoder for survival analysis. Sci- entific Reports, 14(1):24567, Oct 2024. ISSN 2045-2322.
Frees, E. W. Regression modeling with actuarial and finan- cial applications. Cambridge University Press, 2009.
doi:
Baricz, Á. Mills' ratio: Monotonicity patterns and func- tional inequalities. Journal of Mathematical Analysis and Applications, 340(2):1362-1377) 2008
Fu, H., Li, C., Liu, X., Gao, J., Celikyilmaz, A., and Carin, Cyclical annealing schedule: A simple approach to mitigating KI. vanishing. In Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers). Association for Computational Linguistics, 2019.
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A. M., Józefowicz, R., and Bengio, S. Generating sentences from a continuous space. In CONI., pp. 10-21. ACL, 2016. ISBN 978-1-9456626-1997
Brier, G. W. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
Giunchiglia, E., Nemchenko, A., and van der Schaar, M. Rnn-surv: A deep recurrent model for survival analysis. In Artificial Neural Networks and Machine Learning- ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part III 27, pp. 23-32. Springer, 2018.
Brown, L. D. Fundamentals of statistical exponential fami- lies: with applications in statistical decision theory. Ims, 1986.
Burda, Y., Grosse, R., and Salakhutdinov, R. Importance weighted autoencoders. arXiv preprint 15X.v05509.005 2015.
Graf, E., Schmoor, C., Sauerbrei, W., and Schumacher, M. Assessment and comparison of prognostic classification schemes for survival data. Statistics in medicine, 18( 18):2529-2544 1999.
Chapfuwa, P., Li, C., Mehta, N., Carin, L., and Henao, R. Survival cluster analysis. In Proceedings of the ACM Conference on Health, Inference, and Learning, pp. 60- 68, 2020.
Gumbel, J. Statistics of Extremes. Columbia University Press, 1958.
Haider, H., Hochn, B., Davis, S., and Greiner, R. Effective ways to build and evaluate individual survival distribu- tions. Journal of Machine Learning Research, 21(85): 1-63, 2020.
Cox, D. R. Regression models and life-tables. Journal of the Royal Statistical Society: Series (Methodological), 34(2):187-203 1972.
10

---

Censor Dependent Variational Inference
Harrell, F. E., Califf, R. M., Pryor, D. B., Lee, K. L., and Rosati, R. A. Evaluating the yield of medical tests. Jama, 247((8))254325446 1982.
Kvamme, H., Borgan, Ø., and Scheel, 1. Time-to-event pre- diction with neural networks and cox regression. Journal of machine learning research, 20(129):1-30, 2019.
Harrell, F. E. et al. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis, volume 608. Springer, 2001.
Lagakos, S. W. General right censoring and its impact on the analysis of survival data. Biometrics, pp. 139-156, 1979.
Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta- VAE: Learning basic visual concepts with a constrained variational framework. International Conference on Learning Representations, 2017.
Lawless, J. F. Statistical Models and Methods for Lifetime Data. Wiley, 2nd edition, 2003. ISBN 9780471372155
Lee, C., Zame, W., Yoon, J., and Van Der Schaar, M. Deep- hit: A deep learning approach to survival analysis with competing risks. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018
Hu. S., Fridgeirsson, E., van Wingen, G., and Welling, M. Transformer- based deep survival analysis. In Survival Prediction-Algorithms Challenges and Applications, pp. 132-148. PMLR, 2021.
Li, Y. and Turner, R. E. Rényi divergence variational infer- ence. Advances in neural information processing systems, 29, 2016.
Ishwaran, H., Kogalur, U. B., Blackstone, E. H., and Lauer, M. S. Random survival forests. 2008.
Liu, C. and Wang, X. Doubly robust conditional VAE via decoder calibration: An implicit KL annealing approach. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
Jiang, L., Xu, C., Bai, Y., Liu, A., Gong, Y., Wang, Y.-P., and Deng, H.-W. Autosurv: interpretable deep learning framework for cancer survival analysis incorporating clin- ical and multi-omics data. NPJ precision oncology, 8(1):
Lucas, J., Tucker, G., Grosse, R., and Norouzi, M. Under- standing posterior collapse generative latent variable models. 2019.
4, 2024.
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. An introduction variational methods for graphical models. Machine learning, 37:183-233 1999.
Manduchi, L.,, Marcinkevičs, R., Massi, M. C., Weikert, T., Sauter, A., Gotta, V., Müller, T., Vasella, F,, Neidert, M. C., Pfister, M., Stieltjes, B., and Vogt, E. A deep variational approach clustering survival data. In Inter- national Conference on Learning Representations, 2022.
Joy, T., Schmon, S., Torr, P., N, S., and Rainforth, T. Cap- turing label characteristics in {vac}s. In International Conference on Learning Representations, 2021.
Miller, R. G. Least squares regression with censored data, Biometrika, 63(3):449-464, 1976.
Kalbfleisch, J. D. and Prentice, R. L. The statistical analysis failure time data. John Wiley & Sons, 2002.
Mitrinovic, D. Analytic inequalities, 1970.
Katzman, L., Shaham, U.. Cloninger, A., Bates, J., Jiang, T., and Kluger, Y. Deepsurv: personalized treatment rec- ommender system using cox proportional hazards deep neural network. BMC medical research methodology, 18: 1-12, 2018
Nagpal, C.,, Li, X., and Dubrawski, A. Deep survival ma- chines: Fully parametric survival regression and repre- sentation learning for censored data with competing risks. IEEE Journal of Biomedical and Health Informatics, 25 (8):3133-317) 2021a
Kingma, D. and Welling, M. Auto-Encodie Encoodnn Variational Bayes. In 2nd International Conference on Learning Representations, 2014.
Nagpal, C., Yadlowsky, S., Rostamzadeh, N., and Heller, K. Deep cox mixtures for survival regression. In Ma- chine Learning for Healthcare Conference, pp. 674-708. PMLR, 2021b.
Kingma, D. P., Mohamed,
S., Jimenez. Rezende, D., and Welling, M. Semi-supervised learning with deep genera- tive models. Advances in neural information processing systems, 27, 2014.
Nagpal, C., Potosnak, W., and Dubrawski, A. auton-survival: an open-source package for regression, counterfactual estimation, evaluation and phenotyping with censored time-to-ever data. arXiv preprint arXi::2204.07276 2022.
Kingma, D. P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M. Improved variational infer- ence with inverse autoregressive flow. Advances in neural information processing systems, 29, 2016.
Nelson, W. B. Applied life data analysis. John Wiley & Sons, 2005.

---

Censor Dependent Variational Inference
Nowozin, S. Debiasing evidence approximations: On importance-weighted autoencoders and jackknife varia- tional inference. In International conference on learning representations. 2018.
Zhang, L. Sample and sample variance: Their covari- ance and their (in) dependence. The American Statisfician, 6 61(2):19-160, 2007.
Pölsterl, S. scikit-survival: library for time-to-ene to-evett anal- ysis built on top of scikit-learn. Journal of Machine Learning Research, 21(212)1-6,, 2020.
Ranganath, R., Gerrish, S., and Blei, D. Black box varia- tional inference. Artificial intelligence and statistics, pp. 814-822. PMLR, 2014.
Ranganath, R., Perotte, A., Elhadad, N., and Blei, D. Deep survival analysis. In Machine Learning for Healthcare Conference, pp. 10. PMLR, 2016.
Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep gen- erative models. In International conference machine learning, pp. 1278-1286. PMLR, 2014.
Rybkin, O., Daniilidis, K., and Levine, S. Simple and ef- fective vae training with calibrated decoders. In Interna- tional Conference on Machine Learning, pp. 1189-9189. PMLR, 2021.
Sohn, K., Lee, H., and Yan, X. Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems, volume 28, 2015.
Teh, Y. Newman, D., and Welling, M. A collapsed vari- ational bayesian inference algorithm for latent dirichlet allocation. Advances in neural information processing systems, 19, 2006.
Uno, H.. Cai, T., Pencina, M. J., D' Agostino, R. B., and Wei, L.-J. On the c-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in medicine, 30(10): 101- 2011.
Wan, N., Li, D., and Hovakimyan, N. F-divergence varia- tional inference. Advances in neural information process- ing systems, 33:1 13:7770-17339, 2020.
Wang, Y., Blei, D., and Cunningham, Posterior collapse and latent variable non-identifiabiiity. In Advances in Neural Information Processing Systems, 2021.
Wiegrebe, S., Kopper, P., Sonabend, R., Bischl, B., and Bender, A. Deep learning for survival analysis: a review. Artificial Intelligence Review, 57(3):65, 2024.
Xiu, Z., Tao, C., and Henao, R. Variational learning of individual survival distributions. In Proceedings of the ACM Conference on Health, Inferecee and Learning, pp. 10-18 18, 2020.
12

---

Censor Dependent Variational Inference
A. Facts and Formal Theorems
A.1. Facts of Probability Theory
Why do we claim "CD-CVAE is interpretable as an infinite LogNormal or Weibull mixture survival regression on positive survival time"? But the choice of & that determines the location-scale family of po(ula, 2) is implemented as Normal or Gumbel-minimum distribution.
Answer: A Weibull (Lognormal) AFT of positive valued survival time T is log-linear regression assuming Gumbel-min (Gaussian) noise (Miller, 1976). In our setting, continuous time-to-event U is considered to be real-valued after the log- -egtrrnnformm of T..
Table 6: Connection between AFT and the degraded LVSM
choice of standard Gaussian standard Gumbel minimum
standarization
( x z)
linear degradation
degraded model w.r.t
v (a.a(
exp( - 3//² ) 2Tσ) exp 1 exp(ū))/e
po(a, 2) TTx
Log-normal AFT
Weibull AFT
A.2. Formal Theorem 3.2.2
We have the following Notations:
1) The product sets (0) = {(01, Ø2) Φ1 E (1), 0z E {2)))} denotes the set of optimal parameters of 46n.0 and
Pp (0 (0) * Φ denotes its support.
2) t (0) ((0, p) d E ( denotes the embedding of tu (0). The support Ogu -0 | & (0) +
Ø). For any 0, (0) C (0). Optimal goi.dz is degenerated to the optimal do in Vanilla if (Ø1, db2) E eeu (0).
Theorem 3.2.2 (Inference optimality of CDVI)
Following the assumptions. If p + Ø, then
(5) Constraint on optimal Φ1 and Φ2- V(o1 02) E @Dp(0), (=/a,y) x2 ho(ulo, 2) (2lor, y).
If 19p\0EE # 2, we have the following results.
(6) Striet better optimal o. 40 E Op)(EEu we have (0)(() C p()),, and more importantly, V(ő1, 02), Φ1
2, (577,65) E (
L(0) = ELBO-CC(,,
ELBO(A, é1, 62).
(7) Non-degraded location parameter. Uf X, 7 is a location-scale distribution parameterized by the location parameter
pa(z, z) and the deterministic scale parameter a, then for all E O there exists 21 * 22, 1s (x, 21) + pa(2 22) for almost all x 8KL[go p=(2]=)] + (1 (LLL pa [x)] 0.
(8) Lazy posterior free. V0 Opp/EEu v(01, 62) E pp)) such that
If= 1. x is assumed, Le., po(2|<) = p(2), optimal censor-dependent VI is posterior collapse free.
Theorem 3.2.2 demonstrates how the CDVI resolves the issues of vanilla VI. Claim (5) states that the optimal fer.0, captures the constraints on the parameters Φ1 and Φ2, preventing it from being reduced to the naive qa. As we show in Remark 3.2, the assumption of d1 = Φ2 is the root cause latent non-identifiability in Proposition 3.1. Claim (6) shows that the optimal 40, enjoys expanded support Op, enabling our d0,,0z to achieve VI optimality specific 0 values where vanilla VI would fail. To be specific, Claim (7) demonstrates that maintains the complexity and expressive power of the latent variable model Sa (y)ar). Consequently, Claim (8) shows that the optimal qo, or goz will not remain lazy or suffer from the posterior collapse issue.
Φ2
A.3. Formal Theorem 4.3.3
Following the definition 4.3.2, let a; := E[( f) and 3r E[(š, -S)* be the 7th central moments of unbiased estimators fm and Šg. Obviously, 0n 0,3 =
13

---

Censor Dependent Variational Inference
Lemma (Asymptotic bias of log f. and log Sy). I and og, is finite for all 2 1, then for oo,
Elllog fml = log f 2
Q2
+
03
302
+ o(m
(19)
m²
3
4
If SS and 8g are finite for all i 2 1, then for k - xc,
1 B2
B3
3/2
Eg log Sy] = log S
o(k-
(20)
2S
12
35
454
The expectation is taken over 21.m or 2i:k for any given æ, y, 0, Φ1 02.
Lemma I demonstrates the asymptotic bias of importance sampling induced loglikelihood estimators log fm and log Sk. which has an order of magnitude of m or k-1
Lemma (Asymptotic variance of log fmy and log St). I and as are finite for all 2 1, then for m + co.
1 a2
502
Vllog fm]
+ =0mm
(21)
m J
m
M
Similarly, if S and Bg are finite for all i 2 L, then for k + oc.
1 1/
B3
532
Vllog Sxl
+ olk: 3).
(22)
1 52
12
53
5
Recall that Lm,x:= 8log fm + (1-11) Lemma 3 (Asymptotic hias of bmm,)). Under the assumplion of Lemma for m, k + co,
81 log Sp.
lemma 1 & get the following.
1 02
1
03
30²
L(0)-ELBOL (LB--ELB-CCm. = -
m
"m
21
2
3
4
(23)
322
833
3
+ (1
5)1
) + o(m ?) + o(k-
2.2
252
1.2
353
4S
Lemma (Asymptotic variance of k.,,k).. Under the assumption of Lemma for m, k oc,
E2 (Z,, - [(()) =
a2 f2
1
a3
20a² + a²
)1 + (1-
82
B3
2032 + 38
} + o(m 3) + o(k: 2).
m
fz
4ft
1:
5
2
S3
4.4
(24)
Theorem 4.3.3 (Formal; Consistency of Lmm,)). Under the assumption in Lemma 1, for m + oo, k + for all 5 > 0,
lim P(Ii.m. - L(6) > {) = 0.
The proof is almost a direct result of Lemma and Lemma
A.4. Theorem for Delta method CDVI
We prove that the Delta method CDVI yields a smaller asymptotic inference gap/bias, as we mentioned after Definition 4.3.2. Following 17 and Bq.15 18, let Lm,x == 8 log f. + (1 1 8) log Sc. Theorem A.4 (less asymptotic bias of delta method CDVI).
Under the assumption in Lemma 1, for k + cc.
E
62
02
033
302
+ o(m 2),
E
32
32
1
83
o(k -2- 25t
382
+
2ml
(mm)
fm
m²
fz
2
2k(Sx)
2k.52
k2
53
and
203
302
2/3
382
L(0) - E2 blog [mm,e] =
51
)] + +1 (1-
(1-
}
+ omm + o 2)
m
3
4f
1.2
3.53
454
Compared with Lemma 3, the asymptotic inference gap/bias is reduced by one order magnitude of m and k.
14

---

Censor Dependent Variational Inference
B. Proofs
B.1. Proof for Proposition 3.1
Proof of (I); From Lemma 3.1 and assumption 1), for any optimal parameter o E Lv, we have
(= u) fo(ulæ (2) x)) So(ule, 2)po(=
which means for any
h x z) = fo =) 2) = To(u)=)/Sauula = (u((x))
Proof of (2): Since U is continuous, h() in the above equation can be replaced by f.F.S, due to the 1-1 relationship, e.g. huu)= log S(a) leading fo (ula,,) = fo(u)z)
Since fo(ula, 2) po(a, + x e, we claim that ,(x, 2) is independent of the value of Proof of (3): Also based on fo 2) = fo (u|2z) for any 2,
q ( az, u) fo(ula xe, z (z) fo(u po(=)
Proof of (4): If a V-structure latent graph is assumed, i.c., x x, then the prior (=(2x) = p(z).
Now, (4) can be drawn immediately from the conclusion of (3).
B.2. Proof for Theorem 3.2.1
In the section 2.1, we mentioned that the partial likelihood fo(y)ar)*So (y)==> although contains all the information of 0, is not a proper density. Here we further emphasize that the appropriate variational distribution cannot be discussed separately on the subspace DE and Do targeting distribution function fo(y)æ>) and (p(1)zz) in the vanilla VI framework (Xiu et al., 2020; Nagpal et al., 2021a), because leads to ignoring the information of 8.
Proof: Abusing the "density" notation p(y. 6, 2|æ) and p(y. 8|z) for the Radon-Nikodym derivative of P(Y, Z( and P(Y, I[X), the general variational bound defined in Domke & Sheldon (2018) is
ply, 8 ())
log p(y, 8|x)
E. log R)
R. log
R
bound
looseness
For simple non-augmented variational bound enabling Jensen's 's inequality, should be R(=) ply. 8, 2/z)
20(=)
tight "looseness" requires the KL divergence being zero, leading to optimal (o- = p(2|2r, y, 8), which the density of P(Z|X, Y, parameterized by both 0 from U and v from
Now we prove that both p(2|z, y, 8 1) and p(zly,& 0, x) will be independent of C and free from r,. Assuming that continuous U|X, C|X have the same support U, 2) conditional independent censoring, 3) independence between C and Z given X, 4) Fubini's theorem is applicable, have
p(u, 8- 1,
Pu.z(u, P(C E2) Po 2)p(z(=)) Pu
p y. = p(y. = 1|=)
1 E
Pu (y(
Reorganizing terms, we get a (z|zr, 8 = 1) p(2|2 5 = 1) = fo fo(l()) We use the similar proof for (0(2|00 y. 0 0). We note that the assumption of the same support is inadmissible, and we can also express 9 as follows
46 (2 Ir, y, )) qs- (2|2 8 1) q0- (2|2, v. 0)2-5
which leads to the notation of Definition 3.2.
Proof for Remark 3.2 follows naturally. The marginalized qo- (z| y) = p(=|=,,, 1) * P(š = 1|2, y) p 0) * P(5 0/., y), which will not equal p(2]2; x. y, 8 = 1) or p(2 xr, y, 5 = 0) unless one of P(8]2r 3) zero.
y.
15

---

Censor Dependent Variational Inference
B.3. Proof for Formal Theorem 3.2.2
Proof of (5); Using the above conclusion of q
fo 20) 2 (2|22, y, = 1) 2,) gor (z|x, y, 0) So (y)r)//(2(z)
Denoting 4o- (z|2, yr 1) by go; (z/2, and reorganizing the terms, we have
(0;(2/a,)) ho h (2/2,, (holul z) <= god (h)l,,)) z).
Yog
Proof of (6): The proof is trivial, following the definition of optimal 20- (=|2,,
Proof of (7): We have already proved in (I) that if 0 E O then fo(ulz, 2) = fo (ulzr (z) and also in (2) that if fo 2) is location-scan family, it leads to p2 2) independent 2. We now prove the reverse is also true: if po 2) is independent of 2, and I (ular, 2) the density of location-scale family, it leads fo(ula, 2) = fo (u(ar)); So (u/ar, =) = (p(u)a)), thus we have
9 (2|a,, y, = go- (z|2, v, 1) = po(z|z))
Equivalently, following the notation, we have oj = 62. Thus, oi oz E &zU, meaning that 0 E OgU.
Then we complete the proof by contrapositive.
Proof of (8): By non-negativity of KL divergence, the KL divergence zero if and only if the above equation in (7) holds true. Thus, it is direct result of (7). V-structure assumed, prior po p=(2x)) is replaced by p(z).
B.4. Proof for Proposition 4.2
Here, we prove that æ is standard normal or standard of 0ELBO-C/äg given the parameter of Ç, 01, Φ2. For notation clarity, we decompose 0 (C, o) where (c 2) the location parameter of the decoder, and its scale parameter.
distribution, there is no closed-form solution
Proof: Notice that KL divergence terms in ELBO-C do not involve and the expectation is taken over Jb1.d2- Given dataset {zi, Vi, 6.
OELBO-C
a log fo(u,lzi,2)
log Sy (r,, 2)
E(
Σ
Oo
Oo
1:6,=1
s:5,=0
Using chain rule and the density in Table 6, we have the following result: (1) If the decoder normal, these two terms can be expressed as
Ø aog (1 2)
Olog So 2))))
2 log 1 +b()))
aij
A(gi) y/
do
Og..
do
where X(s) is the hazard function of the standard normal distribution that 1) has no closed- form expression, 2) is convex, 3)
be bounded. One naive bound is A(s) st a tighter bound X(s) 3 v==48 for 0s provided by Baricz (2008)
via Mill's ratio Mitrinovic (1970)
(2) If the decoder is Gumbel-miniman (S(s) = exp(-expp) 10000-spp ex)(s))), these two terms can be expressed as
i log fo(y) 2., 2) do
a(ū, exp():)) 0g, 51 + 1 Hexp(1)
y
a llog Sp(ya(z,)2)
2 cxp(ūi)
Og. Oo
(1)
exp(1.)
do
Oyj
Neither of these expressions leads to a closed form solution of when
ØELBO.C
80
16

---

Censor Dependent Variational Inference
B.5. Proof for Theorem ??
The proof mainly follows (Burda al., 2015): are going to prove the monotonicity of ELBO-Cm.. instead of (mm, k).
Proof: Given m, k, let m', k' be any integers less than m, k, respectively. Denote the subset of index m' {di..... dne) C {1,2,3, ... m as a uniformly distributed subset of distinct indices where |mm]l = m'. Tx follows the same definition for 2, kj. For any bounded sequence of a1 .... Om
E 04g + ... + ai
01 + a2 + + am
m
Therefore,
ELBO-Cm.k = Ellog( fd k
51- 01 = 6E.... [lg[fmm) + (1 - 5)B.... [+]](
8 log f1(21) f22) +.. f((2m) + (1 d) [log(Sk]
111
f1(2t)) + f(((zt) +.. + f1 ) + -6)Es1 [lo Ss
8Bamm log Er,.'
mn
8E Zii Br [log f1(z))) + fi((z)) + + fi fi
11 + (1 8)E [log(Se)
ml
= 8E 8E [log mm + (1 - 5) (og(S))) [ = ELBO-Cm',k
Similarly, ELBO-Cm, m. ELBO-C mm,00 Thus,
ELBO-Cm,k 2 max (ILBOOOn- ELBO-Cm,N)) > min min (ELBO-Cm kk: ELBO-Cm, 2 ELBO-Cm: Z ELBO-C.
Here ELBO-Cm. S L(0) is ensured by Jensen's 's inequality and Fubini's theorem.
Assuming bounded f1; $1, the strong law of large numbers: for co.fm -3 E[f] f.
Similar results apply to Sg. The results imply convergence in expectation
lim 100000
ELBO-Cm,t = L(0).
Then, using the definition of B(m, k)) == L(0) ELBO-Cm..s we complete the proof by reversing the inequality.
B.6. Proof for Theorem 4.3.2
This theorem is a corrected extension of Theorem I in Domke & Sheldon (2018). Our proof follows a similar structure, but we first highlight the mistake the original proof in Theorem 1 in Domke & Sheldon (2018). In their original proof, the definition of Eq.5 is not consistent in Theorem 1, where the expectation in Eq.5 is taken over 21, 21,..2m from (as while the expectation in Theorem 1 is taken over the augmented variational distribution, shown on Page 15. Thus, when generalizing their results, must warn the reader that the KL divergence term does not correspond to the inference gap defined in Thm. 4.3.1
Proof: Some basic facts from the definition: 1) (1) = go, (z|z,,). = 4o2 (2|a, y): (1) = foly, 2|x); 3) P2(1) = Sp(y) =)oo((() 4) For any m > o.f (m))d log (y/=x) 5) For any k > 0.f J2(k)danı log S(y(z))
Since p(r) = Ey(2) log pla.z) s()) + [L(((2)(p(2] we have
Ji (m)
J2(m) (L(Q log S(ylz) (B)(m) log Q2(m) + (L((2(())]
log f(ylz) (m) log (m)
By definition, we have
EQ (m) log (1(m)
J [i(m)
(Rq (m) flog
1 Blg f.). JR (m(m)//m
(m)
17

---

Censor Dependent Variational Inference
Similar results for (2(m) can be obtained. Adding these two equations with multiplication of é or 6, we have
(())= log fyy x) (1 -)
6) log S(ylz) Eg.. $
The above equation completes the proof. We highlight that the mentioned mistake limits further interpretation. It is easy to see that
B(1, = XIC..//CCCIPPP/)////////////00)
=
However, cannot subtract these two KL divergences by the chain rule of KL divergence in Domke & Sheldon (2018). Since L(0) - EQq.q Qu llog f S does not correspond to (m,, k), the subtraction does not give any meaningful interpretation.
B.7. Proof for Lemma 1-6 and Formal Theorem 4.3.3
The chain of proof' follows the same structure (Nowozin, 2018), with minor corrections and better consistency of notations.
B.7.1. PROOF OF LEMMA 1
Given the assumptions, which are sufficient for Fubini's theorem to apply, the Taylor expansion of [E fl at log f is given
as
(-1) iʃi
Ellog fmn] = Ellog[ - (fm - [n)] log f
El(mm - f) log /
From Theorem of Angelova (2012), using the definition of a, 3r we can get the relationship between af and ag:
a2 =
022
033
a3
a
m² a + o(m 3).
m
m
By substituting af with ag we have
Ellog sm]] = =lo=]]og1/ ʃ
02
+
03
az) 4f4 m.
1
+ a0mm=)
2 m
3f3 m
After rearrangement, we complete the proof for Eloog fun) By applying the same proof as for B(log Si.,. we complete the whole proof. We denote Bllog fm. = 2/2 a2
3/3 m² + 1fa ( (n a²) and similarly for B B(log Sx).
1
B.7.2. PROOF OF LEMMA 2
By the definition of variance and using the same expansion on both log fm and its expectation log f. have
(-1) (E(( 53 (fm (11-)))) ))
Vllog [m] B((og f. E Elog pm = E
pm if
By expanding the above equation the third order, have
Vilog fmi a22 1 f2 fz (a's a()))) + 3f4
2 (a's a(aa)) + 4f4 (a's - (a))²) 3 (a's asagg) + 9 (ajs - (ag))))
1
By substituting af with as, we complete the proof for V(log fm]. By applying the same proof as for Vilog Se], we complete the whole proof.
18

---

Censor Dependent Variational Inference
B.7.3. PROOF OF LEMMA 3
Notice that ò is binary valued and finite, thus for m - oo and l: oo, where the sequence of limitation doesn't matter, we have
ELBO-Cm.t E[Lm,k]]
5Ellog fm] + (1 - [)g Sk]
1
1
302
6 f + (1 5) log S - 61
}]
m
2
m
3f
4
32
1
3
+ (1 - 5)1 k 2.5
) + o(m 2 + o(k: 2).
ka
353
48
By substituting L(0) log J + (1 - 6) log S. we complete the proof.
B.7.4. PROOF OF LEMMA 4
Notice that 1) o1; Φ here optimally constrained in Claim (5) of Theorem 3.2.2, 2) the expectation w.r. 21:m and 21:k can be separated due to independence. 3) L(0) is not function of and 4) 8 = 6. For m - co, k oo.
[[[Lm, [(0)) = E= (5 f' - 8 log /) ) + EZI. (11 (1-
21
5) log Ss - (1 - 6) log 5) 2)
}
5B[(log f... Eloog./ J. + Bllog fml log 1 + (1 ()og Sy. - Ellog Scc + Eloog Szl - log 5)
= 5Vilog fm]
+ + log fmm)²² + (1 - 6)Vilog Szl + (1 - 6) 1),
6 m f
1 01
m
03
f3
50g
+
m 2
02 ) + (1 -11-6]]
52
8/
12
1
53
333
5
$
1:25
1 82 )²1 + o(m 2) + o(k=2)
8
1 022
03
200a
(1 5)
82
1
3/
203a
32
}} + o(m 2) olk 3).
m
f
m
4ft
k 52
1
53
454
B.7.5. PROOF OF FORMAL THEOREM 4.3.3
Proof:
P([Lm,k L(0) 2 2) P([Lm, E[ k.] + E[LmmA] L(0) +)
S P((Lmmk k - 18 Lmm x]l + [B[Lm, - L(0) 2
(25)
< P((Lmmk Em E/2) + P((E(Lm. L(O)) 2 e/2).
Notice that [|m[Lm, x] L(0) is not random, and based on the result of Lemma 3, for sufficiently large mi, ki, we have [E[[_m.]] - L(0)| E/2, regardless of the value of 8. This proves that 0 as m, Ie
By Chebyshev's inequalities,
4 V(mm
P( - R[(im, 2 E/2) <
Based on the result of Lemma 4, have $ Viimm) 0 m, + oo, regardless of the value of 8, This proves that ① 0 as m,k k - co.
Together, we establish the convergence in probability and hence consistency of Lm.k.
B.7.6. PROOF OF THEOREM A.4
Following Definition 4.3.2, we consider the induced log- ligellhoo estimator
i 5og i' + (1 - 5) log St.
19

---

Censor Dependent Variational Inference
Recall lemma 1, we have proved that
Ellog fm] log f
1 a2 m 2f
ag
3az
o(m 2).
m
3
4
By Definition 4.3.2, we have defined the log-likelihood estimator:
82
log fm =- log fm 02 2mfm.
log S4 log Sy.
(26)
2k55%
where 02, and f are the sample variance and sample mean of fi (z:); 32, and S are the sample variance and sample mean of S1(2)). Here, we show how this extra term in log fm or log Sm. leads to the cancellation of the leading terms in the bias, e.g.,
02 2-
Proof: The is considered as a function of g(z, y) in the form of x/y²
fm
We expand its second-order Taylor expansion at (a2, f):
62 9(22 (0v - a2);; + (fm f)) 2
1 + (ár 02)
202 (Jm -s-
2
(āz - a2) (á2--x2)fmm ʃ) + 6a2
f3 (ā2
2 (Sm - 1)2
f2.
f p
Notice that [[fm]
a22
Taking expectation on both sides and after the rearrangement, we have
à2
2
302 Bl(f.. -1)
E
El
az)(fm f]+
fz
f2
f3
fa
Using the results Zhang (2007), we have El((2 - a2) - f]] = as/m and E((fm. - f) = a2/m. Thus, by substituting.
E
022
02
1
203
303
+ o(m
f2
m
f
f
Finally,
à2
log f Ellog fml log
Ellog fm]
E
2m
f
02
03
3a
1
02
203
3a
}]
+
o(m -2)
m 2
m
353
4
2m
g2
m
f3
f
203
303
+ o(m -2)
m²
3
4f
By applying the same proof as for Ellog St), and f we complete the whole proof.
20

---

Censor Dependent Variational Inference
C. Details of Experiments
C.1. Experimental Setup
The experiments are on Python 3.9 with Pytorch on the Windows 11 system. GPU is not required.
C.2. Simulated dataset (SD1-SD6)
The dimension of 2, U, e is 1. The latent dimension of 2 (21, 22) 2. The Gibbs sampling process is designed as follows:
First, in the P(X,Y,I12 step, we have sample of X,U, C.Y. as follows
The prior of x: p(x) (11, 1), which is independent of Z.
Given 2, P(U X, Z) ~ N(u(z, o?) where μlz, z) * + x 22-
Given Z, PC Z) ~ N(p,, e²). The mean for SD1-SD6 is reported in the Table 1, which controls the rate of censoring.
We compare the sampled u, c to get y and the event indicator 6.
Second, in the P,Z|X,Y, step, we define the distribution follows:
For 5 = 0, P(Z|x,, is normal distributed with mean μı = (26 ()3 exp(x + v). exp((x + y)).
Covariance is fixed as identity matrix for both cases.
Then we start the simulation at = = (0, 0) and burn the first 10k observations.
C.3. Hyper-parameterr training CD-CVAE and the variants
The details the models can be found the model folder via the repository link. In model specification, we have tuned the following hyper-parameters:
Distribution Family of decoders: choose from normal gumbel-minimum.
Network structure: the size of encoder and decoder networks and their depth.
Dropout: the probability of dropout in the last layer of both encoder and decoders network. We select it from (0.0.2, 0.5, 0.9}.
Latent dimension: the dimension of Z: we select from 2 * dim(æ) or 0.5 dim((r).
For the variants with importance sampling, we set m = k and choose it from (10,30,100).
In the training stage, have tuned the following hyper-parameters:
Learning rate: 0.01, 0.001.
batch size: 20, 100, 250, 500, 1000.
Patience: the maximum number of epoch waiting until we stop the algorithm if no better validated metric is found. This helps reduce training time on overfitting the model.
reweighting parameter for the loss of censored observation, as introduced in Deep survival machine (Nagpal et al., 2021a). We choose 1.3 or 0.9.
Temperature:
21

---

Censor Dependent Variational Inference
C.4. Details in training-validatiin--esiii stages of the experiments
For the simulation dataset and inference gap in Table
We set the hyper-parameter m = k = 10 for IS and DVI variants.
No validation and testing,since know the truth. Best metric throughout the training process is reported.
We use Normal family for the decoder that aligns with the truth. Encoder/Decoder network shares the same network structure. Technical adhoc hyper-parane parameerassss are avoided, e.g., temperature set at 1, dropout is 0.
For the evaluation experiments on C/O /Brs in Table 4.
Train-validation validation--est split ratio is 0.6, 0.2, 0.2. Experiment repetition is 5, using the same seeds of dataset split.
Best model selected from best cross-validated C' 'ideex or Brier score of the model taking on quantiles of survival time, predicting from validation æ. We select for overall good fitting of the model, which is not the best validated metries Ord and Brier Score are valuated specific test times prevent overfitting.
The hyper-parameters tuned for training SOTA models in the training stage follows the recommendations from Nagpal et al. (2022). For details, please refer to the package website or the source codes attached.
For the implementation of metrics, we note that
C index is implemented via Python package Pycox by the authors of Kvamme et al. (2019)
Old and Brier score is implemented via Python package Scikit-Surviis (Pölsterl, 2020).
22

---

Censor Dependent Variational Inference
C.5. Exp Trick of Computing Bias Term in Delta Method CDVI: A Step-by-Step Derivation
In this section, we introduce numerically stable approach to calculate the bias terms Delta Method Variational Inference (Teh al., 2006) that is used in the delta method (DVI) variant of CD-CVAE. Inspired by the normalized importance weighting technique of Burda et al. (2015), approach leverages Log-Sum-Eai formulation that implicitly avoids over/underflow problems, which offers a more stable computation for gradient-based optimization.
Take 2q.26 above as an example, denoting / (zi) as wy:
di2
bias term
(27)
2m fm.
where fm = m -m w; = E[wo], = m- E (w;
=[w))>
Goal: reformulate Eq.27 by the logarithm of normalized weight vector, avoiding overflow during exponentiation.
W4
log w; log
x プイ=1 W/
Step I: observe that w; E[w] ui; Σ, w B(w]] ih mB/ww] variance:
E(w] [|/] (mi; 1). Thus, plug into the sample
E(w 2
E
V(w)=
m 2 wy
E[u])²
[uo] (mai - 1)]²
(mi;; - 1)²
m
m
Step 2: substituting the above equation into the bias formula:
E Tww 2
1
m
bias term
m Σ (mi; 1)2
(mi; - 1)²
2m
E [w]
2mm(m - 1)
Since(moi; - 1)2² m² 2mm; + 1; Em Wz = 1, summing over all E [l....,m;
m
m
m
m
m
m 0; 1)3
(m² 2 w 2mmi;; + +1 m
2m
W
+ m
m
w
m.
Thus, the delta method bias term for the density becomes:
bias term
2(m 1)
1=1
Step 3: Compute Em wi via sum exp(log w.)) or exp(log sum exp w;)) safely with 1-2 exponential operations using the Log-Sum-Exp trick.
(Recommended) Step 4: Compute the same bias term of the log survival function, since the bias term must be computed separately for censored observations. (The bias terms in survival function estimates are generally well-behaved and bounded.)
23

---

Censor Dependent Variational Inference
D. Additional Experiments
D.I. Additional Experiments on Simulated Datasets
Table 7: Experiments are repeated 5 times with a 0.6, 0.2, 0.2 train-validation-test split ratio. The best model is selected using pross-validated C-index. Test C'index are reported as mean + standard deviation. Higher is better. Survival times for SD3-5 were transformed using the exponential function.
Model
SD3 (20% cens)
SD4 (30% cens)
SD5 (50% cens)
CoxPH
0.772 + 0.008
0.739 + 0.005
0.663 0.010
SAVAE
0,765 0.012
0.722 0.013
0.653 0.014
DeepSurv
0.010
0.748 0.010
0.005
DSM
0.777 + 0.009
0.742
0.005
RSF
0.789 0.009
0
0.677 0.007
DCM
0.000
0.008
0.675 0.007
CD-CVAE
0.789 0.013
0.758 0.014
00.012
CD-CVAE+B
0.791 0.015
0.761 0.01
0.602
CD-CVAE DVI
0.709
0.741 0.008
0.665 +0.008
24

---

Censor Dependent Variational Inference
D.2. Additional Experiments Real-World Datasets
Table 8: Experiments are repeated five times with 0.6, 0.2, 0.2 train-validation-tte split ratio. Test C-index are reported as mean standard deviation. Higher is better. Bold entries indicate the highest mean per column(dataset).
Model
SUPPORT
FLCHAIN
NWTCO
METABRIC
WHAS
GBSG
PBC
CoxPH
0.661 + 0.006
0.623 + 0.207
0.685 0.009
0.637 0.000
0,756 0.018
0.675 + 0.006
0.829 0.014
SAVAE
0.625 0.008
0.745 0.023
0.689
0.643 0.000
0.751 0.0022
0.668 0.010
0.811 0.028
DeepSurv
0.644 + 0.005
0.775 # 0.005
0.019
00.019
0.758 0.021
0.597=0.000
0.828 = 0.017
DSM
0.668 1: 0.008
0.782 0.010
0.691 1.0024
0.673 10010
0.789 0.022
0.618 0.017
0.836 0.018
RSF
0.677 1: 0.005
0.762 0.005
0.0013
0.679 100005
0.791 0.015
0.685
0.841 = 0.018
DCM
0.676 : 0.004
0.003
0.678
0.670
0.015
0.604
0.841 0.018
CD-CVAE
0.679 + 0.003
0.794
0.018
0.012
0.842 = 0.026
0.678
0.845 = 0.011
CD-CVAE is
0.672 + 0.007
0.787 + 0.012
0.002
0.661 +0.015
0.822 - 0.019
0.663+000112
0.839 0.010
CD-CVAE
0.670 + 0.004
0.791 + 0.013
0.671 0.0225
0.669 +0.015
0.814 = 0.014
0.675 0.011
0.828 = 0.016
Table 9: Experiments are repeated five times with a 0.6, 0.2, 0.2 train- train-validation-ee split ratio. Test Ctd are reported as mean standard deviation. Higher is better. Bold entries indicate the highest mean per column(dataset).
Model
SUPPORT
FLCHAIN
NWTCO
METABRIC
WHAS
GBSG
PBC
CoxPH
0.664 1: 0.006
0.626 +: 0.202
0.688 0.0010
0.637 1.0009
0.761 0.016
0.676 1 0.004
0.831 0.012
SAVAE
0.631 0.012
0.773 0.016
0.675
0.657 #0012
0.770
0.655 0.018
0.815
DeepSurv
0.644 + 0.005
0.795 0.012
0.656 0.001
0.742 0.022
0.602
DSM
0.668 + 0.008
0.789 + 0.007
0.683
0.660
0.805 0.014
0.653
0.858 0.011
RSF
0.651 + 0.005
0.782 + 0.015
0.713
0.680 +00005
0.802 = 0.013
0.718
0.859 = 0.009
DCM
0.672 + 0.006
0.793 0.010
0.723
#0.014
0.801 0.002
0.624
0.858 0.012
CD-CVAE
0.672 1: 0.005
0.789 0.015
0.741 0.012
0.660 0.002
0,802 0.013
0,692 + 0.015
0,845 = 0.018
CD-CVAE+*
0.677 1: 0.006
0.756 0.022
0.717 0.022
0.655 1.001
0.796 0.015
0.663 1 0.012
0.859 = 0.012
CD-CVAE+DVT
0.663 + 0.004
0.751 # 0.014
0.709 + 0.017
0.659
0.788 0.012
0.688 = 0.017
0.858 = 0.012
Table 10: Experiments are repeated five times with a 0.6, 0.2, 0.2 train-validation-tee split ratio. Test Brier Scores are reported as mean- standard deviation. Lower is better. Bold entries indicate the lowest mean per column(dataset).
Model
SUPPORT
FLCHAIN
NWTCO
METABRIC
WHAS
GBSG
PBC
CoxPH
0.218 + 0.002
0.146 + 0.058
0.099
0.206
0.011
0.225 0.008
0.126 0.010
SAVAE
0.224 + 0.005
0.122 + 0.006
0.117
0.218 00001
0.169 0.012
0.224 + 0.005
0.138 0.01
DeepSurv
0.215 + 0.008
0.1 + 0.005
0.083
0.235 + 0.006
0.209 0.010
0.245 + 0.005
0.135 0.007
DSM
0.240 1 0.012
0.121 1 0.007
0.0041000011
0.235 0.008
0.181 0.006
0.248 1 0.006
0.136 0.008
RSF
0.225 1 0.001
0.127 0.006
0.088 0.000
0.221 0.000
0.166 0.006
0.223 1 0.007
0.123 - 0.005
DCM
0.220 + 0.004
0.118 + 0.006
0.081
0.221 0.006
0.175 0.006
0.233 = 0.006
0.141 0.005
CD-CVAE
0.218 + 0.005
0.115 + 0.005
0.081 0.006
0.211 000099
0.178 = 0.007
0,234 + 0.005
0.127 = 0.004
1D-CVAE+15
0.211 0.006
0.122 0.005
0.222 00012
0.172 0.005
0.235 + 0.005
0.144 0.010
CD-CVAE DVI
0.228 + 0.005
0.125 + 0.004
0.005
0.226 0.006
0.169 = 0.022
0.224 + 0.005
0.138 + 0.011
25


Reviews:
[{"review_id": "4ZrYrmfREg", "reviewer": "Reviewer_acdJ", "summary": "This paper analyzes the current practices to apply variational inference to latent variable models for survival analysis, provides insights into why the naive application of VI may be insufficient, and presents a new VI formulation that can potentially sidestep some of those challenges. The authors also include some experimental results that corroborate the improved performance from using the new framework.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We truly appreciate your thoughtful and constructive feedback and your \"willing to accept\" recommendation. Your reasoning between what is well-explained and what is unclear is greatly appreciated. The following responses address the points you raised.\n# Evaluation: Best metrics, average metrics, more experiments.\n\n### This is a general reponse. We value all reviewers’ thoughtful feedback on experimental design. While reporting best performance is a standard practice for VAEs (Burda et al. 2015; Tomczak & Welling 2018, Apellaniz et al. 2024), we agree that additional experiments are helpful. 4 additional experiments can be found via this [anonymous link](https://imgur.com/a/71JKyHg) with Apellaniz et al. 2024 included. Strong results across several benchmarks show our method consistently outperforms baselines on both best and average metrics. Meaningful exploration on simulated datasets is provided too.\n>Claims[1]: Question on vanilla VI solution, discussion around Remark 3.1, and its main takeaway\n\nYour interpretation of the vanilla VI failure case is spot on. Assuming the survival dataset contains two datapoints (x,y,1) and (x,y,0), optimal vanilla VI solution requires both inequalities (4) and (5) to hold as equalities at the same point (x,y) to perfect bound log-likelihood Eq.2. This leads to problems in Prop. 3.1. Remark 3.1 extends the VI optimality analysis to the population level by considering overlapping sample spaces. Specifically, it shows that, under non-informative censoring, an global optimal vanilla VI solution may fail to exist for non-degenerate θ (claim 1-2)  or may be trivial (claim 3-4).\n>Claims[2]: Section 3.2 is too fast. Interpreting Thm 3.2.1.\n\nThank you for the comments. We conduct a standard VI optimality analysis (Domke & Sheldon, 2018) for log-likelihood Eq.2 in Section 3.2. Thm 3.2.1 formalizes the structure of the optimal variational posterior $q_ϕ$ that achieves a zero inference gap under general censoring assumptions. It shows that the optimal parameter ϕ inherently depends on both the event and censoring time distributions ($U$ and $C$). In addition, under certain censoring assumptions, its dependence on censoring time distribution $C$ can be *eliminated*, offering practical insights into model design.\n>Does this imply separating variational parameters for the censored/uncensored data?\n\nA general answer is no. A similar misconception was raised by Reviewer KuLt (#3). The encoder in Eq. 10 is a dense/joint network as shown in Fig3(a). Conditioning on 𝛿 = 0,1, it yields two branches within the same network, producing variational parameters $\\phi_1$ and $\\phi_2$, which are used for notational consistency, as noted in Line 203, with vanilla VI setup and Thm 4.3.1 to 4.3.3. That said, our codebase includes a *de facto* split encoder (`Delta_encoder`) that disables parameter sharing, though it should be used with discretion. For clarity, we have added an appendix section discussing why a joint encoder is theoretically preferred, as the two branches are inherently coupled under the VI optimality conditions.\n> Why $2−i$ is used in Remark 3.2?\n\n$2−i$ is used to unify the indices in the results. For $i=2, q(z|x,y) =q_{\\phi_2}$ if and only if there is no event observation $p(\\delta=2-2=0|y)=1$.\n>Claims[3]: How is the S implemented? Is S simple distributions? \n\nThis likely refers to S(u∣x) in Eq.12. We compute S(u∣x) via the standard reparameterization trick by sampling z from the prior distribution and then computing closed-form $S(u|x,z) using the learned decoder (See `.predict()` in `model/cd-cvae.py`). While the decoder $f(u∣x,z)$ follows simple parametric forms (and so does S(u|x,z)), $f(u|x)$ and $S(u∣x)$ in Eq.2 are generally intractable, as they correspond to an infinite mixture over z [Rasmussen, 1999].\n>What is ζ below fig.3? \n\nFor notation clarity, we decompose the decoder parameter θ into a location parameter $μ_ζ$ and a scale parameter σ. Specifically, ζ parameterizes the neural network that outputs the decoder mean $\\mu(x, z)$.\n>What purpose is proposition 4.2 (no close-form update of $\\sigma$)?\n\nAs noted by Reviewer oybt (#2), stable training in VAE is non-trivial. While robust training strategies for VAEs are well-established in general settings (See Related Work, Liu & Wang, 2025), Proposition 4.2 identifies a fundamental pitfall: censored likelihoods in survival analysis make the dual-step optimization (Rybkin et al., 2021) *inapplicable*, justifying our use of standard $\\sigma$ training in our proposed method of CDVI. \n> a motivating example/a detailed technical summary/a main take-away/add multiple references; \n\nWe appreciate the valuable suggestions. Due to rebuttal length constraints, we will provide a detailed summary, supporting references, and an illustrative figure during the discussion phase.\n\n>  Suggestions [1-6]\n\nWe appreciate the valuable comments and have addressed the noted minor issues. For clarification, DVI is our proposed delta method variant.", "claims_and_evidence": "Overall claims made at the start of the paper are well supported. I do have specific concerns and raise them in the subsequent sections.", "methods_and_evaluation_criteria": "While the benchmarks and the metrics might be enough, I am confused about the lack uncertainty estimates in the numbers in Table 4 and Table 5. Can the authors comment on why they used the maximum and the minimum numbers across the trials without any uncertainty quantification?", "theoretical_claims": "Overall, I had trouble understanding the final take-away from the analysis. Here is what I understand right now and I would appreciate if the authors can help me understand the rest of it. Moreover, I would strongly encourage the authors to provide a high detailed technical summary of their contributions in the introduction (more suggestions in a later section).\n- Section 3.1: Lemma 3.1 provides the conditions that need to be met for the optimal solution of eq. 4 and eq. 5. Proposition 3.1 provides what follows if those conditions are met by a solution under the assumption on the model's functional form. Then, I do not fully understand the claims around Remark 3.1. Does the problem of no good vanilla VI solution only happens if there is data point that has the same $x$ and $y$ but different $\\delta$? The whole discussion around the overlapping sample spaces was very confusing for me. Overall, what is the main takeaway from this? \n- Section 3.2: Overall, the things moved fast in Section 3.2 without a lot of commentry on what is happening and why. Several variables are introduced with no explanation of how to take this information. In particular, how do I interpret results of theorem 3.2.1? Does this imply that we need to keep separate variational parameters for the censored and the uncensored data? The formulation of remark 3.2 is weird. Why are we using $2-i$ here? \n- Section 4.2: How is the $S$ implemented? Do we assume use of simple distributions where $S$ is available in closed form? What is $\\zeta$ below figure 3? What purpose is proposition 4.2 serving in the training procedure?", "experimental_designs_or_analyses": "Please see the section above about the methods and evaluation.", "relation_to_broader_scientific_literature": "Survival analysis models are crucial in various scientific fields. As their applications increase, it becomes increasingly important to learn not only the models themselves but also the uncertainty associated with unobserved variables. Variational inference (VI) plays a pivotal role in addressing this challenge. The primary contribution of this work lies in theoretical analysis of what happens when naive VI is applied and how a more cautious approach can enhance its effectiveness. By providing essential foundations, this work paves the way for future research and development in mechanistic models for survival analysis.", "essential_references_not_discussed": "I think references are missing from some important places in this paper. A few instances are\n- The initial paragraphs of the introduction make claims about the applications of survival analysis. \n- In lines 100-102, the authors are talking about a well-established way to define the likelihood in the survival analysis. It will be great to see some references here. \n- In lines 175-180, there is discussion of posterior collapse without any referencing. \n- In lines 195-205, the authors talk about different types of censoring without describing or referencing. \nIn general a pass over the paper may reveal even more places that would benefit from citations to provide support and additional context for readers not very well-read within the survival analysis literature.", "other_strengths_and_weaknesses": "Overall, I am want to accept this paper. However, I am can not recommend a clear accept in its current state. I think the paper is well written for most parts. But I also had trouble understanding the main take-aways from the analysis. Here are the few things that I think can help this paper a lot. \n- Add references where needed to provided appropriate context \n- Summarized the main results and take-aways from the analysis in the introduction with forward referencing. \n- Add more context around the technical contributions in the Section 4. Also, please add forward referencing to proofs in the main body. \n- Close the loop on the final algorithm for the proposed CD-CVAE approach with details about $S$.\n- Motivate the main benefits of the proposed approach using a working example. I think a clear, simple demonstration of where the vanilla VI fails and where the proposed approach will succeed will go a long way in understanding the scope of the contributions. \n- Add the proper uncertainty estimates in the empirical results.", "other_comments_or_suggestions": "Some typos and minor errors: \n- $S_\\theta$ in Eq. 2 started as $S_{U, \\theta}$ and is then never used like that, again. \n- $h$ is not defined before usage in 1) of Proposition 3.1\n- $\\pi$ in 4) of Proposition 3.1\n- Lines 155-157, second column the wording is strange. I think the authors want to claim to be the first to identity the problem latent non-identifiability in survival analysis. Wording can be changed to make that more precise. Also, similarly, a more precise claim for correcting the result of Domke and Sheldon can be used in lines 301-303. \n- Trailing bracket in equation for proposition 4.2\n- What is DVI in the header part of section 5.", "questions_for_authors": "Please see the previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AuU9mbYdkj", "reviewer": "Reviewer_oybt", "summary": "This paper builds upon prior work (Nagpal et al. 2021a, Apellaniz et al. 2024) that uses the variational distribution $q_\\phi (z \\mid x, y)$ as a posterior approximation for $p_\\theta (z\\mid x)$, without accounting for censoring (i.e., $y$ and $\\delta$). The authors propose a variational inference method that explicitly incorporates censoring, providing a tighter bound on the log-likelihood. The proposed approach is validated through extensive experiments on six simulation studies and seven real-world datasets, demonstrating empirical effectiveness.\n\n## Update after rebuttal\nThank you for the detailed response and for conducting the additional experiments — they address most of my concerns.\n\nAdditionally, I realize my earlier statement, \"using the predicted probability as score is problematic,\" may have been unclear. What I intended to convey is that referring to the current version of the concordance index as \"Harrell’s C-index\" can be misleading, as it differs from the formulation originally described in Harrell’s paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your thoughtful feedback and valuable suggestions. We truly appreciate the recognition of the contribution and your overall support of our work. We have improved our manuscript based on your recommendations. In what follows, we respond to your comments point by point, with deeper discussions where needed.\n\n> **A dependence between censoring times and event times contradicts the assumed DAG**\n\nThis misconception is also raised by Reviewer KuLt (#3). The assumed DAG aligns with a conditional independence between uncensored time-to-event $U$ and censoring times $C$ as noted in line 93. A fact in probability theory is that the independence between $U$ and $C$ *does not* imply an independence between the **observed** censored times ${Y|\\delta=0}$ and the **observed** events times ${Y|\\delta=1}$ due to Eq.1.\n>VI relies on good parameter initialization; more experiements\n\nThank you for sharing your insights. We found that tuning hyperparameters effectively reduces seed variability on datasets like NWTCO. Happy to share our logsumexp and xvaier tricks for stability or train script if needed. Ranganath et al. (2016) do not have a open script. The link for 4 additional experiments is in our reponse to Reviewer acdJ (#1).\n> **Computing of Harrell's C-index via predicted probabilities for non-cox models violates the one-to-one mapping condition**\n\nWe appreciate the opportunity to having a deeper discussion regarding Harrell’s C-index. The issue you raised was carefully discussed during our research. We invite your additional feedback to our step-by-step reasoning below: \n\n1.  We view the C-index, when computed using a particular risk score, as a generalized ranking metric/statistic that applies across all models considered. While Harrell’s C-index is defined using predicted survival times, the notion of concordance naturally extends to ranking via risk scores such as survival probabilities [Uno et al. 2011]. \n\n2. Proportional hazards (PH) assumption brings ordering guarantees. In particular, it ensures time-consistent ordering i.e., $S(u│x_i)≤S(u│x_j) ↔ ∀u, S(u|x_i)≤S(u│x_j)$ and risk-consistent ordering across different scores: $S(u│x_i)≤S(u│x_j) ↔ E(u|x_i)≤E(u│x_j)↔z(x_i)\\geq z(x_j)$, which is refered to as one-to-one mapping [Harrell et al 1996]. The latter allows for efficient concordance computation via hazard ratios for Cox models, which is an advantage over models like RSF, DeepSurv, DSM, and our own. From this perspective, the averaged C-index offers a practical and robust measure of overall ranking performance.\n\n3.  Without PH assumption, the inconsistency across *risk scores* is expected, and C-index based on a meaningful score remains valid and is not a \"misuse\"; the inconsistency across *test time* motivates the adoption of Antolini et al.’s [2005] in our work. Nevertheless, outperforming Cox models under both average and time-dependent C-indices demonstrates effective ranking without relying on the PH assumption.\n\nIn conclusion, we disagree with the claim that \"using the predicted probability as score is problematic\" for non-Cox models\n\n> Clarification on the motivation and limitations of prior work\n\nThank you for the feedback.  As discussed in Lines 142–150, our motivation stems from the lack of variational inference (VI) optimality analysis [Cremer et al., 2018] in prior work. The limitations in achieving VI optimality are shown in Prop. 3.1, while Remarks 3.1 and 3.2 point out issues in existing assumptions and model designs.\n\n> Why the sequential graph as x→z→u? Why x necessary? How reasonable is the v-structure?\n\nThis is an insightful question—one we have also considered. To clarify, only Fig1(a) represents a general generative DAG; Fig2(b) and 3(b) depict the v-structured latent process used in our method. The sequential graph x→z→u corresponds to a D-separated latent structure, as in Ranganath et al. (2016) and Apellaniz et al. (2024), where $u$ depends solely on z. In contrast, the v-structure, assuming z⊥x, requires x to generate u, thereby enabling individual survival modeling. \n\nWhile both latent structures are equivalent [Zheng et al. 2022, Remark 1], the v-structure is the default setup of CVAE [Sohn et al. 2015, Section 3]. For survival analysis tasks, we conjecture that the absence of v-structure stems from the prevailing use of vanilla VI (since Ranganath et al. 2016), and its inferior VI optimality compared to D-separation as noted in Line 175-180.\n\n> Specify which type of location-scale family used for optimal modeling.\n\nAs noted in Line 1023, it is treated as a hyperparameter with no consistent advantage observed; results depend on settings of  others hyperparameters.\n\n>Suggestions: Line 90: \"surjective\" → \"subjective\"\n\nIt is not a typo. In mathematics, a function $f:A→B$ is surjective if for every b∈B, there exists at least one a∈A such that $f(a)=b$\n\n> Suggestions [2-4]\n\nThank you for the valuable suggestions. We have revised the relevant section accordingly.", "claims_and_evidence": "The theoretical contributions and experimental results strongly support the paper’s claims.", "methods_and_evaluation_criteria": "The overall methodological approach is reasonable—optimizing two distinct variational distributions for censored and event groups. However, I have two concerns:\n\n* This approach seems to suggest a dependence between censoring times and event times, which contradicts the assumed DAG. The authors should clarify this apparent inconsistency.\n* The paper only compares against one variational inference method (Nagpal et al. 2021a). Why are other relevant methods, such as Ranganath et al. (2016) and Apellaniz et al. (2024), excluded? A broader comparison would strengthen the evaluation.", "theoretical_claims": "I did not verify the correctness of the proofs in the appendix.", "experimental_designs_or_analyses": "Yes. I've checked the experimental design.\n\nHowever, there are two issues:\n* The Harrell's concordance index this paper claims using is problematic. The correct way of calculating Harrell's C-index is by comparing the predicted times, not the predicted probability at a single time. Harrell's paper ([link](https://onlinelibrary.wiley.com/doi/10.1002/(SICI)1097-0258(19960229)15:4%3C361::AID-SIM168%3E3.0.CO;2-4), Section 5.5) clearly states that the predicted survival time is the default choice and the predicted probability can serve as a substitute with **conditions**. This substitution is only allowed when the predicted probabilities and times have a one-to-one mapping (e.g., proportional hazard is satisfied). However, in the experiments, some of the baselines (RSF, DSM) as well as the proposed method do not have the proportional hazard assumption and therefore using the predicted probability as the risk score is problematic and therefore not the correct Harrell's C-index.\n* The reported performance is based on the highest value across five random seeds, which raises concerns about fairness. Variational inference methods are known to be challenging to optimize, as they heavily depend on good parameter initialization -- a well-documented issue in the literature. In my previous experiments, different random seeds significantly affected performance, leading to high variance. Selecting the best-performing result rather than reporting the average (or another robust statistic) gives an unfair advantage over more stable and robust methods.", "relation_to_broader_scientific_literature": "This paper extends previous work (Nagpal et al. 2021a, Apellaniz et al. 2024) by explicitly incorporating censoring into the variational inference framework.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "The paper is difficult to follow, particularly regarding the motivation and limitations of prior work. The introduction uses vague terms such as “remain unclear” without specifying concrete shortcomings.", "other_comments_or_suggestions": "* Line 90: \"surjective\" → \"subjective\"\n* Lines 99, 126: \"partial log-likelihood\" → \"log-likelihood\" (The term partial log-likelihood in survival analysis specifically refers to Cox's proportional hazards model.)\n* Figure 2: The meaning of solid vs. dashed lines is unclear. Suggestions: (1) make the caption self-contained, explicitly defining these lines.\nand (2) use dashed lines for the encoder in Figure 3(a) to maintain consistency.\n* Lines 352-353: \"Deep Survival Forest (DSF)\" → \"Random Survival Forest (RSF)\"\n* The term *Censor-dependent* Conditional VAE suggests a dependent censoring assumption, which is misleading. A name like Censor-Aware VAE might be clearer.", "questions_for_authors": "* In the general generative DAG of $U$ (Figure 1(a), Figure 2(b), and Figure 3(b)), why do you represent the sequential graph as $x \\rightarrow z \\rightarrow u$? Specifically, in your decoder, why is $x$ necessary? Additionally, how reasonable is the structure of the graph you are using?\n* You mentioned that the event distribution should belong to a location-scale family. However, in your experiments, you did not explicitly specify which type of distribution you used for optimal modeling.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h12mNxVLzc", "reviewer": "Reviewer_KuLt", "summary": "The paper proposes a censor-dependent conditional VAE (CD-VAE), where two variational posteriors—for censored and non-censored events—are inferred given covariates and observed times, instead of the typical single posterior assumed in baseline methods. Further, the paper provides theoretical results to support the decomposed posterior. Experimental results on synthetic datasets show (i) smaller KL divergence between the prior and posterior ;  (ii) competitive performance in terms of the C-Index and Brier score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their time and effort in engaging with our work. Your recognition of the importance of variational survival analysis is encouraging. We've added further experimental results and endeavor to clarify a few points that may have been misinterpreted.\n\n## Response to Experimental Concerns\nSee extended experimental results in response to Reviewer acdJ (#1). \n\n>Evidence[2]: reporting C-index on the synthetic datasets\n\nOur experiments aim to evaluate both the inference optimality of our method and its effectiveness in survival modeling. Our simulation datasets are essential for assessing the former. Nonetheless, we’ve included additional results for completeness. \n\n>Evidence[1,3,4]:significance metrics; comparison with VSI. report variants on benchmark\n>Methods[2]:SEER datasets \n\nStatistical significance testing is not a common practice. VSI lacks a training script. SEER used in Nagpal et al. (2021a) is against our modeling assumptions as a dataset for competing risks analysis. \n\nIn response, we added the Apellaniz et al. (2024) baseline, reported average metrics with uncertainties, and included full performance results for all variants.\n>Evidence[2]:misleading KLD\n\n>Weaknesses: lack of sensitivity exploration\n\nWe would like to clarify a misunderstanding: the KL divergence reported in Table 2 is with respect to the true posterior, not the prior, as noted in the caption. Simulated datasets were used to explore the optimality of CDVI under varying censoring rates, while the seven benchmark datasets naturally cover a wide range of censoring scenarios.\n\n## Addressing Theoretical Questions\n1. Methods[1] & Weekness[1]: A potential issue with the proposed variational distribution is the assumption that the observed time Y depends on the censoring indicator δ, which lacks theoretical justification. \n2. Theoretical Claims[1] & Questions[2]: Justifying the need to decompose the posterior in Theorem 3.2.1 could be problematic: Why necessary for Eq4 and Eq5 to be equal?\n3. Methods[1] & Other Weaknesses[1]: Separating network parameters reduces the number of samples used to learn the inference and can introduce sensitivity to the censoring rate.\n\nWe appreciate the opportunity to clarify all these points regarding our theory: \n1. The dependency between Y and δ is not assumed in the posterior. See also Reviewer oybt (#2)'s first question. \n2. Eq. 4 and 5 are not required to be equal. They are inequalities that can be required to hold equal on both sides simutanuously on a same data point (x,y). See Claims [1] in Reviewer acdJ (#1) for a complete reasoning.\n3. Variational parameter separation is not forced or by default — see our reply to Reviewer acdJ (#1). We use a joint encoder and our proposed variational posterior approximates the true posterior $p(z|x,y,\\delta)$ as a whole; thus, sample imbalance is a general issue in (variational) conditional density estimation, not unique to our method; we address it following standard strategies (Nagpal et al., 2021a, Line 1042).\n\nWe hope these clarifications resolve the concerns and misunderstandings that you mentioned as a major weakness. Nonetheless, we remain happy to provide further detail if helpful! \n\n>Suggestions[1]: Importance of Theorems 4.3.1–4.3.3 \n\nThank you for the comments. While these results build on existing work, our analysis are concise—occupying less than half a page—and are carefully presented to support, rather than distract from, the central idea of censor-dependent variational inference (CDVI). They offer additional theoretical justification for our proposed CDVI by demonstrating its alignment with the broader augmented VI framework. They also offer rigorous foundations for our proposed variants, offering theoretical grounding of design choices that has not been established in prior work. \n\nImportantly, extending these results to the survival setting required nontrivial technical work. As mentioned in line 300, Theorem 4.3.2 corrects a key issue in the original framework by identifying a notational inconsistency between the variational posterior and the augmented variational posterior. To our knowledge, this inconsistency has not been previously addressed in the literature. As a result, Theorem 2 from Domke & Sheldon (2018) could not be directly extended to our setting. For reference, we invite the reviewer to compare our proof in Appendix B.6 with the proof of Theorem 1 in Domke & Sheldon (2018).\n> Brier score is not necessarily a calibration;  Adding References.\n\nThank you for highlighting this extremely helpful detail. We have revised the interpretation in line with Haider et al. and added the mentioned references along with [Sohn et al. 2015] for conditional variational posterior/VAEs. \n\n*We thank the reviewer again for the valuable feedback. Given the clarifications and efforts made, we would appreciate your consideration of reassessing the score. Please don’t hesitate to reach out with further questions.", "claims_and_evidence": "- The paper claims significant performance improvements over baselines:\n\n1) In general, the experimental results demonstrate marginal or comparable performance compared to baselines. It's unclear whether the marginal gains are statistically significant, as confidence intervals are not provided.\n\n2) It's unclear why C-Index and Brier Score results are not reported on the synthetic datasets, where only KL divergence is provided. This could be misleading, as a small KL divergence between the prior and posterior could also imply posterior collapse and is not necessarily indicative of survival model performance.\n\n3) It's unclear why the paper does not benchmark against VSI (Xiu et al., 2020), which proposes a principled, single shared posterior for censored and non-censored approaches on real-world datasets (Table 4).\n\n4) It's unclear why the real-world experiments (Table 4) do not include the importance sampling variations of the proposed CD-VAE approach.", "methods_and_evaluation_criteria": "- The decomposed posterior seems problematic, as only the observed time depends on the censoring indicator, which is already accounted for in the log-likelihood. This also reduces the number of samples used to learn the inference, where the quality of the learned posterior becomes a function of the censoring rate. I encourage the authors to provide complete results (C-Index, Brier score, KL divergence, and calibration) on both synthetic and real-world datasets to comprehensively evaluate the effect of censoring on these metrics. Additionally, including the censoring rate in Table 3 would be more informative than simply reporting the number of censored samples.\n\n- The paper should also consider evaluating the approach on larger datasets with more covariates, such as the SEER dataset, to provide insights into how the proposed method scales with the number of covariates and sample size.\n\n- The paper states that the Brier score is a measure of calibration, which is not necessarily true. See Haider et al. (2020) for D-calibration and Chapfuwa et al. (2020) for KM-calibration as metrics for survival calibration.", "theoretical_claims": "- Lemma 3.1: Justifying the need to decompose the posterior in Theorem 3.2.1 could be problematic:\n\n1) The paper begins by establishing conditions of equality for equations (4) and (5) assuming *non-censored* events. However, it is unclear why it is necessary for these equations to be equal, as the likelihood for observed events is already decomposed in equation (6) according to the censoring indicator. Moreover, note the relationship  $f(t|x) = h(t|x) \\cdot S(t|x)$, which implies that (4) and (5) are equal only if the hazard function is constant at 1.\n\n2) Other theoretical claims appear to be straightforward extensions of previously proposed theorems.", "experimental_designs_or_analyses": "- See methods and evaluation section.", "relation_to_broader_scientific_literature": "- Variational inference is an important generative modeling approach, where most methodological contributions focus on the posterior/prior distribution. This paper focuses on justifying the proposed decomposed censor-dependent posterior for variational survival analysis.", "essential_references_not_discussed": "- The paper should discuss other variational approaches outside survival analysis focusing on *conditional* posterior distributions, including Siddharth et al. (2017), Kingma et al. (2014), and Joy et al. (2021).", "other_strengths_and_weaknesses": "- A major strength of this paper lies in its connections to previously proposed variational survival analysis methods and variational inference. However, the justification of the proposed decomposed censor-dependent variational distribution and its practical implications—particularly the sensitivity to the censoring rate, which has not been thoroughly explored in the paper—represent a major weakness.", "other_comments_or_suggestions": "- Given that Theorems 4.3.1–4.3.3 are straightforward extensions of previously proposed theorems and do not appear to be directly related to the key contribution on the decomposed censor-dependent posterior, it is unclear whether they should be included in the main paper. The writing could be improved to clarify the connections; otherwise, it is not clear what purpose the proposed theorems serve in strengthening the key contributions.\n\n**Minor**\n- Remove empty [] from Brier score equation", "questions_for_authors": "- Could you provide comprehensive results on synthetic and real-world datasets, including the impact of the censoring rate on the learned posteriors?\n- Could you clarify why, in Lemma 3.1, the equivalence between equations (4) and (5) is necessary for *non-censored* events, given that only equation (4) is used in the likelihood estimation in equation (6) when events are *non-censored*?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After the discussion period, the reviewers unanimously recommended a "weak accept". It seems like the authors have largely addressed reviewer concerns although reviewers also pointed out there being clarity issues with the paper that seem to be hard to check during this reviewing process as to whether these issues are fixed (to be clear, this is not an issue with the reviewers not having enough background or lacking expertise; I think here it's on the authors to make their paper substantially clearer in exposition and motivation). I am recommending a weak accept for this paper as a result.
Decision: Accept (poster)