Original: N EURAL F RAILTY
Revised: N EURAL F RAILTY

Original: MACHINE
Revised: M

Original:  : B EYOND PROPOR - TIONAL HAZARD ASSUMPTION IN NEURAL SURVIVAL REGRESSIONS
Revised: ACHINE : B EYOND PROPORTIONAL HAZARD ASSUMPTION IN NEURAL SURVIVAL REGRESSIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present neural frailty machine (NFM), a powerful and flexible neural modeling framework for survival regressions.
Revised: We present neural frailty machine (NFM), a powerful and flexible neural modeling framework for survival regressions.

Original: The NFM framework utilizes the classical idea of multiplicative frailty in survival analysis to capture unobserved heterogeneity among individuals, at the same time being able to leverage the strong approximation power of neural architectures for handling nonlinear covariate dependence.
Revised: The NFM framework utilizes the classical idea of multiplicative frailty in survival analysis to capture unobserved heterogeneity among individuals, at the same time being able to leverage the strong approximation power of neural architectures for handling nonlinear covariate dependence.

Original: Two concrete models are derived under the framework that extends neural proportional hazard models and nonparametric hazard regression models.
Revised: Two concrete models are derived under the framework that extends neural proportional hazard models and nonparametric hazard regression models.

Original: Both models allow efficient training under the likelihood objective.
Revised: Both models allow efficient training under the likelihood objective.

Original: Theoretically, for both proposed models, we establish statistical guarantees of neural function approximation with respect to nonparametric components via characterizing their rate of convergence.
Revised: Theoretically, for both proposed models, we establish statistical guarantees of neural function approximation with respect to nonparametric components via characterizing their rate of convergence.

Original: Empirically, we provide synthetic experiments that verify our theoretical statements.
Revised: Empirically, we provide synthetic experiments that verify our theoretical statements.

Original: We also conduct experimental evaluations over 6 benchmark datasets of different scales, showing that the proposed NFM models outperform state-of-the-art survival models in terms of predictive performance.
Revised: We also conduct experimental evaluations over 6 benchmark datasets of different scales, showing that the proposed NFM models outperform state-of-the-art survival models in terms of predictive performance

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Regression analysis of time-to-event data (Kalbfleisch & Prentice, 2002) has been  the most important modeling tool for clinical studies and has witnessed a growing interest in areas like corporate finance (Duffie et al., 2009), recommendation systems (Jing & Smola, 2017), and computational advertising (Wu et al., 2015).
Revised: Regression analysis of time-to-event data (Kalbfleisch & Prentice, 2002) has been among the most important modeling tools for clinical studies and has witnessed a growing interest in areas like corporate finance (Duffie et al., 2009), recommendation systems (Jing & Smola, 2017), and computational advertising (Wu et al., 2015).

Original: The key feature that differentiates time-to-event data from other types of data is that they are  incompletely observed , with the most prevailing form of incompleteness being the right censoring mechanism (Kalbfleisch & Prentice, 2002).
Revised: The key feature that differentiates time-to-event data from other types of data is that they are often incompletely observed , with the most prevailing form of incompleteness being the right censoring mechanism (Kalbfleisch & Prentice, 2002).

Original: In the right censoring mechanism, the duration time of a sampled subject is (sometimes) only known to be larger than the observation time instead of being recorded precisely.
Revised: In the right censoring mechanism, the duration time of a sampled subject is (sometimes) only known to be larger than the observation time instead of being recorded precisely.

Original: It is well known in the community of survival analysis that even in the case of linear regression, naively discarding the censored observations produces estimation results that are statistically biased (Buckley & James, 1979), at the same time losses sample efficiency if the censoring proportion is high.
Revised: It is well known in the community of survival analysis that even in the case of linear regression, naively discarding the censored observations produces estimation results that are statistically biased (Buckley & James, 1979), at the same time losses sample efficiency if the censoring proportion is high.

Original: Cox’s proportional hazard (CoxPH ) model (Cox, 1972) using the convex objective of negative partial likelihood (Cox, 1975) is the de facto choice in modeling right censored time-to-event data (hereafter abbreviated as censored data without misunderstandings).
Revised: Cox’s proportional hazard (CoxPH ) model (Cox, 1972) using the convex objective of negative partial likelihood (Cox, 1975) is the de facto choice in modeling right censored time-to-event data (hereafter abbreviated as censored data without misunderstandings).

Original: The model is semiparametric (Bickel et al., 1993) in the sense that the baseline hazard function needs no parametric assumptions.
Revised: The model is semiparametric (Bickel et al., 1993) in the sense that the baseline hazard function needs no parametric assumptions.

Original: The original formulation of CoxPH model assumes a linear form and therefore has limited flexibility since the truth is not necessarily linear.
Revised: The original formulation of CoxPH model assumes a linear form and therefore has limited flexibility since the truth is not necessarily linear.

Original: Subsequent studies extended CoxPH model to nonlinear variants using ideas from nonparametric regression (Huang, 1999; Cai et al., 2007; 2008), ensemble learning (Ishwaran et al., 2008), and neural networks (Faraggi & Simon, 1995; Katzman et al., 2018).
Revised: Subsequent studies extended CoxPH model to nonlinear variants using ideas from nonparametric regression (Huang, 1999; Cai et al., 2007; 2008), ensemble learning (Ishwaran et al., 2008), and neural networks (Faraggi & Simon, 1995; Katzman et al., 2018).

Original: While such extensions allowed a more flexible nonlinear dependence structure with the covariates, the learning objectives were still derived under the proportional hazards (PH) assumption, which was shown to be inadequate in many real-world scenarios (Gray, 2000).
Revised: While such extensions allowed a more flexible nonlinear dependence structure with the covariates, the learning objectives were still derived under the proportional hazards (PH) assumption, which was shown to be inadequate in many real-world scenarios (Gray, 2000).

Original: The most notable case was the failure of modeling the phenomenon of crossing hazards (Stablein & Koutrouvelis, 1985).
Revised: The most notable case was the failure of modeling the phenomenon of crossing hazards (Stablein & Koutrouvelis, 1985).

Original: It is thus of significant interest to explore extensions of CoxPH that both allow nonlinear dependence over covariates and relaxations of the PH assumption.
Revised: It is thus of significant interest to explore extensions of CoxPH that both allow nonlinear dependence over covariates and relaxations of the PH assumption.

Original: Frailty models (Wienke, 2010; Duchateau & Janssen, 2007) are among the most important research topics in modern survival analysis, in that they provide a principled way of extending CoxPH model via incorporating a multiplicative random effect to capture unobserved heterogeneity.
Revised: Frailty models (Wienke, 2010; Duchateau & Janssen, 2007) are among the most important research topics in modern survival analysis, in that they provide a principled way of extending CoxPH model via incorporating a multiplicative random effect to capture unobserved heterogeneity.

Original: The resulting parameterization contains many useful variants of CoxPH like the proportional odds model (Bennett, 1983), under specific choices of frailty families.
Revised: The resulting parameterization contains many useful variants of CoxPH like the proportional odds model (Bennett, 1983), under specific choices of frailty families.

Original: While the theory of frailty models has been wellestablished (Murphy, 1994; 1995; Parner, 1998; Kosorok et al., 2004), most of them focused on the linear case.
Revised: While the theory of frailty models has been wellestablished (Murphy, 1994; 1995; Parner, 1998; Kosorok et al., 2004), most of them focused on the linear case.

Original: Recent developments on applying neural approaches to survival analysis (Katzman et al., 2018; Kvamme et al., 2019; Tang et al., 2022; Rindt et al., 2022) have shown promising results in terms of empirical predictive performance, with most of them lacking theoretical discussions.
Revised: Recent developments on applying neural approaches to survival analysis (Katzman et al., 2018; Kvamme et al., 2019; Tang et al., 2022; Rindt et al., 2022) have shown promising results in terms of empirical predictive performance, with most of them lacking theoretical discussions.

Original: Therefore, it is of significant interest to build more powerful frailty models via adopting techniques in modern deep learning (Goodfellow et al., 2016) with provable statistical guarantees.
Revised: Therefore, it is of significant interest to build more powerful frailty models via adopting techniques in modern deep learning (Goodfellow et al., 2016) with provable statistical guarantees.

Original: In this paper, we present a general framework for neural extensions of frailty models called the neural frailty machine (NFM) .
Revised: In this paper, we present a general framework for neural extensions of frailty models called the neural frailty machine (NFM) .

Original: Two concrete neural architectures are derived under the framework: The first one adopts the proportional frailty assumption, allowing an intuitive interpretation of the neural CoxPH model with a multiplicative random effect.
Revised: Two concrete neural architectures are derived under the framework: The first one adopts the proportional frailty assumption, allowing an intuitive interpretation of the neural CoxPH model with a multiplicative random effect.

Original: The second one further relaxes the proportional frailty assumption and could be viewed as an extension of nonparametric hazard regression (NHR) (Cox & O’Sullivan, 1990; Kooperberg et al., 1995), sometimes referred to as ”fully neural” models under the context of neural survival analysis (Omi et al., 2019).
Revised: The second one further relaxes the proportional frailty assumption and could be viewed as an extension of nonparametric hazard regression (NHR) (Cox & O’Sullivan, 1990; Kooperberg et al., 1995), sometimes referred to as ”fully neural” models under the context of neural survival analysis (Omi et al., 2019).

Original: We summarize our contributions as follows.
Revised: We summarize our contributions as follows.

Original: • We propose the neural frailty machine (NFM) framework as a principled way of incorporating unobserved heterogeneity into neural survival regression models.
Revised: • We propose the neural frailty machine (NFM) framework as a principled way of incorporating unobserved heterogeneity into neural survival regression models.

Original: The framework includes many commonly used survival regression models as special cases.
Revised: The framework includes many commonly used survival regression models as special cases.

Original: • We derive two model architectures based on the NFM framework that extend neural CoxPH models and neural NHR models.
Revised: • We derive two model architectures based on the NFM framework that extend neural CoxPH models and neural NHR models.

Original: Both models allow stochastic training and scale to large datasets.
Revised: Both models allow stochastic training and scale to large datasets.

Original: • We show theoretical guarantees for the two proposed models via characterizing the rates of convergence of the proposed nonparametric function estimators.
Revised: • We show theoretical guarantees for the two proposed models via characterizing the rates of convergence of the proposed nonparametric function estimators.

Original: The proof technique is different from previous theoretical studies on neural survival analysis and is applicable to many other types of neural survival models.
Revised: The proof technique is different from previous theoretical studies on neural survival analysis and is applicable to many other types of neural survival models.

Original: • We conduct extensive studies on various benchmark datasets at different scales.
Revised: • We conduct extensive studies on various benchmark datasets at different scales.

Original: Under standard performance metrics, both models are empirically shown to perform competitively, matching or outperforming state-of-the-art neural survival models.
Revised: Under standard performance metrics, both models are empirically shown to perform competitively, matching or outperforming state-of-the-art neural survival models.

Original:  R ELATED WORKS 2. N ONLINEAR EXTENSIONS OF C OX PH
Revised:  R ELATED WORKS 2. N ONLINEAR EXTENSIONS OF C OX PH

Original: Most nonlinear extensions of CoxPH model stem from the equivalence of partial likelihood and semiparametric profile likelihood (Murphy & Van der Vaart, 2000) of CoxPH model, resulting in nonlinear variants that essentially replaces the linear term in partial likelihood with nonlinear variants: Huang (1999) used smoothing splines, Cai et al.
Revised: Most nonlinear extensions of CoxPH model stem from the equivalence of partial likelihood and semiparametric profile likelihood (Murphy & Van der Vaart, 2000) of CoxPH model, resulting in nonlinear variants that essentially replaces the linear term in partial likelihood with nonlinear variants: Huang (1999) used smoothing splines, Cai et al.

Original: (2007; 2008) used local polynomial regression (Fan & Gijbels, 1996).
Revised: (2007; 2008) used local polynomial regression (Fan & Gijbels, 1996).

Original: The empirical success of tree-based models inspired subsequent developments like Ishwaran et al.
Revised: The empirical success of tree-based models inspired subsequent developments like Ishwaran et al.

Original: (2008) that equip tree-based models such as gradient boosting trees and random forests with losses in the form of negative log partial likelihood.
Revised: (2008) that equip tree-based models such as gradient boosting trees and random forests with losses in the form of negative log partial likelihood.

Original: Early developments of neural survival analysis Faraggi & Simon (1995) adopted similar extension strategies and obtained neural versions of partial likelihood.
Revised: Early developments of neural survival analysis Faraggi & Simon (1995) adopted similar extension strategies and obtained neural versions of partial likelihood.

Original: Later attempts Katzman et al.
Revised: Later attempts Katzman et al.

Original: (2018) suggest using the successful practice of stochastic training which is believed to be at the heart of the empirical success of modern neural methods (Hardt et al., 2016).
Revised: (2018) suggest using the successful practice of stochastic training which is believed to be at the heart of the empirical success of modern neural methods (Hardt et al., 2016).

Original: However, stochastic training under the partial likelihood objective is highly non-trivial, as mini-batch versions of log partial likelihood Katzman et al.
Revised: However, stochastic training under the partial likelihood objective is highly non-trivial, as mini-batch versions of log partial likelihood Katzman et al.

Original: (2018) are no longer valid stochastic gradients of the full-sample log partial likelihood (Tang et al., 2022).
Revised: (2018) are no longer valid stochastic gradients of the full-sample log partial likelihood (Tang et al., 2022).

Original: 2. B EYOND C OX PH IN SURVIVAL ANALYSIS
Revised: 2. B EYOND C OX PH IN SURVIVAL ANALYSIS

Original: In linear survival modeling, there are standard alternatives to CoxPH such as the accelerated failure time (AFT) model (Buckley & James, 1979;
Revised: In linear survival modeling, there are standard alternatives to CoxPH such as the accelerated failure time (AFT) model (Buckley & James, 1979;

Original: Ying, 1993), the extended hazard regression model (Etezadi-Amoli & Ciampi, 1987), and the family of linear transformation models (Zeng & Lin,
Revised: Ying, 1993), the extended hazard regression model (Etezadi-Amoli & Ciampi, 1987), and the family of linear transformation models (Zeng & Lin,

Original: While these models allow certain types of nonlinear extensions, the resulting form of (conditional) hazard function is still restricted to be of a specific form.
Revised: While these models allow certain types of nonlinear extensions, the resulting form of (conditional) hazard function is still restricted to be of a specific form.

Original: The idea of nonparametric hazard regression (NHR) (Cox & O’Sullivan, 1990; Kooperberg et al., 1995; Strawderman & Tsiatis, 1996) further improves the flexibility of nonparametric survival analysis via directly modeling the conditional hazard function by nonparametric regression techniques such as spline approximation.
Revised: The idea of nonparametric hazard regression (NHR) (Cox & O’Sullivan, 1990; Kooperberg et al., 1995; Strawderman & Tsiatis, 1996) further improves the flexibility of nonparametric survival analysis via directly modeling the conditional hazard function by nonparametric regression techniques such as spline approximation.

Original: Neural versions of NHR have been developed lately such as the CoxTime model Kvamme et al.
Revised: Neural versions of NHR have been developed lately such as the CoxTime model Kvamme et al.

Original: Rindt et al.
Revised: Rindt et al.

Original: (2022) used a neural network to approximate the conditional survival function and could be thus viewed as another trivial extension of NHR.
Revised: (2022) used a neural network to approximate the conditional survival function and could be thus viewed as another trivial extension of NHR.

Original: Aside from developments in NHR, Lee et al.
Revised: Aside from developments in NHR, Lee et al.

Original: (2018) proposed a discrete-time model with its objective being a mix of the discrete likelihood and a rank-based score; Zhong et al.
Revised: (2018) proposed a discrete-time model with its objective being a mix of the discrete likelihood and a rank-based score; Zhong et al.

Original: (2021a) proposed a neural version of the extended hazard model, unifying both neural CoxPH and neural AFT model;
Revised: (2021a) proposed a neural version of the extended hazard model, unifying both neural CoxPH and neural AFT model;

Original: Tang et al.
Revised: Tang et al.

Original: (2022) used an ODE approach to model the hazard and cumulative hazard functions.
Revised: (2022) used an ODE approach to model the hazard and cumulative hazard functions.

Original: 2. T HEORETICAL JUSTIFICATION OF NEURAL SURVIVAL MODELS
Revised: 2. T HEORETICAL JUSTIFICATION OF NEURAL SURVIVAL MODELS

Original: Despite the abundance of neural survival models, assessment of their theoretical properties remains nascent.
Revised: Despite the abundance of neural survival models, assessment of their theoretical properties remains nascent.

Original: In Zhong et al.
Revised: In Zhong et al.

Original: (2021b), the authors developed minimax theories of partially linear cox model using neural networks as the functional approximator.
Revised: (2021b), the authors developed minimax theories of partially linear cox model using neural networks as the functional approximator.

Original: Zhong et al.
Revised: Zhong et al.

Original: (2021a) provided convergence guarantees of neural estimates under the extended hazard model.
Revised: (2021a) provided convergence guarantees of neural estimates under the extended hazard model.

Original: The theoretical developments therein rely on specific forms of objective function (partial likelihood and kernel pseudolikelihood) and are not directly applicable to the standard likelihood-based objective which is frequently used in survival analysis.
Revised: The theoretical developments therein rely on specific forms of objective function (partial likelihood and kernel pseudolikelihood) and are not directly applicable to the standard likelihood-based objective which is frequently used in survival analysis.

Original:  M ETHODOLOGY 3. T HE NEURAL FRAILTY MACHINE FRAMEWORK
Revised:  M ETHODOLOGY 3. T HE NEURAL FRAILTY MACHINE FRAMEWORK

Original: Let ˜ T ≥ 0 be the interested event time with survival function denoted by S ( t )
Revised: Let ˜ T ≥ 0 be the interested event time with survival function denoted by S ( t )

Original: = P ( ˜ T > t ) associated with a feature(covariate) vector Z ∈ R d . Suppose that ˜ T is a continuous random variable and let f ( t ) be its density function.
Revised: = P ( ˜ T > t ) associated with a feature(covariate) vector Z ∈ R d . Suppose that ˜ T is a continuous random variable and let f ( t ) be its density function.

Original: Then λ ( t ) =
Revised: Then λ ( t ) =

Original: f ( t ) /S
Revised: f ( t ) /S

Original: ( t ) is the hazard function and Λ( t ) =
Revised: ( t ) is the hazard function and Λ( t ) =

Original: (cid:82) t 

Revised: (cid:82) t

Original: λ ( s ) ds is the cumulative hazard function.
Revised: λ ( s ) ds is the cumulative hazard function.

Original: Aside from the covariate Z , we use a positive scalar random variable ω ∈ R + to express the unobserved heterogeneity corresponding to individuals, or frailty ..
Revised: Aside from the covariate Z , we use a positive scalar random variable ω ∈ R + to express the unobserved heterogeneity corresponding to individuals, or frailty ..

Original: In this paper we will assume the following generating scheme of ˜ T via specifying its conditional hazard function:
Revised: In this paper we will assume the following generating scheme of ˜ T via specifying its conditional hazard function:

Original: [Equation]
Revised: [Equation]

Original: Here (cid:101)
Revised: Here (cid:101)

Original: ν is an unspecified non-negative function, and we let the distribution of ω be parameterized by a one-dimensional parameter θ ∈ R .
Revised: ν is an unspecified non-negative function, and we let the distribution of ω be parameterized by a one-dimensional parameter θ ∈ R .

Original: The formulation (1) is quite general and contains several important models in both traditional and neural survival analysis:
Revised: The formulation (1) is quite general and contains several important models in both traditional and neural survival analysis:

Original: When ω follows parametric distributional assumptions
Revised: When ω follows parametric distributional assumptions

Original: and (cid:101) ν
Revised: and (cid:101) ν

Original: ( t, Z ) =
Revised: ( t, Z ) =

Original: λ ( t ) e β ⊤ Z , (1) reduces to the standard proportional frailty model (Kosorok et al., 2004).
Revised: λ ( t ) e β ⊤ Z , (1) reduces to the standard proportional frailty model (Kosorok et al., 2004).

Original: A special case is when ω is degenerate,
Revised: A special case is when ω is degenerate,

Original: i
Revised: i

Original: e., it has no randomness, then the model corresponds to the classic CoxPH model.
Revised: e., it has no randomness, then the model corresponds to the classic CoxPH model.

Original: When ω is degenerate and (cid:101) ν is arbitrary, the model becomes equivalent to nonparametric hazard regression (NHR) (Cox & O’Sullivan, 1990; Kooperberg et al., 1995).
Revised: When ω is degenerate and (cid:101) ν is arbitrary, the model becomes equivalent to nonparametric hazard regression (NHR) (Cox & O’Sullivan, 1990; Kooperberg et al., 1995).

Original: In NHR, the function parameter of interest is usually the logarithm of the (conditional) hazard function.
Revised: In NHR, the function parameter of interest is usually the logarithm of the (conditional) hazard function.

Original: In this paper we construct neural approximations to the logarithm of (cid:101) ν , i.e., ν ( t, Z ) = log (cid:101)
Revised: In this paper we construct neural approximations to the logarithm of (cid:101) ν , i.e., ν ( t, Z ) = log (cid:101)

Original: ν
Revised: ν

Original: ( t, Z ) .
Revised: ( t, Z ) .

Original: The resulting models are called Neural Frailty Machines (NFM) .
Revised: The resulting models are called Neural Frailty Machines (NFM) .

Original: Depending on the prior knowledge of the function ν , we propose two function approximation schemes: is mostly for simplicity and clearness of theoretical derivations.
Revised: Depending on the prior knowledge of the function ν , we propose two function approximation schemes: is mostly for simplicity and clearness of theoretical derivations.

Original: Note that there exist multi-dimensional frailty families like the PVF family (Wienke, 2010).
Revised: Note that there exist multi-dimensional frailty families like the PVF family (Wienke, 2010).

Original: Generalizing our theoretical results to such kinds of families would require additional sets of regularity conditions, and will be left to future explorations.
Revised: Generalizing our theoretical results to such kinds of families would require additional sets of regularity conditions, and will be left to future explorations.

Original: The proportional frailty (PF) scheme assumes the dependence of ν on event time and covariates to be completely decoupled , i.e.,
Revised: The proportional frailty (PF) scheme assumes the dependence of ν on event time and covariates to be completely decoupled , i.e.,

Original: [Equation]
Revised: [Equation]

Original: Proportional-style assumption over hazard functions has been shown to be a useful inductive bias in survival analysis.
Revised: Proportional-style assumption over hazard functions has been shown to be a useful inductive bias in survival analysis.

Original: We will treat both h and m in (2) as function parameters, and device two multi-layer perceptrons (MLP) to approximate them separately.
Revised: We will treat both h and m in (2) as function parameters, and device two multi-layer perceptrons (MLP) to approximate them separately.

Original: The fully neural (FN) scheme imposes no a priori assumptions over ν and is the most general version of NFM.
Revised: The fully neural (FN) scheme imposes no a priori assumptions over ν and is the most general version of NFM.

Original: It is straightforward to see that the most commonly used survival models, such as
Revised: It is straightforward to see that the most commonly used survival models, such as

Original: CoxPH , AFT, EH, or PF models are included in the proposed model space as special cases.
Revised: CoxPH , AFT, EH, or PF models are included in the proposed model space as special cases.

Original: We treat ν = ν ( t, Z ) as the function parameter with input dimension d + 1 and use a multi-layer perceptron (MLP) as the function approximator to ν .
Revised: We treat ν = ν ( t, Z ) as the function parameter with input dimension d + 1 and use a multi-layer perceptron (MLP) as the function approximator to ν .

Original: Similar approximation schemes with respect to the hazard function have been proposed in some recent works (Omi et al., 2019; Rindt et al., 2022), referred to as ”fully neural approaches” without theoretical characterizations.
Revised: Similar approximation schemes with respect to the hazard function have been proposed in some recent works (Omi et al., 2019; Rindt et al., 2022), referred to as ”fully neural approaches” without theoretical characterizations.

Original: The choice of frailty family There are many commonly used families of frailty distributions (Kosorok et al., 2004; Duchateau & Janssen, 2007; Wienke, 2010), among which the most popular one is the gamma frailty , where ω follows a gamma distribution with mean 1 and variance θ .
Revised: The choice of frailty family There are many commonly used families of frailty distributions (Kosorok et al., 2004; Duchateau & Janssen, 2007; Wienke, 2010), among which the most popular one is the gamma frailty , where ω follows a gamma distribution with mean 1 and variance θ .

Original: We briefly introduce some other types of frailty families in appendix A.
Revised: We briefly introduce some other types of frailty families in appendix A.

Original: 3. P ARAMETER LEARNING UNDER CENSORED OBSERVATIONS
Revised: 3. P ARAMETER LEARNING UNDER CENSORED OBSERVATIONS

Original: In time-to-event modeling scenarios, the event times are typically observed under right censoring.
Revised: In time-to-event modeling scenarios, the event times are typically observed under right censoring.

Original: Let C be the right censoring time which is assumed to be conditionally independent of the event time ˜
Revised: Let C be the right censoring time which is assumed to be conditionally independent of the event time ˜

Original: T given Z , i.e., ˜ T ⊥⊥ C | Z .
Revised: T given Z , i.e., ˜ T ⊥⊥ C | Z .

Original: In data collection, one can observe the minimum of the survival time and the censoring time, that is, observe T = ˜ T ∧ C as well as the censoring indicator δ
Revised: In data collection, one can observe the minimum of the survival time and the censoring time, that is, observe T = ˜ T ∧ C as well as the censoring indicator δ

Original: = I ( ˜ T ⩽ C ) , where a ∧ b = min( a, b ) for constants a and b
Revised: = I ( ˜ T ⩽ C ) , where a ∧ b = min( a, b ) for constants a and b

Original: and I ( · ) stands for the indicator function.
Revised: and I ( · ) stands for the indicator function.

Original: We assume n independent and identically distributed (i.i.d.)
Revised: We assume n independent and identically distributed (i.i.d.)

Original: copies of ( T, δ, Z ) are used as the training sample ( T i , δ i , Z i ) , i ∈ [ n ] , where we use [ n ] to denote the set { 1 , 2 , . . . , n } .
Revised: copies of ( T, δ, Z ) are used as the training sample ( T i , δ i , Z i ) , i ∈ [ n ] , where we use [ n ] to denote the set { 1 , 2 , . . . , n } .

Original: Additionally, we assume the unobserved frailties are independent and identically distributed, i.e., ω i i.i.d.
Revised: Additionally, we assume the unobserved frailties are independent and identically distributed, i.e., ω i i.i.d.

Original: ∼ f θ ( ω ) , i ∈ [ n ] .
Revised: ∼ f θ ( ω ) , i ∈ [ n ] .

Original: Next, we derive the learning procedure based on the observed log-likelihood (OLL) objective under both
Revised: Next, we derive the learning procedure based on the observed log-likelihood (OLL) objective under both

Original: PF and FN scheme.
Revised: PF and FN scheme.

Original: To obtain the observed likelihood, we first integrate the conditional survival function given the frailty:
Revised: To obtain the observed likelihood, we first integrate the conditional survival function given the frailty:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Here the frailty transform G θ ( x ) = − log ( E ω ∼ f θ [ e − ωx ]) is defined as the negative of the logarithm of the Laplace transform of the frailty distribution.
Revised: Here the frailty transform G θ ( x ) = − log ( E ω ∼ f θ [ e − ωx ]) is defined as the negative of the logarithm of the Laplace transform of the frailty distribution.

Original: The conditional cumulative hazard function is thus Λ( t | Z )
Revised: The conditional cumulative hazard function is thus Λ( t | Z )

Original: =
Revised: =

Original: G θ ( (cid:82) t 0 e ν ( s,Z ) ds ) .
Revised: G θ ( (cid:82) t 0 e ν ( s,Z ) ds ) .

Original: For the PF scheme of NFM, we use two MLPs (cid:98) h
Revised: For the PF scheme of NFM, we use two MLPs (cid:98) h

Original: = (cid:98) h ( t ; W h , b h ) and (cid:98) m
Revised: = (cid:98) h ( t ; W h , b h ) and (cid:98) m

Original: = (cid:98) m
Revised: = (cid:98) m

Original: ( Z ; W m , b m ) as function approximators to ν and m , parameterized by ( W h , b h ) and ( W m , b m ) , respectively.
Revised: ( Z ; W m , b m ) as function approximators to ν and m , parameterized by ( W h , b h ) and ( W m , b m ) , respectively.

Original: According to standard results on censored data likelihood (Kalbfleisch & Prentice, 2002), we write the learning objective under the PF scheme as:
Revised: According to standard results on censored data likelihood (Kalbfleisch & Prentice, 2002), we write the learning objective under the PF scheme as:

Original: [Equation]
Revised: [Equation]

Original: Here we define g θ ( x ) =
Revised: Here we define g θ ( x ) =

Original: ∂∂x G θ ( x ) .
Revised: ∂∂x G θ ( x ) .

Original: Let ( (cid:99) W hn , (cid:98)
Revised: Let ( (cid:99) W hn , (cid:98)

Original: b hn , (cid:99) W mn , (cid:98) b mn , (cid:98) θ n ) be the maximizer of (4) and further denote (cid:98) h n ( t )
Revised: b hn , (cid:99) W mn , (cid:98) b mn , (cid:98) θ n ) be the maximizer of (4) and further denote (cid:98) h n ( t )

Original: = (cid:98) h ( t ; (cid:99) W h n , (cid:98)
Revised: = (cid:98) h ( t ; (cid:99) W h n , (cid:98)

Original: b h n ) and (cid:98) m n ( Z ) = (cid:98) m
Revised: b h n ) and (cid:98) m n ( Z ) = (cid:98) m

Original: ( Z ; (cid:99) W m n , (cid:98) b m n ) .
Revised: ( Z ; (cid:99) W m n , (cid:98) b m n ) .

Original: The resulting estimators for conditional cumulative hazard and survival functions are:
Revised: The resulting estimators for conditional cumulative hazard and survival functions are:

Original: [Equation]
Revised: [Equation]

Original: For the FN scheme, we use (cid:98) ν
Revised: For the FN scheme, we use (cid:98) ν

Original: = (cid:98) ν
Revised: = (cid:98) ν

Original: ( t, Z ; W ν , b ν ) to approximate ν ( t, Z ) parameterized by ( W ν , b ν ) .
Revised: ( t, Z ; W ν , b ν ) to approximate ν ( t, Z ) parameterized by ( W ν , b ν ) .

Original: The OLL objective is written as:
Revised: The OLL objective is written as:

Original: [Equation]
Revised: [Equation]

Original: Let ( (cid:99)
Revised: Let ( (cid:99)

Original: W νn , (cid:98)
Revised: W νn , (cid:98)

Original: b νn , (cid:98) θ n ) be the maximizer of (6), and further denote (cid:98) ν n ( t, Z ) =
Revised: b νn , (cid:98) θ n ) be the maximizer of (6), and further denote (cid:98) ν n ( t, Z ) =

Original: (cid:98) ν
Revised: (cid:98) ν

Original: ( t, Z
Revised: ( t, Z

Original: ; (cid:99) W νn , (cid:98)
Revised: ; (cid:99) W νn , (cid:98)

Original: b νn ) .
Revised: b νn ) .

Original: The conditional cumulative hazard and survival functions are therefore estimated as:
Revised: The conditional cumulative hazard and survival functions are therefore estimated as:

Original: [Equation]
Revised: [Equation]

Original: The evaluation of objectives like (6) and its gradient requires computing a definite integral of an exponentially transformed MLP function.
Revised: The evaluation of objectives like (6) and its gradient requires computing a definite integral of an exponentially transformed MLP function.

Original: Instead of using exact computations that are available for only a restricted type of activation functions and network structures, we use numerical integration for such kinds of evaluations, using the method of Clenshaw-Curtis quadrature (Boyd, 2001), which has shown competitive performance and efficiency in recent applications to monotonic neural networks (Wehenkel & Louppe, 2019).
Revised: Instead of using exact computations that are available for only a restricted type of activation functions and network structures, we use numerical integration for such kinds of evaluations, using the method of Clenshaw-Curtis quadrature (Boyd, 2001), which has shown competitive performance and efficiency in recent applications to monotonic neural networks (Wehenkel & Louppe, 2019).

Original: Remark 1.
Revised: Remark 1.

Original: The interpretation of frailty terms differs in the two schemes.
Revised: The interpretation of frailty terms differs in the two schemes.

Original: In the PF scheme, introducing the frailty effect strictly increases the modeling capability (i.e., the capability of modeling crossing hazard) in comparison to CoxPH or neural variants of CoxPH (Kosorok et al., 2004).
Revised: In the PF scheme, introducing the frailty effect strictly increases the modeling capability (i.e., the capability of modeling crossing hazard) in comparison to CoxPH or neural variants of CoxPH (Kosorok et al., 2004).

Original: In the
Revised: In the

Original: FN scheme, it is arguable that in the i.i.d.
Revised: FN scheme, it is arguable that in the i.i.d.

Original: case, the marginal hazard function is a reparameterization of the hazard function in the context of NHR.
Revised: case, the marginal hazard function is a reparameterization of the hazard function in the context of NHR.

Original: Therefore, we view the incorporation of frailty effect as injecting a domain-specific inductive bias that has proven to be useful in survival analysis and time-to-event regression modeling and verify this claim empirically in section 5.2.
Revised: Therefore, we view the incorporation of frailty effect as injecting a domain-specific inductive bias that has proven to be useful in survival analysis and time-to-event regression modeling and verify this claim empirically in section 5.2.

Original: Moreover, frailty becomes especially helpful when handling correlated or clustered data where the frailty term is assumed to be shared among certain groups of individuals (Parner, 1998).
Revised: Moreover, frailty becomes especially helpful when handling correlated or clustered data where the frailty term is assumed to be shared among certain groups of individuals (Parner, 1998).

Original: Extending NFM to such scenarios is valuable and we left it to future explorations.
Revised: Extending NFM to such scenarios is valuable and we left it to future explorations.

Original:  T HEORETICAL RESULTS
Revised:  T HEORETICAL RESULTS

Original: In this section, we present theoretical properties of both NFM estimates by characterizing their rates of convergence when the underlying event data follows corresponding model assumptions.
Revised: In this section, we present theoretical properties of both NFM estimates by characterizing their rates of convergence when the underlying event data follows corresponding model assumptions.

Original: The proof technique is based on the method of sieves (Shen & Wong, 1994; Shen, 1997; Chen, 2007) that views neural networks as a special kind of nonlinear sieve (Chen, 2007) that satisfies desirable approximation properties (Yarotsky, 2017).
Revised: The proof technique is based on the method of sieves (Shen & Wong, 1994; Shen, 1997; Chen, 2007) that views neural networks as a special kind of nonlinear sieve (Chen, 2007) that satisfies desirable approximation properties (Yarotsky, 2017).

Original: Since both models produce estimates of function parameters, we need to specify a suitable function space to work with.
Revised: Since both models produce estimates of function parameters, we need to specify a suitable function space to work with.

Original: Here we choose the following
Revised: Here we choose the following

Original: H ¨ older ball as was also used in previous works on nonparametric estimation using neural networks (Schmidt-Hieber, 2020; Farrell et al., 2021; Zhong et al., 2021b)
Revised: H ¨ older ball as was also used in previous works on nonparametric estimation using neural networks (Schmidt-Hieber, 2020; Farrell et al., 2021; Zhong et al., 2021b)

Original: [Equation]
Revised: [Equation]

Original: where the domain X is assumed to be a subset of d -dimensional euclidean space.
Revised: where the domain X is assumed to be a subset of d -dimensional euclidean space.

Original: α =
Revised: α =

Original: ( α 1 , . .
Revised: ( α 1 , . .

Original: . , α d ) is a d -dimensional tuple of nonnegative integers satisfying | α | = α 1 + · · · + α d and D α f = ∂ | α | f ∂x α 11 ··· x αdd is the weak derivative of f .
Revised: . , α d ) is a d -dimensional tuple of nonnegative integers satisfying | α | = α 1 + · · · + α d and D α f = ∂ | α | f ∂x α 11 ··· x αdd is the weak derivative of f .

Original: Now assume that M is a reasonably large constant, and let Θ be a closed interval over the real line.
Revised: Now assume that M is a reasonably large constant, and let Θ be a closed interval over the real line.

Original: We make the following assumptions for the true parameters under both schemes:
Revised: We make the following assumptions for the true parameters under both schemes:

Original: Condition 1 (True parameter, PF scheme) .
Revised: Condition 1 (True parameter, PF scheme) .

Original: The euclidean parameter θ 0 ∈ Θ ⊂ R , and the two function parameters m 0 ∈ W βM ([ − 1 , 1] d ) , h 0 ∈ W βM ([0 , τ ]) , and τ > 0 is the ending time of the study duration, which is usually adopted in the theoretical studies in survival analysis (Van der Vaart, 2000).
Revised: The euclidean parameter θ 0 ∈ Θ ⊂ R , and the two function parameters m 0 ∈ W βM ([ − 1 , 1] d ) , h 0 ∈ W βM ([0 , τ ]) , and τ > 0 is the ending time of the study duration, which is usually adopted in the theoretical studies in survival analysis (Van der Vaart, 2000).

Original: Condition 2 (True parameter, FN scheme) .
Revised: Condition 2 (True parameter, FN scheme) .

Original: The euclidean parameter θ 0 ∈ Θ ⊂ R , and the function parameter ν 0 ∈ W βM ([0 , τ ] × [ − 1 , 1] d ) ,
Revised: The euclidean parameter θ 0 ∈ Θ ⊂ R , and the function parameter ν 0 ∈ W βM ([0 , τ ] × [ − 1 , 1] d ) ,

Original: Next, we construct sieve spaces for function parameter approximation via restricting the complexity of the MLPs to ”scale” with the sample size n .
Revised: Next, we construct sieve spaces for function parameter approximation via restricting the complexity of the MLPs to ”scale” with the sample size n .

Original: Condition 3 (Sieve space, PF scheme) .
Revised: Condition 3 (Sieve space, PF scheme) .

Original: The sieve space H n is constructed as a set of MLPs satisfying (cid:98)
Revised: The sieve space H n is constructed as a set of MLPs satisfying (cid:98)

Original: h ∈ W βM
Revised: h ∈ W βM

Original: h
Revised: h

Original: ([0 , τ ]) , with depth of order O (log n ) and total number of parameters of order O ( n 1 β + d log n ) .
Revised: ([0 , τ ]) , with depth of order O (log n ) and total number of parameters of order O ( n 1 β + d log n ) .

Original: The sieve space M n is constructed as a set of MLPs satisfying (cid:98) m ∈ W βM m ([ − 1 , 1] d ) , with depth of order
Revised: The sieve space M n is constructed as a set of MLPs satisfying (cid:98) m ∈ W βM m ([ − 1 , 1] d ) , with depth of order

Original: O (log n ) and total number of parameters of order
Revised: O (log n ) and total number of parameters of order

Original: O ( n dβ + d log n ) .
Revised: O ( n dβ + d log n ) .

Original: Here M h and M m are sufficiently large constants such that every function in
Revised: Here M h and M m are sufficiently large constants such that every function in

Original: W βM ([ − 1 , 1] d ) and W βM ([0 , τ ]) could be accurately approximated by functions inside H n and M n , according to (Yarotsky, 2017, Theorem 1).
Revised: W βM ([ − 1 , 1] d ) and W βM ([0 , τ ]) could be accurately approximated by functions inside H n and M n , according to (Yarotsky, 2017, Theorem 1).

Original: Condition 4 (Sieve space, FN scheme) .
Revised: Condition 4 (Sieve space, FN scheme) .

Original: The sieve space V n is constructed as a set of MLPs satisfying (cid:98) ν
Revised: The sieve space V n is constructed as a set of MLPs satisfying (cid:98) ν

Original: ∈ W βM ν ([0 , τ ]) , with depth of order O (log n ) and total number of parameters of order
Revised: ∈ W βM ν ([0 , τ ]) , with depth of order O (log n ) and total number of parameters of order

Original: O ( n d +1 β + d +1 log n ) .
Revised: O ( n d +1 β + d +1 log n ) .

Original: Here M ν is a sufficiently large constant such that V n satisfies approximation properties, analogous to condition 3.
Revised: Here M ν is a sufficiently large constant such that V n satisfies approximation properties, analogous to condition 3.

Original: For technical reasons, we will assume the nonparametric function estimators are constrained to fall inside the corresponding sieve spaces, i.e., (cid:98) h n ∈ H n , (cid:98) m n ∈ M n and (cid:98) ν ∈ V n .
Revised: For technical reasons, we will assume the nonparametric function estimators are constrained to fall inside the corresponding sieve spaces, i.e., (cid:98) h n ∈ H n , (cid:98) m n ∈ M n and (cid:98) ν ∈ V n .

Original: This will not affect the implementation of optimization routines as was discussed in Farrell et al.
Revised: This will not affect the implementation of optimization routines as was discussed in Farrell et al.

Original: Furthermore, we restrict the estimate (cid:98) θ n ∈ Θ in both PF and FN schemes.
Revised: Furthermore, we restrict the estimate (cid:98) θ n ∈ Θ in both PF and FN schemes.

Original: Additionally, we need the following regularity condition on the function G θ ( x ) :
Revised: Additionally, we need the following regularity condition on the function G θ ( x ) :

Original: Condition 5.
Revised: Condition 5.

Original: G θ ( x ) is viewed as a bivariate function G : Θ × B
Revised: G θ ( x ) is viewed as a bivariate function G : Θ × B

Original: (cid:55)→ R , where B is a compact set on
Revised: (cid:55)→ R , where B is a compact set on

Original: R .
Revised: R .

Original: The functions G θ ( x ) , ∂∂θ G θ ( x ) , ∂∂x G θ ( x ) , log g θ ( x ) , ∂∂θ log g θ ( x ) , ∂∂x log g θ ( x ) are bounded on Θ × B .
Revised: The functions G θ ( x ) , ∂∂θ G θ ( x ) , ∂∂x G θ ( x ) , log g θ ( x ) , ∂∂θ log g θ ( x ) , ∂∂x log g θ ( x ) are bounded on Θ × B .

Original: We define two metrics that measures convergence of parameter estimates: For the PF scheme, let ϕ 

Revised: We define two metrics that measures convergence of parameter estimates: For the PF scheme, let ϕ

Original: = ( h 0 , m 0 , θ 0 ) be the true parameters and (cid:98)
Revised: = ( h 0 , m 0 , θ 0 ) be the true parameters and (cid:98)

Original: ϕ n = ( (cid:98)
Revised: ϕ n = ( (cid:98)

Original: h n , (cid:98) m n , (cid:98) θ n ) be the estimates.
Revised: h n , (cid:98) m n , (cid:98) θ n ) be the estimates.

Original: We abbreviate P ϕ 0 ,Z = z as the conditional probability distribution of ( T, δ ) given Z =
Revised: We abbreviate P ϕ 0 ,Z = z as the conditional probability distribution of ( T, δ ) given Z =

Original: z under the true parameter, and P (cid:98) ϕ n ,Z = z as the conditional probability distribution of ( T, δ ) given Z =
Revised: z under the true parameter, and P (cid:98) ϕ n ,Z = z as the conditional probability distribution of ( T, δ ) given Z =

Original: z under the estimates.
Revised: z under the estimates.

Original: Define the following metric
Revised: Define the following metric

Original: [Equation]
Revised: [Equation]

Original: where H 2 ( P ∥ Q ) = (cid:82) (cid:16) √ d P − √ d Q (cid:17) 2 is the squared Hellinger distance between probability distributions P and Q .
Revised: where H 2 ( P ∥ Q ) = (cid:82) (cid:16) √ d P − √ d Q (cid:17) 2 is the squared Hellinger distance between probability distributions P and Q .

Original: The case for the FN scheme is similar: Let ψ 

Revised: The case for the FN scheme is similar: Let ψ

Original: =
Revised: =

Original: ( ν 0 , θ 0 ) be the parameters and (cid:98) ν n =
Revised: ( ν 0 , θ 0 ) be the parameters and (cid:98) ν n =

Original: ( (cid:98) ν n , (cid:98) θ n ) be the estimates.
Revised: ( (cid:98) ν n , (cid:98) θ n ) be the estimates.

Original: Analogous to the definitions above, we define P ψ 0 ,
Revised: Analogous to the definitions above, we define P ψ 0 ,

Original: Z = z as the true conditional distribution given Z
Revised: Z = z as the true conditional distribution given Z

Original: = z , and P (cid:98) ψ n ,Z = z be the estimated conditional distribution, we will use the following metric in the FN scheme:
Revised: = z , and P (cid:98) ψ n ,Z = z be the estimated conditional distribution, we will use the following metric in the FN scheme:

Original: [Equation]
Revised: [Equation]

Original: Now we state our main theorems.
Revised: Now we state our main theorems.

Original: We denote P as the data generating distribution and use (cid:101) O to hide poly-logarithmic factors in the big-O notation.
Revised: We denote P as the data generating distribution and use (cid:101) O to hide poly-logarithmic factors in the big-O notation.

Original: Theorem 1 (Rate of convergence, PF scheme) .
Revised: Theorem 1 (Rate of convergence, PF scheme) .

Original: In the PF scheme, under condition 1, 3, 5, we have that
Revised: In the PF scheme, under condition 1, 3, 5, we have that

Original: [Equation]
Revised: [Equation]

Original: Theorem 2 (Rate of convergence, FN scheme) .
Revised: Theorem 2 (Rate of convergence, FN scheme) .

Original: In the FN scheme, under condition 2, 4, 5, we have that
Revised: In the FN scheme, under condition 2, 4, 5, we have that

Original: [Equation]
Revised: [Equation]

Original: Remark 2.
Revised: Remark 2.

Original: The idea of using Hellinger distance to measure the convergence rate of sieve MLEs was proposed in Wong & Shen (1995).
Revised: The idea of using Hellinger distance to measure the convergence rate of sieve MLEs was proposed in Wong & Shen (1995).

Original: Obtaining rates under a stronger topology such as L 2 is possible if the likelihood function satisfies certain conditions such as the curvature condition (Farrell et al., 2021).
Revised: Obtaining rates under a stronger topology such as L 2 is possible if the likelihood function satisfies certain conditions such as the curvature condition (Farrell et al., 2021).

Original: However, such kind of conditions are in general too stringent for likelihood-based objectives, instead, we use Hellinger convergence that has minimal requirements.
Revised: However, such kind of conditions are in general too stringent for likelihood-based objectives, instead, we use Hellinger convergence that has minimal requirements.

Original: Consequently, our proof strategy is applicable to many other survival models that rely on neural function approximation such as Rindt et al.
Revised: Consequently, our proof strategy is applicable to many other survival models that rely on neural function approximation such as Rindt et al.

Original: (2022), with some modification to the regularity conditions.
Revised: (2022), with some modification to the regularity conditions.

Original: For proper choices of metrics in sieve theory, see also the discussion in Chen (2007, Chapter 2).
Revised: For proper choices of metrics in sieve theory, see also the discussion in Chen (2007, Chapter 2).

Original: [Figure]
Revised: [Figure]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section
Revised: In this section

Original: [Figure]
Revised: [Figure]

Original: we assess the empirical performance of NFM.
Revised: we assess the empirical performance of NFM.

Original: We first conduct synthetic experiments for verifying the theoretical convergence guarantees developed in section 4.
Revised: We first conduct synthetic experiments for verifying the theoretical convergence guarantees developed in section 4.

Original: To further illustrate the empirical efficacy of NFM, we evaluate the predictive performance of NFM over 6 benchmark datasets ranging from small scale to large scale, against state-of-the-art baselines.
Revised: To further illustrate the empirical efficacy of NFM, we evaluate the predictive performance of NFM over 6 benchmark datasets ranging from small scale to large scale, against state-of-the-art baselines.

Original: 5. S YNTHETIC EXPERIMENTS
Revised: 5. S YNTHETIC EXPERIMENTS

Original: We conduct synthetic experiments to validate our proposed theory.
Revised: We conduct synthetic experiments to validate our proposed theory.

Original: The underlying data generating scheme is as follows:
Revised: The underlying data generating scheme is as follows:

Original: First, we generate a 5 -dimensional feature Z that is independently sampled from the uniform distribution over the interval [0 , 1] .
Revised: First, we generate a 5 -dimensional feature Z that is independently sampled from the uniform distribution over the interval [0 , 1] .

Original: The (true) conditional hazard function of the event time takes the form of the proportional frailty model (2), with h ( t )
Revised: The (true) conditional hazard function of the event time takes the form of the proportional frailty model (2), with h ( t )

Original: =
Revised: =

Original: t and m ( Z ) = sin( ⟨ Z, β ⟩ ) + ⟨ sin( Z ) , β ⟩ , where β
Revised: t and m ( Z ) = sin( ⟨ Z, β ⟩ ) + ⟨ sin( Z ) , β ⟩ , where β

Original: = (0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5) .
Revised: = (0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5) .

Original: The frailty ω is generated according to a gamma distribution with mean and variance equal to 1.
Revised: The frailty ω is generated according to a gamma distribution with mean and variance equal to 1.

Original: We use this generating model to assess the recovery guarantee of both NFM modeling schemes via inspecting the empirical recovery of ν ( t, Z ) .
Revised: We use this generating model to assess the recovery guarantee of both NFM modeling schemes via inspecting the empirical recovery of ν ( t, Z ) .

Original: For the PF scheme, we have more underlying information about the generating model, and we present an additional assessment regarding the recovery of m ( Z ) in appendix D.1.
Revised: For the PF scheme, we have more underlying information about the generating model, and we present an additional assessment regarding the recovery of m ( Z ) in appendix D.1.

Original: We generate three training datasets of different scales, with n ∈ { 1000 , 5000 , 10000 } .
Revised: We generate three training datasets of different scales, with n ∈ { 1000 , 5000 , 10000 } .

Original: A censoring mechanism is applied such that the censoring ratio is around 40% for each dataset.
Revised: A censoring mechanism is applied such that the censoring ratio is around 40% for each dataset.

Original: The assessment will be made on a fixed test sample of 100 hold-out points that are independently drawn from the generating scheme of the event time.
Revised: The assessment will be made on a fixed test sample of 100 hold-out points that are independently drawn from the generating scheme of the event time.

Original: We report a more detailed description of the implementation of the data generating scheme and model architectures in appendix C.2.
Revised: We report a more detailed description of the implementation of the data generating scheme and model architectures in appendix C.2.

Original: We present the results of our synthetic data experiments in figure 1.
Revised: We present the results of our synthetic data experiments in figure 1.

Original: The evaluation results suggest that both NFM schemes are capable of approximating complicated nonlinear functions using a moderate amount of data, i.e., n ≥ 1000 .
Revised: The evaluation results suggest that both NFM schemes are capable of approximating complicated nonlinear functions using a moderate amount of data, i.e., n ≥ 1000 .

Original: [Table]
Revised: [Table]

Original: 5. R EAL - WORLD DATA EXPERIMENTS
Revised: 5. R EAL - WORLD DATA EXPERIMENTS

Original: Datasets We use five survival datasets and one non-survival dataset for evaluation.
Revised: Datasets We use five survival datasets and one non-survival dataset for evaluation.

Original: The survival datasets include the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC)
Revised: The survival datasets include the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC)

Original: (Curtis et al., 2012), the Rotterdam tumor bank and German Breast Cancer Study Group (RotGBSG)(Knaus et al., 1995), the Assay Of Serum Free Light Chain (FLCHAIN) (Dispenzieri et al., 2012), the Study to Understand Prognoses Preferences Outcomes and Risks of Treatment (SUPPORT) (Knaus et al., 1995), and the Medical Information Mart for Intensive Care (MIMIC-III) (Johnson et al., 2016).
Revised: (Curtis et al., 2012), the Rotterdam tumor bank and German Breast Cancer Study Group (RotGBSG)(Knaus et al., 1995), the Assay Of Serum Free Light Chain (FLCHAIN) (Dispenzieri et al., 2012), the Study to Understand Prognoses Preferences Outcomes and Risks of Treatment (SUPPORT) (Knaus et al., 1995), and the Medical Information Mart for Intensive Care (MIMIC-III) (Johnson et al., 2016).

Original: For all the survival datasets, the event of interest is defined as the mortality after admission.
Revised: For all the survival datasets, the event of interest is defined as the mortality after admission.

Original: In our experiments, we view METABRIC, RotGBSG, FLCHAIN, and SUPPORT as small-scale datasets and MIMIC-III as a moderate-scale dataset.
Revised: In our experiments, we view METABRIC, RotGBSG, FLCHAIN, and SUPPORT as small-scale datasets and MIMIC-III as a moderate-scale dataset.

Original: We additionally use the KKBOX dataset (Kvamme et al., 2019) as a large-scale evaluation.
Revised: We additionally use the KKBOX dataset (Kvamme et al., 2019) as a large-scale evaluation.

Original: In this dataset, an event time is observed if a customer churns from the KKBOX platform.
Revised: In this dataset, an event time is observed if a customer churns from the KKBOX platform.

Original: We summarize the basic statistics of all the datasets in table 3.
Revised: We summarize the basic statistics of all the datasets in table 3.

Original: Baselines We compare NFM with 9 baselines.
Revised: Baselines We compare NFM with 9 baselines.

Original: The first one is the linear CoxPH model (Cox, 1972).
Revised: The first one is the linear CoxPH model (Cox, 1972).

Original: Gradient Boosting Machine (GBM) (Friedman, 2001; Chen & Guestrin, 2016) and Random Survival
Revised: Gradient Boosting Machine (GBM) (Friedman, 2001; Chen & Guestrin, 2016) and Random Survival

Original: Forests (RSF) (Ishwaran et al., 2008) are two tree-based nonparametric survival regression methods.
Revised: Forests (RSF) (Ishwaran et al., 2008) are two tree-based nonparametric survival regression methods.

Original: DeepSurv (Katzman et al., 2018) and CoxTime (Kvamme et al., 2019) are two models that adopt neural variants of partial likelihood as objectives.
Revised: DeepSurv (Katzman et al., 2018) and CoxTime (Kvamme et al., 2019) are two models that adopt neural variants of partial likelihood as objectives.

Original: SuMo-net (Rindt et al., 2022) is a neural variant of NHR.
Revised: SuMo-net (Rindt et al., 2022) is a neural variant of NHR.

Original: We additionally chose three latest state-of-the-art neural survival models: DeepHit (Lee et al., 2018), DeepEH (Zhong et al., 2021a), and SODEN (Tang et al., 2022).
Revised: We additionally chose three latest state-of-the-art neural survival models: DeepHit (Lee et al., 2018), DeepEH (Zhong et al., 2021a), and SODEN (Tang et al., 2022).

Original: Among the chosen baselines, DeepSurv and SuMo-net are viewed as implementations of neural CoxPH and neural
Revised: Among the chosen baselines, DeepSurv and SuMo-net are viewed as implementations of neural CoxPH and neural

Original: NHR and are therefore of particular interest for the empirical verification of the efficacy of frailty.
Revised: NHR and are therefore of particular interest for the empirical verification of the efficacy of frailty.

Original: A more thorough performance comparison with a larger set of baselines is provided in appendix D.3.
Revised: A more thorough performance comparison with a larger set of baselines is provided in appendix D.3.

Original: Evaluation strategy We use two standard metrics in survival predictions for evaluating model performance: integrated Brier score (IBS) and integrated negative binomial log-likelihood (INBLL).
Revised: Evaluation strategy We use two standard metrics in survival predictions for evaluating model performance: integrated Brier score (IBS) and integrated negative binomial log-likelihood (INBLL).

Original: Both metrics are derived from the following:
Revised: Both metrics are derived from the following:

Original: [Equation]
Revised: [Equation]

Original: Where (cid:98) S C ( t ) is an estimate of the survival function S C ( t ) of the censoring variable, obtained by the Kaplan-Meier estimate (Kaplan & Meier, 1958) of the censored observations on the test data.
Revised: Where (cid:98) S C ( t ) is an estimate of the survival function S C ( t ) of the censoring variable, obtained by the Kaplan-Meier estimate (Kaplan & Meier, 1958) of the censored observations on the test data.

Original: ℓ : { 0 , 1 } × [0 , 1] (cid:55)→ R + is some proper loss function for binary classification (Gneiting & Raftery, 2007).
Revised: ℓ : { 0 , 1 } × [0 , 1] (cid:55)→ R + is some proper loss function for binary classification (Gneiting & Raftery, 2007).

Original: The IBS metric corresponds to ℓ being the square loss, and the INBLL metric corresponds to ℓ being the negative binomial (Bernoulli) log-likelihood (Graf et al., 1999).
Revised: The IBS metric corresponds to ℓ being the square loss, and the INBLL metric corresponds to ℓ being the negative binomial (Bernoulli) log-likelihood (Graf et al., 1999).

Original: Both IBS and INBLL are proper scoring rules if the censoring times and survival times are independent.
Revised: Both IBS and INBLL are proper scoring rules if the censoring times and survival times are independent.

Original: 4 We additionally
Revised: 4 We additionally

Original: [Table]
Revised: [Table]

Original: report the result of another widely used metric, the concordance index (C-index), in appendix D.
Revised: report the result of another widely used metric, the concordance index (C-index), in appendix D.

Original: Since all the survival datasets do not have standard train/test splits, we follow previous practice (Zhong et al., 2021a) that uses 5 -fold cross-validation (CV): 1 fold is for testing, and 20% of the rest is held out for validation.
Revised: Since all the survival datasets do not have standard train/test splits, we follow previous practice (Zhong et al., 2021a) that uses 5 -fold cross-validation (CV): 1 fold is for testing, and 20% of the rest is held out for validation.

Original: In our experiments, we observed that a single random split into 5 folds does not produce stable results for most survival datasets.
Revised: In our experiments, we observed that a single random split into 5 folds does not produce stable results for most survival datasets.

Original: Therefore we perform 10 different CV runs for each survival dataset and report average metrics as well as their standard deviations.
Revised: Therefore we perform 10 different CV runs for each survival dataset and report average metrics as well as their standard deviations.

Original: For the
Revised: For the

Original: KKBOX dataset, we use the standard train/valid/test splits that are available via the pycox package (Kvamme et al., 2019) and report results based on 10 trial runs.
Revised: KKBOX dataset, we use the standard train/valid/test splits that are available via the pycox package (Kvamme et al., 2019) and report results based on 10 trial runs.

Original: Experimental setup We follow standard preprocessing strategies (Katzman et al., 2018; Kvamme et al., 2019; Zhong et al., 2021a)
Revised: Experimental setup We follow standard preprocessing strategies (Katzman et al., 2018; Kvamme et al., 2019; Zhong et al., 2021a)

Original: that standardize continuous features into zero mean and unit variance, and do one-hot encodings for all categorical features.
Revised: that standardize continuous features into zero mean and unit variance, and do one-hot encodings for all categorical features.

Original: We adopt MLP with ReLU activation for all function approximators, including (cid:98) h , (cid:98) m in PF scheme, and (cid:98) ν in FN scheme, across all datasets, with the number of layers (depth) and the number of hidden units (width) within each layer being tunable.
Revised: We adopt MLP with ReLU activation for all function approximators, including (cid:98) h , (cid:98) m in PF scheme, and (cid:98) ν in FN scheme, across all datasets, with the number of layers (depth) and the number of hidden units (width) within each layer being tunable.

Original: We tune the frailty transform over several standard choices detailed in appendix C.3.
Revised: We tune the frailty transform over several standard choices detailed in appendix C.3.

Original: We find that the gamma frailty configuration performs reasonably well across all tasks and is recommended to be the default choice.
Revised: We find that the gamma frailty configuration performs reasonably well across all tasks and is recommended to be the default choice.

Original: A more detailed description of the tuning procedure, as well as training configurations for baseline models, are reported in appendix C.3.
Revised: A more detailed description of the tuning procedure, as well as training configurations for baseline models, are reported in appendix C.3.

Original: Results we report experimental results of small-scale datasets in table 1, and results of two larger datasets in table 2.
Revised: Results we report experimental results of small-scale datasets in table 1, and results of two larger datasets in table 2.

Original: The proposed NFM framework achieves the best performance on 5 of thedatasets.
Revised: The proposed NFM framework achieves the best performance on 5 of thedatasets.

Original: The improvement over baselines is particularly evident in METABRIC, SUPPORT, and
Revised: The improvement over baselines is particularly evident in METABRIC, SUPPORT, and

Original: MIMIC-III datasets.
Revised: MIMIC-III datasets.

Original: Benefits of frailty to better understand the additional benefits of introducing the frailty formulation, we compute the (relative) performance gain of NFM-PF and NFM-FN, against their non-frailty counterparts, namely DeepSurv (Katzman et al., 2018) and SuMo-net
Revised: Benefits of frailty to better understand the additional benefits of introducing the frailty formulation, we compute the (relative) performance gain of NFM-PF and NFM-FN, against their non-frailty counterparts, namely DeepSurv (Katzman et al., 2018) and SuMo-net

Original: (Rindt et al., 2022).
Revised: (Rindt et al., 2022).

Original: The evaluation is conducted for all three metrics mentioned in this paper.
Revised: The evaluation is conducted for all three metrics mentioned in this paper.

Original: The results are shown in table 7.
Revised: The results are shown in table 7.

Original: The results suggest a solid improvement in incorporating frailty, as the relative increase in performance could be over 10% for both NFM models.
Revised: The results suggest a solid improvement in incorporating frailty, as the relative increase in performance could be over 10% for both NFM models.

Original: A more detailed discussion is presented in section D.5.
Revised: A more detailed discussion is presented in section D.5.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we make principled explorations on applying the idea of frailty models in modern survival analysis to neural survival regressions.
Revised: In this paper, we make principled explorations on applying the idea of frailty models in modern survival analysis to neural survival regressions.

Original: A flexible and scalable framework called NFM is proposed that includes many useful survival models as special cases.
Revised: A flexible and scalable framework called NFM is proposed that includes many useful survival models as special cases.

Original: Under the framework, we study two derived model architectures both theoretically and empirically.
Revised: Under the framework, we study two derived model architectures both theoretically and empirically.

Original: Theoretically, we obtain the rates of convergences of the nonparametric function estimators based on neural function approximation.
Revised: Theoretically, we obtain the rates of convergences of the nonparametric function estimators based on neural function approximation.

Original: Empirically, we demonstrate the superior predictive performance of the proposed models by evaluating several benchmark datasets.
Revised: Empirically, we demonstrate the superior predictive performance of the proposed models by evaluating several benchmark datasets.

Original: A E XAMPLES OF FRAILTY SPECIFICATIONS
Revised: A E XAMPLES OF FRAILTY SPECIFICATIONS

Original: We list several commonly used frailty models, and specify their corresponding characteristics via their frailty transform G θ :
Revised: We list several commonly used frailty models, and specify their corresponding characteristics via their frailty transform G θ :

Original: Gamma frailty: Arguably the gamma frailty is the most widely used frailty model Murphy (1994; 1995); Parner (1998); Wienke (2010); Duchateau & Janssen (2007), with
Revised: Gamma frailty: Arguably the gamma frailty is the most widely used frailty model Murphy (1994; 1995); Parner (1998); Wienke (2010); Duchateau & Janssen (2007), with

Original: [Equation]
Revised: [Equation]

Original: When θ = 0 , G 0 ( x ) =
Revised: When θ = 0 , G 0 ( x ) =

Original: lim θ → 0 G θ ( x ) is defined as the (pointwise) limit.
Revised: lim θ → 0 G θ ( x ) is defined as the (pointwise) limit.

Original: A notable fact of the gamma frailty specification is that when the proportional frailty (PF) assumption (2) is met, if θ = 0 , the model degenerates to CoxPH .
Revised: A notable fact of the gamma frailty specification is that when the proportional frailty (PF) assumption (2) is met, if θ = 0 , the model degenerates to CoxPH .

Original: Otherwise if θ = 1 , the model corresponds to the proportional odds (PO) model (Bennett, 1983).
Revised: Otherwise if θ = 1 , the model corresponds to the proportional odds (PO) model (Bennett, 1983).

Original: Box-Cox transformation frailty: Under this specification, we have
Revised: Box-Cox transformation frailty: Under this specification, we have

Original: [Equation]
Revised: [Equation]

Original: The case of θ = 0 is defined analogously to that of gamma frailty, which corresponds to the PO model under the PF assumption.
Revised: The case of θ = 0 is defined analogously to that of gamma frailty, which corresponds to the PO model under the PF assumption.

Original: When θ = 1 , the model reduces to CoxPH under the PF assumption.
Revised: When θ = 1 , the model reduces to CoxPH under the PF assumption.

Original: IGG ( α ) frailty: This is an extension of gamma frailty (Kosorok et al., 2004) and includes other types of frailty specifications like the inverse gaussian frailty Hougaard (1984), with
Revised: IGG ( α ) frailty: This is an extension of gamma frailty (Kosorok et al., 2004) and includes other types of frailty specifications like the inverse gaussian frailty Hougaard (1984), with

Original: [Equation]
Revised: [Equation]

Original: In the one-dimensional parameter paradigm, the parameter α is assumed known instead of being learnable.
Revised: In the one-dimensional parameter paradigm, the parameter α is assumed known instead of being learnable.

Original: When α = 1 / 2 , we obtain the gamma frailty model.
Revised: When α = 1 / 2 , we obtain the gamma frailty model.

Original: When α → 0 , the limit corresponds to the inverse Gaussian frailty.
Revised: When α → 0 , the limit corresponds to the inverse Gaussian frailty.

Original: Satistiability of regularity condition 

Revised: Satistiability of regularity condition

Original: In Kosorok et al.
Revised: In Kosorok et al.

Original: (2004, Proposition 1), the authors verified the regularity condition of gamma and IGG ( α ) frailties.
Revised: (2004, Proposition 1), the authors verified the regularity condition of gamma and IGG ( α ) frailties.

Original: Using a similar argument, it is straightforward to verify the regularity of Box-Cox transformation frailty.
Revised: Using a similar argument, it is straightforward to verify the regularity of Box-Cox transformation frailty.

Original: B P ROOFS OF THEOREMS B. P RELIMINARY Additional definitions
Revised: B P ROOFS OF THEOREMS B. P RELIMINARY Additional definitions

Original: The theory of empirical processes (van der Vaart et al., 1996) will be involved heavily in the proof.
Revised: The theory of empirical processes (van der Vaart et al., 1996) will be involved heavily in the proof.

Original: Therefore we briefly introduce some common notations: For a function class F , define N ( ϵ, F , ∥ · ∥ ) to be the covering number of F with respect to norm ∥·∥ under radius ϵ , and define N [] ( ϵ, F , ∥ · ∥ ) to be the bracketing number of F with respect to norm ∥ · ∥ under radius ϵ .
Revised: Therefore we briefly introduce some common notations: For a function class F , define N ( ϵ, F , ∥ · ∥ ) to be the covering number of F with respect to norm ∥·∥ under radius ϵ , and define N [] ( ϵ, F , ∥ · ∥ ) to be the bracketing number of F with respect to norm ∥ · ∥ under radius ϵ .

Original: We use VC ( F ) to denote the VC-dimension of F .
Revised: We use VC ( F ) to denote the VC-dimension of F .

Original: Moreover, we use the notation a ≲ b to denote a ≤ Cb for some positive constant C .
Revised: Moreover, we use the notation a ≲ b to denote a ≤ Cb for some positive constant C .

Original: Before proving theorem 1 and 2, we introduce some additional notations that will be useful throughout the proof process.
Revised: Before proving theorem 1 and 2, we introduce some additional notations that will be useful throughout the proof process.

Original: In the PF scheme, define
Revised: In the PF scheme, define

Original: [Equation]
Revised: [Equation]

Original: where we denote g θ = G ′ ( θ ) .
Revised: where we denote g θ = G ′ ( θ ) .

Original: Under the definition of the sieve space stated in condition 3, we restate the parameter estimates as
Revised: Under the definition of the sieve space stated in condition 3, we restate the parameter estimates as

Original: [Equation]
Revised: [Equation]

Original: Similarly, in the FN scheme, we define
Revised: Similarly, in the FN scheme, we define

Original: [Equation]
Revised: [Equation]

Original: Under the definition of the sieve space stated in condition
Revised: Under the definition of the sieve space stated in condition

Original: [Equation]
Revised: [Equation]

Original: we restate the parameter estimates as
Revised: we restate the parameter estimates as

Original: [Equation]
Revised: [Equation]

Original: We denote the conditional density function and survival function of the event time ˜ T given Z by f ˜ T | Z ( t ) and S ˜ T | Z ( t ) , respectively.
Revised: We denote the conditional density function and survival function of the event time ˜ T given Z by f ˜ T | Z ( t ) and S ˜ T | Z ( t ) , respectively.

Original: Similarly, we denote the conditional density function and survival function of the censoring time C given Z by f C | Z ( t ) and S C | Z ( t ) .
Revised: Similarly, we denote the conditional density function and survival function of the censoring time C given Z by f C | Z ( t ) and S C | Z ( t ) .

Original: Under the assumption that ˜ T ⊥⊥ C | Z , the joint conditional density of the observed time T and the censoring indicator δ given
Revised: Under the assumption that ˜ T ⊥⊥ C | Z , the joint conditional density of the observed time T and the censoring indicator δ given

Original: Z can be expressed as the following:
Revised: Z can be expressed as the following:

Original: [Equation]
Revised: [Equation]

Original: where λ ˜ T | Z ( T ) is the conditional hazard function of the survival time ˜ T given Z .
Revised: where λ ˜ T | Z ( T ) is the conditional hazard function of the survival time ˜ T given Z .

Original: Under the model assumption of PF scheme
Revised: Under the model assumption of PF scheme

Original: [Equation]
Revised: [Equation]

Original: p ( T, δ | Z ) can be expressed by
Revised: p ( T, δ | Z ) can be expressed by

Original: [Equation]
Revised: [Equation]

Original: | Z
Revised: | Z

Original: [Equation]
Revised: [Equation]

Original: For ϕ 0 = ( h 0 , m 0 , θ 0 ) and an estimator (cid:98) ϕ
Revised: For ϕ 0 = ( h 0 , m 0 , θ 0 ) and an estimator (cid:98) ϕ

Original: = ( (cid:98) h, (cid:98) m, (cid:98) θ ) , the defined distance d PF (cid:16)(cid:98) ϕ, ϕ 0 (cid:17) can be explicitly expresses by
Revised: = ( (cid:98) h, (cid:98) m, (cid:98) θ ) , the defined distance d PF (cid:16)(cid:98) ϕ, ϕ 0 (cid:17) can be explicitly expresses by

Original: [Equation]
Revised: [Equation]

Original: Here the dominating measure µ is defined such that for any (measurable) function r ( T, δ )
Revised: Here the dominating measure µ is defined such that for any (measurable) function r ( T, δ )

Original: [Equation]
Revised: [Equation]

Original: Under the model assumption of FN scheme, p ( T, δ | Z ) can be expressed by
Revised: Under the model assumption of FN scheme, p ( T, δ | Z ) can be expressed by

Original: [Equation]
Revised: [Equation]

Original: For ψ 0 = ( ν 0 , θ 0 ) and an estimator (cid:98) ψ
Revised: For ψ 0 = ( ν 0 , θ 0 ) and an estimator (cid:98) ψ

Original: = ( (cid:98) ν, (cid:98) θ ) , the defined distance d FN (cid:16) (cid:98) ψ, ψ 0 (cid:17) can be explicitly expresses by
Revised: = ( (cid:98) ν, (cid:98) θ ) , the defined distance d FN (cid:16) (cid:98) ψ, ψ 0 (cid:17) can be explicitly expresses by

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: B. T ECHNICAL LEMMAS
Revised: B. T ECHNICAL LEMMAS

Original: The following lemmas are needed for the proof of Theorem 1 and 2.
Revised: The following lemmas are needed for the proof of Theorem 1 and 2.

Original: Hereafter for notational convenience, we will use (cid:98) h, (cid:98) m for arbitrary elements in the corresponding sieve space listed in condition 3, (cid:98) ν for an arbitrary element in the sieve space listed in condition 4, and (cid:98) θ for an arbitrary element in Θ .
Revised: Hereafter for notational convenience, we will use (cid:98) h, (cid:98) m for arbitrary elements in the corresponding sieve space listed in condition 3, (cid:98) ν for an arbitrary element in the sieve space listed in condition 4, and (cid:98) θ for an arbitrary element in Θ .

Original: Lemma 1.
Revised: Lemma 1.

Original: Under condition 1, 3, 5, for ( T, δ, Z ) ∈ [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d , the following terms are bounded:
Revised: Under condition 1, 3, 5, for ( T, δ, Z ) ∈ [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d , the following terms are bounded:

Original: l ( T, δ, Z ; h 0 , m 0 , θ 0 ) with true parameter ( h 0 , m 0 , θ 0 ) 2.
Revised: l ( T, δ, Z ; h 0 , m 0 , θ 0 ) with true parameter ( h 0 , m 0 , θ 0 ) 2.

Original: l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) with parameter estimates ( (cid:98) h, (cid:98) m, (cid:98) θ ) in any sieve space listed in condition 3.
Revised: l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) with parameter estimates ( (cid:98) h, (cid:98) m, (cid:98) θ ) in any sieve space listed in condition 3.

Original: Lemma 2.
Revised: Lemma 2.

Original: Under condition 2, 4, 5, for ( T, δ, Z ) ∈ [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d , the following terms are bounded:
Revised: Under condition 2, 4, 5, for ( T, δ, Z ) ∈ [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d , the following terms are bounded:

Original: l ( T, δ, Z ; ν 0 , θ 0 ) with true parameter ( ν 0 , θ 0 ) 2.
Revised: l ( T, δ, Z ; ν 0 , θ 0 ) with true parameter ( ν 0 , θ 0 ) 2.

Original: l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) with parameter estimates ( (cid:98) ν, (cid:98) θ ) in any sieve space listed in condition 4.
Revised: l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) with parameter estimates ( (cid:98) ν, (cid:98) θ ) in any sieve space listed in condition 4.

Original: Lemma 3.
Revised: Lemma 3.

Original: Under condition 1, 3, 5, let ( (cid:98) h, (cid:98) m, (cid:98) θ ) ,
Revised: Under condition 1, 3, 5, let ( (cid:98) h, (cid:98) m, (cid:98) θ ) ,

Original: ( (cid:98) h 1 , (cid:98) m 1 , (cid:98) θ 1 ) , and ( (cid:98) h 2 , (cid:98) m 2 , (cid:98) θ 2 ) be arbitrary three parameter triples inside the sieve space defined in condition 3, the following two inequalities hold.
Revised: ( (cid:98) h 1 , (cid:98) m 1 , (cid:98) θ 1 ) , and ( (cid:98) h 2 , (cid:98) m 2 , (cid:98) θ 2 ) be arbitrary three parameter triples inside the sieve space defined in condition 3, the following two inequalities hold.

Original: ∥
Revised: ∥

Original: [Equation]
Revised: [Equation]

Original: ∥
Revised: ∥

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: θ
Revised: θ

Original: [Equation]
Revised: [Equation]

Original: Lemma 4.
Revised: Lemma 4.

Original: Under condition 2, 4, 5, let ( (cid:98) ν, (cid:98) θ ) , ( (cid:98) ν 1 , (cid:98) θ 1 ) , and ( (cid:98) ν 2 , (cid:98) θ 2 ) be arbitrary three parameter tuples inside the sieve space defined in condition 4,, the following inequalities hold.
Revised: Under condition 2, 4, 5, let ( (cid:98) ν, (cid:98) θ ) , ( (cid:98) ν 1 , (cid:98) θ 1 ) , and ( (cid:98) ν 2 , (cid:98) θ 2 ) be arbitrary three parameter tuples inside the sieve space defined in condition 4,, the following inequalities hold.

Original: [Equation]
Revised: [Equation]

Original: Lemma 5 (Approximating error of PF scheme) .
Revised: Lemma 5 (Approximating error of PF scheme) .

Original: In the PF scheme, for any n , there exists an element in the corresponding sieve space π n ϕ 

Revised: In the PF scheme, for any n , there exists an element in the corresponding sieve space π n ϕ

Original: = ( π n h 0 , π n m 0 , π n θ 0 ) , satisfying d PF ( π n ϕ 0 , ϕ 0 ) =
Revised: = ( π n h 0 , π n m 0 , π n θ 0 ) , satisfying d PF ( π n ϕ 0 , ϕ 0 ) =

Original: O (cid:16) n − ββ + d (cid:17) .
Revised: O (cid:16) n − ββ + d (cid:17) .

Original: Lemma 6 (Approximating error of FN scheme) .
Revised: Lemma 6 (Approximating error of FN scheme) .

Original: In the FN scheme, for any n , there exists an element in the corresponding sieve space π n ψ = ( π n ν 0 , π n θ 0 ) satisfying d FN ( π n ψ 0 , ψ 0 )
Revised: In the FN scheme, for any n , there exists an element in the corresponding sieve space π n ψ = ( π n ν 0 , π n θ 0 ) satisfying d FN ( π n ψ 0 , ψ 0 )

Original: =
Revised: =

Original: O (cid:16) n − β β + d +1 (cid:17) .
Revised: O (cid:16) n − β β + d +1 (cid:17) .

Original: Lemma 7.
Revised: Lemma 7.

Original: Suppose F is a class of functions satisfying that N ( ε, F , ∥ · ∥ ) <
Revised: Suppose F is a class of functions satisfying that N ( ε, F , ∥ · ∥ ) <

Original: ∞ for ∀ ε > 0 .
Revised: ∞ for ∀ ε > 0 .

Original: We define (cid:101) N ( ε, F , ∥ · ∥ ) to be the minimal number of ε -balls B ( f, ε ) =
Revised: We define (cid:101) N ( ε, F , ∥ · ∥ ) to be the minimal number of ε -balls B ( f, ε ) =

Original: { g : ∥ g − f ∥ < ε } needed to cover F with radius ε and further constrain that f ∈ F .
Revised: { g : ∥ g − f ∥ < ε } needed to cover F with radius ε and further constrain that f ∈ F .

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: Lemma 8.
Revised: Lemma 8.

Original: Suppose F is a class of functions satisfying that N [] ( ε, F , ∥ · ∥ ∞ ) < ∞ for ∀ ε > 0 .
Revised: Suppose F is a class of functions satisfying that N [] ( ε, F , ∥ · ∥ ∞ ) < ∞ for ∀ ε > 0 .

Original: We define (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) to be the minimal number of brackets [ l, u ] needed to cover F with ∥ l − u ∥ ∞ < ε and further constrain that f ∈ F , l = f − ε 2 and u = f
Revised: We define (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) to be the minimal number of brackets [ l, u ] needed to cover F with ∥ l − u ∥ ∞ < ε and further constrain that f ∈ F , l = f − ε 2 and u = f

Original: + ε 2 .
Revised: + ε 2 .

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: Furthermore, we have (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) =
Revised: Furthermore, we have (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) =

Original: (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .
Revised: (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .

Original: Lemma 9 (Model capacity of PF scheme) .
Revised: Lemma 9 (Model capacity of PF scheme) .

Original: Let F n = { l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ )
Revised: Let F n = { l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ )

Original: : (cid:98) h ∈
Revised: : (cid:98) h ∈

Original: H
Revised: H

Original: n , (cid:98) m ∈ M n , (cid:98) θ ∈ Θ } .
Revised: n , (cid:98) m ∈ M n , (cid:98) θ ∈ Θ } .

Original: Under condition 5, with s h = 2 β 2 β
Revised: Under condition 5, with s h = 2 β 2 β

Original: +

Revised: +

Original: and s m = 2 β 2 β + d , there exist constants c h and c m > 0 such that
Revised: and s m = 2 β 2 β + d , there exist constants c h and c m > 0 such that

Original: [Equation]
Revised: [Equation]

Original: Lemma 10 (Model capacity of FN scheme) .
Revised: Lemma 10 (Model capacity of FN scheme) .

Original: Let G n = { l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) : (cid:98) ν ∈ V n , (cid:98) θ ∈ Θ } .
Revised: Let G n = { l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) : (cid:98) ν ∈ V n , (cid:98) θ ∈ Θ } .

Original: Under condition 5, with s ν = 2 β 2 β + d +1 , there exists a constant c ν > 0 such that
Revised: Under condition 5, with s ν = 2 β 2 β + d +1 , there exists a constant c ν > 0 such that

Original: [Equation]
Revised: [Equation]

Original: B. P ROOFS OF THEOREM 1 AND
Revised: B. P ROOFS OF THEOREM 1 AND

Original: Proof of theorem 1.
Revised: Proof of theorem 1.

Original: The proof is divided into four steps.
Revised: The proof is divided into four steps.

Original: Step 1 We denote ϕ 

Revised: Step 1 We denote ϕ

Original: = ( h 0 , m 0 , θ 0 ) and (cid:98) ϕ
Revised: = ( h 0 , m 0 , θ 0 ) and (cid:98) ϕ

Original: = ( (cid:98) h, (cid:98) m, (cid:98) θ ) , where (cid:98) h ∈
Revised: = ( (cid:98) h, (cid:98) m, (cid:98) θ ) , where (cid:98) h ∈

Original: H n , (cid:98) m ∈ M n and (cid:98) θ ∈ Θ .
Revised: H n , (cid:98) m ∈ M n and (cid:98) θ ∈ Θ .

Original: For arbitrary small ε > 0 , we have that
Revised: For arbitrary small ε > 0 , we have that

Original: [Equation]
Revised: [Equation]

Original: Using the fact that D KL (cid:16) P (cid:98) ϕ,Z ∥ P ϕ 0 ,Z (cid:17) ≥ 2 H 2 ( P (cid:98) ϕ
Revised: Using the fact that D KL (cid:16) P (cid:98) ϕ,Z ∥ P ϕ 0 ,Z (cid:17) ≥ 2 H 2 ( P (cid:98) ϕ

Original: [Equation]
Revised: [Equation]

Original: Z ∥ P ϕ 0 ,Z ) .
Revised: Z ∥ P ϕ 0 ,Z ) .

Original: Thus, we further obtain that
Revised: Thus, we further obtain that

Original: [Equation]
Revised: [Equation]

Original: Step 2 Consider the following derivation.
Revised: Step 2 Consider the following derivation.

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original: 
Revised: 

Original: By Taylor’s expansion on log x
Revised: By Taylor’s expansion on log x

Original: [Equation]
Revised: [Equation]

Original: there exists ξ ( T, δ, Z ) between p 12 ( T, δ, Z ; h 0 , m 0 , θ 0 ) and p 12 ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) pointwisely such that p
Revised: there exists ξ ( T, δ, Z ) between p 12 ( T, δ, Z ; h 0 , m 0 , θ 0 ) and p 12 ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) pointwisely such that p

Original: [Equation]
Revised: [Equation]

Original: m
Revised: m

Original: [Equation]
Revised: [Equation]

Original: θ
Revised: θ

Original: [Equation]
Revised: [Equation]

Original: ) p ( T, δ, Z
Revised: ) p ( T, δ, Z

Original: [Equation]
Revised: [Equation]

Original: (cid:98) h, (cid:98) m, (cid:98) θ ) (cid:33)= p ( T, δ, Z ; h 0 , m 0 , θ 0 ) (cid:18) log (cid:112) p ( T, δ, Z ; h 0 , m 0 , θ 0 ) − log (cid:113) p ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) (cid:19)= p ( T, δ, Z ; h 0 , m 0 , θ 0 ) ξ ( T, δ, Z )(cid:18)(cid:112) p ( T, δ, Z ; h 0 , m 0 , θ 0 ) − (cid:113) p ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) (cid:19)
Revised: (cid:98) h, (cid:98) m, (cid:98) θ ) (cid:33)= p ( T, δ, Z ; h 0 , m 0 , θ 0 ) (cid:18) log (cid:112) p ( T, δ, Z ; h 0 , m 0 , θ 0 ) − log (cid:113) p ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) (cid:19)= p ( T, δ, Z ; h 0 , m 0 , θ 0 ) ξ ( T, δ, Z )(cid:18)(cid:112) p ( T, δ, Z ; h 0 , m 0 , θ 0 ) − (cid:113) p ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) (cid:19)

Original: Since p ( T, δ, Z ; h 0 , m 0 , θ 0 ) p (
Revised: Since p ( T, δ, Z ; h 0 , m 0 , θ 0 ) p (

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (cid:98)
Revised: (cid:98)

Original: [Equation]
Revised: [Equation]

Original: (cid:98)
Revised: (cid:98)

Original: [Equation]
Revised: [Equation]

Original: (cid:98)
Revised: (cid:98)

Original: [Equation]
Revised: [Equation]

Original: ) =
Revised: ) =

Original: e l ( T,
Revised: e l ( T,

Original: [Equation]
Revised: [Equation]

Original: Z
Revised: Z

Original: [Equation]
Revised: [Equation]

Original: h 0 ,m
Revised: h 0 ,m

Original: [Equation]
Revised: [Equation]

Original: 0 ) −
Revised: 0 ) −

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ,δ
Revised: ,δ

Original: [Equation]
Revised: [Equation]

Original: (cid:98)
Revised: (cid:98)

Original: [Equation]
Revised: [Equation]

Original: , (cid:98)
Revised: , (cid:98)

Original: [Equation]
Revised: [Equation]

Original: , (cid:98) θ ) by lemma 1, l ( T, δ, Z ; h 0 , m 0 , θ 0 ) and l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) are bounded among [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d uniformly on all (cid:98) ϕ
Revised: , (cid:98) θ ) by lemma 1, l ( T, δ, Z ; h 0 , m 0 , θ 0 ) and l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) are bounded among [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d uniformly on all (cid:98) ϕ

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: there exist constants C 1 and C 2 such that
Revised: there exist constants C 1 and C 2 such that

Original: [Equation]
Revised: [Equation]

Original: This leads to the fact that p
Revised: This leads to the fact that p

Original: [Equation]
Revised: [Equation]

Original: T,
Revised: T,

Original: [Equation]
Revised: [Equation]

Original: , Z ;
Revised: , Z ;

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: m
Revised: m

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: is bounded.
Revised: is bounded.

Original: We further obtained that
Revised: We further obtained that

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: we have that sup
Revised: we have that sup

Original: [Equation]
Revised: [Equation]

Original: ≤ ε 2 .
Revised: ≤ ε 2 .

Original: Step 

Revised: Step

Original: We define that (cid:101) F n = { l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) − l ( T, δ, Z ; π n h 0 , π n m 0 , π n θ 0 ) : (cid:98) h ∈
Revised: We define that (cid:101) F n = { l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) − l ( T, δ, Z ; π n h 0 , π n m 0 , π n θ 0 ) : (cid:98) h ∈

Original: H
Revised: H

Original: n , (cid:98) m ∈
Revised: n , (cid:98) m ∈

Original: M n , (cid:98) θ ∈ Θ } .
Revised: M n , (cid:98) θ ∈ Θ } .

Original: Here ( π n h 0 , π n m 0 , π n θ 0 ) have been defined in lemma 5.
Revised: Here ( π n h 0 , π n m 0 , π n θ 0 ) have been defined in lemma 5.

Original: Obviously, we have that log N [] ( ε, (cid:101) F n , ∥ · ∥ ∞ )
Revised: Obviously, we have that log N [] ( ε, (cid:101) F n , ∥ · ∥ ∞ )

Original: = log N [] ( ε, F n , ∥ · ∥ ∞ ) , where F is defined in lemma 9.
Revised: = log N [] ( ε, F n , ∥ · ∥ ∞ ) , where F is defined in lemma 9.

Original: By lemma 9, we further have that
Revised: By lemma 9, we further have that

Original: [Equation]
Revised: [Equation]

Original: According to Bartlett et al.
Revised: According to Bartlett et al.

Original: (2019, Theorem 7), under condition 3, we have that the VC-dimension of
Revised: (2019, Theorem 7), under condition 3, we have that the VC-dimension of

Original: H n and M n satisfy that VC ( H n ) ≲
Revised: H n and M n satisfy that VC ( H n ) ≲

Original: [Equation]
Revised: [Equation]

Original: 1 β
Revised: 1 β

Original: [Equation]
Revised: [Equation]

Original: n and VC ( M n )
Revised: n and VC ( M n )

Original: [Equation]
Revised: [Equation]

Original: n
Revised: n

Original: [Equation]
Revised: [Equation]

Original: d log 3 n .
Revised: d log 3 n .

Original: Thus, we obtain that
Revised: Thus, we obtain that

Original: [Equation]
Revised: [Equation]

Original: and log N
Revised: and log N

Original: [Equation]
Revised: [Equation]

Original: Thus, we obtain that log N [] ( ε, (cid:101) F n , ∥ · ∥ ∞ ) ≲ n dβ + d log 3 n log 1 ε
Revised: Thus, we obtain that log N [] ( ε, (cid:101) F n , ∥ · ∥ ∞ ) ≲ n dβ + d log 3 n log 1 ε

Original: [Equation]
Revised: [Equation]

Original: Step 4 By the Cauchy-Schwartz inequality, we have that
Revised: Step 4 By the Cauchy-Schwartz inequality, we have that

Original: [Equation]
Revised: [Equation]

Original: Similar to the second part and by lemma 5, we further have that
Revised: Similar to the second part and by lemma 5, we further have that

Original: [Equation]
Revised: [Equation]

Original: Now let
Revised: Now let

Original: [Equation]
Revised: [Equation]

Original: By Step 1,2,3 and Shen & Wong (1994, Theorem 1), we have
Revised: By Step 1,2,3 and Shen & Wong (1994, Theorem 1), we have

Original: [Equation]
Revised: [Equation]

Original: By lemma 5, d PF ( π n ϕ 0 , ϕ 0 )
Revised: By lemma 5, d PF ( π n ϕ 0 , ϕ 0 )

Original: =
Revised: =

Original: O ( n − ββ + d ) .
Revised: O ( n − ββ + d ) .

Original: By Step 4, (cid:112) E [ l ( T, δ, Z ; π n h 0 , π n m 0 , π n θ 0 ) − l ( T, δ, Z ; h 0 , m 0 , θ 0 )]
Revised: By Step 4, (cid:112) E [ l ( T, δ, Z ; π n h 0 , π n m 0 , π n θ 0 ) − l ( T, δ, Z ; h 0 , m 0 , θ 0 )]

Original: = O (cid:16) n − β 2 β +2 d (cid:17) .
Revised: = O (cid:16) n − β 2 β +2 d (cid:17) .

Original: Thus, we have d PF
Revised: Thus, we have d PF

Original: [Equation]
Revised: [Equation]

Original: Proof of theorem 2.
Revised: Proof of theorem 2.

Original: The proof is divided into four steps.
Revised: The proof is divided into four steps.

Original: Step 1 We denote ψ 0 = ( ν 0 , θ 0 ) and (cid:98) ψ
Revised: Step 1 We denote ψ 0 = ( ν 0 , θ 0 ) and (cid:98) ψ

Original: = ( (cid:98) ν, (cid:98) θ ) , where (cid:98) ν ∈ V n and (cid:98) θ ∈ Θ .
Revised: = ( (cid:98) ν, (cid:98) θ ) , where (cid:98) ν ∈ V n and (cid:98) θ ∈ Θ .

Original: For arbitrary 0 < ε ≤ 1 , we have that
Revised: For arbitrary 0 < ε ≤ 1 , we have that

Original: [Equation]
Revised: [Equation]

Original: Using the fact that KL ( P (cid:98) ψ,Z ∥ P ψ 0 ,Z ) ≥ 2 H 2 ( P (cid:98) ψ,Z ∥ P ψ 0 ,Z ) .
Revised: Using the fact that KL ( P (cid:98) ψ,Z ∥ P ψ 0 ,Z ) ≥ 2 H 2 ( P (cid:98) ψ,Z ∥ P ψ 0 ,Z ) .

Original: Thus, we further obtain that
Revised: Thus, we further obtain that

Original: [Equation]
Revised: [Equation]

Original: Step 

Revised: Step

Original: We consider the following derivation.
Revised: We consider the following derivation.

Original: [Equation]
Revised: [Equation]

Original: By Taylor’s expansion on log x
Revised: By Taylor’s expansion on log x

Original: [Equation]
Revised: [Equation]

Original: there exists η ( T
Revised: there exists η ( T

Original: [Equation]
Revised: [Equation]

Original: δ
Revised: δ

Original: [Equation]
Revised: [Equation]

Original: Z ) between (cid:112) p ( T, δ, Z ; ν 0 , θ 0 ) and (cid:113) p ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) pointwisely such that
Revised: Z ) between (cid:112) p ( T, δ, Z ; ν 0 , θ 0 ) and (cid:113) p ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) pointwisely such that

Original: [Equation]
Revised: [Equation]

Original: Since p ( T, δ, Z ; ν 0 , θ 0 )
Revised: Since p ( T, δ, Z ; ν 0 , θ 0 )

Original: /p
Revised: /p

Original: ( T, δ, Z ; (cid:98) ν, (cid:98) θ )
Revised: ( T, δ, Z ; (cid:98) ν, (cid:98) θ )

Original: =
Revised: =

Original: e l ( T,δ,Z ; ν 0 ,θ 0 ) − l ( T,δ,Z ; (cid:98) ν, (cid:98) θ ) , by lemma 2, l ( T, δ, Z ; ν 0 , θ 0 ) and l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) are bounded on [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d uniformly for all (cid:98) ψ
Revised: e l ( T,δ,Z ; ν 0 ,θ 0 ) − l ( T,δ,Z ; (cid:98) ν, (cid:98) θ ) , by lemma 2, l ( T, δ, Z ; ν 0 , θ 0 ) and l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) are bounded on [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d uniformly for all (cid:98) ψ

Original: = ( (cid:98) ν, (cid:98) θ ) .
Revised: = ( (cid:98) ν, (cid:98) θ ) .

Original: Thus there exist constants C 3 and C 4 such that 0 < C 3 ≤ p ( T, δ, Z ; ν 0 , θ 0 ) /p
Revised: Thus there exist constants C 3 and C 4 such that 0 < C 3 ≤ p ( T, δ, Z ; ν 0 , θ 0 ) /p

Original: ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) ≤ C 4 .
Revised: ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) ≤ C 4 .

Original: This leads to the fact that p ( T, δ, Z ; ν 0 , θ 0 ) 1 η ( T,δ,Z ) 2 is bounded.
Revised: This leads to the fact that p ( T, δ, Z ; ν 0 , θ 0 ) 1 η ( T,δ,Z ) 2 is bounded.

Original: We further have that
Revised: We further have that

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: Thus, we have that
Revised: Thus, we have that

Original: [Equation]
Revised: [Equation]

Original: ≤
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: Step 

Revised: Step

Original: We define that (cid:101) G n = { l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) − l ( T, δ, Z ; π n ν 0 , π n θ 0 ) : (cid:98) ν ∈ V n , θ ∈ Θ } .
Revised: We define that (cid:101) G n = { l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) − l ( T, δ, Z ; π n ν 0 , π n θ 0 ) : (cid:98) ν ∈ V n , θ ∈ Θ } .

Original: Here ( π n ν 0 , π n θ 0 ) have been defined in lemma 6.
Revised: Here ( π n ν 0 , π n θ 0 ) have been defined in lemma 6.

Original: Obviously, we have that log N [] ( ε, (cid:101) G n , ∥ · ∥ ∞ )
Revised: Obviously, we have that log N [] ( ε, (cid:101) G n , ∥ · ∥ ∞ )

Original: =
Revised: =

Original: log N [] ( ε, G n , ∥ · ∥ ∞ ) , where G is defined in lemma 10.
Revised: log N [] ( ε, G n , ∥ · ∥ ∞ ) , where G is defined in lemma 10.

Original: By lemma 10, we further obtain that
Revised: By lemma 10, we further obtain that

Original: [Equation]
Revised: [Equation]

Original: According to Bartlett et al.
Revised: According to Bartlett et al.

Original: (2019, Theorem 7), under condition 4, we have that the VC-dimension of V n satisfies that VC ( V n ) ≲ n d +1 β + d +1 log 3 n .
Revised: (2019, Theorem 7), under condition 4, we have that the VC-dimension of V n satisfies that VC ( V n ) ≲ n d +1 β + d +1 log 3 n .

Original: Thus, we obtain that
Revised: Thus, we obtain that

Original: [Equation]
Revised: [Equation]

Original: Furthermore, we get that log N [] ( ε, (cid:101) G n , ∥ · ∥ ∞ ) ≲ n d +1 β +
Revised: Furthermore, we get that log N [] ( ε, (cid:101) G n , ∥ · ∥ ∞ ) ≲ n d +1 β +

Original: [Equation]
Revised: [Equation]

Original: +1 log 3 n log 1 ε .
Revised: +1 log 3 n log 1 ε .

Original: Step 4 By the Cauchy-Schwartz inequality, we have that
Revised: Step 4 By the Cauchy-Schwartz inequality, we have that

Original: [Equation]
Revised: [Equation]

Original: Similar to the second part and by lemma 6, we further obtain that
Revised: Similar to the second part and by lemma 6, we further obtain that

Original: [Equation]
Revised: [Equation]

Original: Now let τ =
Revised: Now let τ =

Original: [Equation]
Revised: [Equation]

Original: By step 1,2,3 and Step 1,2,3 and Shen & Wong (1994, Theorem 1),
Revised: By step 1,2,3 and Step 1,2,3 and Shen & Wong (1994, Theorem 1),

Original: [Equation]
Revised: [Equation]

Original: By lemma6, d FN ( π n ψ 0 , ψ 0 )
Revised: By lemma6, d FN ( π n ψ 0 , ψ 0 )

Original: =
Revised: =

Original: O ( n − β β + d +1 )
Revised: O ( n − β β + d +1 )

Original: By Step 4, (cid:112) E [ l ( T, δ, Z ; π n ν 0 , π n θ 0 ) − l ( T, δ, Z ; ν 0 , θ 0 )]
Revised: By Step 4, (cid:112) E [ l ( T, δ, Z ; π n ν 0 , π n θ 0 ) − l ( T, δ, Z ; ν 0 , θ 0 )]

Original: = O ( n − β 2 β +2 d +2 ) .
Revised: = O ( n − β 2 β +2 d +2 ) .

Original: Thus, we have d FN (cid:16) (cid:98) ψ n , ψ 0 (cid:17) =
Revised: Thus, we have d FN (cid:16) (cid:98) ψ n , ψ 0 (cid:17) =

Original: O ( n − β
Revised: O ( n − β

Original: [Equation]
Revised: [Equation]

Original: +2 log 2 n ) = (cid:101) O ( n − β 2 β +2 d +2 ) .
Revised: +2 log 2 n ) = (cid:101) O ( n − β 2 β +2 d +2 ) .

Original: B. P ROOFS OF TECHNICAL LEMMAS
Revised: B. P ROOFS OF TECHNICAL LEMMAS

Original: Proof of lemma 1.
Revised: Proof of lemma 1.

Original: Since h 0 ( T ) ∈ W βM ([0 , τ ]) and m 0 ( Z ) ∈ W βM ([ − 1 , 1] d ) , we have that h 0 ( T ) ≤
Revised: Since h 0 ( T ) ∈ W βM ([0 , τ ]) and m 0 ( Z ) ∈ W βM ([ − 1 , 1] d ) , we have that h 0 ( T ) ≤

Original: M , m 0 ( Z ) ≤ M and e m 

Revised: M , m 0 ( Z ) ≤ M and e m

Original: ( Z ) (cid:82) T 

Revised: ( Z ) (cid:82) T

Original: h 0 ( s ) ds ≤ τe 2 M .
Revised: h 0 ( s ) ds ≤ τe 2 M .

Original: Let B = [0 , τe 2 M ] , we have that | l
Revised: Let B = [0 , τe 2 M ] , we have that | l

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ≤
Revised: ≤

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: θ
Revised: θ

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ds (cid:33)(cid:12) (cid:12) (cid:12) (cid:12)(cid:12) ≤ 2 M + sup x ∈B | log g θ 

Revised: ds (cid:33)(cid:12) (cid:12) (cid:12) (cid:12)(cid:12) ≤ 2 M + sup x ∈B | log g θ

Original: ( x ) | + sup x ∈B | G θ 0 ( x ) |
Revised: ( x ) | + sup x ∈B | G θ 0 ( x ) |

Original: By condition 5, we have that l ( T, δ, Z ; h 0 , m 0 , θ 0 ) is bounded for ( T, δ, Z ) ∈ [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d .
Revised: By condition 5, we have that l ( T, δ, Z ; h 0 , m 0 , θ 0 ) is bounded for ( T, δ, Z ) ∈ [0 , τ ] × { 0 , 1 } × [ − 1 , 1] d .

Original: The proof of the boundness of l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) is similar.
Revised: The proof of the boundness of l ( T, δ, Z ; (cid:98) h, (cid:98) m, (cid:98) θ ) is similar.

Original: Proof of lemma 2.
Revised: Proof of lemma 2.

Original: Since ν 0 ( T, Z ) ∈ W βM ([0 , τ ] × [ − 1 , 1] d ) , we have ν 

Revised: Since ν 0 ( T, Z ) ∈ W βM ([0 , τ ] × [ − 1 , 1] d ) , we have ν

Original: ( T, Z ) ≤ M and (cid:82) T 0 e ν ( s,Z ) ds ≤ τe M .
Revised: ( T, Z ) ≤ M and (cid:82) T 0 e ν ( s,Z ) ds ≤ τe M .

Original: Let B = [0 , τe M ] , we have that
Revised: Let B = [0 , τe M ] , we have that

Original: [Equation]
Revised: [Equation]

Original: By condition 5, we have that l ( T, δ, Z ; ν 0 , θ 0 ) is bounded among ( T, δ, Z ) ∈ [0 , τ ] ×{ 0 , 1 }× [ − 1 , 1] d
Revised: By condition 5, we have that l ( T, δ, Z ; ν 0 , θ 0 ) is bounded among ( T, δ, Z ) ∈ [0 , τ ] ×{ 0 , 1 }× [ − 1 , 1] d

Original: The proof of the boundness of l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) is similar.
Revised: The proof of the boundness of l ( T, δ, Z ; (cid:98) ν, (cid:98) θ ) is similar.

Original: Proof of lemma 3.
Revised: Proof of lemma 3.

Original: By definition, we have that
Revised: By definition, we have that

Original: [Equation]
Revised: [Equation]

Original: Let B = [0 , τ max( e 2 M , e M h + M m )] .
Revised: Let B = [0 , τ max( e 2 M , e M h + M m )] .

Original: By Taylor’s expansion
Revised: By Taylor’s expansion

Original: [Equation]
Revised: [Equation]

Original: we can further show that
Revised: we can further show that

Original: [Equation]
Revised: [Equation]

Original: Again, by Taylor’s expansion
Revised: Again, by Taylor’s expansion

Original: [Equation]
Revised: [Equation]

Original: we have that (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
Revised: we have that (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: we obtain that
Revised: we obtain that

Original: [Equation]
Revised: [Equation]

Original: (cid:98) h ∥
Revised: (cid:98) h ∥

Original: [Equation]
Revised: [Equation]

Original: M h · e max( M
Revised: M h · e max( M

Original: [Equation]
Revised: [Equation]

Original: M m ) ∥ m 0 − (cid:98) m ∥ ∞ (cid:105)
Revised: M m ) ∥ m 0 − (cid:98) m ∥ ∞ (cid:105)

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: (cid:12)(cid:12) (cid:12)(cid:12) ∂ log g ˜ θ (˜ x ) ∂ ˜ θ (cid:12)(cid:12) (cid:12)(cid:12) · (cid:12)(cid:12) (cid:12) θ 0 − (cid:98) θ (cid:12)(cid:12) (cid:12) + (cid:12)(cid:12) (cid:12) h 0 (
Revised: (cid:12)(cid:12) (cid:12)(cid:12) ∂ log g ˜ θ (˜ x ) ∂ ˜ θ (cid:12)(cid:12) (cid:12)(cid:12) · (cid:12)(cid:12) (cid:12) θ 0 − (cid:98) θ (cid:12)(cid:12) (cid:12) + (cid:12)(cid:12) (cid:12) h 0 (

Original: [Equation]
Revised: [Equation]

Original: ) − (cid:98) h (
Revised: ) − (cid:98) h (

Original: [Equation]
Revised: [Equation]

Original: ) (cid:12)(cid:12) (cid:12)
Revised: ) (cid:12)(cid:12) (cid:12)

Original: [Equation]
Revised: [Equation]

Original: m
Revised: m

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (cid:98) m
Revised: (cid:98) m

Original: [Equation]
Revised: [Equation]

Original: ) | +
Revised: ) | +

Original: [Equation]
Revised: [Equation]

Original: (cid:12)(cid:12)(cid:12)(cid:12) ∂G ˜
Revised: (cid:12)(cid:12)(cid:12)(cid:12) ∂G ˜

Original: [Equation]
Revised: [Equation]

Original: (˜ x ) ∂ ˜ θ (cid:12)(cid:12)(cid:12)(cid:12) · (cid:12)(cid:12)(cid:12) θ 0 − (cid:98) θ (cid:12)(cid:12)(cid:12) + sup
Revised: (˜ x ) ∂ ˜ θ (cid:12)(cid:12)(cid:12)(cid:12) · (cid:12)(cid:12)(cid:12) θ 0 − (cid:98) θ (cid:12)(cid:12)(cid:12) + sup

Original: [Equation]
Revised: [Equation]

Original: ˜
Revised: ˜

Original: [Equation]
Revised: [Equation]

Original: ∈B (cid:12)(cid:12)(cid:12)(cid:12) ∂G
Revised: ∈B (cid:12)(cid:12)(cid:12)(cid:12) ∂G

Original: ˜ θ
Revised: ˜ θ

Original: (˜ x ) ∂ ˜ x (cid:12)(cid:12)(cid:12)(cid:12) · (cid:104) e M · τe max( M,M h ) ∥
Revised: (˜ x ) ∂ ˜ x (cid:12)(cid:12)(cid:12)(cid:12) · (cid:104) e M · τe max( M,M h ) ∥

Original: h 0 − (cid:98) h
Revised: h 0 − (cid:98) h

Original: ∥ ∞ + τe M h · e max( M,M m )
Revised: ∥ ∞ + τe M h · e max( M,M m )

Original: ∥ m 0 − (cid:98) m ∥ ∞ (cid:105) .
Revised: ∥ m 0 − (cid:98) m ∥ ∞ (cid:105) .

Original: Taking supremum on both sides, we conclude that
Revised: Taking supremum on both sides, we conclude that

Original: [Equation]
Revised: [Equation]

Original: The proof of the second inequality is similar.
Revised: The proof of the second inequality is similar.

Original: Proof of lemma 4.
Revised: Proof of lemma 4.

Original: By definition, we have that
Revised: By definition, we have that

Original: [Equation]
Revised: [Equation]

Original: Let B = [0 , τ max( e M , e M ν )] .
Revised: Let B = [0 , τ max( e M , e M ν )] .

Original: By Taylor’s expansion
Revised: By Taylor’s expansion

Original: [Equation]
Revised: [Equation]

Original: we can further show that
Revised: we can further show that

Original: [Equation]
Revised: [Equation]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ) −
Revised: ) −

Original: [Table] [Table]
Revised: [Table] [Table]

Original: , ˜ x
Revised: , ˜ x

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: Again, by Taylor’s expansion,
Revised: Again, by Taylor’s expansion,

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: we obtain that
Revised: we obtain that

Original: [Equation]
Revised: [Equation]

Original: Taking supremum on both sides, we conclude that
Revised: Taking supremum on both sides, we conclude that

Original: [Equation]
Revised: [Equation]

Original: The proof of the second inequality is similar.
Revised: The proof of the second inequality is similar.

Original: Proof of lemma 5.
Revised: Proof of lemma 5.

Original: According to Yarotsky (2017, Theorem 1), there exist approximating functions (cid:98) h ∗ and (cid:98) m ∗ such that ∥ (cid:98) h ∗ − h 

Revised: According to Yarotsky (2017, Theorem 1), there exist approximating functions (cid:98) h ∗ and (cid:98) m ∗ such that ∥ (cid:98) h ∗ − h

Original: ∥ ∞ =
Revised: ∥ ∞ =

Original: O (cid:16) n − ββ + d (cid:17) and ∥ (cid:98) m ∗ − m 

Revised: O (cid:16) n − ββ + d (cid:17) and ∥ (cid:98) m ∗ − m

Original: ∥ ∞ =
Revised: ∥ ∞ =

Original: O (cid:16) n − ββ + d (cid:17) .
Revised: O (cid:16) n − ββ + d (cid:17) .

Original: Let π n h 

Revised: Let π n h

Original: = (cid:98) h ∗ , π n m 

Revised: = (cid:98) h ∗ , π n m

Original: = (cid:98) m ∗ , and π n θ = θ 0 .
Revised: = (cid:98) m ∗ , and π n θ = θ 0 .

Original: We have that
Revised: We have that

Original: [Equation]
Revised: [Equation]

Original: By lemma 1 and
Revised: By lemma 1 and

Original: [Equation]
Revised: [Equation]

Original: we have that
Revised: we have that

Original: [Equation]
Revised: [Equation]

Original: Since f C | Z ( T ) 1 − δ ≤ f C | Z ( T ) + 1 and S C | Z ( T ) δ ≤ 1 , we also have that
Revised: Since f C | Z ( T ) 1 − δ ≤ f C | Z ( T ) + 1 and S C | Z ( T ) δ ≤ 1 , we also have that

Original: [Equation]
Revised: [Equation]

Original: Thus, we obtain that d PF ( π n ϕ 0 , ϕ 0 ) =
Revised: Thus, we obtain that d PF ( π n ϕ 0 , ϕ 0 ) =

Original: O (cid:16) n − ββ + d (cid:17) .
Revised: O (cid:16) n − ββ + d (cid:17) .

Original: Proof of lemma 6.
Revised: Proof of lemma 6.

Original: According to Yarotsky (2017, Theorem 1), there exists an approximating function (cid:98) ν ∗ such
Revised: According to Yarotsky (2017, Theorem 1), there exists an approximating function (cid:98) ν ∗ such

Original: that ∥ (cid:98) ν ∗ − ν 0 ∥ ∞ = O (cid:16) n − β β + d +1 (cid:17) .
Revised: that ∥ (cid:98) ν ∗ − ν 0 ∥ ∞ = O (cid:16) n − β β + d +1 (cid:17) .

Original: Let π n ν 

Revised: Let π n ν

Original: = (cid:98) ν ∗ and π n θ 

Revised: = (cid:98) ν ∗ and π n θ

Original: = θ 0 .
Revised: = θ 0 .

Original: We have that d
Revised: We have that d

Original: [Equation]
Revised: [Equation]

Original: ∞
Revised: ∞

Original: [Equation]
Revised: [Equation]

Original: By lemma 2 and 4, we have that (cid:13)(cid:13)(cid:13) e 12 l
Revised: By lemma 2 and 4, we have that (cid:13)(cid:13)(cid:13) e 12 l

Original: [Equation]
Revised: [Equation]

Original: T,δ,Z ; π n ν 0 ,π n θ 0 ) − e 12 l
Revised: T,δ,Z ; π n ν 0 ,π n θ 0 ) − e 12 l

Original: [Equation]
Revised: [Equation]

Original: T,δ,Z ; ν 0 ,θ 0 )
Revised: T,δ,Z ; ν 0 ,θ 0 )

Original: [Equation]
Revised: [Equation]

Original: ∞ ≲
Revised: ∞ ≲

Original: [Equation]
Revised: [Equation]

Original: π n θ 0 − θ 

Revised: π n θ 0 − θ

Original: ∥ + ∥
Revised: ∥ + ∥

Original: π n ν 0 − ν 0 ∥ ∞ =
Revised: π n ν 0 − ν 0 ∥ ∞ =

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original: n − β β
Revised: n − β β

Original: + d +1 (cid:17) .
Revised: + d +1 (cid:17) .

Original: Since f C | Z ( T ) 1 − δ ≤ f C | Z ( T ) + 1 and S C | Z ( T ) δ ≤ 1 , we also have that
Revised: Since f C | Z ( T ) 1 − δ ≤ f C | Z ( T ) + 1 and S C | Z ( T ) δ ≤ 1 , we also have that

Original: [Equation]
Revised: [Equation]

Original: Thus, we obtain that d FN ( π n ψ 0 , ψ 0 ) =
Revised: Thus, we obtain that d FN ( π n ψ 0 , ψ 0 ) =

Original: O (cid:16) n − β
Revised: O (cid:16) n − β

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: Proof of lemma 7.
Revised: Proof of lemma 7.

Original: The left inequality is trivial according to the definition of covering number.
Revised: The left inequality is trivial according to the definition of covering number.

Original: We need to show that the correctness of the right inequality.
Revised: We need to show that the correctness of the right inequality.

Original: Suppose that we have { B ( g i , ε 2 ) } , i = 1 . . .
Revised: Suppose that we have { B ( g i , ε 2 ) } , i = 1 . . .

Original: , N , where N = N ( ε 2 , F , ∥·∥ ) , are the minimal number of ε 2 -ball that covers F .
Revised: , N , where N = N ( ε 2 , F , ∥·∥ ) , are the minimal number of ε 2 -ball that covers F .

Original: Then there exists at least one f i ∈ F
Revised: Then there exists at least one f i ∈ F

Original: such that f i ∈ B ( g i , ε ) .
Revised: such that f i ∈ B ( g i , ε ) .

Original: Consider the following ε − balls { B ( f i , ε ) } , i = 1 . .
Revised: Consider the following ε − balls { B ( f i , ε ) } , i = 1 . .

Original: N .
Revised: N .

Original: For arbitrary f ∈ F ∩ B ( g i , ε 2 ) , we have that ∥ f − f i ∥ ≤ ∥ f − g i ∥ + ∥ f
Revised: For arbitrary f ∈ F ∩ B ( g i , ε 2 ) , we have that ∥ f − f i ∥ ≤ ∥ f − g i ∥ + ∥ f

Original: i − g i ∥ ≤ ε .
Revised: i − g i ∥ ≤ ε .

Original: Thus { B ( f i , ε ) } , i = 1 . . .
Revised: Thus { B ( f i , ε ) } , i = 1 . . .

Original: , N forms a ε -covering of F .
Revised: , N forms a ε -covering of F .

Original: By definition, we have that (cid:101) N ( ε, F , ∥ · ∥ ) ≤ N ( ε 2 , F , ∥ · ∥ ) .
Revised: By definition, we have that (cid:101) N ( ε, F , ∥ · ∥ ) ≤ N ( ε 2 , F , ∥ · ∥ ) .

Original: Proof of lemma 8.
Revised: Proof of lemma 8.

Original: The proof of the first two inequalities follows exactly the same steps of lemma 7.
Revised: The proof of the first two inequalities follows exactly the same steps of lemma 7.

Original: Here we just need to mention the rest of the statement that (cid:101) N [] ( ε, F , ∥ · ∥ ∞ )
Revised: Here we just need to mention the rest of the statement that (cid:101) N [] ( ε, F , ∥ · ∥ ∞ )

Original: = (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .
Revised: = (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .

Original: We first choose a set of ε 2 -covering balls { B ( f i , ε 2 ) } , i = 1 , . .
Revised: We first choose a set of ε 2 -covering balls { B ( f i , ε 2 ) } , i = 1 , . .

Original: N 1 , where N 1 = (cid:101) N ( ε 2 , F , ∥·∥ ∞ ) .
Revised: N 1 , where N 1 = (cid:101) N ( ε 2 , F , ∥·∥ ∞ ) .

Original: Now we construct a set of brackets { [ l i , u i ] } , i = 1 . . .
Revised: Now we construct a set of brackets { [ l i , u i ] } , i = 1 . . .

Original: , N 1 , where l i = f i − ε 2 and u i =
Revised: , N 1 , where l i = f i − ε 2 and u i =

Original: f i + ε 2 .
Revised: f i + ε 2 .

Original: Noting that the bracket { [ l i , u i ] } is exactly the same as B ( f i , ε 2 ) , The set { [ l i , u i ] } , i = 1 , . .
Revised: Noting that the bracket { [ l i , u i ] } is exactly the same as B ( f i , ε 2 ) , The set { [ l i , u i ] } , i = 1 , . .

Original: . , N 1 covers
Revised: . , N 1 covers

Original: F , which leads to (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) ≤ (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .
Revised: F , which leads to (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) ≤ (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .

Original: Likewise, we have that (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) ≥ (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .
Revised: Likewise, we have that (cid:101) N [] ( ε, F , ∥ · ∥ ∞ ) ≥ (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .

Original: Consequently, we have that (cid:101) N [] ( ε, F , ∥ · ∥ ∞ )
Revised: Consequently, we have that (cid:101) N [] ( ε, F , ∥ · ∥ ∞ )

Original: = (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .
Revised: = (cid:101) N ( ε 2 , F , ∥ · ∥ ∞ ) .

Original: Proof of lemma 9.
Revised: Proof of lemma 9.

Original: By lemma 8, first we have that N [] ( ε, F n , ∥·∥ ∞ ) ≤ (cid:101) N [] ( ε, F n , ∥·∥ ∞ ) .
Revised: By lemma 8, first we have that N [] ( ε, F n , ∥·∥ ∞ ) ≤ (cid:101) N [] ( ε, F n , ∥·∥ ∞ ) .

Original: By lemma 3, there exists a constant c 1 > 0 such that for arbitrary (cid:98) h 1 , (cid:98) h 2 ∈ H n , (cid:98) m 1 , (cid:98) m 2 ∈ M n and (cid:98) θ 1 , (cid:98) θ 2 ∈ Θ , we have that
Revised: By lemma 3, there exists a constant c 1 > 0 such that for arbitrary (cid:98) h 1 , (cid:98) h 2 ∈ H n , (cid:98) m 1 , (cid:98) m 2 ∈ M n and (cid:98) θ 1 , (cid:98) θ 2 ∈ Θ , we have that

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original: h 2 ∥ ∞ + ∥ (cid:98) m 1 − (cid:98) m 2 ∥ ∞ ] , which indicates that as long as | (cid:98) θ 1 − (cid:98) θ 2 | ≤ ε 3 c 1 , ∥ (cid:98) h 1 − (cid:98) h 2 ∥ ∞ ≤ ε 3 c 1 and ∥ (cid:98) m 1 − (cid:98) m 2 ∥ ∞ ≤ ε 3 c 1 , we have that ∥ l ( T, δ, Z ; (cid:98) h 1 , (cid:98) m 1 , θ 1 ) − l ( T, δ, Z ; (cid:98) h 2 , (cid:98) m 2 , θ 2 ) ∥ ∞ ≤ ε .
Revised: h 2 ∥ ∞ + ∥ (cid:98) m 1 − (cid:98) m 2 ∥ ∞ ] , which indicates that as long as | (cid:98) θ 1 − (cid:98) θ 2 | ≤ ε 3 c 1 , ∥ (cid:98) h 1 − (cid:98) h 2 ∥ ∞ ≤ ε 3 c 1 and ∥ (cid:98) m 1 − (cid:98) m 2 ∥ ∞ ≤ ε 3 c 1 , we have that ∥ l ( T, δ, Z ; (cid:98) h 1 , (cid:98) m 1 , θ 1 ) − l ( T, δ, Z ; (cid:98) h 2 , (cid:98) m 2 , θ 2 ) ∥ ∞ ≤ ε .

Original: Consequently, we have that
Revised: Consequently, we have that

Original: [Equation]
Revised: [Equation]

Original: N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) × (cid:101) N [] ( ε 3 c 1 , M n , ∥ · ∥ ∞ ) .
Revised: N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) × (cid:101) N [] ( ε 3 c 1 , M n , ∥ · ∥ ∞ ) .

Original: Since Θ is a compact set on R , by lemma 8 and traditional volume argument, we have that (cid:101) N [] ( ε 3 c 1 , Θ , ∥ · ∥ ∞ ) ≤ N [] ( ε 6 c 1 , Θ , ∥ · ∥ ∞ ) ≲ 1 ε .
Revised: Since Θ is a compact set on R , by lemma 8 and traditional volume argument, we have that (cid:101) N [] ( ε 3 c 1 , Θ , ∥ · ∥ ∞ ) ≤ N [] ( ε 6 c 1 , Θ , ∥ · ∥ ∞ ) ≲ 1 ε .

Original: For (cid:101) N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) , by lemma 8, we have that (cid:101) N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ )
Revised: For (cid:101) N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) , by lemma 8, we have that (cid:101) N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ )

Original: =
Revised: =

Original: (cid:101) N ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) .
Revised: (cid:101) N ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) .

Original: By Chen & Shen (1998, Lemma 2), there exists a constant c 2 > 0 such that ∥ (cid:98) h 1 − (cid:98) h 2 ∥ ∞ ≤ c 2 ∥ (cid:98) h 1 − (cid:98) h 

Revised: By Chen & Shen (1998, Lemma 2), there exists a constant c 2 > 0 such that ∥ (cid:98) h 1 − (cid:98) h 2 ∥ ∞ ≤ c 2 ∥ (cid:98) h 1 − (cid:98) h

Original: ∥ s h 2 , which leads to (cid:101) N ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) ≤ (cid:101) N ( ε 1 /sh
Revised: ∥ s h 2 , which leads to (cid:101) N ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) ≤ (cid:101) N ( ε 1 /sh

Original: (3 c 1 c 2 ) 1 /sh , H n , ∥ · ∥ 2 ) .
Revised: (3 c 1 c 2 ) 1 /sh , H n , ∥ · ∥ 2 ) .

Original: By lemma 7 we further have that (cid:101) N ( ε 1 /sh (3 c 1 c 2 ) 1 /sh , H n , ∥ · ∥ 2 ) ≤ N ( ε 1 /sh 2(3 c 1 c 2 ) 1 /sh , H n , ∥ · ∥ 2 ) .
Revised: By lemma 7 we further have that (cid:101) N ( ε 1 /sh (3 c 1 c 2 ) 1 /sh , H n , ∥ · ∥ 2 ) ≤ N ( ε 1 /sh 2(3 c 1 c 2 ) 1 /sh , H n , ∥ · ∥ 2 ) .

Original: Let c h
Revised: Let c h

Original: = 1 2(3 c 1 c 2 ) 1 /sh .
Revised: = 1 2(3 c 1 c 2 ) 1 /sh .

Original: We have that (cid:101) N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) ≤ N ( c h ε 1 /s h , H n , ∥ · ∥ 2 ) .
Revised: We have that (cid:101) N [] ( ε 3 c 1 , H n , ∥ · ∥ ∞ ) ≤ N ( c h ε 1 /s h , H n , ∥ · ∥ 2 ) .

Original: Similarly, there exists a constant c m > 0 such that (cid:101) N [] ( ε 3 c 1 , M n , ∥·∥ ∞ ) ≤ N ( c m ε 1 /s m , M n , ∥·∥ 2 ) .
Revised: Similarly, there exists a constant c m > 0 such that (cid:101) N [] ( ε 3 c 1 , M n , ∥·∥ ∞ ) ≤ N ( c m ε 1 /s m , M n , ∥·∥ 2 ) .

Original: Thus, finally we can obtain that
Revised: Thus, finally we can obtain that

Original: N [] (
Revised: N [] (

Original: [Equation]
Revised: [Equation]

Original: , F n , ∥ · ∥ ∞ ) ≲
Revised: , F n , ∥ · ∥ ∞ ) ≲

Original: [Equation]
Revised: [Equation]

Original: εN
Revised: εN

Original: [Equation]
Revised: [Equation]

Original: c h
Revised: c h

Original: [Equation]
Revised: [Equation]

Original: h
Revised: h

Original: [Equation]
Revised: [Equation]

Original: H n
Revised: H n

Original: [Equation]
Revised: [Equation]

Original: ∥ · ∥
Revised: ∥ · ∥

Original: [Equation]
Revised: [Equation]

Original: × N
Revised: × N

Original: [Equation]
Revised: [Equation]

Original: m
Revised: m

Original: [Equation]
Revised: [Equation]

Original: M n
Revised: M n

Original: [Equation]
Revised: [Equation]

Original: ∥ · ∥
Revised: ∥ · ∥

Original: [Equation]
Revised: [Equation]

Original: Proof of lemma 10.
Revised: Proof of lemma 10.

Original: By lemma 8, first we have N [] ( ε, G n , ∥ · ∥ ∞ ) ≤ (cid:101) N [] ( ε, G n , ∥ · ∥ ∞ ) .
Revised: By lemma 8, first we have N [] ( ε, G n , ∥ · ∥ ∞ ) ≤ (cid:101) N [] ( ε, G n , ∥ · ∥ ∞ ) .

Original: By lemma 4, there exists a constant c 3 > 0 such that for arbitrary (cid:98) ν 1 , (cid:98) ν 2 ∈ V n and (cid:98) θ 1 , (cid:98) θ 2 ∈ Θ , we have that ∥ l
Revised: By lemma 4, there exists a constant c 3 > 0 such that for arbitrary (cid:98) ν 1 , (cid:98) ν 2 ∈ V n and (cid:98) θ 1 , (cid:98) θ 2 ∈ Θ , we have that ∥ l

Original: [Equation]
Revised: [Equation]

Original: δ
Revised: δ

Original: [Equation]
Revised: [Equation]

Original: (cid:98)
Revised: (cid:98)

Original: [Equation]
Revised: [Equation]

Original: 1 , (cid:98)
Revised: 1 , (cid:98)

Original: [Equation]
Revised: [Equation]

Original: ] , which indicates that as long as | (cid:98) θ 1 − (cid:98) θ 2 | ≤ ε 2 c 3 and ∥ (cid:98) ν 1 − (cid:98) ν 2 ∥ ∞ ≤ ε 2 c 3 , we have that ∥ l ( T, δ, Z ; (cid:98) ν 1 , (cid:98) θ 1 ) − l ( T, δ, Z ; (cid:98) ν 2 , (cid:98) θ 2 ) ∥ ∞ ≤ ε .
Revised: ] , which indicates that as long as | (cid:98) θ 1 − (cid:98) θ 2 | ≤ ε 2 c 3 and ∥ (cid:98) ν 1 − (cid:98) ν 2 ∥ ∞ ≤ ε 2 c 3 , we have that ∥ l ( T, δ, Z ; (cid:98) ν 1 , (cid:98) θ 1 ) − l ( T, δ, Z ; (cid:98) ν 2 , (cid:98) θ 2 ) ∥ ∞ ≤ ε .

Original: Thus, we have:
Revised: Thus, we have:

Original: [Equation]
Revised: [Equation]

Original: Since Θ is a compact set on R , by lemma 8 and traditional volume argument, we have that
Revised: Since Θ is a compact set on R , by lemma 8 and traditional volume argument, we have that

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ε .
Revised: ε .

Original: [Table]
Revised: [Table]

Original: For (cid:101) N [] ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) , by lemma 8, we have that (cid:101) N [] ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) =
Revised: For (cid:101) N [] ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) , by lemma 8, we have that (cid:101) N [] ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) =

Original: (cid:101) N ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) .
Revised: (cid:101) N ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) .

Original: By
Revised: By

Original: Chen & Shen (1998, Lemma 2), there exists a constant c 4 > 0 such that ∥ (cid:98) ν 1 − (cid:98) ν 2 ∥ ∞ ≤ c 4 ∥ (cid:98) ν 1 − (cid:98) ν 2 ∥ s h 2 , which leads to (cid:101) N ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) ≤ (cid:101) N ( ε 1 /sν
Revised: Chen & Shen (1998, Lemma 2), there exists a constant c 4 > 0 such that ∥ (cid:98) ν 1 − (cid:98) ν 2 ∥ ∞ ≤ c 4 ∥ (cid:98) ν 1 − (cid:98) ν 2 ∥ s h 2 , which leads to (cid:101) N ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) ≤ (cid:101) N ( ε 1 /sν

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4 ) 1 /sν , V n , ∥ · ∥ 2 ) .
Revised: 4 ) 1 /sν , V n , ∥ · ∥ 2 ) .

Original: By lemma 7 we further have (cid:101) N ( ε 1 /sν
Revised: By lemma 7 we further have (cid:101) N ( ε 1 /sν

Original: (2 c 3 c 4 ) 1 /sν , V n , ∥ · ∥ 2 ) ≤ N ( ε 1 /sν 2(2 c
Revised: (2 c 3 c 4 ) 1 /sν , V n , ∥ · ∥ 2 ) ≤ N ( ε 1 /sν 2(2 c

Original: 3 c 4 ) 1 /sν , V n , ∥ · ∥ 2 ) .
Revised: 3 c 4 ) 1 /sν , V n , ∥ · ∥ 2 ) .

Original: Let c ν = 1 2(2 c
Revised: Let c ν = 1 2(2 c

Original: 3 c 4 ) 1 /sν , we have that (cid:101) N [] ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) ≤ N ( c ν ε 1 /s ν , V n , ∥ · ∥ 2 ) .
Revised: 3 c 4 ) 1 /sν , we have that (cid:101) N [] ( ε 2 c 3 , V n , ∥ · ∥ ∞ ) ≤ N ( c ν ε 1 /s ν , V n , ∥ · ∥ 2 ) .

Original: Thus, finally we can obtain that
Revised: Thus, finally we can obtain that

Original: [Equation]
Revised: [Equation]

Original: C E XPERIMENTAL DETAILS C. D ATASET SUMMARY
Revised: C E XPERIMENTAL DETAILS C. D ATASET SUMMARY

Original: We report summaries of descriptive statistics of the 6 benchmark datasets used in section 5.2 in table 3.
Revised: We report summaries of descriptive statistics of the 6 benchmark datasets used in section 5.2 in table 3.

Original: C. D ETAILS OF SYNTHETIC EXPERIMENTS
Revised: C. D ETAILS OF SYNTHETIC EXPERIMENTS

Original: Since the true model is assumed to be of PF form, we generate event time according to the following transformed regression model (Dabrowska & Doksum, 1988):
Revised: Since the true model is assumed to be of PF form, we generate event time according to the following transformed regression model (Dabrowska & Doksum, 1988):

Original: [Equation]
Revised: [Equation]

Original: where H ( t ) =
Revised: where H ( t ) =

Original: (cid:82) t 

Revised: (cid:82) t

Original: e h ( s ) ds with h defined in (2).
Revised: e h ( s ) ds with h defined in (2).

Original: The error term ϵ is generated such that e ϵ has cumulative hazard function G θ .
Revised: The error term ϵ is generated such that e ϵ has cumulative hazard function G θ .

Original: The formulation (15) is the equivalent to (2) (Dabrowska & Doksum, 1988; Cuzick, 1988;
Revised: The formulation (15) is the equivalent to (2) (Dabrowska & Doksum, 1988; Cuzick, 1988;

Original: Kosorok et al., 2004).
Revised: Kosorok et al., 2004).

Original: In our experiments, the covariates are of dimension 5 , sampled independently from the uniform distribution over [0 , 1] .
Revised: In our experiments, the covariates are of dimension 5 , sampled independently from the uniform distribution over [0 , 1] .

Original: We set h ( t ) =
Revised: We set h ( t ) =

Original: t and hence
Revised: t and hence

Original: H ( t ) =
Revised: H ( t ) =

Original: e t .
Revised: e t .

Original: The function form of m ( Z ) is set to be m ( Z )
Revised: The function form of m ( Z ) is set to be m ( Z )

Original: = sin( ⟨ Z, β ⟩ ) + ⟨ sin( Z ) , β ⟩ , where β
Revised: = sin( ⟨ Z, β ⟩ ) + ⟨ sin( Z ) , β ⟩ , where β

Original: = (0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5) .
Revised: = (0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5) .

Original: Then censoring time C is generated according to
Revised: Then censoring time C is generated according to

Original: [Equation]
Revised: [Equation]

Original: which reuses covariate Z , and draws independently a noise vector ϵ C such that the censoring ratio is controlled at around 40% .
Revised: which reuses covariate Z , and draws independently a noise vector ϵ C such that the censoring ratio is controlled at around 40% .

Original: We generate three datasets with n ∈ { 1000 , 5000 , 10000 } respectively.
Revised: We generate three datasets with n ∈ { 1000 , 5000 , 10000 } respectively.

Original: Hyperparameter configurations We specify below the network architectures and optimization configurations used in all the tasks:
Revised: Hyperparameter configurations We specify below the network architectures and optimization configurations used in all the tasks:

Original: PF scheme:
Revised: PF scheme:

Original: For both (cid:98) m and (cid:98) h , we use 64 hidden units for n = 1000 , 128 hidden units for n = 5000 and 256 hidden units for n = 10000 .
Revised: For both (cid:98) m and (cid:98) h , we use 64 hidden units for n = 1000 , 128 hidden units for n = 5000 and 256 hidden units for n = 10000 .

Original: We train each model for 100 epochs with batch size 128 , optimized using Adam with learning rate 0 . 0001 , and no weight decay.
Revised: We train each model for 100 epochs with batch size 128 , optimized using Adam with learning rate 0 . 0001 , and no weight decay.

Original: FN scheme:
Revised: FN scheme:

Original: For both (cid:98) ν , we use 64 hidden units for n =
Revised: For both (cid:98) ν , we use 64 hidden units for n =

Original: , 128 hidden units for n = 5000 and 256 hidden units for n = 10000 .
Revised: , 128 hidden units for n = 5000 and 256 hidden units for n = 10000 .

Original: We train each model for 100 epochs with batch size 128 , optimized using Adam with learning rate 0 . 0001 , and no weight decay.
Revised: We train each model for 100 epochs with batch size 128 , optimized using Adam with learning rate 0 . 0001 , and no weight decay.

Original: C. D ETAILS OF PUBLIC DATA EXPERIMENTS
Revised: C. D ETAILS OF PUBLIC DATA EXPERIMENTS

Original: Dataset preprocessing For METABRIC, RotGBSG, FLCHAIN, SUPPORT and KKBOX dataset, we take the version provided in the pycox package (Kvamme et al., 2019).
Revised: Dataset preprocessing For METABRIC, RotGBSG, FLCHAIN, SUPPORT and KKBOX dataset, we take the version provided in the pycox package (Kvamme et al., 2019).

Original: We standardize continuous features into zero mean and unit variance and do one-hot encodings for all categorical features.
Revised: We standardize continuous features into zero mean and unit variance and do one-hot encodings for all categorical features.

Original: For the MIMIC-III dataset, we follow the preprocessing routines in Purushotham et al.
Revised: For the MIMIC-III dataset, we follow the preprocessing routines in Purushotham et al.

Original: (2018) which extracts 26 features.
Revised: (2018) which extracts 26 features.

Original: The event of interest is defined as the mortality after admission, and the censored time is defined as the last time of being discharged from the hospital.
Revised: The event of interest is defined as the mortality after admission, and the censored time is defined as the last time of being discharged from the hospital.

Original: The definition is similar to that in Tang et al.
Revised: The definition is similar to that in Tang et al.

Original: But since the dataset is not open sourced, according to our implementation the resulting dataset exhibits a much higher censoring rate ( 90 . 2% as compared to 61 . 0% as reported in the SODEN paper (Tang et al., 2022)).
Revised: But since the dataset is not open sourced, according to our implementation the resulting dataset exhibits a much higher censoring rate ( 90 . 2% as compared to 61 . 0% as reported in the SODEN paper (Tang et al., 2022)).

Original: Since the major purpose of this paper is for the proposal of the NFM framework, We use our own version of the processed dataset to further verify the predictive performance of NFM.
Revised: Since the major purpose of this paper is for the proposal of the NFM framework, We use our own version of the processed dataset to further verify the predictive performance of NFM.

Original: Hyperparameter configurations We follow the general training template that uses MLP as all nonparametric function approximators (i.e., (cid:98) m and (cid:98) h in the PF scheme, and (cid:98)
Revised: Hyperparameter configurations We follow the general training template that uses MLP as all nonparametric function approximators (i.e., (cid:98) m and (cid:98) h in the PF scheme, and (cid:98)

Original: ν in the FN scheme), and train for 100 epochs across all datasets using Adam as the optimizer.
Revised: ν in the FN scheme), and train for 100 epochs across all datasets using Adam as the optimizer.

Original: The tunable parameters and their respective tuning ranges are reported as follows:
Revised: The tunable parameters and their respective tuning ranges are reported as follows:

Original: Number of layers (network depth)
Revised: Number of layers (network depth)

Original: We tune the network depth L ∈ { 2 , 3 , 4 } .
Revised: We tune the network depth L ∈ { 2 , 3 , 4 } .

Original: Typically, the performance of two-layer MLPs is sufficiently satisfactory.
Revised: Typically, the performance of two-layer MLPs is sufficiently satisfactory.

Original: Number of hidden units in each layer (network width)
Revised: Number of hidden units in each layer (network width)

Original: We tune the network width W ∈ { 2 k , 5 ≤ k ≤ 10 } .
Revised: We tune the network width W ∈ { 2 k , 5 ≤ k ≤ 10 } .

Original: Optional dropout
Revised: Optional dropout

Original: We optionally apply dropout with probability p ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 5 , 0 . 7 } .
Revised: We optionally apply dropout with probability p ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 5 , 0 . 7 } .

Original: Batch size We tune batch size within the range
Revised: Batch size We tune batch size within the range

Original: { 128 , 256 , 512 } , in the KKBOX dataset, we also tested with larger batch sizes { 1024 } .
Revised: { 128 , 256 , 512 } , in the KKBOX dataset, we also tested with larger batch sizes { 1024 } .

Original: Learning rate and weight decay We tune both the learning rate and weight decay coefficient of
Revised: Learning rate and weight decay We tune both the learning rate and weight decay coefficient of

Original: Adam within range { 0 . 01 , 0 . 001 , 0 . 0001 } .
Revised: Adam within range { 0 . 01 , 0 . 001 , 0 . 0001 } .

Original: Frailty specification We tested gamma frailty, Box-Cox transformation frailty, and IGG ( α ) frailty with α ∈ { 0 , 0 . 25 , 0 . 75 } .
Revised: Frailty specification We tested gamma frailty, Box-Cox transformation frailty, and IGG ( α ) frailty with α ∈ { 0 , 0 . 25 , 0 . 75 } .

Original: Here note that IGG (0 . 5) is equivalent to gamma frailty.
Revised: Here note that IGG (0 . 5) is equivalent to gamma frailty.

Original: We also empirically tried to set α to be a learnable parameter and found that this additional flexibility provides little performance improvement regarding the datasets used for evaluation.
Revised: We also empirically tried to set α to be a learnable parameter and found that this additional flexibility provides little performance improvement regarding the datasets used for evaluation.

Original: C. I MPLEMENTATIONS
Revised: C. I MPLEMENTATIONS

Original: We use pytorch to implement NFM.
Revised: We use pytorch to implement NFM.

Original: The source code is provided in the supplementary material .
Revised: The source code is provided in the supplementary material .

Original: For the baseline models:
Revised: For the baseline models:

Original: • We use the implementations of CoxPH , GBM, and RSF from the sksurv package P¨olsterl (2020), for the KKBOX dataset, we use the XGBoost library (Chen & Guestrin, 2016) to implement GBM and RSF, which might yield some performance degradation.
Revised: • We use the implementations of CoxPH , GBM, and RSF from the sksurv package P¨olsterl (2020), for the KKBOX dataset, we use the XGBoost library (Chen & Guestrin, 2016) to implement GBM and RSF, which might yield some performance degradation.

Original: • We use the pycox package to implement DeepSurv, CoxTime, and DeepHit models.
Revised: • We use the pycox package to implement DeepSurv, CoxTime, and DeepHit models.

Original: • We use the official code provided in the SODEN paper (Tang et al., 2022) to implement SODEN.
Revised: • We use the official code provided in the SODEN paper (Tang et al., 2022) to implement SODEN.

Original: • We obtain results of SuMo and DeepEH based on our re-implementations.
Revised: • We obtain results of SuMo and DeepEH based on our re-implementations.

Original: D A DDITIONAL EXPERIMENTS D. R ECOVERY ASSESSMENT OF m ( Z ) IN PF SCHEME
Revised: D A DDITIONAL EXPERIMENTS D. R ECOVERY ASSESSMENT OF m ( Z ) IN PF SCHEME

Original: We plot empirical recovery results targeting the m function in (2) in figure 2.
Revised: We plot empirical recovery results targeting the m function in (2) in figure 2.

Original: The result demonstrates satisfactory recovery with a moderate amount of data, i.e., n ≥ 1000 .
Revised: The result demonstrates satisfactory recovery with a moderate amount of data, i.e., n ≥ 1000 .

Original: D. R ECOVERY ASSESSMENT OF SURVIVAL FUNCTIONS
Revised: D. R ECOVERY ASSESSMENT OF SURVIVAL FUNCTIONS

Original: To assess the recovery performance of NFM with respect to survival functions, we consider the following setup: under the same data generation framework as in section C.2, we compute the test
Revised: To assess the recovery performance of NFM with respect to survival functions, we consider the following setup: under the same data generation framework as in section C.2, we compute the test

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: feature ¯ Z as the sample mean of all the 100 hold-out test points.
Revised: feature ¯ Z as the sample mean of all the 100 hold-out test points.

Original: And plot (cid:98) S ( t | ¯ Z ) against the ground truth S ( t | ¯ Z )
Revised: And plot (cid:98) S ( t | ¯ Z ) against the ground truth S ( t | ¯ Z )

Original: regarding both PF and FN schemes.
Revised: regarding both PF and FN schemes.

Original: The results are shown in figure 3.
Revised: The results are shown in figure 3.

Original: The results suggest that both scheme provides accurate estimation of survival functions when the sample size is sufficiently large.
Revised: The results suggest that both scheme provides accurate estimation of survival functions when the sample size is sufficiently large.

Original: D. P ERFORMANCE EVALUATION UNDER IBS AND INBLL
Revised: D. P ERFORMANCE EVALUATION UNDER IBS AND INBLL

Original: In this subsection we augment the experimental results in section 5 with three recent state-of-the-art baseline models: SurvNode Groha et al.
Revised: In this subsection we augment the experimental results in section 5 with three recent state-of-the-art baseline models: SurvNode Groha et al.

Original: , DCM (Nagpal et al., 2021) and DeSurv (Danks &
Revised: , DCM (Nagpal et al., 2021) and DeSurv (Danks &

Original: Yau, 2022).
Revised: Yau, 2022).

Original: The results are demonstrated in table 4 for four small scale datasets, and table 5 for two larger datasets.
Revised: The results are demonstrated in table 4 for four small scale datasets, and table 5 for two larger datasets.

Original: The proposed NFM framework achieves the best performance on 5 of the 6 datasets, which is consistent with the findings in table 1.
Revised: The proposed NFM framework achieves the best performance on 5 of the 6 datasets, which is consistent with the findings in table 1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: +1 .
Revised: +1 .

Original: 33% +1 .
Revised: 33% +1 .

Original: 56% +
Revised: 56% +

Original: [Table]
Revised: [Table]

Original: 61% +
Revised: 61% +

Original: [Table]
Revised: [Table]

Original: 30% +3 .
Revised: 30% +3 .

Original: 08% +2 .
Revised: 08% +2 .

Original: RotGBSG +1 .
Revised: RotGBSG +1 .

Original: 11% +0 .
Revised: 11% +0 .

Original: 95% +
Revised: 95% +

Original: [Table]
Revised: [Table]

Original: D. P ERFORMANCE EVALUATIONS UNDER THE CONCORDANCE INDEX (C- INDEX )
Revised: D. P ERFORMANCE EVALUATIONS UNDER THE CONCORDANCE INDEX (C- INDEX )

Original: The concordance index (C-index) (Antolini et al., 2005) is yet another evaluation metric that is commonly used in survival analysis.
Revised: The concordance index (C-index) (Antolini et al., 2005) is yet another evaluation metric that is commonly used in survival analysis.

Original: The C-index estimates the probability that, for a random pair of individuals, the predicted survival times of the two individuals have the same ordering as their true survival times.
Revised: The C-index estimates the probability that, for a random pair of individuals, the predicted survival times of the two individuals have the same ordering as their true survival times.

Original: Formally, C-index is defined as
Revised: Formally, C-index is defined as

Original: [Equation]
Revised: [Equation]

Original: We report performance evaluations based on C-index over all the 6 benchmark datasets in table 6.
Revised: We report performance evaluations based on C-index over all the 6 benchmark datasets in table 6.

Original: From table 6, it appears that there’s no clear winner regarding the C-index metric across the 6 selected datasets.
Revised: From table 6, it appears that there’s no clear winner regarding the C-index metric across the 6 selected datasets.

Original: We conjecture this phenomenon to be closely related to the loose correlation between the C-index and the likelihood-based learning objective, as was observed in Rindt et al.
Revised: We conjecture this phenomenon to be closely related to the loose correlation between the C-index and the likelihood-based learning objective, as was observed in Rindt et al.

Original: Therefore we compute the average rank of each model as an overall assessment of performance, as illustrated in the last column in table 6.
Revised: Therefore we compute the average rank of each model as an overall assessment of performance, as illustrated in the last column in table 6.

Original: The results suggest that the two NFM models perform better on average.
Revised: The results suggest that the two NFM models perform better on average.

Original: D. B ENEFITS OF FRAILTY
Revised: D. B ENEFITS OF FRAILTY

Original: We compute the (relative) performance gain of NFM-PF and NFM-FN, against their non-frailty counterparts, namely DeepSurv (Katzman et al., 2018) and SuMo-net
Revised: We compute the (relative) performance gain of NFM-PF and NFM-FN, against their non-frailty counterparts, namely DeepSurv (Katzman et al., 2018) and SuMo-net

Original: (Rindt et al., 2022)
Revised: (Rindt et al., 2022)

Original: based on results in table 1, table 2 and table 6.
Revised: based on results in table 1, table 2 and table 6.

Original: The results are shown in table 

Revised: The results are shown in table

Original: The results suggest solid improvement in incorporating frailty, especially for IBS and INBLL metrics, as the relative increase in performance could be over 10% for both NFM models.
Revised: The results suggest solid improvement in incorporating frailty, especially for IBS and INBLL metrics, as the relative increase in performance could be over 10% for both NFM models.

Original: For the IBS and INBLL metrics, the performance improvement is consistent across all datasets.
Revised: For the IBS and INBLL metrics, the performance improvement is consistent across all datasets.

Original: The only performance degradation appears on the MIMIC-III dataset evaluated under C-index.
Revised: The only performance degradation appears on the MIMIC-III dataset evaluated under C-index.

Original: This phenomenon is also understandable: Since the DeepSurv model utilized a variant of partial likelihood (PL) for model training, as previous works (Steck et al., 2007) pointed out that PL type objective is closely related to the ranking problem.
Revised: This phenomenon is also understandable: Since the DeepSurv model utilized a variant of partial likelihood (PL) for model training, as previous works (Steck et al., 2007) pointed out that PL type objective is closely related to the ranking problem.

Original: As C-index could be considered a certain type of ranking measure, it is possible that DeepSurv obtains better ranking performance than NFM-type models which are trained using scale-sensitive likelihood objective.
Revised: As C-index could be considered a certain type of ranking measure, it is possible that DeepSurv obtains better ranking performance than NFM-type models which are trained using scale-sensitive likelihood objective.