SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $H(t) = \int_0^t e^{h(s)}ds$  with  $h$  defined in (2). The error term  $\epsilon$  is generated such that  $e^{\epsilon}$  has cumulative hazard function  $G_{\theta}$ . The formulation (15) is the equivalent to (2) [18, 17, 45]. In our experiments, the covariates are of dimension 5, sampled independently from the uniform distribution over [0, 1]. We set  $h(t) = t$  and hence  $H(t) = e^{t}$ . The function form of  $m(Z)$  is set to be  $m(Z) = \sin (\langle Z,\beta \rangle) + \langle \sin (Z),\beta \rangle$ , where  $\beta = (0.1,0.2,0.3,0.4,0.5)$ . Then censoring time  $C$  is generated according to
$$
\log H (C) = - m (Z) + \epsilon_ {C}, \tag {16}
$$
which reuses covariate  $Z$ , and draws independently a noise vector  $\epsilon_{C}$  such that the censoring ratio is controlled at around  $40\%$ . We generate three datasets with  $n \in \{1000, 5000, 10000\}$  respectively.
Hyperparameter configurations We specify below the network architectures and optimization configurations used in all the tasks:
PF scheme: For both  $\widehat{m}$  and  $\widehat{h}$ , we use 64 hidden units for  $n = 1000$ , 128 hidden units for  $n = 5000$  and 256 hidden units for  $n = 10000$ . We train each model for 100 epochs with batch size 128, optimized using Adam with learning rate 0.0001, and no weight decay.
FN scheme: For both  $\widehat{\nu}$ , we use 64 hidden units for  $n = 1000$ , 128 hidden units for  $n = 5000$  and 256 hidden units for  $n = 10000$ . We train each model for 100 epochs with batch size 128, optimized using Adam with learning rate 0.0001, and no weight decay.
C.3 Details of public data experiments
Dataset preprocessing For METABRIC, RotGBSG, FLCHAIN, SUPPORT and KKBOX dataset, we take the version provided in the pycox package [46]. We standardize continuous features into zero mean and unit variance and do one-hot encodings for all categorical features. For the MIMIC-III dataset, we follow the preprocessing routines in [55] which extracts 26 features. The event of interest is defined as the mortality after admission, and the censored time is defined as the last time of being discharged from the hospital. The definition is similar to that in [64]. But since the dataset is not open sourced, according to our implementation the resulting dataset exhibits a much higher censoring rate (90.2% as compared to 61.0% as reported in the SODEN paper [64]). Since the major purpose of this paper is for the proposal of the NFM framework, We use our own version of the processed dataset to further verify the predictive performance of NFM.
Hyperparameter configurations We follow the general training template that uses MLP as all nonparametric function approximators (i.e.,  $\hat{m}$  and  $\hat{h}$  in the PF scheme, and  $\widehat{\nu}$  in the FN scheme), and train for 100 epochs across all datasets using Adam as the optimizer. The tunable parameters and their respective tuning ranges are reported as follows:
Number of layers (network depth) We tune the network depth  $L \in \{2, 3, 4\}$ . Typically, the performance of two-layer MLPs is sufficiently satisfactory.
Number of hidden units in each layer (network width) We tune the network width  $W \in \{2^k, 5 \leq k \leq 10\}$ .
Optional dropout We optionally apply dropout with probability  $p \in \{0.1, 0.2, 0.3, 0.5, 0.7\}$ .
Batch size We tune batch size within the range  $\{128, 256, 512\}$ , in the KKBOX dataset, we also tested with larger batch sizes  $\{1024\}$ .
Learning rate and weight decay We tune both the learning rate and weight decay coefficient of Adam within range  $\{0.01, 0.001, 0.0001\}$ .
Frailty specification We tested gamma frailty, Box-Cox transformation frailty, and  $\mathrm{IGG}(\alpha)$  frailty with  $\alpha \in \{0, 0.25, 0.75\}$ . Here note that  $\mathrm{IGG}(0.5)$  is equivalent to gamma frailty. We also empirically tried to set  $\alpha$  to be a learnable parameter and found that this additional flexibility provides little performance improvement regarding the datasets used for evaluation.
C.4 Implementations
We use pytorch to implement NFM. The source code is provided in the supplementary material. For the baseline models:
- We use the implementations of CoxPH, GBM, and RSF from the sksrv package [54], for the KKBOX dataset, we use the XGBoost library [10] to implement GBM and RSF, which might yield some performance degradation.
- We use the pycox package to implement DeepSurv, CoxTime, and DeepHit models.
- We use the official code provided in the SODEN paper [64] to implement SODEN.
- We obtain results of SuMo and DeepEH based on our re-implementations.
D.1 Recovery assessment of  $m(Z)$  in PF scheme
We plot empirical recovery results targeting the  $m$  function in (2) in figure 2. The result demonstrates satisfactory recovery with a moderate amount of data, i.e.,  $n \geq 1000$ .
D.2 Recovery assessment of survival functions
To assess the recovery performance of NFM with respect to survival functions, we consider the following setup: under the same data generation framework as in section C.2, we compute the test feature  $\bar{Z}$  as the sample mean of all the 100 hold-out test points. And plot  $\widehat{S}(t|\bar{Z})$  against the ground
Figure 2: Visualizations of synthetic data results under the PF scheme of NFM framework, regarding empirical recovery of the  $m$  function in (2)
Figure 3: Visualizations of synthetic data results under the NFM framework. The plots in the first row compare the empirical estimates of the survival function  $S(t|\bar{Z})$  against its true value with  $\bar{Z}$  being the average of the features of the 100 hold-out points, under the PF scheme. The plots in the second row are obtained using the FN scheme, with analogous semantics to the first row.
truth  $S(t|\bar{Z})$  regarding both PF and FN schemes. The results are shown in figure 3. The results suggest that both schemes provide accurate estimation of survival functions when the sample size is sufficiently large.
D.3 Numerical results of the synthetic experiments
Following [75], we report the relative integrated mean squared error (RISE) of the estimated survival function against the ground truth and list the results in table. The results suggest that the goodness of fit becomes better with a larger sample size. Moreover, since the true model in the simulation is generated as an PF model, we found PF to perform slightly better than FN, which is reasonable since the inductive bias of PF is more correct in this setup.
Table 4: RISE of the estimated survival function in synthetic experiments
N = 1000	N = 5000	N = 10000
NFM-PF	0.0473	0.0145	0.0137
NFM-FN	0.0430	0.0184	0.0165
Table 5: Survival prediction results measured in C-index (\%) on all the 6 benchmark datasets. In each column, the boldfaced score denotes the best result and the underlined score represents the second-best result. The average rank of each model is reported in the rightmost column. We did not manage to obtain reasonable results for DeepEH and SODEN on two larger datasets MIMIC-III and KKBOX, and we set corresponding ranks to be the worst on those datasets.
Model	METABRIC	RotGBSG	FLCHAIN	SUPPORT	MIMIC-III	KKBOX	Ave. Rank
CoxPH	63.42±1.81	66.14±1.46	79.09±1.11	56.89±0.91	74.91±0.00	83.01±0.00	11.33
GBM	64.02±1.79	67.35±1.16	79.47±1.08	61.46±0.80	75.20±0.00	85.84±0.00	7.17
RSF	64.47±1.82	67.33±1.34	78.75±1.07	61.63±0.84	75.47±0.17	85.79±0.00	8.00
DeepSurv	63.95±2.12	67.20±1.22	79.04±1.14	60.91±0.85	80.08±0.44	85.59±0.08	8.50
CoxTime	66.22±1.69	67.41±1.35	78.95±1.01	61.54±0.87	78.78±0.62	87.31±0.24	5.00
DeepHit	66.33±1.61	66.38±1.07	78.48±1.09	63.20±0.85	79.16±0.59	86.12±0.26	6.50
DeepEH	66.59±2.00	67.93±1.28	78.71±1.44	61.51±1.04	-	-	6.33
SuMo-net	64.82±1.80	67.20±1.31	79.28±1.02	62.18±0.78	76.23±1.06	84.77±0.02	7.00
SODEN	64.82±1.05	66.97±0.75	79.00±0.96	61.10±0.59	-	-	10.17
SurvNode	64.64±4.91	67.30±1.65	76.11±0.98	55.37±0.77	-	-	11.50
DCM	65.76±1.25	66.75±1.35	78.61±0.79	62.19±0.95	76.45±0.34	83.48±0.07	8.33
DeSurv	65.88±2.02	67.30±1.45	78.97±1.64	61.47±0.97	80.97±0.30	86.11±0.05	5.67
NFM-PF	64.98±1.87	67.77±1.35	79.45±1.03	61.33±0.83	79.56±0.15	86.23±0.01	4.67
NFM-FN	66.63±1.82	67.73±1.29	79.29±0.93	62.21±0.41	80.18±0.20	86.61±0.01	2.16
Table 6: Relative improvement of NFM models in comparison to their non-frailty counterparts, measured in IBS, INBLL, and C-index.
Dataset	NFM-PF vs DeepSurv	NFM-FN vs SuMo-net
IBS	INBLL	C-index	IBS	INBLL	C-index
METABRIC	+1.33%	+1.56%	+1.61%	+2.30%	+3.08%	+2.79%
RotGBSG	+1.11%	+0.95%	+0.84%	+0.62%	+0.40%	+0.79%
FLCHAIN	+1.29%	+1.32%	+0.52%	+0.20%	+0.27%	+0.01%
SUPPORT	+0.31%	+0.23%	+0.69%	+2.22%	+1.76%	+0.05%
MIMIC-III	+12.38%	+12.15%	-0.64%	+6.18%	+5.56%	+5.18%
KKBOX	+2.56%	+0.51%	+0.75%	+8.20%	+10.38%	+2.17%
D.4 Performance evaluations under the concordance index (C-index)
The concordance index (C-index) [1] is yet another evaluation metric that is commonly used in survival analysis. The C-index estimates the probability that, for a random pair of individuals, the predicted survival times of the two individuals have the same ordering as their true survival times. Formally, C-index is defined as
$$
\mathcal {C} = \mathbb {P} \left[ \widehat {S} \left(T _ {i} \mid Z _ {i}\right) <   \widehat {S} \left(T _ {j} \mid Z _ {j}\right) \mid T _ {i} <   T _ {j}, \delta_ {i} = 1 \right]. \tag {17}
$$
We report performance evaluations based on C-index over all the 6 benchmark datasets in table 5. From table 5, it appears that there's no clear winner regarding the C-index metric across the 6 selected datasets. We conjecture this phenomenon to be closely related to the loose correlation between the C-index and the likelihood-based learning objective, as was observed in [56]. Therefore we compute the average rank of each model as an overall assessment of performance, as illustrated in the last column in table 5. The results suggest that the two NFM models perform better on average.
D.5 Benefits of frailty
We compute the (relative) performance gain of NFM-PF and NFM-FN, against their non-frailty counterparts, namely DeepSurv [42] and SuMo-net [56] based on results in table 1, table 2 and table 5. The results are shown in table 6 The results suggest solid improvement in incorporating frailty, especially for IBS and INBLL metrics, as the relative increase in performance could be over  $10\%$  for both NFM models. For the IBS and INBLL metrics, the performance improvement is consistent across all datasets. The only performance degradation appears on the MIMIC-III dataset evaluated under C-index. This phenomenon is also understandable: Since the DeepSurv model utilized a variant
of partial likelihood (PL) for model training, as previous works [62] pointed out that PL type objective is closely related to the ranking problem. As C-index could be considered a certain type of ranking measure, it is possible that DeepSurv obtains better ranking performance than NFM-type models which are trained using scale-sensitive likelihood objective.
D.6 Limitations
In this section we discuss the limitations of this paper form both theoretical and empirical standpoints.
Theoretical limitations As we have established formal statistical guarantees regarding the estimation properties of NFM, the guarantees do not necessarily imply that NFM perform well on prediction tasks under metrics such as IBS and INBLL. Following the spirit of classical learning theory [58], for prediction guarantees it is ideal to directly optimize the underlying metric or its surrogates, which is difficult in survival problems since the metrics involve both a model over the event time and a working model on the censoring time. So far as we have noticed, the only effort that aims to address this issue is the method of inversely-weighted survival games [33]. However, the authors in [33] did not provide rigorous learning-theoretic statements, which is a promising research direction for future works.
Empirical limitations While NFM is shown to be a competitive survival model for prognosis empirically, we have observed from the empirical results that we can hardly obtain statistically significant improvements over the baselines, which is a common problem exhibited in previous works on neural survival regressions [75, 56]. We conjecture that this phenomenon is primarily due to two facts: Firstly, there lacks open-to-public large-scale survival datasets that allows scalable evaluation of neural survival models. Secondly, for most of the current available datasets, there are no authoritative train-test splits. Consequently, most experiments are done using cross-validation on moderate scale datasets, causing the resulting variability of the modeling algorithms to be relatively large. Therefore we suggest the survival analysis community to release (in a privacy-preserving manner) more large-scale, sanitized datasets equipped with standard train-test splits, which we believe will greatly benefit the state-of-the-art for neural survival modeling.