# Promoting Variable Effect Consistency in Mixture Cure Model for Credit Scoring

Authors: Chenlu Zheng; Jianping Zhu; Xinyan Fan; Song Chen; Zhiyuan Zhang
Journal: Discrete Dynamics in Nature and Society
 (2022)

---

## Abstract

Mixture cure models are widely adopted in credit scoring. Mixture cure models consist of two parts: an incident part which predicts the probability of default and a latency part which predicts when they are likely to default. The two model parts describe two quite relevant credit aspects. So, it is reasonable to expect that the two sets of the coefficients are somewhat related. Moreover, in practical cases, it is difficult to interpret the results when the two sets of the coefficients of the same variables have conflicting signs. Most existing works either ignore the interconnections of the two sets of coefficients or impose a strict constraint between them. We proposed a mixture cure model considering the variable effect consistency using a sign-based penalty. It is a more flexible model that allows the two sets of coefficients to be in different distributions and magnitudes. To accommodate high-dimensional credit data, a group lasso penalty is also imposed for variable selection. Simulation shows that the proposed method has competitive performance compared with alternative methods in terms of estimation and prediction. Furthermore, the empirical study illustrates that the proposed method outperforms the alternative method and can improve the interpretability of the results.


---

## Body

## 1. Introduction

Credit scoring is an effective and crucial approach for evaluating credit risk [1, 2]. A slight improvement in the prediction precision of credit scoring models can bring considerable benefits. Therefore, credit scoring has attracted increasing attention of scholars and practitioners. Many studies treat it as a classification problem to distinguish noncreditworthy customers from creditworthy ones [2, 3]. These studies focus on classification techniques including logistic regression, support vector machine, neural network, and random forest [4, 5]. For example, Zhou et al. [6] proposed a logistic regression method with clustering analysis for credit risk evaluation. Zhang et al. [7] proposed a cost-sensitive logistic regression model to assess the credit risk. Considering the high cost and time consumption of credit scoring, a credit granting process using three-way decisions is proposed to make efficient credit decisions [8]. Since the exposure to risk and the losses caused by default are strongly related to the time when they default [9], credit risk prediction overtime is of great significance for timely risk management.Survival analysis, with its ability of predicting the probability of default over time, has been first applied in credit scoring in 1992 [10]. It can be more informative than the binary classification model. Subsequently, various survival analysis models are proposed to predict credit risk over time. For instance, Cox proportional hazards (PH) model is adopted to predict early repayment and time to default in personal loans and investigated the effect of different variables on time to default [11, 12]. In addition, macroeconomic factors and time-varying data are also incorporated in survival analysis to improve the performance of prediction in credit scoring [13, 14]. And the models are further extended by a survival gradient boosting decision tree approach to enhance the prediction performance [15].However, standard survival analysis assumes that the loan term is long enough and every customer will eventually default. In practice, a substantial proportion of customers will not default during the entire loan term. Since mixture cure models applied in medicine assume that some patients have been cured and will not die during the follow-up period, it is more appropriate in the credit market and was first introduced to credit scoring by Tong et al. [16].Recently, the mixture cure model, an extension of the standard survival analysis, is widely adopted in credit scoring for its ability of predicting not only whether customers will default but also when they are likely to default. Results showed that the mixture cure model is more suitable for credit data compared with standard survival analysis models and the mixture cure model incorporating penalized spline has better performance in prediction [17]. Mixture cure models have been further developed by identifying different risk patterns of customers, considering the influence of competitive risk, and the relationship between the default times and the variables [18–20].Mixture cure models consist of two parts: an incident part which predicts the probability of default and a latency part which predicts when they are likely to default. In the two model parts, the two sets of the coefficients indicate the two sets of the variable effects on the credit risk. The two model parts describe two quite relevant credit aspects. Nevertheless, most of the existing studies ignore the relations between the two sets of coefficients in two model parts. These works generally assume that there are no direct constraints between the two sets of coefficients, which may get conflicting results of variable effects. For example, Dirick et al. [21] propose a mixture cure model incorporating macroeconomic factors to predict credit risk. The results show that the customers’ annual income has the opposite effect on whether and when to default. In other words, according to the results, customers with lower annual income have a lower probability of default, but they are more likely to default earlier. It is difficult to interpret the conflicting results and apply them in practice.In fact, the two model parts describe two quite relevant credit aspects, namely, the probability of default and survival (nondefault) time. Customers with high default probability are more likely to default earlier. So, it is reasonable to expect that the two sets of the coefficients are somewhat related. Theoretical derivations [22] and empirical analysis [23] also suggest that relaxing the independence of two sets of coefficients can improve the model performance. The assumption has been relaxed by establishing a joint distribution of the defaulting predictor and the logarithm of the hazard rate in [23]. Note that the two model parts still describe two different aspects of default. The assumption of a joint distribution may be too strict. So, we consider a more flexible model that allows the two sets of coefficients to be in different distributions and magnitudes. Sign consistency penalty is proposed to promote the similarity in sign to get more interpretable results by Zhang et al. [24]. In this paper, we propose a variable effect consistency mixture cure model with a sign-based penalty. The proposed method can promote the similarity in the signs of variable effect in the two model parts to improve interpretability. To accommodate high-dimensional credit data, a group lasso penalty is also imposed for variable selection [25].The contributions of this paper are as follows. First, we propose a variable effect consistency mixture cure model. The proposed method can lead to more interpretable results by promoting the similarity in the signs of coefficients in the two parts of the mixture cure model. Second, a group lasso penalty is imposed to select important variable subgroups and accommodate the high-dimensional data. Third, simulation and empirical analysis of credit data illustrate that the proposed method can improve the prediction accuracy as well as interpretability, which has important practical significance for applying the prediction results to the credit business.The remainder of the paper is organized as follows. Section2 introduces the variable effect consistency mixture cure model. Computational algorithm is presented in Section 3. Simulation is carried out in Section 4. Empirical study is presented in Section 5. Finally, conclusions are discussed in Section 6.


## 2. Methods

In this paper, we consider credit data withn customers and p variables. Denote U as the time to default and C as the time of censoring. Let Y be the unobservable binary variable with Y=0 indicating that the customer is cured and will not default (U=∞), whereas Y=1 indicates an uncured customer and will eventually default.Denoteδi=ΙUi<Ci as the censoring indicator of customer i, where Ui and Ci are the time to default and censoring time of customer i, respectively. δi=0 if censored and δi=1 otherwise. Note that there are three possible credit states of customers. (a) δi=0 and Yi=0: censored, cured customers who will not default; (b) δi=0 and Yi=1: censored, uncured customers who will eventually default and have not been observed to default in censoring time Ci; (c) δi=1 and Yi=1: uncensored, uncured customers who have been observed to default.Denoteti=minUi,Ci. Note that, in many practical cases, variables can be naturally grouped. For instance, many categorical variables may have several levels and can be represented by subgroups of dummy variables [26]. The additive model with polynomial or nonparametric components can be expressed as groups of basis functions [27]. In addition, grouping structure can also be introduced by taking advantage of prior knowledge. For example, genes belonging to the same biological pathway can also be considered a group [28]. Let x=x1Τ,…,xJΤΤ be the variable vector with J subgroups. xj=xj1,…,xjpjΤ is the j-th subgroup of variable vector, and ∑j=1Jpj=p. The observable data are ti,δi,x, i=1,…,n.The incident part of the mixture cure model describes the probability of default, for which we adopt a logistic regression model. Letπi be the probability of cured (nondefault) customer i.(1)πi=PrYi=0|x=11+expα0+xΤα,where α0 is the intercept, α=α1Τ,…,αJΤΤ is the vector of unknown regression coefficients, and the j-th subgroup of the coefficient vector is αj=αj1,…,αjpjΤ.In the latency part, for uncured customers, we adopt an exponential model for survival. Note that the exponential model has been commonly adopted in mixture cure models [29, 30]. It is easy to capture the relations between the probability and the time to default for it includes only one parameter [23]. The survival function is(2)Sit|Yi=1,x=exp−λit,λi=expβo+xTβ,where λi is the hazard function of customer i, β0 is the intercept, β=β1Τ,…,βJΤT is the vector of unknown regression coefficients, and the j-th subgroup of the coefficient vector is βj=βj1,…,βjpjΤ. Survival function Sit|Yi=1,x=PUi>t|Yi=1,x indicates the survival probability of uncured customers in time t, that is, the probability of default in time t given the customer will default.The mixture cure model can be given by(3)Sit|x=πi+1−πiSit|Yi=1,x,where Sit|x is the survival probability of customer i in time t.For observable datati,δi,x, the objective function can be written as follows.(4)Qα0,α,β0,β=−Lα0,α,β0,β+Pα,β,where Lα0,α,β0,β is the log-likelihood function, which is(5)Lα0,α,β0,β=∑i=1nδilog1−πi+logλi−λiti+1−δilog1−πiexp−λiti+πi,with πi=1/1+expα0+xTα, and λi=expβo+xTβ. Here, Pα,β is the penalty function, which is(6)Pα,β=μ1∑j=1Jpjαj+βj+μ22∑j=1J∑k=1pjsignαjk−signβjk2,where μ1>0 and μ2>0 are tuning parameters, · is the l2 norm, and sign· is the sign function. In many practical cases, grouping structure arises naturally. In addition, it is hard to interpret the results when coefficients corresponding to the same variables have conflicting signs. Therefore, we consider a flexible mixture cure model with sign consistency and group variable selection penalties. The first penalty is a group lasso penalty. It can conduct estimation and group variable selection by shrinking the coefficients of insignificant groups to 0. It considers grouping structures and has good prediction performance [26]. The second penalty is the sign consistency penalty. It promotes the sign consistency of α and β in the two parts of the model, which can lead to more interpretable results [31].


## 3. Computational Algorithm

In this section, the Expectation Coordinate Descent (ECD) algorithm is developed to optimize the objective function. In E-step, a latent unobservedYi is introduced to obtain a complete log-likelihood function. In CD-step, group coordinate descent is adopted to iteratively update a single parameter with the remaining parameters fixed at their most recent values. Sign function sign· is difficult to optimize for its discontinuity and nondifferentiability. Therefore, referring to [24, 32], we propose the approximation as follows:(7)signαjk−signβjk2≈αjkαjk+τ−βjkβjk+τ2,where τ is a small positive constant (more discussions below).The ECD algorithm updatesα0,α,β0,β in the m–th iteration as follows.


### 3.1. E-Step

Denote the observation of the latentYi as yi and denote the complete data as ti,δi,yi,x,i=1,…,n. The complete log-likelihood is(8)Lm=∑i=1n1−yilogπim+yilog1−πim+∑i=1nδiβ0m+xTβm−yitiexpβ0m+xTβm≔L1m+L2m,where(9)πim=11+expα0m+xTαm.The expectation ofyi is(10)Eyi=1δi=11−πimexp−λimtiπim+1−πimexp−λimtiδi=0,where(11)λim=expβ0m+xTβm.When customeri is observed to default (δi=1), the unobserved yi=1, whereas the expectation of yi is related to the probability of cured and the uncured but censored customers.In E-step, we take the expectation ofLm with respect to Eyi given the complete data ti,δi,yi,x,i=1,⋯,n.(12)ELm=EL1m+EL2m,where(13)l1m=EL1m=∑i=1n1−Eyilogπim+Eyilog1−πim,l2m=EL2m=∑i=1nδiβ0m+xTβm−Eyitiexpβ0m+xTβm.


### 3.2. CD-Step

In CD-step, group coordinate descent is adopted to iteratively updateα0,α,β0,β. The intercept α0m+1 is updated by(14)α0m+1=α0m−∂2l1m∂α0m2−1∂l1m∂α0m.Forαjm+1∈Rpj, we adopted a fast unified algorithm, Groupwise Majorization Descent (GMD) proposed in [33] to solve the group lasso penalized objective function in (4). The upper bound of the objective function is as follows:(15)−αjm+1−αjmΤαl1m∂αjm+μ2V1jm+M1jm2αjm+1−αjmΤαjm+1−αjmΤ+μ1pjαjm+1.Here,V1jm is pj-length vector, and M1jm is a constant as follows:(16)V1jm=1αjkm+τβjkmβjkm+τ−αjkmαjkm+τ1≤k≤pj,M1jm=ψ−∂2l1m∂αjk1m∂αjk2m1≤k1,k2≤pj+maxkμ21αjkm+τ2,where ψ· is the maximum eigenvalue function.Similarly, the interceptβ0m+1 is updated by(17)β0m+1=β0m−∂2l2m∂β0m2−1∂l2m∂β0m.Forβjm+1∈Rpj, consider the optimization function:(18)−βjm+1−βjmΤ∂l2m∂βjm+μ2V2jm+M2jm2βjm+1−βjmΤβjm+1−βjm+μ1pjβjm+1.Here,V2jm is pj-length vector, and M2jm is a constant as follows:(19)V2jm=1βjkm+ταjkm|αjkm|+τ−βjkmβjkm+τ1≤k≤pj,M2jm=ψ−∂2l2m∂βjk1m∂βjk2m1≤k1,k2≤pj+maxkμ21βjkm+τ2.The tuning parameters,μ1 and μ2, are selected by 5-fold cross-validation. The parameter τ in the approximation of the sign function controls the degree of approximation [24]. A smaller τ leads to a better approximation but less stable estimation. The proposed method is valid as long as τ is not too large, and the parameters with different signs can be distinguished [34]. Therefore, as suggested in [31], we set τ=0.1, which leads to satisfactory results.The ECD algorithm is summarized in Table1.Table 1
ECD algorithm.
Expectation Coordinate Descent algorithmInitializem=0, α0m=β0m=0, and αm=βm=0J×p. Compute πim and λim from (9) and (11)RepeatE-stepUpdateEyi from (10)CD-stepUpdateα0m+1 from (14)Forj=1,…,p, update αjm+1 from (15); update πim+1 from (9)Updateβ0m+1 from (17);Forj=1,…,p, update βjm+1 from (18); update λim+1 from (11)m=m+1Untilmaxαjm+1−αjm,βjm+1−βjm≤10−3


## 3.1. E-Step

Denote the observation of the latentYi as yi and denote the complete data as ti,δi,yi,x,i=1,…,n. The complete log-likelihood is(8)Lm=∑i=1n1−yilogπim+yilog1−πim+∑i=1nδiβ0m+xTβm−yitiexpβ0m+xTβm≔L1m+L2m,where(9)πim=11+expα0m+xTαm.The expectation ofyi is(10)Eyi=1δi=11−πimexp−λimtiπim+1−πimexp−λimtiδi=0,where(11)λim=expβ0m+xTβm.When customeri is observed to default (δi=1), the unobserved yi=1, whereas the expectation of yi is related to the probability of cured and the uncured but censored customers.In E-step, we take the expectation ofLm with respect to Eyi given the complete data ti,δi,yi,x,i=1,⋯,n.(12)ELm=EL1m+EL2m,where(13)l1m=EL1m=∑i=1n1−Eyilogπim+Eyilog1−πim,l2m=EL2m=∑i=1nδiβ0m+xTβm−Eyitiexpβ0m+xTβm.


## 3.2. CD-Step

In CD-step, group coordinate descent is adopted to iteratively updateα0,α,β0,β. The intercept α0m+1 is updated by(14)α0m+1=α0m−∂2l1m∂α0m2−1∂l1m∂α0m.Forαjm+1∈Rpj, we adopted a fast unified algorithm, Groupwise Majorization Descent (GMD) proposed in [33] to solve the group lasso penalized objective function in (4). The upper bound of the objective function is as follows:(15)−αjm+1−αjmΤαl1m∂αjm+μ2V1jm+M1jm2αjm+1−αjmΤαjm+1−αjmΤ+μ1pjαjm+1.Here,V1jm is pj-length vector, and M1jm is a constant as follows:(16)V1jm=1αjkm+τβjkmβjkm+τ−αjkmαjkm+τ1≤k≤pj,M1jm=ψ−∂2l1m∂αjk1m∂αjk2m1≤k1,k2≤pj+maxkμ21αjkm+τ2,where ψ· is the maximum eigenvalue function.Similarly, the interceptβ0m+1 is updated by(17)β0m+1=β0m−∂2l2m∂β0m2−1∂l2m∂β0m.Forβjm+1∈Rpj, consider the optimization function:(18)−βjm+1−βjmΤ∂l2m∂βjm+μ2V2jm+M2jm2βjm+1−βjmΤβjm+1−βjm+μ1pjβjm+1.Here,V2jm is pj-length vector, and M2jm is a constant as follows:(19)V2jm=1βjkm+ταjkm|αjkm|+τ−βjkmβjkm+τ1≤k≤pj,M2jm=ψ−∂2l2m∂βjk1m∂βjk2m1≤k1,k2≤pj+maxkμ21βjkm+τ2.The tuning parameters,μ1 and μ2, are selected by 5-fold cross-validation. The parameter τ in the approximation of the sign function controls the degree of approximation [24]. A smaller τ leads to a better approximation but less stable estimation. The proposed method is valid as long as τ is not too large, and the parameters with different signs can be distinguished [34]. Therefore, as suggested in [31], we set τ=0.1, which leads to satisfactory results.The ECD algorithm is summarized in Table1.Table 1
ECD algorithm.
Expectation Coordinate Descent algorithmInitializem=0, α0m=β0m=0, and αm=βm=0J×p. Compute πim and λim from (9) and (11)RepeatE-stepUpdateEyi from (10)CD-stepUpdateα0m+1 from (14)Forj=1,…,p, update αjm+1 from (15); update πim+1 from (9)Updateβ0m+1 from (17);Forj=1,…,p, update βjm+1 from (18); update λim+1 from (11)m=m+1Untilmaxαjm+1−αjm,βjm+1−βjm≤10−3


## 4. Simulations

In this section, some experiment examples are given to illustrate the performance of the proposed method compared to alternative methods. The proposed method is a mixture cure model with group lasso and sign consistency (MCGS). Two alternative methods are the standard mixture cure model without variable selection and sign consistency penalty (Full) and the mixture cure model with group lasso penalty (MCG), respectively. For comparison, alternative methods both adopt the logistic regression in the incident part and the exponential model in the latency part.Here, we set sample sizen=1000 and consider low-dimensional data with p=60 and high-dimensional data with p=200. The censoring time is generated from an exponential distribution with censoring rates η=0.25,5. We consider three examples regarding different grouping structures of coefficients and different types of variables. The true values of coefficients are generated according to the following settings in three examples:Example 1: for each subgroup, we setpj=10. Intragroup variables xjkm and xjkn are generated from a multivariate normal distribution with the correlation coefficient ρ=0.3km−kn, whereas intergroup variables are independent. Denote the true coefficients as αtrue and βtrue. The coefficients of the two scenarios are shown as follows:Scenario 1:(20)αtrue=0.1,…,0.1︸10,−0.4,…,−0.4︸10,0,…,0︸p−20Τ,βtrue=0.4,…,0.4︸10,−0.3,…,−0.3︸10,0,…,0︸p−20Τ.Scenario 2:(21)αtrue=0.5,…,0.5︸10,0.2,…,0.2︸8,−0.2,−0.2,0,…,0︸p−20Τ,βtrue=0.4,…,0.4︸10,−0.3,…,−0.3︸8,0,…,0︸p−20Τ.Example 2: the settings are similar to Example 1 except for the subgroup settings. We set 15 variables in the first subgroup, 5 variables in the second subgroup, and 10 variables for the remaining subgroups. The coefficients are shown as follows:(22)αtrue=0.5,…,0.5︸15,−0.4,…,−0.4︸5,0,…,0︸p−20Τ,βtrue=0.4,…,0.4︸15,−0.3,…,−0.3︸5,0,…,0︸p−20Τ.Example 3: consider the case with some discrete variables. For each subgroup, we setpj=10. A latent variable Zj is generated from a multivariate normal distribution with the intragroup correlation coefficient ρ=0.3km−kn and intergroup correlation independent. The coefficient setting is the same as Example 2. xjk is defined as follows:(23)xjk=IZj>0j≤p2,Zjj>p2.The performance of each model is me asured by 5 measures. Denoteθ∈α,β,αΤ,βΤΤ, θ^ as the estimation of θ, π^ as the estimation of π, and λ^ as the estimation of λ. The true positive rate (TPR), false positive rate (FPR), and mean square error (MSE) with respect to α, β, and αΤ,βΤΤ can be written as follows:(24)TPRθ=TPTP+FN,FPRθ=FPTN+FP,MSEθ=∑j=1pθjtrue−θ^j2∑j=1pθjtrue2,where(25)TP=∑j=1pIθjtrue≠0∩θ^j≠0,TP+FN=∑j=1pIθjtrue≠0,FP=∑j=1pIθjtrue=0∩θ^j≠0,TN+FP=∑j=1pIθjtrue=0.The relative root mean square error of the cure rate estimation (RMSEπ) and the relative root mean square error of the hazard function estimation (RMSEλ) are(26)RMSEπ=∑i=1nπ^i−πi2∑i=1nπi2,RMSEλ=∑i=1nλ^i−λi2∑i=1nλi2.Tables2–5 show the mean TPR, FPR, and MSE of the coefficients, as well as the standard deviations over the 100 replicates for each example.Table 2
Results of the estimation performance in scenario 1, Example 1.
ηpαβαΤ,βΤΤFullMCGMCGSFullMCGMCGSFullMCGMCGS0.2560MSE0.650.130.070.130.040.030.340.070.05(0.23)(0.04)(0.02)(0.03)(0.01)(0.01)(0.10)(0.02)(0.01)TPR—0.921.00—1.001.00—0.961.00(0.14)(0.01)(0.00)(0.00)(0.07)(0.00)FPR—0.180.28—0.590.48—0.380.38(0.18)(0.19)(0.16)(0.18)(0.13)(0.17)200MSE29.040.170.081.050.040.0412.380.090.06(9.28)(0.04)(0.02)(0.18)(0.01)(0.01)(3.76)(0.02)(0.01)TPR—0.831.00—1.001.00—0.911.00(0.20)(0.00)(0.00)(0.00)(0.10)(0.00)FPR—0.020.07—0.280.20—0.150.14(0.03)(0.05)(0.09)(0.09)(0.05)(0.07)560MSE9.170.420.230.510.070.074.010.210.13(3.02)(0.21)(0.04)(0.14)(0.02)(0.02)(1.23)(0.09)(0.02)TPR—0.551.00—1.001.00—0.771.00(0.24)(0.00)(0.00)(0.00)(0.12)(0.00)FPR—0.030.04—0.670.06—0.350.05(0.08)(0.07)(0.15)(0.11)(0.09)(0.09)200MSE17.301.000.122.380.050.0411.560.630.09(2.12)(0.00)(0.02)(0.48)(0.01)(0.01)(1.26)(0.00)(0.02)TPR—0.001.00—1.001.00—0.501.00(0.00)(0.00)(0.00)(0.00)(0.00)(0.00)FPR—0.000.02—0.350.02—0.170.02(0.00)(0.01)(0.08)(0.01)(0.04)(0.01)Note. In each cell, there is mean (standard deviation).Table 3
Results of the estimation performance in scenario 2, Example 1.
ηpαβαΤ,βΤΤFullMCGMCGSFullMCGMCGSFullMCGMCGS0.2560MSE0.510.110.110.130.040.040.330.080.08(0.19)(0.03)(0.03)(0.03)(0.01)(0.01)(0.11)(0.02)(0.02)TPR—0.990.99—1.001.00—0.990.99(0.03)(0.03)(0.00)(0.00)(0.01)(0.01)FPR—0.330.33—0.710.71—0.520.52(0.23)(0.23)(0.13)(0.13)(0.14)(0.14)200MSE24.300.160.161.010.050.0513.400.110.11(6.60)(0.03)(0.03)(0.19)(0.01)(0.01)(3.50)(0.02)(0.02)TPR—0.980.98—1.001.00—0.990.99(0.03)(0.03)(0.00)(0.00)(0.02)(0.02)FPR—0.040.04—0.350.35—0.190.19(0.05)(0.05)(0.08)(0.08)(0.06)(0.06)560MSE5.560.320.330.530.100.113.210.220.23(1.83)(0.10)(0.07)(0.15)(0.03)(0.03)(0.97)(0.06)(0.04)TPR—0.630.87—1.001.00—0.810.93(0.19)(0.20)(0.01)(0.01)(0.10)(0.10)FPR—0.070.08—0.700.36—0.380.22(0.13)(0.11)(0.15)(0.33)(0.11)(0.19)200MSE24.460.600.394.520.110.1315.130.370.27(3.45)(0.25)(0.04)(0.85)(0.03)(0.03)(1.71)(0.14)(0.02)TPR—0.380.93—1.001.00—0.690.96(0.21)(0.17)(0.00)(0.02)(0.11)(0.08)FPR—0.000.02—0.320.08—0.160.05(0.00)(0.04)(0.12)(0.13)(0.06)(0.08)Note. In each cell, there is mean (standard deviation).Table 4
Results of the estimation performance in Example 2.
ηpαβαΤ,βΤΤFullMCGMCGSFullMCGMCGSFullMCGMCGS0.2560MSE0.350.130.070.090.040.020.250.090.05(0.13)(0.03)(0.02)(0.02)(0.01)(0.01)(0.08)(0.02)(0.01)TPR—1.001.00—1.001.00—1.001.00(0.00)(0.00)(0.00)(0.00)(0.00)(0.00)FPR—0.370.08—0.760.12—0.570.10(0.24)(0.06)(0.09)(0.07)(0.13)(0.06)200MSE20.240.230.090.680.040.0212.700.160.06(4.57)(0.05)(0.02)(0.12)(0.01)(0.01)(2.81)(0.03)(0.01)TPR—1.001.00—1.001.00—1.001.00(0.00)(0.00)(0.00)(0.00)(0.00)(0.00)FPR—0.020.04—0.370.06—0.200.05(0.04)(0.02)(0.09)(0.03)(0.05)(0.02)560MSE4.720.980.100.300.040.033.020.620.07(1.08)(0.10)(0.02)(0.07)(0.01)(0.01)(0.66)(0.06)(0.02)TPR—0.051.00—1.001.00—0.531.00(0.22)(0.00)(0.00)(0.00)(0.11)(0.00)FPR—0.000.03—0.640.04—0.320.03(0.00)(0.03)(0.17)(0.03)(0.09)(0.03)200MSE17.301.000.122.380.050.0411.560.630.09(2.12)(0.00)(0.02)(0.48)(0.01)(0.01)(1.26)(0.00)(0.02)TPR—0.001.00—1.001.00—0.501.00(0.00)(0.00)(0.00)(0.00)(0.00)(0.00)FPR—0.000.02—0.350.02—0.170.02(0.00)(0.01)(0.08)(0.01)(0.04)(0.01)Note. In each cell, there is mean (standard deviation).Table 5
Results of the estimation performance in Example 3.
ηpαβαΤ,βΤΤFullMCGMCGSFullMCGMCGSFullMCGMCGS0.2560MSE0.330.190.100.040.030.020.220.130.07(0.08)(0.05)(0.02)(0.01)(0.01)(0.01)(0.05)(0.03)(0.01)TPR—1.001.00—1.001.00—1.001.00(0.01)(0.00)(0.00)(0.00)(0.00)(0.00)FPR—0.530.29—0.660.29—0.600.29(0.15)(0.12)(0.08)(0.12)(0.09)(0.12)200MSE2.420.480.170.200.050.031.560.310.11(0.57)(0.14)(0.02)(0.03)(0.01)(0.01)(0.35)(0.08)(0.01)TPR—0.951.00—1.001.00—0.981.00(0.18)(0.00)(0.00)(0.00)(0.09)(0.00)FPR—0.050.14—0.570.16—0.310.15(0.07)(0.06)(0.10)(0.06)(0.07)(0.06)560MSE1.310.990.430.080.030.030.830.620.27(0.33)(0.08)(0.29)(0.02)(0.01)(0.01)(0.20)(0.05)(0.18)TPR—0.010.83—1.001.00—0.510.92(0.10)(0.38)(0.00)(0.00)(0.05)(0.19)FPR—0.000.14—0.510.36—0.250.25(0.02)(0.14)(0.13)(0.17)(0.07)(0.12)200MSE6.941.000.280.510.040.054.470.630.19(1.12)(0.00)(0.06)(0.07)(0.01)(0.01)(0.68)(0.00)(0.04)TPR—0.001.00—1.001.00—0.501.00(0.00)(0.00)(0.00)(0.00)(0.00)(0.00)FPR—0.000.17—0.340.22—0.170.19(0.00)(0.06)(0.08)(0.06)(0.04)(0.06)Note. In each cell, there is mean (standard deviation).As indicated in Tables2–5, the two group selection methods (MCGS and MCG) perform significantly better than the Full method. This is expected since the group lasso can select important subgroups of variables. Comparing the two group methods, the proposed method has competitive performance compared with the MCG method. It indicates that promoting sign consistency improves the performance in terms of estimation. For instance, under Scenario 1 in Example 1 with η=5 and p=200 in Table 2, the mean MSEs of α, β, and αΤ,βΤΤ for the proposed method are 0.12, 0.04, and 0.09, respectively, compared to 1.00, 0.05, and 0.63 for the MCG method and 17.3, 2.38, and 11.56 for the Full method.Tables6–8 show the mean RMSE of π and λ, as well as the standard deviations over the 100 replicates for each example. The results illustrate the performance in terms of prediction of the probability of nondefault and survival.Table 6
Results of the prediction performance in Example 1.
ηpRMSEπRMSEλFullMCGMCGSFullMCGMCGSScenario 10.25600.060.020.010.570.100.12(0.02)(0.01)(0.00)(0.80)(0.08)(0.08)2000.370.030.0224.100.120.16(0.04)(0.01)(0.01)(57.28)(0.08)(0.10)5600.330.190.106.320.230.18(0.05)(0.18)(0.03)(20.47)(0.28)(0.09)200.410.960.02>10 ^ 30.150.10(0.03)(0.04)(0.01)(>10 ^ 3)(0.15)(0.08)Scenario 20.25600.060.020.020.520.120.12(0.01)(0.01)(0.01)(0.71)(0.07)(0.07)2000.310.030.0372.580.150.15(0.03)(0.01)(0.01)(387.68)(0.10)(0.10)5600.300.080.145.720.210.27(0.06)(0.04)(0.06)(10.42)(0.18)(0.15)2000.530.260.17>10 ^ 30.260.35(0.04)(0.24)(0.04)(>10 ^ 3)(0.17)(0.17)Note. In each cell, there is mean (standard deviation).Table 7
Results of the prediction performance in Example 2.
ηpRMSEπRMSEλFullMCGMCGSFullMCGMCGS0.25600.050.020.010.360.100.06(0.01)(0.01)(0.00)(0.35)(0.07)(0.05)2000.270.040.0110.010.120.07(0.03)(0.01)(0.00)(33.80)(0.17)(0.06)5600.260.880.021.980.210.09(0.03)(0.16)(0.01)(4.56)(0.30)(0.08)2000.410.960.02>10 ^ 30.150.10(0.03)(0.04)(0.01)(>10 ^ 3)(0.15)(0.08)Note. In each cell, there is mean (standard deviation).Table 8
Results of the prediction performance in Example 3.
ηpRMSEπRMSEλFullMCGMCGSFullMCGMCGS0.25600.210.110.060.040.030.02(0.04)(0.03)(0.02)(0.01)(0.01)(0.01)2000.830.280.100.300.050.03(0.10)(0.08)(0.02)(0.07)(0.01)(0.01)5601.000.770.340.090.030.03(0.19)(0.09)(0.21)(0.03)(0.01)(0.01)2002.620.720.210.990.040.04(0.21)(0.06)(0.10)(0.33)(0.01)(0.01)Note. In each cell, there is mean (standard deviation).As shown in Tables6–8 the prediction performance of group selection methods is significantly better than that of the Full method, and the proposed method has competitive performance compared with the MCG method. For example, in Example 2 with η=0.25 and p=200 in Table 8, the mean RMSEπ and RMSEλ for the proposed method are 0.01, and 0.07, respectively, compared to 0.04 and 0.12 for the MCG method and 0.27 and 10.01 for the Full method. In addition, compared with the results regarding low- and high-dimensional settings, the group selection methods have greater advantages in prediction performance when the dimensionality is higher.Results of simulation reveal that the proposed variable effect consistency mixture cure model can improve the performance in terms of estimation and prediction compared with alternative methods.


## 5. Empirical Study

In this section, we applied our proposed method to real data on credit loans. The data come from the personal loan department of a city commercial bank in China, which contains 4796 personal loan samples from 2014 to 2019 after preprocessing. The data include mortgage loans and credit loans, covering consumer durables, personal housing decoration loans, and other personal consumption loans. Censoring timeCi is the interval between the loan value date and either default or the end of observation (June 1, 2019). Therefore, censoring times of customers vary from individual to individual. It has a mean of 1.93 years and a standard deviation of 0.8. Customers whose time to default Yi is longer than the censoring time Ci are censored (δi=0). 47 out of 4796 customers are censored. By transforming the discrete variables into dummy variables, the data contain 27 variables. Table 9 provides a list of variables and their descriptions.Table 9
Description of variables.
VariablesDescriptionBusiness typeConsumer durables, personal housing decoration loans, and other personal consumption loansInterest rate0.029,0.095Loan line0,+∞Loan term0,+∞Early repaymentYes, noEntrusted paymentYes, noAge18,70GenderMale, femaleEducationMaster/doctor, bachelor, vocational education, high school, and belowMedical insuranceYes, noHousing statusSelf-purchasing (with mortgage), self-purchasing (without mortgage), and othersEmploymentEmployed and othersType of workplaceGovernment organization/institution, firm, and othersOccupationManagers, commercial and service workers, and othersProfessional titleAdvanced, intermediate, primary, and no professional titleAnnual household income (RMB)≤200,000, 200,000−400,000, 400,000−600,000, ≥600,000, and unknownIn this section, the alternative method is a mixture cure model with group lasso (MCG). Different from the simulation, the real values of parametersα and β are unknown in real data. Referring to [30, 35], we adopt the (1) log-rank statistics and (2) negative log-likelihood to evaluate the performance of the models instead of TPRθ, FPRθ, MEθ, RMSEπ, and RMSEλ in simulation.Log-rank statistics is a commonly used indicator in survival analysis to test the null hypothesis that there is no significant difference in survival distribution between two or more independent groups. It is calculated by cross-validation. We sequentially take1/10 samples as the validation set and the remaining as the training set. Apply the proposed and alternative methods to obtain the estimation of α and β and then calculate the xΤα and xΤβ for the validation set. Results of xΤα and xΤβ are based on 10 replicates. Divide the calculated xΤα into two groups at the median and calculate the log-rank statistics. Similarly, divide the calculated xΤβ into two groups at the median and calculate the log-rank statistics. The mean log-rank of the proposed and the alternative methods is 5.6 and 4.3 respectively, indicating better performance of the proposed method.Figure1 shows the Kaplan–Meier curves stratified by different groups. Kaplan–Meier curves are commonly used to describe the change of the survival probability overtime in survival analysis [36, 37]. The probability of being cured is negatively related to xΤα, and the survival time is negatively related to xΤβ. In Figure 1, a group is denoted by “low risk” with lower xΤα (a) or lower xΤβ (b) whereas another group is denoted by “high risk.” As indicated in Figure 1, there are clearly different trends in the curves in different groups. Customers with lower xΤα and xΤβ have lower risk and are less likely to default.Figure 1
Kaplan–Meier curves stratified by different groups: (a) divided by the median ofxΤα and (b) divided by the median of xΤβ.
(a)(b)To assess the performance of the model, the data are randomly divided into training set and test set by 2 : 1. The training set is used for fitting the model and the test set is used for evaluating the prediction performance of the fitted model. The tuning parameters are selected by 5-fold cross-validation. The mean (standard deviation) negative log-likelihood of the proposed method (MCGS) and the alternative method (MCG) is 106.04 (16.04) and 118.60 (19.09), respectively. The result is based on 100 duplicates. It indicates that the proposed method performs better than the alternative in terms of model fit and prediction.Table10 reports the estimations of the MCGS method and the MCG method. A positive coefficient α indicates that the variable is positively related to the probability of default, and a positive coefficient β indicates that the variable is negatively related to default time. Both probabilities of default and default time are two quite relevant credit aspects. Compared with the alternative method, the signs of the α and β of the proposed method are promoted to be more consistent, whereas the business type in the MCG model has an opposite effect on the probability and time to default. The results show that promoting variable effect consistency can improve prediction performance as well as interpretability.Table 10
Estimation of coefficients of the methods.
MCGMCGSαβαβα0/β0−1.17−4.07−1.43−3.67Business typeConsumer durables0.150.700.110.12Personal housing decoration loans−0.020.2100Interest rate0.540.170.340.28Loan line−0.060−0.03−0.03Loan term0000Early repayment (yes)0−0.2400Entrusted payment (yes)−0.13000Age0000Gender (male)00.820.250.30EducationMaster/doctor−0.03−0.32−0.02−0.02Bachelor−0.04−0.22−0.06−0.06Vocational education0.020.320.050.05Medical insurance (yes)00.3400Housing statusSelf-purchasing (with mortgage)0−0.0100Self-purchasing (without mortgage)00.0400Employment (employed)−0.08−0.31−0.24−0.21Type of workplaceGovernment organization and institution−0.14−1.5300Firm−0.14−0.4000OccupationManagers0.090.090.040.04Commercial and service workers−0.25−0.29−0.11−0.11Professional titleAdvanced0.040.410.040.04Intermediate−0.02−0.25−0.04−0.04Primary0−0.0300Annual household income (RMB)≤200,00000.0500200,000−400,00000.3800400,000−600,0000−0.2900≥600,0000−0.1900The coefficient results of the proposed method reveal that interest rate, loan line, business type, gender, education, and employment status are important variables that affect the probability of default and the time to default. Loan term, age, medical insurance, entrusted payment, early repayment, annual household income, type of workplace, and housing status have no significant impact on credit. The impact of occupation and professional title on credit is not clear.From the perspective of loan products, we find that interest rate has a negative impact. This is not surprising, as higher interest rates lead to higher costs, and the customers are more likely to default. The loan line has a positive effect. One possible explanation is that low-risk customers are more likely to obtain a higher loan line. Loan term, entrusted payment, and early repayment have no effect on the credit. Different coefficient of business type reveals that, compared with other personal loans, consumer durables are more likely to default.From the perspective of the influence of the variables of the customers, employed customers have a positive impact. Age, annual household income, housing status, and type of workplace have no significant effect on the credit. Compared with women, men are more likely to default. This is consistent with the results of [38] and the personality characteristics of men’s risk preference [39]. Customers with higher education are less likely to default. Bachelor degree or above has a positive effect on credit. Generally, customers with higher education have a higher chance of getting decent jobs and income. They tend to maintain good credit records and are less likely to default. Compared with other employment groups such as self-employed, freelance, and unemployed ones, the employed group has a more stable income and is less likely to default.


## 6. Conclusions

The mixture cure model is widely adopted in credit scoring for its ability of predicting whether customers will default and when they are likely to default. However, most of the existing studies ignore the relations between the two sets of variable effects in the two model parts which may get conflicting results of variable effects. It can be difficult to interpret the results and apply them in practice.In this paper, we propose a variable effect consistency mixture cure model, to promote the similarity of the sign of variables in the two model parts by imposing a sign consistency penalty. Meanwhile, to accommodate the high-dimensional credit data, we also impose a group lasso penalty to conduct variable selection and parameter estimation. Simulation shows that the proposed method has competitive performance compared with the MCG method and significantly outperforms the Full method in terms of estimation and prediction. Furthermore, the empirical study illustrates that the proposed method can improve prediction performance as well as interpretability. The results of the variable effect consistency mixture cure model also offer additional insights into the relationship between the variable effect before and after loan.



---

*Source: 3112987-2022-02-21.xml*