# A Deep Learning Approach with Feature Derivation and Selection for Overdue Repayment Forecasting
## 3 Proposed System
### Feature Derivation and Selection  
Generally, the process of feature derivation is essential when the features of the input are not abundant. Feature derivation requires manual design and professional prior knowledge, thus making the derivation procedure stochastic and insufficient. Generally, derivation methods are based on statistical information and expert diagnosis. Statistics-based methods calculate some common mathematical statistical values such as the maximum value, mean value and variance value of part of  
Figure 1: Structure of the AD-BLSTM model. Input raw data are pre-processed into dynamic label sequences and static features. Feature derivation and a selection approach are operated on static features for better informative representation. Two-layer bi-directional long short-term memory (BiLSTM) encodes the sequential input, while the deep neural network (DNN) encodes selected features. The attention mechanism balances the importance of each hidden LSTM layer, and two encoded vectors are concatenated into the output layer. the existing features. Expert diagnosis-based methods introduce new features by professional prior knowledge and manual inference. Motivated by the statistics-based feature derivation methods and the problem of insufficiency, our approach improves upon the original method by categorizing features into various major categories and expanding the mathematical statistical values in each major class. Our proposed feature engineering framework is illustrated in Figure 2.  
The purpose of feature derivation is to extend the number of input features from \(\mathcal{N}_{0}\) to \(\mathcal{N}_{d}\), where \(\mathcal{N}_{0}\) is the number of input features and \(\mathcal{N}_{d}\gg\mathcal{N}_{0}\). First of all, we manually design \(\mathcal{N}_{c}\) major categories according to the content of the input features. Besides this, mathematical statistics are set as one major category. Specifically, in this research, we set financial indicators, products, mathematical statistics, periods and time conditions as our major categories. Secondly, we add a number of subindexes as adequately as possible in each major category. For instance, the category of mathematical statistics includes subindexes of the cumulative value, cycle proportions, variance value, etc. Finally, the connection of one subindex from each major category is extracted as a new derived feature. Each customer's repayment behavior sequence is mapped into the derived features, and the mapping value is the corresponding feature value. A negative number is filled as a missing value signal when there is no accurate mapping value between the input data and derived feature. The total amount of newly derived features can be calculated by Equation (2):  
\[\mathcal{N}_{d}=\prod_{i=1}^{\mathcal{N}_{c}}n_{i} \tag{2}\]  
where \(n_{i}\) is the amount of subindexes in the \(i\)-th major category. To maximize the size of derived features, \(n_{i}\) is supposed to be relatively large when designing the major categories and subindexes. One specific instance is illustrated in Figure 3. By combining the \"overdue days\" subindex in the financial indicator major category, \"non-payday loan\" in product, \"maximum\" in mathematical statistics, \"weekday\" in time condition and \"one week\" in period, we can obtain a newly derived feature (Feature 1): the maximal overdue days for a non-payday loan on weekdays of the recent one week. Our proposed derivation approach involves the standardization and extension of common measures, and the problem of aimlessly selecting combining variables is solved by our method. Although feature combinatorial representation can be accomplished by the neural network automatically due to deep learning's characteristic of end-to-end learning and its powerful representation ability, the experimental result in our research shows that the derived features can improve the model performance despite requiring additional work.