# Optimizing Flight Delay Predictions with Scorecard Systems
## 4 Research Method
### Model-Building Process  
The model-building process involves a structured sequence of steps designed to transform raw data into a predictive framework, ensuring a robust and reliable model tailored to the analysis objectives. The process consists of the following steps:  
* Random sample and data partition: Two datasets are created, a training set and a validation set, based on a simple random sample method (without duplications) in proportions of 60%/40%.
* Binning, categorization, and grouping: Groups and categories of variables are created. Based on entropy statistics, every interval variable is split into categories. This is a typical method used in decision tree techniques. Nominal variables are also grouped, especially when the number of unique categories is too big. After binning, every variable is transformed into a logit variable.
* Variable preselection: Based on univariate statistics for every variable, unimportant variables are indicated, such as those with too small of a predictive power or that are too unstable over time.
* Multifactor analysis and multidimensional variable selection: The heuristic method, called branch and bound, available in SAS software, is used [48] along with RFE (recursive feature elimination), available in Python 3.13.  
Before diving into specific steps such as model assessment, implementation, and monitoring, it is essential to highlight that each stage plays a crucial role in ensuring the model's reliability, relevance, and practical applicability. The process consists of the following steps:  
* Model assessment: There are no unique model criteria. The most popular are the following: predictive power (Gini); stability, such as R, Gini, or delta Gini-relative difference between Gini on training and validating datasets; collinearity measures, such as Max VIF--maximal variance inflation factor [49], MAX Pearson--maximal Pearson correlation statistics on a pair of variables, and MAX Con Index--maximal condition index [50]; and variable significance measures (\(p\)-values).
* Model implementation: It is recommended to calculate all innovation factors, especially the predicted innovation rate, on a yearly basis and use them as measures of innovation change.
* Monitoring and testing: After every yearly calculation, some factors can be tested by additionally organized surveys, maybe on a lower scale, or only for extreme cases, like the highest or the lowest predicted innovation rates, to verify and monitor the equality of the observed and predicted innovation measures. Also, what should be emphasized, based on predicted innovation rates, can be made into a ranking of all variables and then can be investigated with a more advanced study for some chosen variables to understand the change and quality of innovation. Another interesting point relates to the possibility to create recommendations dedicated to a particular variable to increase its innovation; namely, it can be done by a partial score analysis. To increase the innovation rate, the variable should change its characteristic existing in the model to obtain a lower partial score. The mentioned ideas are described further in the following subsections.  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Dataset** & **Nobs** & **Delayed** & **On Time** & **DR [\%]** \\ \hline Training & 1,233,489 & 61,170 & 1,172,319 & 5.2 \\ Validating & 822,334 & 40,785 & 781,549 & 5.2 \\ All & 2,055,823 & 101,955 & 1,953,868 & 5.2 \\ \hline \end{tabular} Source: Authors’ own elaboration.  
\end{table}
Table 3: Datasets used in the modelling process.