# Exploiting Data Analytics and Deep Learning Systems to Support Pavement Maintenance Decisions
## 3 Methodology
### Use of Deep Learning for Tabular Sets--Use of Fastai Deep Learning Library  
Once the important features were identified, the next step was developing a model that could attempt to predict the year for intervention when roads would reach a similar degradation level as those identified in the dataset, based on the available characteristics. To accomplish this, a deep learning model was developed for the application on a tabular dataset. Deep Learning is a machine learning technique based on \"learn by example\" principle. A Deep Learning model commonly uses large sets of labeled data which is processed through neural network architectures built-up with many layers. By exploiting the input data, through synaptic connections between adjacent layers with feed-forward propagation, the output data are computed. The output quality depends on the recognition accuracy, which in turn depends on the consistency of both the neural structure and process data. Therefore, it is clear that the choice of the input data (features) and the neural network architecture, including the typical training and testing parameters, are important to achieve decision-making targets. Whilst, a significant amount of research has been focused on the utilization of deep learning for image analysis, there is a lot of merit to leverage the power of deep learning for tabular dataset analysis as well. New base architectures have shown a lot of worth in this type of analysis over traditional classification and grouping methods [60] and by considering a deep learning model, this workflow can leverage both the resources of the gradient boosting for feature selection and deep learning for the model development. Deep learning models also have shown advantages over traditional approaches when datasets have high cardinality categorical variables. This is as a result of the model using embeddings for these categorical features [61] and this is important given the presence of these variables in the dataset. For this study, the open source FastAI framework [60] was utilized. This library was chosen because of its ease of use, therefore, making it easily interpretable and easily replicated for authorities and users. The library was designed around two main goals: to make its implementation and use approachable and productive whilst maintaining flexible configurability. This is important as there is not a significant level of training needed to create the models and therefore explaining its implementation for road professionals would not be difficult. FastAI also offers excellent support for tabular datasets with built-in loaders to handle this type of data. Within the FastAI library, there is high-level API support for tabular datasets which allows the creation of models based on characteristics of the dataset available. Within the tabular data setup, there are also provisions to denote categorical elements, which is an added advantage, considering the dataset at hand. The tabular API is made up of the components shown in Figure 3.  
The library also has model-fitting methods within its structure namely the use of 'lr_find', which allows the establishment of a good learning rate to be used rather than going through an iterative process at the training level. Within the model, the training was regularized utilizing dropouts for dense layers and embeddings (0.05). These were features applied to avoid overfitting of the model. A random seed value of 47 was utilized for the beginning of the training of the model. Three preprocessors were also utilized--FillMissing, Categorify and Normalize; FillMissing searches for missing values, Categorify finds the categorical variables and Normalize carries out a normalization for continuous variables within the dataset. The dependent variable ('dep_var) for this study was the year and the features used were grouped based on them being either categorical (cat_vars) or continuous variables (cont_vars).