# Towards Low-Cost Pavement Condition Health Monitoring and Analysis Using Deep Learning
## 2 Materials and Methods  
### Background of Deep Learning Techniques  
Recently, there has been a universal rise in the research and use of deep learning applications to solve complex and different problems across varying research fields [15]. This has been due to the increases in the accuracy of these methodologies. The accuracies in some instances are now considered even better than those of human abilities as demonstrated by the ImageNet large scale visual recognition challenge [16] in which the Human benchmark for recognizing objects was beaten in 2015 by a developed deep learning framework [17] and whose accuracy mark has continued to be improved upon every year of the challenge as new frameworks and algorithms have been developed. The basic concept of Machine learning is one in which a computer system can automatically carry out tasks such as object detection, image classification and speech recognition based on supplying it with datasets centered around the individual task. Deep Learning is a subset of Machine learning which utilizes neural networks rather than traditional handcrafted features. Deep Learning involves the use of computationally based models which are made of several processing layers which learn data representations with several layers of generalization [18]. Artificial Neural Networks (ANN) are based on the biological neural network and consist of input layers, hidden layers and finally output layers. Deep learning networks figure out the complex structures within datasets by the utilization of backpropagation algorithms which alert the computer how to alter its internal parameters to yield the best representation of the next layer and eventually the final model. Neurons within the network receive inputs then process them and feed it forward to other neurons in successive layers. The hidden neurons take outputs from previous layers, compute new outputs with the use of activation functions which are then fed forward to the next layer with the use of different weights applied during the applications. A simple graphic of this is demonstrated in Figure 1.  
They essentially work to understand how the data is built up and thus how to identify, predict or classify future similar but unknown datasets. For image processing, convolutions are utilized. Convolutions utilize filters which help extract different features from the image such as edge information. In neural networks, features are extracted using filters with weights that undergo learning during the training process. The retrieved features are then summed together to make choices. The spatial relationship of pixels within an image is also considered in the convolution which helps to identify particular objects that have defined spatial relationships with other objects. The most common type of system is called supervised learning wherein a convolutional neural network is fed annotated datasets and learns to make similar predictions based on the learning data. Convolutional neural networks (CNN) have been vastly utilized for this and they are designed in such a way to process data in the form of arrays. CNN's are generally utilized for detection, image segmentation and classification and identifying objects and regions within an image. A typical depiction of the workflow for deploying one of these models and network is shown in Figure 2.  
Figure 1: Basic structure within a neural network. There are existing networks which have been trained on millions of images and utilizing high-performance PC systems which are very deep with complex convolutional layers. Using these as building blocks, Transfer learning has been utilized as a quicker way to develop models without starting over the process, thus reducing time and computational costs. It is an approach wherein knowledge developed in one task is transferred and used to improve the learning of different target tasks using a pre-trained model as a baseline [19]. Within the CNN, each layer extracts features from an image and successive layers further extract more complex features.