# A Review of Deep Learning Advancements in Road Analysis for Autonomous Driving
## 4 Road Surface Classification
### The Nature of Surface Material  
The composition of a road's surface material plays an important role in its characteristics, durability, and suitability for various environmental conditions and traffic demands. Understanding these distinctions is crucial for a multitude of applications, ranging from infrastructure maintenance and urban planning to the calibration of autonomous vehicle navigation systems. As we review the SOTA in this domain, we will highlight innovative approaches and methodologies that have been developed to accurately identify and classify road surface types, underscoring the significance of material composition in the context of RSC. In [86], the authors developed a deep ensemble network utilizing various sensor data obtained from the vehicle's Controller Area Network, such as data from IMU sensors, steering angle sensors, vehicle speed sensors, and acceleration sensors. The dataset comprised four distinct road types: flat road, sinusoidal road, manhole and pothole, and bump, which is training information for a deep ensemble network that incorporates Long-Short Term Memory (LSTM) units. In the same direction, but working with accelerometer data and GPS information as input information, in [87], the authors utilize CNNs, LSTM network, and reservoir computing models to analyze the data and predict road conditions like asphalt, concrete, cobblestone, and dirt roads. The best result is obtained with the custom CNN, which was designed to process the sequences of oscillations produced by the accelerometers of mobile devices, adapting the inputs (temporal multidimensional 1D samples) into a form suitable for a 2D convolutional process. Similarly, the work from [88] lies in the application of CNN to time-frequency representations of data from accelerometers and gyroscopes. This method, termed CNN-SP, is tested against traditional CNN applications on time-domain data (CNN-1D) and shallow ML algorithms on a dataset collected from 12 different vehicles over more than 40 km of road, achieving an impressive 97.2% identification _Acc_. In a similar manner, in [89], the author uses data collected from accelerometers and gyroscopes from nine different vehicles, resulting in nine datasets. They tested 34 computational models, employing techniques like KMC, SVM, KNN, LSTM, CNN, and hybrid CNN-LSTM models. The best-performing model in the study was a CNN-based deep neural network. This model achieved a validation _Acc_ of 93.17% in classifying surfaces between segments of dirt, cobblestone, or asphalt roads. An innovative approach regarding the dataset presented in [90], where the authors use as neural input of a 2D representation of the tire-pavement interaction noise signals. These signals are acoustic noises generated by the interaction between the tire and road surface, which are captured using a microphone, and transformed into 2D images through a continuous wavelet transform. By doing so, 11,040 images were generated, considering two classes: asphalt and snowy roads. On this dataset, a simple CNN architecture was based on [91] for which an _Acc_ of 91.84% was obtained on the data from the front wheel and 91.18% on the data from the rear wheel. A different approach is presented in [92] where the authors introduce a dataset called Road Traversing Knowledge (RTK) with 77,547 road images from Brazil, classified into three classes: asphalt, paved, and unpaved. On this dataset, they trained a CNN consisting of a simple structure for which the training stage was also performed on 5093 images from the KITTI dataset and 1401 images from CaRINA [93]. An _Acc_ of 94.57% was obtained on the RTK dataset with augmentation, trying to leave each surface type with a similar amount of samples. Using the same dataset, which also includes the semantic segmentation data for each label, the authors from [94] utilize the U-Net [95] architecture paired with different ResNet architectures.