# A Review of Deep Learning Advancements in Road Analysis for Autonomous Driving
## 4 Road Surface Classification
### The Nature of Surface Material
Using eleven semantic segmentation classes--asphalt, paved, unpaved, markings, speed-bump, cats-eye, storm-drain, patch, water-puddle, pothole, and cracks, they trained and tested multiple neural architectures, the best result being obtained with ResNet50 with weight added in the loss function. In [96], the authors trained and tested AlexNet on 500 original images from Poland roads from which nine types of road surfaces were labeled: asphalt, concrete, tryinka (road surface made of large hexagonal concrete paver blocks), concrete paver blocks, granite paver blocks, openwork surface, gravel, sand, and grass. The study conducted two sets of experiments: one for recognizing specific types of road surfaces and another for determining the general condition of the surfaces. For general condition classification, the images were divided into three types: good ones, i.e., asphalt and concrete; medium ones,i.e., trylinka, concrete cubes, granite cubes, and openwork surfaces; and bad ones, i.e., sand, gravel, and grass. The results showed high accuracy rates, over 85% in identifying specific road types and over 91% in determining their general condition. In comparison, in [97], the authors innovatively employ a deep CNN using a ResNet50 architecture, coupled with a \"one-cycle\" learning rate (LR) optimization technique, to classify five pavement types: asphalt, concrete, interlocking, cobblestone, and unpaved. The dataset used consists of 49,173 images sourced from highways and local roads in Ceara State, Brazil, for which, in the testing phase, the authors claimed near-perfect accuracy, \(P\), and \(R\) values close to 100%. The model's success lies in its use of transfer learning with pre-trained ResNet50 weights, fine-tuned for pavement surface type classification. An innovative approach is presented in [98], where the authors integrate visual and tactile data in an early fusion method (VTF). This integration is achieved through a CNN-Transformer network, for which the visual data are obtained from road images, while tactile data come from a specially designed intelligent tire system equipped with piezoelectric sensors. The CNN-Transformer architecture effectively fuses these two modalities. The network comprises modality-specific SE-CNNs (Squeeze-and-Excitation Convolutional Neural Networks) for initial feature extraction from both visual and tactile inputs. This method reports an accuracy of 99.48% on a dataset consisting of 40,000 images, which includes four classes of road surfaces: marble road, painted road, gravel road, and asphalt road. Combining classes that define the surface material with those that characterize the ambient circumstances resulted in the proposal of various techniques. For example, in [99], the authors used a dataset consisting of 12,440 images captured on Lithuanian roads, which were categorized into six classes, representing combinations of three road surface types (asphalt, cobblestone, and gravel) under two different conditions (wet and dry), namely: gravel wet, gravel dry, cobblestone wet, cobblestone dry, asphalt wet, and asphalt dry. Using AlexNet as a baseline network, they obtained an _Acc_ of 88.3% on the dataset. In the same manner of creating a comprehensive dataset (Road Surface Condition Dataset--RSCD), in [100], the authors conducted a real-vehicle image acquisition on Beijing roads, resulting in approximately 1 million image samples. Using three detailed road level annotations: material (asphalt, concrete, dirt/mud, gravel), friction (dry, wet, water, fresh snow, melted snow, and ice) and unevenness (smooth, slight, or severe), they obtained 27 combined classes for which an EfficientNet-B0-based CNN was trained, with a decision-level fusion method based on the Dempster-Shafer evidence theory [101]. The paper reports an _Acc_ of 92.05% for the CNN model and 97.50% after the fusion process. Using the same dataset, the authors from [102] introduce Attention-ReXNet, an advanced CNN that builds upon the ReXNet [103] architecture.