# Applications of Artificial Intelligence Enhanced Drones in Distress Pavement, Pothole Detection, and Healthcare Monitoring with Service Delivery

Authors: Yue Wang; Tian Ye
Journal: Journal of Engineering
 (2022)

---

## Abstract

Artificial Intelligence (AI) has fascinated the present study assigned to multiple areas such as distress detection on the pavement, pothole detection, and healthcare. The distress detection on pavement and roads and delivering healthcare and medical services need to be monitored through state-of-the-art technology, i.e., drone technology. Improvement in construction sites and healthcare delivery are of serious concern. Nowadays, computer vision techniques are commonly used in this area utilizing images and videos of construction sites. Due to confined data, researchers are using Unmanned Aerial Vehicles (UAVs) or Drone to get maximum information through 360° monitoring. This review article presents the useful monitoring techniques using AI-enabled drones for scholars around the world. In this comprehensive review, initially, the image acquisition equipment along with the perks and limitations has been presented. Second, the main constraints related to different computer vision techniques are highlighted for detecting distress in the pavement. Then, the possible research solution to some of the distress issues such that detection of pavement texture, cracks or potholes, joint faulting, temperature segregation, and rutting issues are predicted. Finally, the application of AI-enhanced drones in the healthcare field is elucidated which showed their significance. Moreover, in this research, the comparative image analysis of pavement and path hole detection was presented for the collection of detailed information and accurate detection. In the future, the work can also be enhanced to monitor the live pavement distress detection, especially for busy roads and highways. Moreover, an analysis to determine and reduce the costs in the healthcare sectors and organizations is required for future work.


---

## Body

## 1. Introduction

Artificial Intelligence-based devices assist researchers to broaden the levels of identification, detection, and delivery services. Drones provide efficient diagnostics, detections, and supervision in aerial cases through high-performance built-in computers, thermal scanners, high-resolution cameras, and other tools [1]. The use of AI-based drones is vital in the healthcare sector. It assists the health professionals with quick and efficient deliveries of the sample, lab reports, and other health-related essentials. Drones have enforced the public to follow mandatory health rules properly via audio and visual alerts. Its working capabilities and patrolling in many developed countries during Covid-19 are remarkable. Drones reported the rules violations and the diagnosed cases and helped to disinfect the public spots.Among many AI approaches, Deep learning (DL) methods attain outstanding results in monitoring road conditions. It determines and characterizes the pavement distress accurately. The most significant advantage is its capability to learn large data quantities. Among many DL approaches, the widely used is deep convolutional neural networks (DCNNs) for vision-based tasks as it provides higher accuracy [2]. All the DL approaches learn more descriptively multiple-level image features. DCNN has many-layered architecture and extracts various features of pavement distress like transfer learning, multiple source sensors, multi-stage CNNs, etc. All the layers provide results for three main tasks, classification, segmentation, and detection. For classification tasks, DL outperforms the other conventional machine learning (ML) techniques with an accuracy of 90–97%. For segmentation tasks, DL performs classification of pixel-level, which enables an in-depth examination of an image. Moreover, it helps to detect the distress morphology as well. It results in high accuracy results of 70–99%, especially in single-class problems. The generative adversarial networks (GANs), connectivity maps, and other approaches enhance the outputs of segmentation. For detection tasks, especially pavement distress detection, it performs with high accuracy of 70–97% for both 2-Directional (2D) and 3-Directional (3D) data. Moreover, it provides classification with defects localization that is significant for navigation. The technologies of depth measurement are used to measure the physical characteristics of distress on the pavement. DL approaches are the most reliable approaches for pothole and pavement distress detections but require further advancements with more innovative techniques, budget-friendly sensors ad platforms. In the healthcare section, DL extracts the features automatically, which reduces the complex processing. To evaluate the link among various factors of a patient’s no-show the DNN model can be employed. It will also determine the correlation between the factors and the results. Many researchers detecting multiple retinal diseases also use the CNN model. Similarly, recurrent neural networks (RNNs) assisted the healthcare providers with their tasks, especially clinical trial cohort selection. Similarly, GANs show great ability and potential to generate synthetic magnetic resonance imaging (MRI) images. Additionally, GANs help in protecting the privacy of patients.However, DL techniques have certain limitations. It requires training on large data volumes and post-processing algorithms for exact and smooth shape extractions. The results using DL techniques for very high-resolution images are discontinuous and possess ambiguous structural semantics [3]. Sometimes the results show noises in segmentation. Moreover, the physical characteristics of pavement distress are still a gap. These are unable to specify the exact distress location, so not suitable for real-time applications. It requires redundant scanning for the entire picture, so it is a computationally expensive method. The main drawback is that these models are costly, time-consuming, require large volumes of sample data, and high degree of technical sophistication. In some cases, DNNs made errors in medical diagnosis based on images, which can alter the detection and diagnosis process.


### 1.1. Literature Review

Currently, computer vision techniques are the recent trend, and they widely opt for several areas and applications such that the biomedical domain, aerospace engineering, construction sites, metrological department of any area and identifying botanical diseases, traffic monitoring, and many other areas [4–9]. Discuss the several types of instruments and devices that are used to identify the distress such that digital cameras that identify the cracks in the pavement and laser screening methods to identify the rutting issue [10]. Moreover, the researchers have proposed thermal imaging schemes to monitor the temperature segregation throughout the asphalt paving along with the same technique is also proposed to extract the texture of pavement pothole identification joint fault and many others [11].The same image-based techniques can be seen, which are later deployed in the vehicles for the same purposes. Techniques such that ground-penetrating radar (GPR) had been applied for checking the thickness of the pavement layer using a depth camera [12]. The same technique is applied to the airport pavement to identify the cracks [13]. Moreover, one may easily deploy these techniques and algorithms over a UAV to check the distress in pavements and identify the pothole [14]. Researchers have been trying to install high-resolution cameras along with LiDAR to detect the roughness of roads [15]. A study [16] used a UAV-assisted ad hoc on-demand distance vector (AODV) routing design to accumulate and share traffic data of the current time. This helps in controlling the congestion earlier than it occurs. This design enhanced the intersection-based routing protocols of the urban environment. Moreover, this routing protocol outperforms the simulation results as it reduces the delays, increases the ratio of packet delivery, improves the instant and average throughput, and saves more energy. Researchers used UAVs in [17] to capture real images and implemented a deep Laplacian pyramid with a super-resolution network and a fine-tuning strategy. These strategies enhance detection accuracy. Among the performances of many object detection models, the proposed flight strategy proves to be a simpler and more efficient one. Moreover, it shows robust diagnostic capabilities and fault recognition. Table 1 gives detail of innovative methodologies and contributions of the current research.Table 1
Literature review.
Sr. No.ReferencesYearsMethodologiesContributions1.Shukla et al. [4]2016AI and imaging(i) Solves many biomedical problems(ii) Increases the diagnosing, studying, understanding, monitoring, and treating abilities for medical disorders2.Chakraborty and Kalyani [5]2020Deep learning(i) Details the recent growth and limitations of the current systems3.Jiang et al. [6]2013Computer vision algorithms and imaging(i) Shows huge potential in the described challenges4.Beltrán-González et al. [7]2020Computer vision and Machine learning(i) Provides encouraging performances5.Seo et al. [8]2015Computer vision(i) Assists in health monitoring and construction safety6.Ismail [9]2010Computer vision(i) Shows effectiveness of a safety treatment(ii) Helps in reducing the traffic events7.Zhang et al. [10]20183D laser profiling(i) Detect cracks, subsides, potholes, shoving, rutting, and shoving with high accuracy8.Kim et al. [11]2017IR thermal imaging(i) Provides statistical correlation between fracture resistance and temperature differential9.Loizos and Plati [12]2007Ground-penetrating radar (GPR)(i) Effectively and accurately estimates pavement asphalt layer thicknesses10.Moropoulou et al. [13]2002IR thermography and GPR(i) Detects cracks, voids, etc. efficiently in airport pavements11.Saad and Tahar [14]2019Multirotor UAV(i) Identifies rut and pothole accurately(ii) boosts the effectiveness of road conditions monitoring (RCM)12.Blasiis et al. [15]2020LiDAR(i) Provides good correlation(ii) Proves to be useful for maintenance works planning13.Jobaer et al. [16]2020UAV-assisted AODV(i) Reduces the delays, and increases the ratio of packet delivery(ii) Saves more energy14.Rahman et al. [17]2021UAVs and a deep Laplacian pyramid with a super-resolution network and a fine-tuning strategy(i) Proves to be a simpler and more efficient one(ii) Shows robust diagnostic capabilities and fault recognition


### 1.2. Motivation and Contribution

Extreme climatic events and intensive usage of vehicles are aging pavements and deteriorating road networks. Many researchers are making huge efforts to develop smart and intelligent roadway infrastructure and healthcare systems keeping reduction in inspection costs and safety as a priority. Emerging AI-enhanced drones possess the potential to minimize the inspection costs of pavement structures and enhance safety levels. The motivation for this study is to combine a comprehensive and systematic review of AI-assisted drone technology in distress pavement, pothole detection, and healthcare. The contribution of this paper is this comprehensive review study provides the latest updates about computer vision-enabled drones and their utilization for inspecting pavement issues. Thus, the image acquisition equipment plays a vital role and therefore one may find the limitations of such equipment as well in this manuscript. In addition to this, the adoption of computer vision algorithms and pre-processing coding techniques along with their constraints are highlighted. This will provide the researchers to go into this field and explore more optimized solutions to identify the issues related to the pavement field.


## 1.1. Literature Review

Currently, computer vision techniques are the recent trend, and they widely opt for several areas and applications such that the biomedical domain, aerospace engineering, construction sites, metrological department of any area and identifying botanical diseases, traffic monitoring, and many other areas [4–9]. Discuss the several types of instruments and devices that are used to identify the distress such that digital cameras that identify the cracks in the pavement and laser screening methods to identify the rutting issue [10]. Moreover, the researchers have proposed thermal imaging schemes to monitor the temperature segregation throughout the asphalt paving along with the same technique is also proposed to extract the texture of pavement pothole identification joint fault and many others [11].The same image-based techniques can be seen, which are later deployed in the vehicles for the same purposes. Techniques such that ground-penetrating radar (GPR) had been applied for checking the thickness of the pavement layer using a depth camera [12]. The same technique is applied to the airport pavement to identify the cracks [13]. Moreover, one may easily deploy these techniques and algorithms over a UAV to check the distress in pavements and identify the pothole [14]. Researchers have been trying to install high-resolution cameras along with LiDAR to detect the roughness of roads [15]. A study [16] used a UAV-assisted ad hoc on-demand distance vector (AODV) routing design to accumulate and share traffic data of the current time. This helps in controlling the congestion earlier than it occurs. This design enhanced the intersection-based routing protocols of the urban environment. Moreover, this routing protocol outperforms the simulation results as it reduces the delays, increases the ratio of packet delivery, improves the instant and average throughput, and saves more energy. Researchers used UAVs in [17] to capture real images and implemented a deep Laplacian pyramid with a super-resolution network and a fine-tuning strategy. These strategies enhance detection accuracy. Among the performances of many object detection models, the proposed flight strategy proves to be a simpler and more efficient one. Moreover, it shows robust diagnostic capabilities and fault recognition. Table 1 gives detail of innovative methodologies and contributions of the current research.Table 1
Literature review.
Sr. No.ReferencesYearsMethodologiesContributions1.Shukla et al. [4]2016AI and imaging(i) Solves many biomedical problems(ii) Increases the diagnosing, studying, understanding, monitoring, and treating abilities for medical disorders2.Chakraborty and Kalyani [5]2020Deep learning(i) Details the recent growth and limitations of the current systems3.Jiang et al. [6]2013Computer vision algorithms and imaging(i) Shows huge potential in the described challenges4.Beltrán-González et al. [7]2020Computer vision and Machine learning(i) Provides encouraging performances5.Seo et al. [8]2015Computer vision(i) Assists in health monitoring and construction safety6.Ismail [9]2010Computer vision(i) Shows effectiveness of a safety treatment(ii) Helps in reducing the traffic events7.Zhang et al. [10]20183D laser profiling(i) Detect cracks, subsides, potholes, shoving, rutting, and shoving with high accuracy8.Kim et al. [11]2017IR thermal imaging(i) Provides statistical correlation between fracture resistance and temperature differential9.Loizos and Plati [12]2007Ground-penetrating radar (GPR)(i) Effectively and accurately estimates pavement asphalt layer thicknesses10.Moropoulou et al. [13]2002IR thermography and GPR(i) Detects cracks, voids, etc. efficiently in airport pavements11.Saad and Tahar [14]2019Multirotor UAV(i) Identifies rut and pothole accurately(ii) boosts the effectiveness of road conditions monitoring (RCM)12.Blasiis et al. [15]2020LiDAR(i) Provides good correlation(ii) Proves to be useful for maintenance works planning13.Jobaer et al. [16]2020UAV-assisted AODV(i) Reduces the delays, and increases the ratio of packet delivery(ii) Saves more energy14.Rahman et al. [17]2021UAVs and a deep Laplacian pyramid with a super-resolution network and a fine-tuning strategy(i) Proves to be a simpler and more efficient one(ii) Shows robust diagnostic capabilities and fault recognition


## 1.2. Motivation and Contribution

Extreme climatic events and intensive usage of vehicles are aging pavements and deteriorating road networks. Many researchers are making huge efforts to develop smart and intelligent roadway infrastructure and healthcare systems keeping reduction in inspection costs and safety as a priority. Emerging AI-enhanced drones possess the potential to minimize the inspection costs of pavement structures and enhance safety levels. The motivation for this study is to combine a comprehensive and systematic review of AI-assisted drone technology in distress pavement, pothole detection, and healthcare. The contribution of this paper is this comprehensive review study provides the latest updates about computer vision-enabled drones and their utilization for inspecting pavement issues. Thus, the image acquisition equipment plays a vital role and therefore one may find the limitations of such equipment as well in this manuscript. In addition to this, the adoption of computer vision algorithms and pre-processing coding techniques along with their constraints are highlighted. This will provide the researchers to go into this field and explore more optimized solutions to identify the issues related to the pavement field.


## 2. Image Procuring Equipment

### 2.1. Types of Camera and Working Principles

#### 2.1.1. Deploying Digital Camera

The study focuses on the pavement detection area; thus, several analysts started their work by studying the properties of road pavement using their image set. This image set is available online as well as can be acquired using digital cameras. One may see pavement surface and texture analysis in [18–20], and asphalt-based crack deformation detection crack detection [21, 22].Moreover, the working principle for such digital cameras is quite simple to understand. At the time when someone presses the shutter, the camera lens starts focusing the light on the image sensor, which is a charge-coupled device (CCD) or super charge-coupled device. This converts a light signal to an electrical signal. This is the way by which one may get the electronic signal interrelated with the photographed environment. This achieved information will go under compression and pre-processed using a microcontroller. It should be noted that through this approach one may set the format of the image as well, i.e., JPEG or PNG etc and can also store it in the memory [23–25].


#### 2.1.2. Utilization of Depth Camera

Furthermore, researchers have started to utilize depth cameras to modernize pavement engineering. In comparison with ordinary two-dimensional cameras, depth cameras have more benefits when it is to acquiring depth information [26]. These depth cameras have been categorized into three types as mentioned below [27]:(i)
Time of Flight (ToF)(ii)
Structure light (SL)(iii)
Binocular Stereo Camera (BSC)Defining ToF, one may see its two types which are again working principle-based such that direct and indirect methods. The light pulses are emitted in the direct method using laser emitters as shown in Figure1. After striking with the object, the light bounces back, then the sensor receives it and the object distance is computed using the ToF term for the light and. Whereas if we notice the indirect measurement method, the light source is modulated. A continuous beam of light is used to emit it and the object distance can be computed with the light phase.Figure 1
Working principle of ToF depth cameras [28].Discussing the structured light camera, their working principle is quite easy to understand, here one may see the light source, which is modulated, and multiple light spots are shown at the time of radiation over the objects. Later, sensors compute the distance to capture the reflected light. The most common example of structured light cameras is the “Kinect” sensors as shown in Figure2.Figure 2
Structured light (SL) camera (Kinect sensors) [29].There is a situation when the eyes’ field angle changes for the object at multiple distances from a similar frame of point. This happens because of the distance between the pupils of our eyes, which is 65 mm for an adult while observing objects. As per the relationship of geometric optics and the projection relation, the point at multiple distances from the observer is evident to be imaged at different retina positions thus such positional difference is known as binocular parallax. This results in the reflection of the depth of objects and can provide a stereoscopic vision [30]. One may see the utilization of two coplanar CMOS cameras for simulating the similar working of eyes in the binocular camera.In above Figure3, one may see the structure and physical representation of the binocular camera. In addition to this, one may also see the fundamental working principle in it. There are four prisms packed in a pair of binoculars. In each tube, one prism is mounted horizontally while the other prism is vertical. The light rays take the path through the lenses and then through these Porro prisms [32]. For the above brief study, it is understood that one may opt depth camera very carefully. Thus, this study proposes the main guidelines with some significant aspects as shown in Table 2.Figure 3
Components and Working Principle of Binocular cameras of Camera [31].Table 2
Significant Aspects for opting for Depth Camera [33].
Depth camera typeSignificant aspectsMeasuring rangePrecisionComplexityOperational in dark modeApplicationBinocularMedium Range0.01–1 mmHighNoObject RecognitionStructured Light0.1 m–10 m0.01–1 mmMediumYesPose estimationToF Type0.15 m–15m1 cmLowNoVirtual Reality


### 2.2. Thermal Image Technology (TIT)

It is defined as a contactless and benign procedure for computing the object’s temperature by calculating the emitted infrared radiations, such a method is known as thermal image technology (TIT) [34]. It is one of the suitable technologies to examine instant temperature change and one of the fast procedures. It is admitted that the infrared (IR) comes in the invisible light bands within the electromagnetic spectrum and has 0.76–100 μm of wavelength. This infrared wave has been categorized as short (1.4–3 μm), the medium is between (3–8 μm) whereas the long wave is greater than 8 μm and the extremely long wave is between 15 and 100 μm. The TIT mechanism can detect the IR radiation in between short and longwave regions [35]. This is because the TIT mechanism uses infrared detectors for capturing the pattern of the radiation energy related to any object. Once it is captured, the incident radiation is transformed into an electrical signal by the detector. In this way, it results in thermal imaging. This acquired image from the thermal imaging technique shows the object’s surface heat distribution along with multiple colors which display the variable temperatures of the considered object. Figure 4 provides the understanding and difference between the thermal and digital images [36]. It should be noted that there are mainly two kinds of thermal imaging techniques such as passive and active. If one is obtaining the information, regarding the object’s temperature with no applications of external energy to the targeted object is known as the passive thermal imaging technique or simply passive thermography. Whereas, if someone is obtaining the imaging by providing external thermal energy to the targeted object such technique is known as active thermography. This has been illustrated in Figure 5.Figure 4
Digital imaging versus thermal imaging [36].Figure 5
Passive and active thermal imaging system [37].Discussing the benefits of the thermal imaging technique, one may enjoy its high portability, and contactless testing, and the radiations are no longer harmful. In addition to this, it can obtain a real-time image without any dependency on light intensity. The only constraint with this technique is the cost as this method is a comparatively expensive method and there is an improvement required for enhancing the resolution of the acquired image.


### 2.3. Laser Technique

Satellite laser scanning is something very popular; researchers are also doing remote sensing and mapping. This is why it is commonly applied for pavement distress detection [38].


#### 2.3.1. Laser-Based Scanning Principle

The principle is quite simple and based on the measurement of laser distance. There is particular lightning that is projected over the targeted object and later is discovered by using the CMOS camera as shown in Figure6. In most cases, the distance at which the camera is placed for projection is known along with the horizontal position by studying the laser line deformation on the object [39]. Moreover, this laser mechanism is strongly coupled with an encoder. The main objective of this encoder is to derive the laser detection vehicle’s direction and in this way the surface image of the targeted commodity is constructed as illustrated in.Figure 6
Optical triangular principle for laser scanning [40].


#### 2.3.2. Fixing Sampling Interval for Laser Scanning

After going through the literature, it has been observed that the changing of sampling frequency may or may not potentially affect the capability to detect the cracks from the pavement using 3D Laser scanning. Hence, one can never neglect this parameter. In addition to this, one may keep this thing in mind that the type of detection device can also affect this sampling frequency. Take an example of a CT meter in which the laser is based on a displacement sensor organized over the arm. This arm rotates over the circumference with a radius of 140 mm while measuring a set sampling interval with the texture. Similar works can be witnessed such that 3D pavement detection [41]. In these research contributions, one may see a scanning speed of 120 Km/h whereas the sampling interval lies in the range from 1 mm to 5 mm. It is observed that there are some vehicular-mounted laser devices with low-frequency laser radiation to acquire the macro level of texture from the targeted object [42]. By studying some manuscripts, it is concluded that this sampling must be lower than 500 mm to ensure the proper detection of stress and roughness of pavement [43].


#### 2.3.3. Perks of Laser Technique

There are several benefits of the laser technique; one of the advantages is its insensitive property to light effect. Second, the high resolution of the acquired images helps to render the images for designing the required machine perception [44]. Moreover, with the deployment of sensor technology, one can be able to detect the 0.5 mm depth. The only constraint with this technique is the high cost. Therefore, its only purpose is data collection in restricted predefined jurisdiction.


### 2.4. Mechanism of Pavement Inspection

In the early 1970s, researchers proposed the GERPHO mechanism [45]. This is the first time that researchers opted camera to capture the surface of roads. This data was first stored and helped the researcher to correspond to the particular position of the targeted object but never commented on the roughness and distress of the pavement or road. This technique later was named as Automatic pavement distress survey system (APDSS) [46]. APDSS has one common disadvantage it does not diagnose the type of crack and may function only in the nighttime. One may further see ARIA, which stands for automated road image analyzer dully developed by American researchers [47]. This technique requires 15 to 20 minutes to analyze 1 lane only, which is almost equal to 1 mile for crack detection. Next in 2015, a proper pavement condition evaluation service was developed which later on merged with unmanned aerial vehicles, which are shown in Figure 7. After the advent of UAVs, this turned out to be one of the promising methods to identify the cracks by acquiring high-quality images. In addition to this, various equipment is used for the same purpose [48].Figure 7
Crack detection with drone software [48].


### 2.5. About GPR

Researchers have opted for the antenna of GPR and the utilization of electromagnetic waves to know about the structure and acquire information about the material [49]. The schematic related to this technique is illustrated in Figure 8 [50]. Moreover, there are mainly three GPR signal types A, B, and C-scan, as illustrated in Figure 9. These waves detect abrupt variation and represent the cross-sectional images of the objects [51].Figure 8
Principle of GPR technique [50].Figure 9
Visualization modes of a GPR signal: (a) A-scan, (b) B-scan, and (c) C-scan [51].
(a)(b)(c)It is said that GPR being a fast and non-destructive method identifies the stress of pavement and above all the data acquired using these techniques is very easy and automatic. It has a limitation over the large data, and one may face a processing delay in identifying the detects.


## 2.1. Types of Camera and Working Principles

### 2.1.1. Deploying Digital Camera

The study focuses on the pavement detection area; thus, several analysts started their work by studying the properties of road pavement using their image set. This image set is available online as well as can be acquired using digital cameras. One may see pavement surface and texture analysis in [18–20], and asphalt-based crack deformation detection crack detection [21, 22].Moreover, the working principle for such digital cameras is quite simple to understand. At the time when someone presses the shutter, the camera lens starts focusing the light on the image sensor, which is a charge-coupled device (CCD) or super charge-coupled device. This converts a light signal to an electrical signal. This is the way by which one may get the electronic signal interrelated with the photographed environment. This achieved information will go under compression and pre-processed using a microcontroller. It should be noted that through this approach one may set the format of the image as well, i.e., JPEG or PNG etc and can also store it in the memory [23–25].


### 2.1.2. Utilization of Depth Camera

Furthermore, researchers have started to utilize depth cameras to modernize pavement engineering. In comparison with ordinary two-dimensional cameras, depth cameras have more benefits when it is to acquiring depth information [26]. These depth cameras have been categorized into three types as mentioned below [27]:(i)
Time of Flight (ToF)(ii)
Structure light (SL)(iii)
Binocular Stereo Camera (BSC)Defining ToF, one may see its two types which are again working principle-based such that direct and indirect methods. The light pulses are emitted in the direct method using laser emitters as shown in Figure1. After striking with the object, the light bounces back, then the sensor receives it and the object distance is computed using the ToF term for the light and. Whereas if we notice the indirect measurement method, the light source is modulated. A continuous beam of light is used to emit it and the object distance can be computed with the light phase.Figure 1
Working principle of ToF depth cameras [28].Discussing the structured light camera, their working principle is quite easy to understand, here one may see the light source, which is modulated, and multiple light spots are shown at the time of radiation over the objects. Later, sensors compute the distance to capture the reflected light. The most common example of structured light cameras is the “Kinect” sensors as shown in Figure2.Figure 2
Structured light (SL) camera (Kinect sensors) [29].There is a situation when the eyes’ field angle changes for the object at multiple distances from a similar frame of point. This happens because of the distance between the pupils of our eyes, which is 65 mm for an adult while observing objects. As per the relationship of geometric optics and the projection relation, the point at multiple distances from the observer is evident to be imaged at different retina positions thus such positional difference is known as binocular parallax. This results in the reflection of the depth of objects and can provide a stereoscopic vision [30]. One may see the utilization of two coplanar CMOS cameras for simulating the similar working of eyes in the binocular camera.In above Figure3, one may see the structure and physical representation of the binocular camera. In addition to this, one may also see the fundamental working principle in it. There are four prisms packed in a pair of binoculars. In each tube, one prism is mounted horizontally while the other prism is vertical. The light rays take the path through the lenses and then through these Porro prisms [32]. For the above brief study, it is understood that one may opt depth camera very carefully. Thus, this study proposes the main guidelines with some significant aspects as shown in Table 2.Figure 3
Components and Working Principle of Binocular cameras of Camera [31].Table 2
Significant Aspects for opting for Depth Camera [33].
Depth camera typeSignificant aspectsMeasuring rangePrecisionComplexityOperational in dark modeApplicationBinocularMedium Range0.01–1 mmHighNoObject RecognitionStructured Light0.1 m–10 m0.01–1 mmMediumYesPose estimationToF Type0.15 m–15m1 cmLowNoVirtual Reality


## 2.1.1. Deploying Digital Camera

The study focuses on the pavement detection area; thus, several analysts started their work by studying the properties of road pavement using their image set. This image set is available online as well as can be acquired using digital cameras. One may see pavement surface and texture analysis in [18–20], and asphalt-based crack deformation detection crack detection [21, 22].Moreover, the working principle for such digital cameras is quite simple to understand. At the time when someone presses the shutter, the camera lens starts focusing the light on the image sensor, which is a charge-coupled device (CCD) or super charge-coupled device. This converts a light signal to an electrical signal. This is the way by which one may get the electronic signal interrelated with the photographed environment. This achieved information will go under compression and pre-processed using a microcontroller. It should be noted that through this approach one may set the format of the image as well, i.e., JPEG or PNG etc and can also store it in the memory [23–25].


## 2.1.2. Utilization of Depth Camera

Furthermore, researchers have started to utilize depth cameras to modernize pavement engineering. In comparison with ordinary two-dimensional cameras, depth cameras have more benefits when it is to acquiring depth information [26]. These depth cameras have been categorized into three types as mentioned below [27]:(i)
Time of Flight (ToF)(ii)
Structure light (SL)(iii)
Binocular Stereo Camera (BSC)Defining ToF, one may see its two types which are again working principle-based such that direct and indirect methods. The light pulses are emitted in the direct method using laser emitters as shown in Figure1. After striking with the object, the light bounces back, then the sensor receives it and the object distance is computed using the ToF term for the light and. Whereas if we notice the indirect measurement method, the light source is modulated. A continuous beam of light is used to emit it and the object distance can be computed with the light phase.Figure 1
Working principle of ToF depth cameras [28].Discussing the structured light camera, their working principle is quite easy to understand, here one may see the light source, which is modulated, and multiple light spots are shown at the time of radiation over the objects. Later, sensors compute the distance to capture the reflected light. The most common example of structured light cameras is the “Kinect” sensors as shown in Figure2.Figure 2
Structured light (SL) camera (Kinect sensors) [29].There is a situation when the eyes’ field angle changes for the object at multiple distances from a similar frame of point. This happens because of the distance between the pupils of our eyes, which is 65 mm for an adult while observing objects. As per the relationship of geometric optics and the projection relation, the point at multiple distances from the observer is evident to be imaged at different retina positions thus such positional difference is known as binocular parallax. This results in the reflection of the depth of objects and can provide a stereoscopic vision [30]. One may see the utilization of two coplanar CMOS cameras for simulating the similar working of eyes in the binocular camera.In above Figure3, one may see the structure and physical representation of the binocular camera. In addition to this, one may also see the fundamental working principle in it. There are four prisms packed in a pair of binoculars. In each tube, one prism is mounted horizontally while the other prism is vertical. The light rays take the path through the lenses and then through these Porro prisms [32]. For the above brief study, it is understood that one may opt depth camera very carefully. Thus, this study proposes the main guidelines with some significant aspects as shown in Table 2.Figure 3
Components and Working Principle of Binocular cameras of Camera [31].Table 2
Significant Aspects for opting for Depth Camera [33].
Depth camera typeSignificant aspectsMeasuring rangePrecisionComplexityOperational in dark modeApplicationBinocularMedium Range0.01–1 mmHighNoObject RecognitionStructured Light0.1 m–10 m0.01–1 mmMediumYesPose estimationToF Type0.15 m–15m1 cmLowNoVirtual Reality


## 2.2. Thermal Image Technology (TIT)

It is defined as a contactless and benign procedure for computing the object’s temperature by calculating the emitted infrared radiations, such a method is known as thermal image technology (TIT) [34]. It is one of the suitable technologies to examine instant temperature change and one of the fast procedures. It is admitted that the infrared (IR) comes in the invisible light bands within the electromagnetic spectrum and has 0.76–100 μm of wavelength. This infrared wave has been categorized as short (1.4–3 μm), the medium is between (3–8 μm) whereas the long wave is greater than 8 μm and the extremely long wave is between 15 and 100 μm. The TIT mechanism can detect the IR radiation in between short and longwave regions [35]. This is because the TIT mechanism uses infrared detectors for capturing the pattern of the radiation energy related to any object. Once it is captured, the incident radiation is transformed into an electrical signal by the detector. In this way, it results in thermal imaging. This acquired image from the thermal imaging technique shows the object’s surface heat distribution along with multiple colors which display the variable temperatures of the considered object. Figure 4 provides the understanding and difference between the thermal and digital images [36]. It should be noted that there are mainly two kinds of thermal imaging techniques such as passive and active. If one is obtaining the information, regarding the object’s temperature with no applications of external energy to the targeted object is known as the passive thermal imaging technique or simply passive thermography. Whereas, if someone is obtaining the imaging by providing external thermal energy to the targeted object such technique is known as active thermography. This has been illustrated in Figure 5.Figure 4
Digital imaging versus thermal imaging [36].Figure 5
Passive and active thermal imaging system [37].Discussing the benefits of the thermal imaging technique, one may enjoy its high portability, and contactless testing, and the radiations are no longer harmful. In addition to this, it can obtain a real-time image without any dependency on light intensity. The only constraint with this technique is the cost as this method is a comparatively expensive method and there is an improvement required for enhancing the resolution of the acquired image.


## 2.3. Laser Technique

Satellite laser scanning is something very popular; researchers are also doing remote sensing and mapping. This is why it is commonly applied for pavement distress detection [38].


### 2.3.1. Laser-Based Scanning Principle

The principle is quite simple and based on the measurement of laser distance. There is particular lightning that is projected over the targeted object and later is discovered by using the CMOS camera as shown in Figure6. In most cases, the distance at which the camera is placed for projection is known along with the horizontal position by studying the laser line deformation on the object [39]. Moreover, this laser mechanism is strongly coupled with an encoder. The main objective of this encoder is to derive the laser detection vehicle’s direction and in this way the surface image of the targeted commodity is constructed as illustrated in.Figure 6
Optical triangular principle for laser scanning [40].


### 2.3.2. Fixing Sampling Interval for Laser Scanning

After going through the literature, it has been observed that the changing of sampling frequency may or may not potentially affect the capability to detect the cracks from the pavement using 3D Laser scanning. Hence, one can never neglect this parameter. In addition to this, one may keep this thing in mind that the type of detection device can also affect this sampling frequency. Take an example of a CT meter in which the laser is based on a displacement sensor organized over the arm. This arm rotates over the circumference with a radius of 140 mm while measuring a set sampling interval with the texture. Similar works can be witnessed such that 3D pavement detection [41]. In these research contributions, one may see a scanning speed of 120 Km/h whereas the sampling interval lies in the range from 1 mm to 5 mm. It is observed that there are some vehicular-mounted laser devices with low-frequency laser radiation to acquire the macro level of texture from the targeted object [42]. By studying some manuscripts, it is concluded that this sampling must be lower than 500 mm to ensure the proper detection of stress and roughness of pavement [43].


### 2.3.3. Perks of Laser Technique

There are several benefits of the laser technique; one of the advantages is its insensitive property to light effect. Second, the high resolution of the acquired images helps to render the images for designing the required machine perception [44]. Moreover, with the deployment of sensor technology, one can be able to detect the 0.5 mm depth. The only constraint with this technique is the high cost. Therefore, its only purpose is data collection in restricted predefined jurisdiction.


## 2.3.1. Laser-Based Scanning Principle

The principle is quite simple and based on the measurement of laser distance. There is particular lightning that is projected over the targeted object and later is discovered by using the CMOS camera as shown in Figure6. In most cases, the distance at which the camera is placed for projection is known along with the horizontal position by studying the laser line deformation on the object [39]. Moreover, this laser mechanism is strongly coupled with an encoder. The main objective of this encoder is to derive the laser detection vehicle’s direction and in this way the surface image of the targeted commodity is constructed as illustrated in.Figure 6
Optical triangular principle for laser scanning [40].


## 2.3.2. Fixing Sampling Interval for Laser Scanning

After going through the literature, it has been observed that the changing of sampling frequency may or may not potentially affect the capability to detect the cracks from the pavement using 3D Laser scanning. Hence, one can never neglect this parameter. In addition to this, one may keep this thing in mind that the type of detection device can also affect this sampling frequency. Take an example of a CT meter in which the laser is based on a displacement sensor organized over the arm. This arm rotates over the circumference with a radius of 140 mm while measuring a set sampling interval with the texture. Similar works can be witnessed such that 3D pavement detection [41]. In these research contributions, one may see a scanning speed of 120 Km/h whereas the sampling interval lies in the range from 1 mm to 5 mm. It is observed that there are some vehicular-mounted laser devices with low-frequency laser radiation to acquire the macro level of texture from the targeted object [42]. By studying some manuscripts, it is concluded that this sampling must be lower than 500 mm to ensure the proper detection of stress and roughness of pavement [43].


## 2.3.3. Perks of Laser Technique

There are several benefits of the laser technique; one of the advantages is its insensitive property to light effect. Second, the high resolution of the acquired images helps to render the images for designing the required machine perception [44]. Moreover, with the deployment of sensor technology, one can be able to detect the 0.5 mm depth. The only constraint with this technique is the high cost. Therefore, its only purpose is data collection in restricted predefined jurisdiction.


## 2.4. Mechanism of Pavement Inspection

In the early 1970s, researchers proposed the GERPHO mechanism [45]. This is the first time that researchers opted camera to capture the surface of roads. This data was first stored and helped the researcher to correspond to the particular position of the targeted object but never commented on the roughness and distress of the pavement or road. This technique later was named as Automatic pavement distress survey system (APDSS) [46]. APDSS has one common disadvantage it does not diagnose the type of crack and may function only in the nighttime. One may further see ARIA, which stands for automated road image analyzer dully developed by American researchers [47]. This technique requires 15 to 20 minutes to analyze 1 lane only, which is almost equal to 1 mile for crack detection. Next in 2015, a proper pavement condition evaluation service was developed which later on merged with unmanned aerial vehicles, which are shown in Figure 7. After the advent of UAVs, this turned out to be one of the promising methods to identify the cracks by acquiring high-quality images. In addition to this, various equipment is used for the same purpose [48].Figure 7
Crack detection with drone software [48].


## 2.5. About GPR

Researchers have opted for the antenna of GPR and the utilization of electromagnetic waves to know about the structure and acquire information about the material [49]. The schematic related to this technique is illustrated in Figure 8 [50]. Moreover, there are mainly three GPR signal types A, B, and C-scan, as illustrated in Figure 9. These waves detect abrupt variation and represent the cross-sectional images of the objects [51].Figure 8
Principle of GPR technique [50].Figure 9
Visualization modes of a GPR signal: (a) A-scan, (b) B-scan, and (c) C-scan [51].
(a)(b)(c)It is said that GPR being a fast and non-destructive method identifies the stress of pavement and above all the data acquired using these techniques is very easy and automatic. It has a limitation over the large data, and one may face a processing delay in identifying the detects.


## 3. Utilization of Computer Vision Strategies

Computer vision technology has been opted a lot nowadays by several researchers at different levels to check the issues of pavement, it seems now digital image processing and computer vision are now a part of pavement distress engineering. This is because it has been used extensively to study the asphalt structures, characterization of aggregates, and condition monitoring of macroscopic scale [52]. Thus, the following study discusses the importance of this technique and discusses the analysis of previously proposed techniques along with equipment in the domain of pavement engineering.


### 3.1. Detection of Distress over Pavement’s Surface

The cracks in pavement usually appear after some time due to traffic and it is one of the common failures of civil engineering utilized to design that pavement. In most cases, this detection is done manually but now to ensure the high detection efficiency and accuracy along with data prediction that when one pavement needs maintenance computer vision technique is heavily proposed and suggested. This is still evolving with the regular advent of several sensors and efficient algorithms [53].This is started by pre-processing the cracks via the segmentation technique. This helps computing gadgets to check the image using thresholding, segmentation, and correlation techniques. Moreover, Table3 describes all such studies that were presented on pavement distress. One may study [54] and can conclude that several more methods are based on thresholding techniques such that OTSU, hybrid thresholding, and histogram technique. Majorly, researchers have used the thresholding technique because it is easy to operate once the threshold value is computed [55]. One may also see the segmentation based on a deep learning algorithm and may compare the method with OTSU. Since computer vision techniques are based on image analysis. Therefore, any image we acquire must have some external noise and non-uniform illumination, which is one of the limitations. This limitation has been addressed using different filters and some of the researchers addressed this issue by introducing the mean and standard deviation values [56]. This is illustrated in Figure 10.Table 3
Thresholding techniques for crack segmentation.
Sr. No.ReferencesYearsTechniquesResults1.Yang et al. [64]2020Deep learningGives superior, accurate, and generalizability results2.Huyan et al. [87]2019Deep learningGives faster and better performance3.Lettsome et al. [88]2012Computer vision using edge detectionPerforms much better Gives significant quantitative improvement4.Somcharean and Phiphobmongkol [61]2008Computer vision using edge detectionEliminates false detection Gives accurate results and strong texture5.Tsai et al. [89]2014Multi-scaled segmentation techniqueGives promising results6.Lu [90]2014Multi-scaled segmentation techniqueGives improved versatility and detection accuracy7.Moussa and Hussain [91]2011Conventional ThresholdingIdentifies cracks automatically and effectively8.Salari and Bao [92]2011Conventional ThresholdingGives more effective and reliable resultsFigure 10
Deep learning-based crack segmentation [56].Due to the roughness of pavements in most cases, the snake model has been proposed to refine the cracks from the image and it has been concluded as better than other video quality. In addition to this, one may witness the snake model to tackle the crack segmentation. It is believed that the snake model is far better than the segmentation threshold.A hybrid algorithm for the thresholding technique is used for crack detection in Figure11. In the list of these thresholding techniques, there are several algorithms such that the canny model, Sobel model, Laplacian of Gaussian methods, Roberts method [57], and Prewitt method are included. These algorithms are proposed for identifying the cracks within the pavement. These methods have been optimized as initially, they had bad processing effects and therefore several segmentation methods such that morphological theory has been proposed. This algorithm not only tackles the noises but also detects the cracks and edges. Moreover, this should be noted that the processing should be faster, and hence in [58], it is noted to reduce the processing one may pre-process the images, i.e., eliminating the image noise from the crack. One may see an edge detection algorithm that is dependent on the grid analysis (GCA) to increase the real-time segmentation. This algorithm gives high accuracy and good repeatability. In most cases, it is used with uniformity for an image. Later, edge detection can also be utilized for the same purpose but need an artificial light source with uniformity. This GCA technique-based edge detection is much improved to reduce the impact of shadow. Figure 12 shows crack detection using discontinuity-based algorithms like Canny edge detection, Valley for image edge detection, and Valley tracing [59–61].Figure 11
(a) Original Image and (b) and (c) Crack detection Using Thresholding [55].
(a)(b)(c)Figure 12
Different Crack Detection Results (a) Canny edge detection (low thresholding). (b) Canny edge detection (high thresholding). (c) Valley edge detection. (d) Valley tracing [59].
(a)(b)(c)(d)The study of this literature provides several algorithms that recommend multi-level segmentation procedures, i.e. wavelet decomposition method and grey level morphology to see cracks [62]. It is observed that the wavelet transform may lead to loss of translation variance and may add some artifacts. This is the reason researchers prefer wavelet for only edge detection by amalgamating the holes algorithm [63]. Diving the image into sub-regions and using the same algorithm may improve the results. Another reason to opt for this procedure is that the conventional methods are lacking in representing the features. As machine and deep learning have a boom nowadays, therefore, several research contributions have proposed neural network-based algorithms to detect the cracks such that feature Pyramid and hierarchical boosting network (FPHBN) [64]. This algorithm is used mostly to create a differentiate a crack with its background. Moreover, it is compared also with the convolutional neural network as well along with edge detection using HED, RCF, and FCN. This has been illustrated in Figure 13. Moreover, to crack length measurement, Ali et al. used the deep convolutional neural networks (DCNN) as can be seen in Figure 14 [65].Figure 13
Crack detection Results Using FPHBN, HED, RCF, and FCN [64].Figure 14
Demonstration of DCNN algorithm [65].Researchers have designed several algorithms such that the convolution network for crack segmentation and the Gaussian conditional random field algorithm that can be used to refine the results [66]. Furthermore, some techniques are based on autoencoders for classifying the cracks as per spatial enhancement by removing the noise. One may see pixel defect analysis applying a U-net Deep encoding method, which is proposed for evaluating the pavement quality [67]. In this method, segmentation-based performance is demonstrated over the classical U-Net method without the significant computational overhead. In recent times, there have been several strategies and techniques used to divide the crack image into the segment as mentioned below:(i)
Minimal Path Section Method (MPSM)(ii)
Dynamic Optimization-based Crack Segmentation (DOCS)(iii)
Regional Growth Segmentation (RGS)Furthermore, researchers have proposed the Ground Penetrating Radar (GPR) to detect the pavement. This GPR is nothing but a real-time NDT technique that utilizes high-frequency radio waves. Some of the researchers have proposed GPR in the networks for pavement distress detection along with crack, water damage pit, and others as illustrated in Figure15 [68].Figure 15
GPR based Distress detection [68]. (a) Reflection crack (b) Water-damage pit (c) Uneven settlement.
(a)(b)(c)In recent, image processing methods are hybridized with deep learning algorithms to increase the efficiency of the results. In such techniques, one may find CNN, RNN, and autoencoders [69]. These algorithms have provided accuracy of up to 85%, less than 2 mm location errors, and are robust in terms of stability but later these algorithms were amended to a faster region convolution neural network for checking pavement distress [65]. The exact number of precision is 89.13% and intersection over union (IoU) is 86.24% which are available [70].


### 3.2. Classification Techniques

More artificial intelligence (AI) techniques are proposed to classify the crack types. In the list of such techniques, one may see the support vector machine method (SVM), artificial neural network (ANN), and various other algorithms. Some techniques like wavelet neural network (WNN) and backpropagation neural network (BPNN) are also proposed to not only classify the crack images but also used to measure the distress [71, 72]. With the implementation of BPNN researchers have classified the distress in the pavement into four types as mentioned below:(i)
Defect-free(ii)
Crack(iii)
Joint and(iv)
BridgedIn addition to this classification, researchers improved the accuracy level up to 90–95 percent [73]. SVM is a popular technique and is mostly opted one. The main usage of these techniques can be studied where SVM is used for structural risk minimization. The classifiers were introduced based on SVM and these classifiers can identify five types of distress in the pavement. Most importantly, SVM techniques have been compared with the WNN algorithm and the results found the SVM to be more accurate than the other algorithm [74]. Researchers, therefore, started improvising SVM and introduced a proximal support vector machine (PSVM) which proved to be easier to apply and more accurate than conventional SVM [75]. Several other classification approaches include wavelet transform along with Radon transform. This algorithm not only classifies the cracks but also identifies the longitudinal and diagonal cracks with transverse types. This is one of the expert systems and operates rapidly.


### 3.3. Pothole Detection

Another major thing along with pavement distress is pothole detection. This affects a lot in the maintenance and management plan for any pavement [76]. In today’s world, the manual detection method is not accurate and in addition to this, it is time-consuming. Thus, researchers have been proposing computer vision techniques to identify this sort of abnormality. Section Table 4 refers to the number of research contributions proposed for pothole detections using image processing. For digital image processing, one may need a digital camera but some researchers proposed laser scanning in the digital camera to develop a system that helps us in detecting potholes. Since the equipment cost of stereo vision acquisition is less, several studies have utilized it for the same purpose [77–79]. Few researchers have combined stereo camera and GPR techniques for pothole image extraction. Convolution Neural Network is one of the most common techniques opted for because it requires no manual assistance, and it is more stable because it does not affect the environment. CNN is a much more robust algorithm because it has noise filtering ability and can detect the road image noise-free.Table 4
Digital image technology for potholes detection.
Sr. No.ReferencesYearDetection methodImage feature extraction methodResults1.She et al. [40]2021Laser, digital camera, and CMOS sensor3D line laser technologyAchieves higher precision and gives sufficiently accurate results2.Silva et al. [93]2020Digital camera(MAS) PANGEA with (YOLO) v4Gives accurate and promising results3.Gao et al. [94]2020Digital cameraLIBSVMGives superior segmentation effects and processing efficiency4.Wang et al. [78]2017Digital cameraMarkov random field model, geometric criteria, and morphological processingEffectively distinguishes and accurately segments potholes5.Li et al. [79]2016GPS, stereovision systems, and GPRRegion-based segmentation and morphological operationsAchieves accuracy in results6.Koch and Brilakis [81]2011Digital cameraElliptic regression and morphological thinningShows reasonable accuracy7.Lin and Liu [80]2010Digital cameraSVMAchieves a high recognition rateThus, training CNN on images with high-resolution takes more process time and memory as well which is one of the constraints, and to overcome this limitation many researchers have proposed a sliding window method. Furthermore, CNN is improvised and had been produced in two versions part-based classification neural network (PCNN) and localization neural network (LCNN). These two versions are proposed to achieve better performance and reduce the load of calculation. There are geometric criteria along with morphological processing that develop the wavelet energy field of the pavement image for distress detection in pavement [80, 81].


## 3.1. Detection of Distress over Pavement’s Surface

The cracks in pavement usually appear after some time due to traffic and it is one of the common failures of civil engineering utilized to design that pavement. In most cases, this detection is done manually but now to ensure the high detection efficiency and accuracy along with data prediction that when one pavement needs maintenance computer vision technique is heavily proposed and suggested. This is still evolving with the regular advent of several sensors and efficient algorithms [53].This is started by pre-processing the cracks via the segmentation technique. This helps computing gadgets to check the image using thresholding, segmentation, and correlation techniques. Moreover, Table3 describes all such studies that were presented on pavement distress. One may study [54] and can conclude that several more methods are based on thresholding techniques such that OTSU, hybrid thresholding, and histogram technique. Majorly, researchers have used the thresholding technique because it is easy to operate once the threshold value is computed [55]. One may also see the segmentation based on a deep learning algorithm and may compare the method with OTSU. Since computer vision techniques are based on image analysis. Therefore, any image we acquire must have some external noise and non-uniform illumination, which is one of the limitations. This limitation has been addressed using different filters and some of the researchers addressed this issue by introducing the mean and standard deviation values [56]. This is illustrated in Figure 10.Table 3
Thresholding techniques for crack segmentation.
Sr. No.ReferencesYearsTechniquesResults1.Yang et al. [64]2020Deep learningGives superior, accurate, and generalizability results2.Huyan et al. [87]2019Deep learningGives faster and better performance3.Lettsome et al. [88]2012Computer vision using edge detectionPerforms much better Gives significant quantitative improvement4.Somcharean and Phiphobmongkol [61]2008Computer vision using edge detectionEliminates false detection Gives accurate results and strong texture5.Tsai et al. [89]2014Multi-scaled segmentation techniqueGives promising results6.Lu [90]2014Multi-scaled segmentation techniqueGives improved versatility and detection accuracy7.Moussa and Hussain [91]2011Conventional ThresholdingIdentifies cracks automatically and effectively8.Salari and Bao [92]2011Conventional ThresholdingGives more effective and reliable resultsFigure 10
Deep learning-based crack segmentation [56].Due to the roughness of pavements in most cases, the snake model has been proposed to refine the cracks from the image and it has been concluded as better than other video quality. In addition to this, one may witness the snake model to tackle the crack segmentation. It is believed that the snake model is far better than the segmentation threshold.A hybrid algorithm for the thresholding technique is used for crack detection in Figure11. In the list of these thresholding techniques, there are several algorithms such that the canny model, Sobel model, Laplacian of Gaussian methods, Roberts method [57], and Prewitt method are included. These algorithms are proposed for identifying the cracks within the pavement. These methods have been optimized as initially, they had bad processing effects and therefore several segmentation methods such that morphological theory has been proposed. This algorithm not only tackles the noises but also detects the cracks and edges. Moreover, this should be noted that the processing should be faster, and hence in [58], it is noted to reduce the processing one may pre-process the images, i.e., eliminating the image noise from the crack. One may see an edge detection algorithm that is dependent on the grid analysis (GCA) to increase the real-time segmentation. This algorithm gives high accuracy and good repeatability. In most cases, it is used with uniformity for an image. Later, edge detection can also be utilized for the same purpose but need an artificial light source with uniformity. This GCA technique-based edge detection is much improved to reduce the impact of shadow. Figure 12 shows crack detection using discontinuity-based algorithms like Canny edge detection, Valley for image edge detection, and Valley tracing [59–61].Figure 11
(a) Original Image and (b) and (c) Crack detection Using Thresholding [55].
(a)(b)(c)Figure 12
Different Crack Detection Results (a) Canny edge detection (low thresholding). (b) Canny edge detection (high thresholding). (c) Valley edge detection. (d) Valley tracing [59].
(a)(b)(c)(d)The study of this literature provides several algorithms that recommend multi-level segmentation procedures, i.e. wavelet decomposition method and grey level morphology to see cracks [62]. It is observed that the wavelet transform may lead to loss of translation variance and may add some artifacts. This is the reason researchers prefer wavelet for only edge detection by amalgamating the holes algorithm [63]. Diving the image into sub-regions and using the same algorithm may improve the results. Another reason to opt for this procedure is that the conventional methods are lacking in representing the features. As machine and deep learning have a boom nowadays, therefore, several research contributions have proposed neural network-based algorithms to detect the cracks such that feature Pyramid and hierarchical boosting network (FPHBN) [64]. This algorithm is used mostly to create a differentiate a crack with its background. Moreover, it is compared also with the convolutional neural network as well along with edge detection using HED, RCF, and FCN. This has been illustrated in Figure 13. Moreover, to crack length measurement, Ali et al. used the deep convolutional neural networks (DCNN) as can be seen in Figure 14 [65].Figure 13
Crack detection Results Using FPHBN, HED, RCF, and FCN [64].Figure 14
Demonstration of DCNN algorithm [65].Researchers have designed several algorithms such that the convolution network for crack segmentation and the Gaussian conditional random field algorithm that can be used to refine the results [66]. Furthermore, some techniques are based on autoencoders for classifying the cracks as per spatial enhancement by removing the noise. One may see pixel defect analysis applying a U-net Deep encoding method, which is proposed for evaluating the pavement quality [67]. In this method, segmentation-based performance is demonstrated over the classical U-Net method without the significant computational overhead. In recent times, there have been several strategies and techniques used to divide the crack image into the segment as mentioned below:(i)
Minimal Path Section Method (MPSM)(ii)
Dynamic Optimization-based Crack Segmentation (DOCS)(iii)
Regional Growth Segmentation (RGS)Furthermore, researchers have proposed the Ground Penetrating Radar (GPR) to detect the pavement. This GPR is nothing but a real-time NDT technique that utilizes high-frequency radio waves. Some of the researchers have proposed GPR in the networks for pavement distress detection along with crack, water damage pit, and others as illustrated in Figure15 [68].Figure 15
GPR based Distress detection [68]. (a) Reflection crack (b) Water-damage pit (c) Uneven settlement.
(a)(b)(c)In recent, image processing methods are hybridized with deep learning algorithms to increase the efficiency of the results. In such techniques, one may find CNN, RNN, and autoencoders [69]. These algorithms have provided accuracy of up to 85%, less than 2 mm location errors, and are robust in terms of stability but later these algorithms were amended to a faster region convolution neural network for checking pavement distress [65]. The exact number of precision is 89.13% and intersection over union (IoU) is 86.24% which are available [70].


## 3.2. Classification Techniques

More artificial intelligence (AI) techniques are proposed to classify the crack types. In the list of such techniques, one may see the support vector machine method (SVM), artificial neural network (ANN), and various other algorithms. Some techniques like wavelet neural network (WNN) and backpropagation neural network (BPNN) are also proposed to not only classify the crack images but also used to measure the distress [71, 72]. With the implementation of BPNN researchers have classified the distress in the pavement into four types as mentioned below:(i)
Defect-free(ii)
Crack(iii)
Joint and(iv)
BridgedIn addition to this classification, researchers improved the accuracy level up to 90–95 percent [73]. SVM is a popular technique and is mostly opted one. The main usage of these techniques can be studied where SVM is used for structural risk minimization. The classifiers were introduced based on SVM and these classifiers can identify five types of distress in the pavement. Most importantly, SVM techniques have been compared with the WNN algorithm and the results found the SVM to be more accurate than the other algorithm [74]. Researchers, therefore, started improvising SVM and introduced a proximal support vector machine (PSVM) which proved to be easier to apply and more accurate than conventional SVM [75]. Several other classification approaches include wavelet transform along with Radon transform. This algorithm not only classifies the cracks but also identifies the longitudinal and diagonal cracks with transverse types. This is one of the expert systems and operates rapidly.


## 3.3. Pothole Detection

Another major thing along with pavement distress is pothole detection. This affects a lot in the maintenance and management plan for any pavement [76]. In today’s world, the manual detection method is not accurate and in addition to this, it is time-consuming. Thus, researchers have been proposing computer vision techniques to identify this sort of abnormality. Section Table 4 refers to the number of research contributions proposed for pothole detections using image processing. For digital image processing, one may need a digital camera but some researchers proposed laser scanning in the digital camera to develop a system that helps us in detecting potholes. Since the equipment cost of stereo vision acquisition is less, several studies have utilized it for the same purpose [77–79]. Few researchers have combined stereo camera and GPR techniques for pothole image extraction. Convolution Neural Network is one of the most common techniques opted for because it requires no manual assistance, and it is more stable because it does not affect the environment. CNN is a much more robust algorithm because it has noise filtering ability and can detect the road image noise-free.Table 4
Digital image technology for potholes detection.
Sr. No.ReferencesYearDetection methodImage feature extraction methodResults1.She et al. [40]2021Laser, digital camera, and CMOS sensor3D line laser technologyAchieves higher precision and gives sufficiently accurate results2.Silva et al. [93]2020Digital camera(MAS) PANGEA with (YOLO) v4Gives accurate and promising results3.Gao et al. [94]2020Digital cameraLIBSVMGives superior segmentation effects and processing efficiency4.Wang et al. [78]2017Digital cameraMarkov random field model, geometric criteria, and morphological processingEffectively distinguishes and accurately segments potholes5.Li et al. [79]2016GPS, stereovision systems, and GPRRegion-based segmentation and morphological operationsAchieves accuracy in results6.Koch and Brilakis [81]2011Digital cameraElliptic regression and morphological thinningShows reasonable accuracy7.Lin and Liu [80]2010Digital cameraSVMAchieves a high recognition rateThus, training CNN on images with high-resolution takes more process time and memory as well which is one of the constraints, and to overcome this limitation many researchers have proposed a sliding window method. Furthermore, CNN is improvised and had been produced in two versions part-based classification neural network (PCNN) and localization neural network (LCNN). These two versions are proposed to achieve better performance and reduce the load of calculation. There are geometric criteria along with morphological processing that develop the wavelet energy field of the pavement image for distress detection in pavement [80, 81].


## 4. Recent State-of-the-Art Work of Drone in Healthcare

Technology is a hub of contexts, algorithms, and platforms, which explores and tunes various possibilities for humankind. Drones can be based on various control techniques, dynamic models, and multiple-purpose schemes like operation-based models, size-based models, sliding window mode, and backstepping. The usage of these techniques depends upon their purpose and requirement in the healthcare systems. Some are applicable and efficient for outdoor activities while others are for indoor activities. Their performances are measured based on their performance, accuracy, effectiveness, efficiency, and security metrics in healthcare. Multiple optimization parameters analyze these performance measurements.The primary focus of this review is to determine how AI-enhanced drones help in transporting medications, treatments, health-related equipment, and biological samples. Moreover, these drones provide services to areas, which are difficult to access. Researchers in [82] applied the deep learning software for the triage of COVID-19 pneumonia. The results showed the applied software detected COVID-19 Pneumonia efficiently with CT findings such as GGOs and lobular lesions. The applied software proved to help localize and quantify lung lesions and for providing treatment as well. Another study [83] employed transfer learning for COVID-19 detection and managed to train CNNs up to high accuracy. The results proved the usefulness of transfer learning for abnormal and normal classifications. Moreover, overall performance was achieved through image augmentation. In [84], researchers proposed an AI-enabled IoT-based drone for secure communication in the healthcare department. The research used a blockchain model with machine learning techniques to make the drone robust and resilient against multiple types of attacks. One can study the usage of drones for intelligent and secure surveillance in the healthcare sector in [85]. The researchers used an edge-AI layer in the UAVs where the Naive Bayes classifier classified the non-relevant data from the relevant data. Then, stored the relevant information in the blockchain layer. Results showed the suggested architecture provided improved communication reliability, and performance. Moreover, these drones made intelligent decisions in healthcare services like medicine delivery, health monitoring, telesurgery, sanitization, and others. Similarly, drones can increase public safety by assisting the users especially the emergency ambulances about the current road conditions. In a recent study [86], a UAV, with a camera and geo-tag, and a system with a YOLOv3 algorithm is employed for pothole detections and to differentiate them from dark patches of road. The UAV clicks and transfers the image to the system, which then detects the pothole immediately with high accuracy. This model enables the authorities to fix and maintain these potholes and enables the public, especially the ambulances to know the current road conditions for safety.


## 5. Conclusion

This manuscript provides a comprehensive review of the different computer vision techniques applied over the roads and pavement to detect potholes and distresses using unmanned aerial vehicles. Image Acquisition devices are also introduced along with the state-of-the-art approaches used for the same analysis. The constraints and limitations related to the image processing are on-site pavement crack, pavement texture, rutting, and others. Moreover, the study addresses the common issues such that the clarity, efficiency, and cost of an image along with pavement distress type and suggest that these issues must be taken into consideration before opting for any suitable detection equipment.In inclusion to this, the features of image acquisition devices are also correlated and applied computer vision technology is also discussed regarding different features of various types of these devices.(i)
When it came to crack detection on the job, the most frequent methods were laser and digital cameras. Several crack detection vehicles were also developed. Existing edge detection methods, like the Sobel approach and the Canny method, might result in poor segmentation results. As a result, deep learning techniques, like SVM, CNN, RNNs, BPNN, were applied. For on-site crack classification, these techniques proved to be an excellent method.(ii)
The current state of analysis into the use of digital image processing for evaluating the texture of pavement was examined. The fractal theory, geometry (number of peaks, mean texture depth, and mean profile depth, for example), and frequency spectrum (fractal theory, fast Fourier transform, and wavelet transform, for example) were mostly used as pavement texture explaining indicators. Furthermore, imaging technology has various evaluation methods and indicators that need to be enhanced due to its characteristics. To improve pavement texture detection in the future, new methodologies and indicators will need to be provided.(iii)
As it is quick and can collect the entire temperature field distribution, thermal imaging technology has grown into a very hopeful approach to detecting temperature segregation. Laser detection equipment, line and point lasers, is essential for pavement detection. A continuous image of a road cross-section is impossible to obtain using point laser equipment. Placing more laser test spots will make the more authentic and accurate, but the expenses rise in tandem. To lower the detection error, the point laser number with the dispersion of its placements requires further research. To obtain the continuous image of a road section one can use equipment of line laser detection; however, the pavement transverse profile elevation shows less accuracy as compared to the point laser approach. The accuracy of the measurement would be influenced by the detection vehicle’s wandering and longitudinal turbulence, pavement mark line, other pavement distresses, and so on.(iv)
Common methods for extracting the pothole from the obtained image include SVM-based and CNN-based approaches. A huge amount of memory that is usually unavailable is essential to train CNN on high-resolution images. As a result, a better machine learning or CNN-based method must be suggested using image technology for reducing the calculation load for pothole detection. The most common methods that detect joint faulting include binocular vision and laser scanning. The laser scanning method gives accurate results but is expensive. Contrary, the vision method of binoculars is more effective together with cheaper, however, requires more research for the joint faulting calculation method.Healthcare professionals have a debate on support and criticism for AI-enhanced drones. This review shows the study of drones in the healthcare sector. The predominant focus shows that drones enhance the efficiencies and capabilities of health care and service providers. Drones not only promote healthcare but also provide a feasible way to provide services to inaccessible areas. Findings indicate how these noise-free and carbon emission-free drones render their services and achieve sustainable development goals. Their usage improves socio-economic situations and safety in society and may further enhance the social and economic lives of people.


## 6. Future Recommendation

The criticism for AI-enhanced drones arises from the early research and the need for public use. AI requires high investment, research processes, training, and programming to authenticate its application, and efficacy regarding its direction and future usage. For future work, we can enhance the study while engaging more healthcare professionals, patients, and communities in drone development and deployment. Moreover, an examination can be conducted that shows how this engagement can affect health and digital studies. This will enable the developers to produce such medical drones that will satisfy the requirements of their users and targeted population. This will ensure the support and acceptance from various communities including the rural population. Furthermore, the financial and healthcare quality cost examinations for integrating different applications of drones into present healthcare services will be studied in the future. This will provide information to the decision and policymakers for integrating medical drones more appropriately.



---

*Source: 7733196-2022-10-10.xml*