# Defect Point Location Method of Civil Bridge Based on Internet of Things Wireless Communication

Authors: Xiaofeng Yan; Zedong Liu; Zijing Zhuang; Yong Miao
Journal: Journal of Electrical and Computer Engineering
 (2022)

---

## Abstract

With the growth of the country’s comprehensive strength, China’s road and bridge traffic is also growing rapidly. Therefore, the maintenance of highway bridge pavement has become extremely important. The main manifestation of highway bridge deck diseases is bridge deck cracks. If the bridge deck cracks are found in the early stage of damage and solved in time, it will undoubtedly greatly reduce the maintenance cost and care and ensure that the road can be driven safely. At present, the detection of highway bridge defects is mainly based on human vision, but this kind of artificial visual inspection is difficult to complete efficiently. The purpose of the article was to study image recognition techniques and measure the surface damage to bridge superstructures. It has also developed an intelligent software system that can measure and identify cracks under bridges. Aiming at the compatibility problem of wireless communication front end caused by the difference in wireless communication protocols, this study designs a high-applicability front-end control interface for wireless communication. After testing, data can be sent and received when the I/O mode rate drops to 10 Kbps. This method is severely limited and is not suitable for IoT applications with low power consumption and low frequency. It uses the SPI interface for communication and can send and receive normally at different rates, with an upper limit of 8 Mbps. This method consumes a little more pins, but the clock signal is stable, and the transmission performance can meet the needs of most applications.


---

## Body

## 1. Introduction

Since the reform and opening up, China’s bridge construction has developed rapidly. Some highways are over 40 years old. According to recent road surveys, road bridges are aging and deteriorating, leading to an increase in dangerous bridges. Seriously, some road bridges built between the 1960s and 1970s had problems such as narrow bridges, severe damage, and cracks. At present, compared with the international test level, China’s road and bridge maintenance management is not enough. With the steady growth of the national economy, the role of road traffic as infrastructure in economic development is growing. Therefore, the maintenance and management of highway bridges can become one of the key issues for further development. In the case of not affecting the vehicle, it has become an urgent problem to find and identify road hidden dangers effectively and quickly.This study focuses on the automatic detection of bridge defects and analyzes the deterioration of bridges in the process of detection. It detects and defines algorithms to detect defects under highway bridges and measure cracks under highway bridges and verify their accuracy. Finally, the study provides an effective and fast measurement method for highway surface defects. The exclusion method can be calibrated according to the defects of the lower part of the bridge. The length of the abutment rupture can also be determined, which provides a reliable basis for evaluating the defect level. Compared with traditional visual recognition, this recognition method has higher efficiency, faster speed, higher accuracy, and lower cost of manpower and material resources. This saves time in highway troubleshooting. Although foreign digital technology uses digital image processing technology to identify the surface defects of highways on bridges, it is not suitable for more complex bridge platforms. Therefore, if an image algorithm with strong adaptability and detection ability can be designed, it will undoubtedly increase the ability to detect bridge defects. Therefore, it is of great significance and value to study the crack identification algorithm of highway bridge deck.


## 2. Related Work

Experts at home and abroad have also conducted a lot of research on IoT communication and defect point location methods for civil bridges. KongL discusses mmWave communication capabilities. As the next generation of wireless technology, mmWave is advanced in its multi-gigabit transmission capability and beamforming technology [1]. Xu Z believes that the future Internet of things (IoT) will make everything smartly connected. To achieve it, there are many challenges. A key challenge is the issue of batteries for small devices such as sensors or tags [2]. Mishra D thinks of multimedia communication through wireless sensor network (WSN). The sensor node transmits the confidential data to the gateway node through the public channel. In such an environment, the security problem is still a serious problem in the past years [3]. Sen S believes that reducing the size of IoT nodes and slowly increasing energy storage density will lead to lower energy availability. Furthermore, smaller size and less energy mean less resources are available for securing IoT nodes [4]. Bennis M focused his attention on various techniques and methodologies related to the requirements of URLLC and their application in selected use cases. These results provide clear insights into the design of low-latency and high-reliability wireless networks [5]. Sairam found that random access operation is performed in a communication environment where multiple communication modes with different transmission rates coexist with small overhead [6]. These methods provide some references for their research. However, due to the short time and small sample size of the relevant research, the study has not been recognized by the public.


## 3. Design of Wireless Communication Front-End Control Interface for IoT Applications

In the design of wireless communication front-end control system for IoT applications, the most basic part of the physical layer is the wireless communication front end. The wireless communication front end is also an important part of the wireless communication composition of the Internet of things. It will design the wireless communication front-end control interface. The control interface designed in this study includes the following functional modules: data communication interface, data processing module, test module, and external signal trigger module. Through the analysis of the wireless communication front end, Table1 lists the comparison of three types of wireless communication front-end parameters. The data communication interface is designed using the SPI bus protocol, and the data communication interface design will refer to this table and discuss the feasibility of other different protocols. Generally, different wireless communication protocols have different requirements on the data frame format. It will refer to several commonly used wireless communication protocols. It designs a data processing module to handle communication protocols in different data frame formats [7].Table 1
Comparison of wireless communication front-end parameters.
AttributesTRF7960 AnRF24L01BK5823Affiliated companyTINordicBekenFrequency range13.56 MHz2.4–2.525 GHz5.77–5.85 GHzData rate106–848 kbps1–2 mbps125,256,512 kbpsData communication interfaceSPISPISPIThe wireless communication front-end control interface usually includes a data communication interface, a data processing module, a test module, and an external signal trigger module [8]. Among them, the data communication interface compares several existing bus protocols. It will select the SPI bus protocol for design. The data processing module is mainly used for processing the data frame format and reserves the data pass-through mode to realize the interface. The external signal trigger module is used to respond to external control signals, such as external wake-up signals and test signals sent by MCU. Figure 1 shows the design framework of the wireless communication front-end control interface.Figure 1
Control interface architecture.


### 3.1. System Software Design

The system software design is divided into three parts: communication interface design, data processing module design, and auxiliary function module design. The module design is listed in Table2, and the system design framework is shown in Figure 2 [9].Table 2
Software system design.
Design typeDesign nameThe main functionNumerical valueCommunication interfaceUART interfaceCommunicate with the host computer0.234656SPI interfaceConfigure the communication front-end control interface0.264610Data processing moduleData sendingSend data to the communication front end1.065210Data receptionReceive the data returned by the communication front end1.3261263Accessibility moduleTiming functionBasic timing use1.2658562Low power modeControl system power consumption5.6546549Figure 2
Embedded wireless communication system architecture.The MCU obtains data from the host computer through the communication interface and configures the wireless communication front end. According to the data transmission mode of the wireless communication front end, three data transceiver functions will be designed. This enables embedded systems to be used for communication interface data transfer in standard mode and special signal write and read in pass-through mode. The timing function and low power consumption mode are designed in the auxiliary function module. This can meet the system’s requirements for low power consumption and realize the system’s timing control [10].


### 3.2. Overall Architecture of LoRa IoT Test System

According to the overall architecture of IoT design, LoRa’s performance test system is divided into three parts: perception layer, network layer, and application layer. The overall architecture topology is shown in Figure3. The perception layer is mainly arranged with terminals, which are used to send test data. It works by laying out the hardware in a test environment. The LoRa terminal periodically sends a fixed amount of data to the gateway according to different types of hardware configuration combinations. It then uploads the data to the server over the network. After the server receives the data, it stores the data in the database. The application layer can call the server port through the browser interface to display the currently received data visually and realize the specific parameter configuration of the hardware [11, 12].Figure 3
System composition topology.The perception layer is at the bottom of the overall architecture. It is for the purpose of testing LoRa performance and establishing an accurate relationship between hardware configuration parameters. The test system just consists of a single LoRa terminal and a single LoRa gateway. Among them, LoRa terminal and gateway are designed with SXl278 wireless communication module. It has the characteristics of long communication distance, low power consumption, and strong anti-interference. It is suitable for self-configured low-power wide-area networks. The LoRa terminal periodically sends test data. The gateway receives the corresponding data through the wireless network, to realize the collection of test data. The network layer is responsible for receiving data from the perception layer. The main device of the network layer is the gateway, which is also an important part of connecting the terminal and the server. It realizes the conversion between LoRa network protocol and different types of communication protocols. The gateway not only needs to receive the data sent by the terminal under different hardware configuration parameters to realize the data collection, but also needs to receive the instructions sent by the server. It sends a series of control commands to the terminal, such as changing configuration parameters and starting and ending the test. The application layer includes the Web server and the computers that connect to the server. The application layer implements functions such as data management, data storage, and control hardware. The computer equipment connected to the server can realize the parameter configuration of the hardware and real-time dynamic display of the specific parameters of the data currently received by the gateway by calling the server port. This provides valuable information for LoRa testing, which can be discovered and dealt with in time in case of emergencies. It provides a real-time and interactive management interface for LoRa testing. Through the cooperative operation of the above three parts, it provides feasibility and convenience for LoRa testing, thus laying a foundation for the subsequent LoRa performance analysis.Among the system parameters, in addition to the number of receiving antennas and the average power of the transmitted signal, which affect the reliability of the system, the modulation order of the transmitted signal also has its influence. On the one hand, a larger modulation order can make each symbol carry more bits of information and improve bandwidth utilization. On the other hand, when the average power of the transmitted signal is limited, a larger modulation order will lead to a closer distance between the transmitted signal constellation points. This increases the transmission symbol error rate and reduces the reliability of the system. Considering the IIoT scenario, the data packets to be transmitted are usually short and the amount of information is small. Therefore, the modulation order does not necessarily need to be too high, but the order can be selected appropriately. It obtains higher system reliability under the condition of satisfying the transmission time requirement. Next, the influence of different modulation orders on the system transmission symbol error rate is studied through experiments. In the experiment, the number of receiving antennas is set toN = 128. The experimental results are shown in Figure 4 [13].Figure 4
Average signal-to-noise ratio of transmitted signals.In practical IoT applications, perception layer targets are often exposed to relatively unstable environments, and there are often no network connections for some unforeseen reasons. Recognizing this situation in time can avoid many problems, such as limiting the use of target power and consumption. In the test, due to the high frequency of sensor data collection, the portal believes that if the node cannot receive data within a certain period of time, it is considered that the node is not connected to the network [14], but in practical applications, the number of many data collection nodes is usually not high. If the amount of data is large, it can be collected once a day. There is no data transfer between nodes at other times. If the node is disconnected from the network, the gateway is not visible. There are two solutions for this situation. One is that the lower node continuously reports its routing status to the gateway. This solves the problem, but increases the load on the coordinator. The other is that the parent node sends a message. If the parent node does not receive the heartbeat packet of the child node within the specified time, the child node is considered disconnected from the network. It reports the information to the gateway in the child node processing. It divides the coordinate weights to avoid the coordinator from processing a large amount of node state information. The heartbeat mechanism flow of the parent node and child node is shown in Figure 5 and Figure 6, respectively [15].Figure 5
Parent node heartbeat mechanism flow.Figure 6
Child node heartbeat mechanism flow.


### 3.3. Composition and Design of Software Algorithms

Usually, to enhance the speed of image operation, it is necessary to reasonably grayscale the collected crack images of highway bridges. A bridge crack image is divided into three channels: red, green, and blue, and only when the gray values of the three channels are the same, the image appears gray. In general, each value of RGB is divided into 0 to 255 interval values. Each value of each component is in the interval of 256 values [16].The method of averaging is to add the values of each channel of the image to obtain the average value.(1)Grayi,j=Ri,j+Gi,j+Bi,j,Grayi,j=0.99×Ri,j+0.587×Gi,j+0.144×Bi,j,Grayi,j=maxRi,j,Gi,j,B,i,j.The desired value is as follows:(2)a5=a1+a2+a3+a4+a6+a7+a8+a9.The binarized pixel gray value relationship of the thresholded crack image,gx, y are as follows:(3)gx,y=a,fx,y>Tb,fx,y≤T..The grayscale histogram component of the crack image is represented by the formula:(4)pq=nqnq=0,1,2,…,L−1.The value of the variance for a large period is as follows:(5)σB2k=P1km1k−mG2+P2km2k−mG2,P1k=∑i=0kPi.The probability of occurrence of the set is as follows:(6)P2k=∑i=k+1L=1Pi=1−P1k.The between-class variance is given by the formula:(7)σB2k=mGP1k−mk2,σB2k=P1k1−P1k.The specific separation metric is shown in the formula:(8)ηk=σB2kσG2,0≤ηk≤1.Let the starting threshold be:(9)T=U1+U22,T0=Zmin+Zmax2,T=Q1+Q22.The formula used to calculate the local threshold of the crack image:(10)Txy=mZxy+nGxy,Txy=mZxy+nG,g,x,y=1,fx,y>mZxy.The essence of morphological processing is that element B filters the entire image A, as follows:(11)A⊕B=xB+x⊆A,A⊕B=xB∩A≠φ.


## 3.1. System Software Design

The system software design is divided into three parts: communication interface design, data processing module design, and auxiliary function module design. The module design is listed in Table2, and the system design framework is shown in Figure 2 [9].Table 2
Software system design.
Design typeDesign nameThe main functionNumerical valueCommunication interfaceUART interfaceCommunicate with the host computer0.234656SPI interfaceConfigure the communication front-end control interface0.264610Data processing moduleData sendingSend data to the communication front end1.065210Data receptionReceive the data returned by the communication front end1.3261263Accessibility moduleTiming functionBasic timing use1.2658562Low power modeControl system power consumption5.6546549Figure 2
Embedded wireless communication system architecture.The MCU obtains data from the host computer through the communication interface and configures the wireless communication front end. According to the data transmission mode of the wireless communication front end, three data transceiver functions will be designed. This enables embedded systems to be used for communication interface data transfer in standard mode and special signal write and read in pass-through mode. The timing function and low power consumption mode are designed in the auxiliary function module. This can meet the system’s requirements for low power consumption and realize the system’s timing control [10].


## 3.2. Overall Architecture of LoRa IoT Test System

According to the overall architecture of IoT design, LoRa’s performance test system is divided into three parts: perception layer, network layer, and application layer. The overall architecture topology is shown in Figure3. The perception layer is mainly arranged with terminals, which are used to send test data. It works by laying out the hardware in a test environment. The LoRa terminal periodically sends a fixed amount of data to the gateway according to different types of hardware configuration combinations. It then uploads the data to the server over the network. After the server receives the data, it stores the data in the database. The application layer can call the server port through the browser interface to display the currently received data visually and realize the specific parameter configuration of the hardware [11, 12].Figure 3
System composition topology.The perception layer is at the bottom of the overall architecture. It is for the purpose of testing LoRa performance and establishing an accurate relationship between hardware configuration parameters. The test system just consists of a single LoRa terminal and a single LoRa gateway. Among them, LoRa terminal and gateway are designed with SXl278 wireless communication module. It has the characteristics of long communication distance, low power consumption, and strong anti-interference. It is suitable for self-configured low-power wide-area networks. The LoRa terminal periodically sends test data. The gateway receives the corresponding data through the wireless network, to realize the collection of test data. The network layer is responsible for receiving data from the perception layer. The main device of the network layer is the gateway, which is also an important part of connecting the terminal and the server. It realizes the conversion between LoRa network protocol and different types of communication protocols. The gateway not only needs to receive the data sent by the terminal under different hardware configuration parameters to realize the data collection, but also needs to receive the instructions sent by the server. It sends a series of control commands to the terminal, such as changing configuration parameters and starting and ending the test. The application layer includes the Web server and the computers that connect to the server. The application layer implements functions such as data management, data storage, and control hardware. The computer equipment connected to the server can realize the parameter configuration of the hardware and real-time dynamic display of the specific parameters of the data currently received by the gateway by calling the server port. This provides valuable information for LoRa testing, which can be discovered and dealt with in time in case of emergencies. It provides a real-time and interactive management interface for LoRa testing. Through the cooperative operation of the above three parts, it provides feasibility and convenience for LoRa testing, thus laying a foundation for the subsequent LoRa performance analysis.Among the system parameters, in addition to the number of receiving antennas and the average power of the transmitted signal, which affect the reliability of the system, the modulation order of the transmitted signal also has its influence. On the one hand, a larger modulation order can make each symbol carry more bits of information and improve bandwidth utilization. On the other hand, when the average power of the transmitted signal is limited, a larger modulation order will lead to a closer distance between the transmitted signal constellation points. This increases the transmission symbol error rate and reduces the reliability of the system. Considering the IIoT scenario, the data packets to be transmitted are usually short and the amount of information is small. Therefore, the modulation order does not necessarily need to be too high, but the order can be selected appropriately. It obtains higher system reliability under the condition of satisfying the transmission time requirement. Next, the influence of different modulation orders on the system transmission symbol error rate is studied through experiments. In the experiment, the number of receiving antennas is set toN = 128. The experimental results are shown in Figure 4 [13].Figure 4
Average signal-to-noise ratio of transmitted signals.In practical IoT applications, perception layer targets are often exposed to relatively unstable environments, and there are often no network connections for some unforeseen reasons. Recognizing this situation in time can avoid many problems, such as limiting the use of target power and consumption. In the test, due to the high frequency of sensor data collection, the portal believes that if the node cannot receive data within a certain period of time, it is considered that the node is not connected to the network [14], but in practical applications, the number of many data collection nodes is usually not high. If the amount of data is large, it can be collected once a day. There is no data transfer between nodes at other times. If the node is disconnected from the network, the gateway is not visible. There are two solutions for this situation. One is that the lower node continuously reports its routing status to the gateway. This solves the problem, but increases the load on the coordinator. The other is that the parent node sends a message. If the parent node does not receive the heartbeat packet of the child node within the specified time, the child node is considered disconnected from the network. It reports the information to the gateway in the child node processing. It divides the coordinate weights to avoid the coordinator from processing a large amount of node state information. The heartbeat mechanism flow of the parent node and child node is shown in Figure 5 and Figure 6, respectively [15].Figure 5
Parent node heartbeat mechanism flow.Figure 6
Child node heartbeat mechanism flow.


## 3.3. Composition and Design of Software Algorithms

Usually, to enhance the speed of image operation, it is necessary to reasonably grayscale the collected crack images of highway bridges. A bridge crack image is divided into three channels: red, green, and blue, and only when the gray values of the three channels are the same, the image appears gray. In general, each value of RGB is divided into 0 to 255 interval values. Each value of each component is in the interval of 256 values [16].The method of averaging is to add the values of each channel of the image to obtain the average value.(1)Grayi,j=Ri,j+Gi,j+Bi,j,Grayi,j=0.99×Ri,j+0.587×Gi,j+0.144×Bi,j,Grayi,j=maxRi,j,Gi,j,B,i,j.The desired value is as follows:(2)a5=a1+a2+a3+a4+a6+a7+a8+a9.The binarized pixel gray value relationship of the thresholded crack image,gx, y are as follows:(3)gx,y=a,fx,y>Tb,fx,y≤T..The grayscale histogram component of the crack image is represented by the formula:(4)pq=nqnq=0,1,2,…,L−1.The value of the variance for a large period is as follows:(5)σB2k=P1km1k−mG2+P2km2k−mG2,P1k=∑i=0kPi.The probability of occurrence of the set is as follows:(6)P2k=∑i=k+1L=1Pi=1−P1k.The between-class variance is given by the formula:(7)σB2k=mGP1k−mk2,σB2k=P1k1−P1k.The specific separation metric is shown in the formula:(8)ηk=σB2kσG2,0≤ηk≤1.Let the starting threshold be:(9)T=U1+U22,T0=Zmin+Zmax2,T=Q1+Q22.The formula used to calculate the local threshold of the crack image:(10)Txy=mZxy+nGxy,Txy=mZxy+nG,g,x,y=1,fx,y>mZxy.The essence of morphological processing is that element B filters the entire image A, as follows:(11)A⊕B=xB+x⊆A,A⊕B=xB∩A≠φ.


## 4. Bridge Surface Defect Detection System and Its Hardware Environment

Starting from software requirement analysis, it is a very important link in the software development cycle to enter the stage of transforming the user’s informal requirements into a complete requirement definition stage. Although it does not involve specific “how to achieve,” it expounds “what to achieve” and some nonfunctional requirements related to it in detail. This is such as the operating environment of the system, the scalability of the system, and the security of the system. It is then organized into a scientific and rigorous demand analysis report with a clear logical structure of the system, clear description, and easy to understand. In actual production practice, system requirement analysis does not necessarily need to output very standard technical documents, but it must give stakeholders and development teams a rough idea first in their minds. What are the requirements, what is the user story, how is the interaction logic between systems arranged, and what is the UI interface design that will enrich the user experience? Because only when the requirements are analyzed and the core spirit can be understood, there will be no dilemma of poor project delivery quality and great differences from the demand side [17].


### 4.1. Design Goals

In the context of the demand for many bridges in China and the shortcomings of manual detection methods, such as low efficiency, it uses artificial intelligence methods. It cooperates with the practice of wireless communication to detect and analyze bridge deck cracks (including bridge pier surface and bridge pavement). It finally achieves the goal of automatic detection and classification of bridge deck cracks by machines. In the process, firstly, a drone equipped with a high-definition camera is used to inspect the bridge, and the video stream of the bridge surface (including the road surface and the pier surface) is collected. It then uses the crack cascade classifier built by itself to detect cracks in real-time video streams. Finally, a convolutional neural network is used to build a model to classify and identify bridge surface defects. The monitoring personnel can watch and control synchronously on the computer.


### 4.2. Hardware Facilities

Hardware Environment: professional aerial photography UAV is the main platform for system application, which completes image set training and recognition of UAV. It requires a camera of at least 5 million pixels, a minimum memory of 12 GB, and a CPU frequency of 4GHZ.


### 4.3. Functional Analysis

The bridge surface defect analysis system is the management of collecting bridge cracks and classifying them scientifically. Among them, the bridge cracks are the main line, which runs through the whole process, including flying drones near the bridges to be inspected. It controls the UAV flight route in the visual interface of the system. It collects the images returned by the drone in real time or imports the saved video into the relevant interface of crack detection through the system. After image preprocessing, the preprocessing must meet certain requirements. It uses the model trained by the system to accurately identify and grade the cracks, and finally, the cracks go through the streaming processing of each module. It realizes the module-to-module series connection, thus making the whole system run. As for the analysis results expected by professional engineers, professionals are also required to manually detect the recognition rate [18].By analyzing the database requirements and understanding the requirements of the storage structure on the system, the result record table can be drawn according to the results of data modeling.User Information Sheet: the user information table includes the user information using the system, the user’s ID, the user name, the user password, and the department where the corresponding user is located. It is shown in Table3.Table 3
User information table.
NumberingNameType of dataLengthDefaultsAllow nullsPrimary foreign keyRemark1userIdInt100NoPrimary keySelf-growth2userNameVARCHAR255NullNo3userPassVARCHAR255NullNo4DepartmentVARCHAR255NullYesPicture Information Record Sheet: it mainly records the information of the collected crack pictures. It includes the picture record number, the picture name, the specific shooting time of the picture, the location where the picture was taken, and the path where the picture is stored. Among them, the picture record number is used as the primary key, which can achieve self-growth and cannot be a null value. Of the parameters mentioned in the appeal, only the shooting location allows null values. Other parameters should be recorded accurately, as shown in Table4 [19].Table 4
Picture information record table.
NumberingNameType of dataLengthDefaultsAllow nullsPrimary foreign keyRemark1picSetIdInt100NoPrimary keySelf-growth2picSetNameVARCHAR255NullNo3picSetTimeDatetime80No4picSetLocationVARCHAR255NullYes5picSetPathVARCHAR255NullNoVideo Information Table: the video information table mainly records the video information with cracks. It mainly includes the video record number, video name, video shooting time, video shooting location, and video storage path. Among them, the video record number is used as the primary key, which can achieve self-growth and cannot be a null value. Among the parameters mentioned in the appeal, only the shooting location and the time of shooting the video allow null values. All other parameters must be recorded accurately. The design of the video information table is similar to the design structure of the picture information table. It is shown in Table5 [20].Table 5
Video information sheet.
NumberingNameType of dataLengthDefaultsAllow nullsPrimary foreign keyRemark1videoIdInt100NoPrimary keySelf-growth2videoNameVARCHAR255NullNO3videoTimeDatetime80Yes4videoLicationVARCHAR255NullYes5videoPathVARCHAR255NullNoThe crack result information detected by the system mainly includes the detection record number, video IDmmc1, detection source path, the level of the crack in this detection result, and the number of detection results in this detection. The test results are divided into three grades: A, B, and C grades, corresponding to grades 1, 2, and 3, respectively. These are the minor, moderate, and severe crack classes mentioned above in the system design. When multiple cracks are detected in a video, the detection record number will increase automatically. The corresponding video IDs are the same, and the specific crack level detected is also recorded according to the data. The number of detection results will also increase according to the number of cracks detected in the same video.After tracking the bridge crack target, it is necessary to quantitatively measure the crack size in order to evaluate the damage to the bridge. In general, the detected crack contour is circumscribed with a minimum rectangle, and the width of the minimum circular rectangle is used as the accounting standard for cracks. However, since this type of crack is more effective for straight lines, it is not suitable for more complex curved cracks. Therefore, the article provides a simpler and more accurate method to represent the size of cracks and their cyclic arcs. The idea of this study is to first calculate the area A and radius H of the associated crack region by automatically tracking the crack and then obtain the skeleton of the crack outline by selecting the box. It then counts the number of white pixels. TheL length of the crack is actually determined by the number of these white pixels. The article uses the above methods to test the size of transverse and vertical cracks, respectively. This result is shown in in Figure 7 [21].Figure 7
Horizontal and longitudinal crack size test.Error Analysis: According to the above data, the actual crack width is actually lower than the tested width value. The error is about 0.03 mm, and the calculated error is about 2.5%–16%. According to the allowable crack value of the bridge, it must be ≤ 0.30 mm, and both the test data and the actual crack width are greater than 0.30 mm. Therefore, both lateral and vertical cracks in the table need to be repaired. The error range of their detection is still within the allowable range, indicating that the detection is effective [22].Figure8 shows the deviations that affect the speed of data retrieval under any requirement. The speed of data retrieval is very important. Because it determines the storage capacity required by the image acquisition system, the most basic image acquisition system has a camera and several cards to record the camera’s output. For example, the inspection vehicle travels at a speed of 1.11 m/s, and the image acquisition system also meets the above requirements. The camera can process images and transfer them to memory at a transfer speed of 11,998,000 pixels per second. If it is determined that the pixel resolution is only 2 mm, the data retrieval rate is reduced to 4,999,500, that is, 40% of the original selection capacity. Of course, lower speed and better pixel resolution can also be set. The transmission rate of such data is greatly reduced, which in turn reduces the capacity of the image acquisition system. It can achieve the purpose and basic principle of identifying bridge defects by reducing the width of the covered deck or reducing the number of movable deck areas [23].Figure 8
Image acquisition rate and resolution.According to the experimental results, it can be seen that the neural network model based on the “block” image histogram (HNN structure) has 15 nodes in the hidden layer, and the learning efficiency is 0.1 or 0.01. Its network performance is optimal after 1500 cycles. The recognition rate in the testing process can reach 73.6% (the recognition results are shown in Figure9).Figure 9
Accuracy comparison based on HNN mode.Figure10 shows the classification accuracy on the test machine during the training of the network structure of the selected model scheme. Finally, the accuracy of the convolutional neural network model is higher than 95.2% and tends to be flat, and the accuracy is high. The abscissa represents the number of iterations. After iterating for 500 epochs, the three lines with different colors represent the increase with the number of iterations. The historical highest accuracy and average accuracy statistics are shown in Figure 10 [24].Figure 10
Convolutional neural network model accuracy.Based on the analysis of bridge surface defects based on wireless communication, the system combines the popular image recognition and convolutional neural network recognition technology. It can realize the management of video files or real-time video streams of drones, to identify the corresponding bridge cracks. In the whole system, the collected crack images are classified by the trained crack classifier, to judge the level of cracks. Of course, with the continuous collection of crack patterns, the recognition rate and classification accuracy of bridge cracks will gradually increase. Therefore, the existing system can also be modified accordingly at the later stage. The system can also be optimized and expanded in terms of external system connection and internal process refinement. In terms of external system docking, the GPS navigation system and live broadcast system can be introduced. In this way, it is more convenient for relevant professional engineering personnel to conduct real-time survey and judgment and to adopt selected engineering strategies. The internal process can be optimized and transformed on the basis of existing functions, and the most advanced and convenient information technology can be cited. It professionalizes, normalizes, and refines the functions of each module of the system. It makes the corresponding modules more in line with professional testing standards. This can provide users with more convenient and refined operation functions and can also add functions such as report making, mathematical statistics, and data visualization. It provides convenience for the relevant staff to the greatest extent and enables engineers in the professional field to have a better understanding of the overall situation of the bridge [25].


## 4.1. Design Goals

In the context of the demand for many bridges in China and the shortcomings of manual detection methods, such as low efficiency, it uses artificial intelligence methods. It cooperates with the practice of wireless communication to detect and analyze bridge deck cracks (including bridge pier surface and bridge pavement). It finally achieves the goal of automatic detection and classification of bridge deck cracks by machines. In the process, firstly, a drone equipped with a high-definition camera is used to inspect the bridge, and the video stream of the bridge surface (including the road surface and the pier surface) is collected. It then uses the crack cascade classifier built by itself to detect cracks in real-time video streams. Finally, a convolutional neural network is used to build a model to classify and identify bridge surface defects. The monitoring personnel can watch and control synchronously on the computer.


## 4.2. Hardware Facilities

Hardware Environment: professional aerial photography UAV is the main platform for system application, which completes image set training and recognition of UAV. It requires a camera of at least 5 million pixels, a minimum memory of 12 GB, and a CPU frequency of 4GHZ.


## 4.3. Functional Analysis

The bridge surface defect analysis system is the management of collecting bridge cracks and classifying them scientifically. Among them, the bridge cracks are the main line, which runs through the whole process, including flying drones near the bridges to be inspected. It controls the UAV flight route in the visual interface of the system. It collects the images returned by the drone in real time or imports the saved video into the relevant interface of crack detection through the system. After image preprocessing, the preprocessing must meet certain requirements. It uses the model trained by the system to accurately identify and grade the cracks, and finally, the cracks go through the streaming processing of each module. It realizes the module-to-module series connection, thus making the whole system run. As for the analysis results expected by professional engineers, professionals are also required to manually detect the recognition rate [18].By analyzing the database requirements and understanding the requirements of the storage structure on the system, the result record table can be drawn according to the results of data modeling.User Information Sheet: the user information table includes the user information using the system, the user’s ID, the user name, the user password, and the department where the corresponding user is located. It is shown in Table3.Table 3
User information table.
NumberingNameType of dataLengthDefaultsAllow nullsPrimary foreign keyRemark1userIdInt100NoPrimary keySelf-growth2userNameVARCHAR255NullNo3userPassVARCHAR255NullNo4DepartmentVARCHAR255NullYesPicture Information Record Sheet: it mainly records the information of the collected crack pictures. It includes the picture record number, the picture name, the specific shooting time of the picture, the location where the picture was taken, and the path where the picture is stored. Among them, the picture record number is used as the primary key, which can achieve self-growth and cannot be a null value. Of the parameters mentioned in the appeal, only the shooting location allows null values. Other parameters should be recorded accurately, as shown in Table4 [19].Table 4
Picture information record table.
NumberingNameType of dataLengthDefaultsAllow nullsPrimary foreign keyRemark1picSetIdInt100NoPrimary keySelf-growth2picSetNameVARCHAR255NullNo3picSetTimeDatetime80No4picSetLocationVARCHAR255NullYes5picSetPathVARCHAR255NullNoVideo Information Table: the video information table mainly records the video information with cracks. It mainly includes the video record number, video name, video shooting time, video shooting location, and video storage path. Among them, the video record number is used as the primary key, which can achieve self-growth and cannot be a null value. Among the parameters mentioned in the appeal, only the shooting location and the time of shooting the video allow null values. All other parameters must be recorded accurately. The design of the video information table is similar to the design structure of the picture information table. It is shown in Table5 [20].Table 5
Video information sheet.
NumberingNameType of dataLengthDefaultsAllow nullsPrimary foreign keyRemark1videoIdInt100NoPrimary keySelf-growth2videoNameVARCHAR255NullNO3videoTimeDatetime80Yes4videoLicationVARCHAR255NullYes5videoPathVARCHAR255NullNoThe crack result information detected by the system mainly includes the detection record number, video IDmmc1, detection source path, the level of the crack in this detection result, and the number of detection results in this detection. The test results are divided into three grades: A, B, and C grades, corresponding to grades 1, 2, and 3, respectively. These are the minor, moderate, and severe crack classes mentioned above in the system design. When multiple cracks are detected in a video, the detection record number will increase automatically. The corresponding video IDs are the same, and the specific crack level detected is also recorded according to the data. The number of detection results will also increase according to the number of cracks detected in the same video.After tracking the bridge crack target, it is necessary to quantitatively measure the crack size in order to evaluate the damage to the bridge. In general, the detected crack contour is circumscribed with a minimum rectangle, and the width of the minimum circular rectangle is used as the accounting standard for cracks. However, since this type of crack is more effective for straight lines, it is not suitable for more complex curved cracks. Therefore, the article provides a simpler and more accurate method to represent the size of cracks and their cyclic arcs. The idea of this study is to first calculate the area A and radius H of the associated crack region by automatically tracking the crack and then obtain the skeleton of the crack outline by selecting the box. It then counts the number of white pixels. TheL length of the crack is actually determined by the number of these white pixels. The article uses the above methods to test the size of transverse and vertical cracks, respectively. This result is shown in in Figure 7 [21].Figure 7
Horizontal and longitudinal crack size test.Error Analysis: According to the above data, the actual crack width is actually lower than the tested width value. The error is about 0.03 mm, and the calculated error is about 2.5%–16%. According to the allowable crack value of the bridge, it must be ≤ 0.30 mm, and both the test data and the actual crack width are greater than 0.30 mm. Therefore, both lateral and vertical cracks in the table need to be repaired. The error range of their detection is still within the allowable range, indicating that the detection is effective [22].Figure8 shows the deviations that affect the speed of data retrieval under any requirement. The speed of data retrieval is very important. Because it determines the storage capacity required by the image acquisition system, the most basic image acquisition system has a camera and several cards to record the camera’s output. For example, the inspection vehicle travels at a speed of 1.11 m/s, and the image acquisition system also meets the above requirements. The camera can process images and transfer them to memory at a transfer speed of 11,998,000 pixels per second. If it is determined that the pixel resolution is only 2 mm, the data retrieval rate is reduced to 4,999,500, that is, 40% of the original selection capacity. Of course, lower speed and better pixel resolution can also be set. The transmission rate of such data is greatly reduced, which in turn reduces the capacity of the image acquisition system. It can achieve the purpose and basic principle of identifying bridge defects by reducing the width of the covered deck or reducing the number of movable deck areas [23].Figure 8
Image acquisition rate and resolution.According to the experimental results, it can be seen that the neural network model based on the “block” image histogram (HNN structure) has 15 nodes in the hidden layer, and the learning efficiency is 0.1 or 0.01. Its network performance is optimal after 1500 cycles. The recognition rate in the testing process can reach 73.6% (the recognition results are shown in Figure9).Figure 9
Accuracy comparison based on HNN mode.Figure10 shows the classification accuracy on the test machine during the training of the network structure of the selected model scheme. Finally, the accuracy of the convolutional neural network model is higher than 95.2% and tends to be flat, and the accuracy is high. The abscissa represents the number of iterations. After iterating for 500 epochs, the three lines with different colors represent the increase with the number of iterations. The historical highest accuracy and average accuracy statistics are shown in Figure 10 [24].Figure 10
Convolutional neural network model accuracy.Based on the analysis of bridge surface defects based on wireless communication, the system combines the popular image recognition and convolutional neural network recognition technology. It can realize the management of video files or real-time video streams of drones, to identify the corresponding bridge cracks. In the whole system, the collected crack images are classified by the trained crack classifier, to judge the level of cracks. Of course, with the continuous collection of crack patterns, the recognition rate and classification accuracy of bridge cracks will gradually increase. Therefore, the existing system can also be modified accordingly at the later stage. The system can also be optimized and expanded in terms of external system connection and internal process refinement. In terms of external system docking, the GPS navigation system and live broadcast system can be introduced. In this way, it is more convenient for relevant professional engineering personnel to conduct real-time survey and judgment and to adopt selected engineering strategies. The internal process can be optimized and transformed on the basis of existing functions, and the most advanced and convenient information technology can be cited. It professionalizes, normalizes, and refines the functions of each module of the system. It makes the corresponding modules more in line with professional testing standards. This can provide users with more convenient and refined operation functions and can also add functions such as report making, mathematical statistics, and data visualization. It provides convenience for the relevant staff to the greatest extent and enables engineers in the professional field to have a better understanding of the overall situation of the bridge [25].


## 5. Discussion

This study studies the existing defect detection models for the practical application scenarios of the bridge defect intelligent detection system. It compares and analyzes the one-stage and two-stage detection models and finally chooses the two-stage Mask R-CNN defect detection model. It designed a new machine vision-based bridge concrete surface defect detection system. To apply Mask R-CNN to the detection of concrete bridges, a dataset containing 5861 actual concrete defect images was collected. Defect detection images vary widely in damage morphology. Most objects are relatively small, and the large-scale variation makes the defect detection task more difficult. The article flags four types of concrete defects: cracks, honeycomb, spalling, and exposed reinforcement. Each image is annotated with a border indicating the defect location and defect type. The article adopts a transfer learning method. It adjusts the weight of sample types through data augmentation and performs category imbalance correction. It uses a deep learning-based detection framework Mask R-CNN network to predict concrete defect locations and categories. It also segmented the bridge defects and imported the detected defect information (image name, defect type, defect area, confidence) into the text file. It then quantified and evaluated linear cracks, a defect. It calculates the length, maximum width, minimum width, and average width of the fracture by extracting the fracture skeleton. The experimental results show that the intelligent detection method of bridge defects based on Mask R-CNN constructed in this study can detect four kinds of defects on the bridge surface well. There is no need for manual feature extraction and calculation in the detection process. Through the defect intelligent detection system designed in the article, only one key is needed to trigger defect detection. The segmentation defect can be automatically identified and the area of the defect can be quantified [26].It designs a wireless communication front-end control system based on the application of the Internet of things by researching the communication technology of the Internet of things. The design is divided into three parts: the design of the front-end control interface of wireless communication for the Internet of things application, the design of the embedded wireless communication system based on the single-chip microcomputer, and the design of the application layer software interface based on the host computer. It analyzes SPI bus, UART bus, and I2C bus protocol. It also comprehensively considers the relevant parameters of the existing communication front end and designs a standard data communication interface based on the SPI bus. According to the data frame encapsulation format of several typical wireless communication protocols, this study designs a general data processing module, which can be widely used in different communication protocols. By studying the external signal trigger mechanism, a low-power external signal trigger module that can meet the needs of IoT applications is designed.The main work of the article includes the following: this study analyzes the HA standard and the ZCL-based Light Link standard and explains the reason for the proliferation of Zigbee application layer standards. It proposes the development of SAPI-based application layer standards and designs communication protocols. From the three aspects of Zigbee network gateway access, address translation, and sensor data, ZXBee protocol are designed first. It proposes the communication protocol frame format and explains the relevant details in the communication protocol. It then defines customized sensor data specifications and data transfer tools that can meet the requirements of different fields of Zigbee applications. It tests the ZXBee protocol and proposes the best solution. Finally, the optimized ZXBee protocol is tested, and the communication protocol is applied to the smart home system. The overall design idea of the system is expounded, and the operating environment and hardware deployment of the system are introduced. It describes the porting process of the ZXBee protocol. It shows the operation effect of the system after the protocol is successfully transplanted. Finally, the system test is carried out, and the test results verify the feasibility of the ZXBee protocol.


## 6. Conclusion

The article is to research and test the image recognition measurement algorithm for the cracks at the bottom of highway bridges, but there are still some deficiencies and errors. Online detection of cracks: since the whole project is still in the experimental stage, the main research object of this study is the cracks of highway bridges. The purpose of this study is to realize the research and realization of the image recognition and measurement algorithm for bridge cracks. The specific structure and control aspects are still in the prototype stage. At present, only offline detection of cracks can be realized, and online synchronous crack detection cannot be realized yet. The next stage of research may focus on this aspect to achieve. The problem with the denoising part of the crack: there is still some noise. So, more samples of the bottom of the bridge are needed. It is used to carry out different types of analysis in different time periods to find the regularity of these noise points. Crack width testing issues: since there are not many test samples, the generalizability of the final test results may not be determined. Therefore, it is necessary to carry out various tests with more samples of bridge bottom cracks to eliminate errors. Simultaneous detection of various road and bridge diseases: the diseases of highway bridges are not limited to the cracks at the bottom. There are many other aspects, such as cracks in the reinforcement of the bridge and cracks on both sides of the bridge. At present, only a single detection of the bottom crack is performed, and the evaluation of the safety level may not be very accurate. Therefore, the next step should be to be able to perform simultaneous detection of multiple diseases. Due to the relationship between experimental conditions and time, defect image processing is carried out on the basis of a single image. It is suggested that the next step is to use image stitching technology to stitch the continuously collected images to solve the technical problems that cannot be solved by one image. Due to the limitations of image acquisition equipment and processing techniques, the depth of the cracks cannot be measured. The depth of cracks in bridge surface defects is also an important detection index. According to the state of bridge surface defects, it is necessary to make a certain level of prediction of the health of the bridge. Therefore, a more complete database should be established to predict the actual health status of bridges.



---

*Source: 8728397-2022-04-11.xml*