title: AI Based Crack Recording and Repair using Autonomous Robot and Bio-Concrete Agent
Creators: name: Xia, Zhongxin
Publisher: name: UNSPECIFIED
Year: 2024

description: Concrete is one of the most widely used building materials globally. Concrete cracks, a common occurrence, can result from numerous factors across the life cycle stages of concrete. With time, unrepaired narrow cracks may enlarge and get deeper, undermining the structure. But manual detection and repair of cracks by large concrete structures are both time-consuming and labor-intensive. Especially for narrow cracks, which are easily overlooked and cumbersome to repair using traditional methods. Moreover, the repair agents employed in traditional techniques are not environmentally friendly. A relatively new method for repairing narrow cracks is the use of the environmentally friendly bio-concrete agent Basilisk ER7, which is easy to use and can seal concrete cracks up to 0.6 mm. However, Basilisk ER7 is generally applied by spraying the agent onto the entire concrete surface to ensures the coverage of narrow cracks, it can be quite wasteful of the repair agent. If workers manually inspects for cracks first and then applies the Basilisk ER7 to each crack individually, it could indeed save on the amount of repair agent. But this process requires significant manual effort and attention to accurately identify and treat each crack.
This doctoral dissertation demonstrates the development of a compact and low-cost robot named “CrackRepairBot”, specifically designed for repairing narrow cracks drying- and shrinkage cracking and wear on flat concrete surfaces using repair agent Basilisk ER7. Assisted by computer vision techniques such as image segmentation and image processing via OpenCV tool, the robot can swiftly identify cracks, distinguish between narrow and wide ones, and then apply the repair agent exclusively to the narrow cracks. Compared to the common application method using entire area spraying, this feature helps to save the repair agent. Since narrow cracks and wide cracks often occur together, and the repair agent Basilisk ER7 is only suitable for narrow cracks, this work has also developed a automatic recording and visualization system for wide cracks to facilitate engineers to take other methods to repair these wide cracks in the future. 
The most significant feature of the robotic system developed in this work is the distribution of various functional modules across different edge computing modules, which achieve communication through wired connections. The high-performance single-board computer, acting as the “brain” of the entire system, is primarily used for deploying machine learning model and achieving real-time inference of this model. The inference results from the machine learning model are sent to the robot’s existing single-board computer and microcontroller, which then control the robot’s movements, crack recording and spraying system. The algorithms used in this robot system are based on open-source frameworks. Such a system design ensures that the entire system is not overly dependent on hardware from a specific brand, facilitating the replacement of hardware in the future based on the needs of new application scenarios. In other words, this system design enhances the system’s flexibility and adaptability, allowing for easier upgrades and integration of new edge computing devices as needed without significantly affecting other devices within the network. Moreover, since the high-performance single-board computer takes on the vast majority of the system’s computational demands, the robot and microcontroller can be selected for cost-effective, low-computational versions, thereby reducing the overall development cost of the robot. According to the system design, the robot used in this work is the relatively inexpensive and open-source JetAuto Pro. Jetson Orin Nano uses the results deduced from the image segmentation algorithm YOLOv8 to control the Arduino and Jetson Nano. When the Jetson Orin Nano detects a concrete crack up 0.6 mm wide via 10x zoomable camera, it will send commend to Arduino to open the normally closed solenoid water valve and allow the repair agent to spray onto the crack from a pressurizable water bottle. If the crack’s width exceeds 0.6 mm, the Jetson Orin Nano will issue commands to Jetson Nano via an Ethernet cable, and Jetson Nano will immediately use the camera to take a picture of the wide crack and obtain coordinates from the map through the Robot Operating System (ROS), and then store the coordinates and picture in MongoDB Atlas. The locations of these cracks and their corresponding pictures will be visualized in a Building Information Modeling (BIM) and Geographic Information Systems (GIS) integrated web application based on 3D geospatial platform Cesium. 
To achieve autonomous navigation, the maps used by robot were converted from IFC (Industry Foundation Classes) model and photo taken by drone. Then, the robot will utilize the LiDAR and the multiple target points set by Python script to comprehensively cover and check the entire ground. The target points set by Python scripts can also be used for route planning during the curing phase of repair agent, allowing the robot to retrace its previous path to spray water for humidity maintenance of cracks. 
The robot initially performed specific functional tests on cracked concrete samples and indoor floor. Subsequently, comprehensive test was conducted on the concrete surface of a skate park, where all functionalities of the robot operated effectively. However, the best-trained YOLOv8 model occasionally misclassifies certain objects as cracks, and its performance still requires further improvement. This work demonstrates that intelligent robot can assist workers in completing time-consuming and labor-intensive task such as narrow crack repair and wider crack recording, thereby reducing labor costs in construction project. It can be seen that the development of robots in the field of civil engineering has significant application potential. In the future, this work aims to deploy the developed system on drones for crack detection and repair in high-rise buildings. To overcome the limitations in the generalization ability of the YOLOv8 model, more advanced algorithms are planned for use to achieve accurate crack detection.