# Sewer Cleaning Robot: A Visually Assisted Cleaning Robot for Sewers
## 3 Software Algorithm Design
### Monocular Visual Localization  
The environment inside sewer is characterized by complexity, homogeneity, and dim lighting over medium and long distances. Additionally, pipe defects exhibit both visual diversity and similarity [28], which can lead to reduced accuracy in defect detection. To address the challenge of portable equipment at operation sites being unable to run real-time defect detection models with high accuracy, a lightweight sewer defect detection model, YOLOv5-Sewer, optimized for pipeline environments, is employed. This model  
Figure 5: Upper computer control interface. Figure 6: Partial images from the sewer pipe internal defect dataset. achieves a lightweight design that enhances its performance on low-resource devices while maintaining detection accuracy [25]. To enable efficient obstacle cleaning by SCR and to provide operators with accurate target location information, target localization plays a critical role in the overall cleaning operation process. Target localization involves utilizing the target feature information from the image plane to obtain the position in 3D space. Monocular vision localization is a technique that uses a single camera to estimate the position of objects in a 3D space. Compared with binocular vision systems, monocular vision systems require only one camera, offering advantages such as a simpler structure, a lower cost, greater flexibility, and convenience [29]. Additionally, monocular vision eliminates the need to consider factors like the optimal baseline distance between two cameras and blind zones, making it a more practical and versatile solution. These features provide monocular vision systems with broader application prospects in real-world scenarios. This paper uses a monocular visual localization method that utilizes edge detection within defect bounding boxes across two images to achieve precise target positioning. The workflow is depicted in Figure 7. First, the camera is calibrated to determine its intrinsic matrix \(K\) and distortion coefficient \(D\):  
\[K= \begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix}  \tag{5}\]  
\[D= \begin{bmatrix}k_{1}&k_{2}&p_{1}&p_{2}&k_{3}\end{bmatrix}  \tag{6}\]  
The intrinsic matrix \(K\) encodes the camera internal parameters, which include the focal lengths \(f_{x}\) and \(f_{y}\) along the \(x\) and \(y\) axes, and the coordinates of the principal point \((c_{x},c_{y})\), which is usually near the center of the image. The intrinsic matrix is essential for converting 3D points in the camera coordinate system into 2D pixel coordinates in the image. The distortion coefficients \(D\) account for lens distortion and include both radial and tangential distortion parameters. Specifically, \(k_{1}\), \(k_{2}\), and \(k_{3}\) are the radial distortion coefficients, while \(p_{1}\) and \(p_{2}\) are the tangential distortion coefficients. Radial distortion causes straight lines to appear curved, commonly observed as barrel or pincushion distortion, while tangential distortion arises due to lens assembly imperfections. These parameters are determined during the camera calibration process using a series of images of a known calibration pattern, such as a checkerboard. Using the OpenCV library cv2.calibrateCamera function, the intrinsic matrix \(K\) and the distortion coefficients \(D\) are computed simultaneously. Figure 7: Target localization flowchart. The camera is subsequently repositioned to capture two images of the scene from distinct viewpoints, ensuring sufficient parallax for accurate depth estimation. A defect detection model is utilized to process the images, identify the target defects, and generate bounding boxes around the areas of defect. Within these bounding boxes, the Canny edge detection algorithm is applied to enhance the edges and extract the structural features of the defect. Canny edge detection operates by analyzing the image gradients to identify areas of rapid intensity changes, which typically correspond to edges.