# Towards Autonomous Bridge Inspection: Sensor Mounting Using Aerial Manipulators
## 6 Blob Detection  
This section presents the methods we propose to detect the hardener blob position and orientation. A modular object detection framework, as shown in Figure 6, is designed to ensure a reliable blob pose detection. Since the detection is to be done on board the UAVs, RGB-D cameras are selected. Therefore, the inputs to the framework are images and organized point clouds obtained from the visual sensor. The remainder of this section introduces the individual components of the framework and adds implementation details where necessary. The sensor message synchronizer is responsible for the time-based synchronization of the given sensor message streams. In the case of blob detection, a module that synchronizes images and organized pointclouds from an RGB-D camera is derived. This is necessary since the algorithm detects the blob in both 2D image space and 3D point clouds, which are not necessarily sampled simultaneously. The underlying implementation uses ROS libraries to synchronize messages with an approximate time policy. An object detector attempts to find a set of object poses using synchronized sensor data. The module used in this paper detects blob poses and is implemented in the following way. First, all the blob positions and radii are found in the image frame using the standard blob detection functionality found in the OpenCV libraries. Second, the depth information corresponding to the detected blobs is isolated from the organized point cloud. Finally, blob positions are calculated as centroids of the corresponding depth positions, while the orientation is obtained through the random sample consensus (RANSAC) algorithm from the Point Cloud Library (PCL). The remaining framework components are independent from synchronizer and detector modules. The pose tracker is used to track the obtained object's poses through multiple frames based on the closest Euclidean distance criterion. This component solves the issue of multiple objects being visible, as it always outputs the pose of the currently tracked object. Moreover, it increases the robustness of the system since it remembers the object poses for a certain number of frames, which allows some leniency with the detector. The goal of the world transformation component is to transform the tracked pose from the sensor to the world frame using the estimated odometry from an external source  
Figure 6: The pipeline for the modular object detection framework. Inputs to the system are an arbitrary number of sensor messages (image, depth, point cloud, etc.) along with sensor–world frame transformations. Output is the detected blob pose in the world frame. The synchronizer and detector are modular components, while the linear Kalman filter, world transformation and pose tracker stay invariant. that any UAV should have access to. Additionally, since the blob poses are to be sent as references to the trajectory planner, it is important to correctly compute the blob orientation. Since the blob is a flat surface, there are two equally correct possible orientations that can be detected. Therefore, the blob orientation is chosen as follows:  
\[R_{\text{blob}}= \begin{cases}R_{\text{blob}}&\text{if }\mathbf{r}_{1B}\cdot \mathbf{r}_{\text{blob}}\geq 0\\ R_{\text{blob}}\cdot R_{180}&\text{otherwise}\end{cases} , \tag{22}\]  
where \(\mathbf{r}_{1B}\) is the heading component of the UAV rotation matrix expressed in world coordinates \(R_{B}= \begin{bmatrix}\mathbf{r}_{1B}&\mathbf{r}_{2B}&\mathbf{r}_{3B}\end{bmatrix} \) and \(\mathbf{r}_{\text{blob}}\) is the heading component of the blob rotation matrix expressed in world coordinates \(R_{\text{blob}}= \begin{bmatrix}\mathbf{r}_{\text{blob}}&\mathbf{r}_{\text{ 2 blob}}&\mathbf{r}_{\text{3 blob}}\end{bmatrix} \) and \(R_{180}=diag(-1,-1,1)\). Finally, a linear Kalman filter with a constant velocity model is used to further increase the robustness of the system and provide smoother blob position estimates.