# Recognition of Manual Welding Positions from Depth Hole Image Remotely Sensed by RGB-D Camera
## 3 Depth Image Analysis and Body Joint Identification  
### Depth Hole Detection  
An image including depth information of the welder's body must be captured with an RGB-D camera to recognize a welding position by extracting body joints. As described earlier, when a welding operation is taken with an RGB-D camera, bright light is generated due to the welding arc, and as a result, a wide range of depth holes are generated. We are interested in the proportion of manual welding's vertical and horizontal welding positions that gradually increases with each assembly step during the assembly process. Because most of the welding for these two positions is done in front of the welder's own body, the depth hole region caused by the arc occurs around the body. As a result, the distinction between the body and the arc is blurred, making it impossible to recognize the exact joint positions, as shown in Figure 5. Therefore, to capture motion for those welding positions, a process of removing depth holes is required. Before removing the depth hole, it is necessary to identify the depth hole that overlaps the body in the depth image. Since welding is carried out in front of the welder for safety, if the scene is taken with an RGB-D camera, the welder's body appears as a shape surrounding the depth hole, as shown in Figure 6. Therefore, we use this characteristic to detect the depth hole. Figure 5: Motion capture from depth images: (**a**) correct without arc, and (**b**) incorrect with depth holes by arc. To isolate the depth hole from the body, the depth hole region must be searched. Because the depth hole is recorded as an undefined region in the depth image, the depth hole region can be specified using image pixel analysis. To improve search efficiency, depth holes of a certain size or less are excluded in this case. We use the fact that the rate of change of brightness occurs in the body's outline to isolate a body part from the surrounding environment. Canny edge detection [58], which is effective for detecting the rate of change of brightness, is utilized. The result of the searched depth hole and the object separated from the background are visualized in Figure 7. A plurality of regions recognized as depth holes exists around the welder. It is necessary to find the depth hole that exists inside the welder's body, among many depth holes found. In this study, the centroid of an object is utilized by following the rule that the body surrounds the depth hole. The proposed assumption is that the contour of the depth hole, if any, is in most cases tangent to or inside the body contour. In this case, the rectangle circumscribing the depth hole will cover more than half of the body contour area. If the depth hole inside the rectangle circumscribes the object's centroid estimated as the body contour, it becomes a depth hole overlapping the body. Otherwise, it is treated as noise. Due to the nature of the arc, an exception may occur where a part of the depth hole contour is outside the body contour. Even in this case, the exceptional effect is greatly reduced and ignored with the proposed assumption. Figure 6: Depth images for two welding positions: (**a**) vertical, and (**b**) horizontal. Figure 7: Separation of depth holes and objects from background: (**a**) depth holes detected, (**b**) edge detection by Canny edge algorithm, and (**c**) segmentation of objects. The centroid of the object contour is simply obtained from the spatial moment Equation (1):  
\[m_{ij}=\sum_{x,\,y}(array(x,\,y)\times x^{i}y^{i}) \tag{1}\]  
\[\overline{x}=\frac{m_{10}}{m_{00}},\overline{y}=\frac{m_{01}}{m_{00}}\]  
where \(m_{ij}\) denotes the general form of the spatial moment, \(m_{00}\) the zero-order moment of the image, \(m_{10}\) the first-order moment of the image concerning the \(x\)-axis, and \(m_{01}\) the first-order moment of the image concerning the \(y\)-axis.