# Recognition of Manual Welding Positions from Depth Hole Image Remotely Sensed by RGB-D Camera
## 2 Previous Research
### Depth Hole Filling  
The ToF sensor creates the depth image by measuring the round-trip time of the infrared signal emitted by the IR projector. This method calculates depth by emitting infrared light, thus it is greatly influenced by weather factors such as light, rain, fog, and environmental factors such as the strength of lighting scattering. As a result, the depth measured by the ToF sensor is inaccurate [37] in the aforementioned environment. It has also been reported that the same type of LiDAR sensor reduces depth calculation accuracy due to ambient lighting and disturbance factors [38; 39]. As a result, obtaining a depth image using the above sensors is limited when a light source is stronger than necessary.  
Various welding methods used in assembly sites include arc, gas, laser, and explosion welding. CO\({}_{2}\) welding, which is mainly used in shipbuilding shops, inevitably generates an arc that emits bright light, as shown in Figure 3.  
Figure 3: Arc generated by CO\({}_{2}\) gas arc welding.  
RGB-D cameras have a limitation in being greatly affected by bright sources such as welding arcs. Therefore, when shooting a welding worker, an arc causes the RGB-D camera to fail to measure the reflected signal, resulting in a region where depth information is lost in the depth image. This region is classified as undefined as the area where the RGB-D camera cannot define depth. For example, the undefined region is defined as a depth hole in the depth image and is expressed in black as shown in the right scene of Figure 4.  
When the welding arc and the welder's body overlap, it is difficult to capture motion accurately because the depth hole obscures the body in the depth image captured by an RGB-D camera. To recognize the body motion or working posture, we must look for a depth hole caused by the welding arc and remove it if it is found.  
In previous studies, image inpainting techniques [40,41] and image filtering methods [42, 43, 44] were used to care for depth holes. Image filtering is an operation that creates a new image by using a square matrix and a convolution technique. A guidance image converted from an RGB image to grayscale is required to remove the depth hole using image filtering. However, a guidance image without a welding arc is unavailable in advance for a welding operation in progress, so utilizing the image filtering is not practical.  
Image inpainting is, in theory, a technique for repairing damaged areas of an image. Normally, the depth image's depth holes are searched for and removed. However, in this study, where motion capture accuracy is critical, high accuracy can be obtained even when only the depth hole caused by the interference element is removed. Furthermore, while the depth hole filling speed is adequate for a small amount of image data, removing all of the depth holes for each frame of moving images takes a significant amount of computing time. As a result, data processing speed should be prioritized.  
In this study, an image inpainting technique is adopted as a method to remove depth holes. Furthermore, we present a method to search only the depth hole that overlaps the body among the depth holes generated in the depth image.