# A 3D Vision-Based Weld Seam Extraction Method for Arbitrarily Placed Spatial Planar Weldments
## 1 Introduction  
The introduction of visual sensors enables the industrial robot as an actuator to perceive the positional information of the surrounding environment. Therefore, vision-based automatic weld seam extraction (WSE) and weld seam tracking have become an indispensable part of robotic welding, which requires the weldment's posture and position to be known to the robot. According to our research on actual production, many welding applications must handle randomly placed weldments with uncertain position information. For these applications, the additional human labor and time cost brought by fixing the weldments with satisfactory accuracy for WSE are unacceptable in terms of efficiency and economy, especially in welding applications with multi-variety and small batches, as exemplified by the samples in Section 4.2. There are two major steps to achieve this goal. The first step is model reconstruction, from which a 2D or 3D model representing the geometric information of the weldment is obtained; the second step is to perform WSE on the model. Model reconstruction can be straightforward for simple weldments. A common approach is capturing a 2D image using a laser profile sensor and extracting the weld seams through pixel analysis [1; 2; 3; 4; 5]. For applications with a more complex weld seam or multiple weld seams to be recognized, a monocular or binocular structured light 3D camera that outputs point clouds is more commonly used [6; 7; 8; 9]. The problem is that when the weldment is too large for the camera's FOV or when there are occlusions between multiple welds, a single frame of point cloud cannot fully express all the necessary geometric features. In this regard, the studies mentioned above focus on the WSE of a single weld seam orseveral weld seams in a single frame of a point cloud, referenced as local weld seams in this paper. There are also scholars who work on solving this practical problem and obtaining global weld seams. To perform 3D reconstruction on a large-size tube sheet, [PERSON] et al. (2018) collect 2D images and laser rangefinder data simultaneously through the sensors installed on a robot. They perform SIFT feature matching on the 2D images to obtain the positional relationship between different frames of images, which, combined with the depth data, leads to a complete point cloud. [PERSON] et al. (2018) first convert the 3D point cloud into a 2D grayscale image and use an edge detection algorithm to detect the pixel set corresponding to the weld seam from the 2D image. The weld seams are then fitted with spline curves. The position errors are within \(\pm\)0.3 mm, and the standard deviations are less than 0.4 mm. Considering the need to extract geometric features from weldments of complex spatial structures, in this paper, we arbitrarily mount the assembled weldments (One of the factors influencing the quality of welded joints is the appropriate sequence and order of assembly ([PERSON] et al., 2018)) on a one-axis turntable. By rotating a weldment along with the turntable and capturing multiple frames of point clouds of the weldment using a binocular structured light 3D camera, a point cloud that represents the complete geometric features of the weldment is created through point cloud registration. WSE can be divided into two types according to the vision sensor used. For research with 2D vision employed, weld seams are mostly obtained by running pixel analysis on 2D images scanned with a laser profile sensor. This approach is particularly suitable for weldments assembled by two workpieces since their geometric features can be expressed in 2D images with decent accuracy. As an example, [PERSON], [PERSON], [PERSON] and [PERSON] (2018) perform WSE on V-shaped butt joints by extracting pixels that represent the center line of a weld seam, with an average positional error of 0.147 mm.