# Colored 3D Path Extraction Based on Depth-RGB Sensor for Welding Robot Trajectory Generation
## 3 Experimental Setup
### Trajectory Extraction Based on Stereo Vision System Embedding Color Data  
Figure 5 shows the steps necessary for the definition of parameters and processing of the images that will carry out the extraction of the points corresponding to the weld bead. Next, the objective of each block was defined.  
Set up the data acquisition parameters: Image acquisition and processing was performed by Intel SDK [32], which, as an open source software program, has support for different programming languages, such as python, through the pyrealsense2 library, the official python wrapper. Since the implemented vision system has different sensors, both color and depth sensors were set to a resolution of 640 \(\times\) 480 pixels and a frame rate of 30 fps, with a depth accuracy between 0.1 to 1 mm.  
Acquire and align depth and color frame information: It is necessary to align the depth and color frames to make a 3D reconstruction faithful to the captured scene. This was achieved through the pyrealsense2 library [32], which has an algorithm that aligns the depth image with another image, in this case, the color image.  
Segment and remove the background data: Sometimes, we seek to process a region of interest (ROI); in this case, the ROI is defined by the distance at which the test object is located relative to the camera. Therefore, we first planned a filter using one of the device's own tools to acquire the images [32], where a depth clipping distance in which all information beyond our ROI was segmented and removed instead of using all the information in the scene.  
Point-cloud calculation from depth and color-aligned frames: The pyrealsense2 library [32] was used to calculate the point cloud since it has the intrinsic values of the stereo vision system and can perform the calculations for the point cloud acquisition, in addition to registering the color of the aligned frame.  
Color segmentation: This block represents the core of the proposed methodology that segment the welding area from the rest of the surface, in which the image was preprocessed considering the brightness of the scene to binarize the color image and look for the threshold at which a single frame of the point-cloud was vectorized to an XYZRGB format to using the Numpy and OpenCV library tools. However, to improve the selection of the points of interest, a change in the color space to hue saturation value (HSV) was used. The threshold was applied to the hue channel to find the color region, as well as to the saturation channel as a parameter for brightness.  
Trajectory planning: In order to calculate the trajectory from the color market segmented data in the previous module, following the methodology of [PERSON] et al. [15], a cubic B-spline interpolation algorithm was implemented to approximate the nonlinear dataset, the function was divided by knot points, and between the knots, the subset of data points a 5 th order polynomial curve was applied to satisfies a smoothness requirement to  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Tool Path** & **Tool-** & **Vc (m/min)** & **RPM** & **F (mm)** \\ \hline Facing & Facer 2.5\({}^{\circ}\) & 650 & 3500 & 300 \\ Pocketing & Flat 0.25\({}^{\circ}\) & 120 & 6000 & 7 \\ Drilling & Drill 0.203\({}^{\prime\prime}\) & 50 & 3048 & 6 \\ Tangent to curve & Flat 1.0\({}^{\circ}\) & 350 & 4500 & 40 \\ Wall machining & Flat 0.5\({}^{\circ}\) & 200 & 5000 & 30 \\ Z level & Flat 0.437\({}^{\circ}\) & 250 & 5500 & 47 \\ Z finishing & Ball 0.25\({}^{\circ}\) & 100 & 6000 & 32 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Machining parameters for the workpiece manufacture. Milling parameters: Vc = cutting speed, RPM = spindle revolution per minute; F = feed rate.  
Figure 5: The flowchart of path extraction.  
the target weld seam points. It was planned that the trajectory would be smooth enough to be applied directly to the robot through a transformation matrix referenced to the welding direction.