# Painting Path Planning for a Painting Robot with a RealSense Depth Sensor
## 5 Experiments and Results  
In this part, the experiments and their outcomes are explained in detail. The experiments were conducted with a RealSense D435 depth camera. We chose the D435 because this camera has a wider field of view [7; 8; 46; 47]. This is an important point because the goal is to cover a wide surface during the painting process. The camera and the robot's tool center point were about 70 cm from the wall. This distance is reported as being the best for the recording the depth via D415 and D435 RealSense cameras in the literature [1; 7; 8; 46; 47]. ROS-based robot navigation is the robotic basis for this experiment and will be explained only in brief in the next paragraph, since a robotics explanation is not within the scope of this paper. The communication between the motor drivers and the control software is achieved via controller area network open (CANopen) protocol. A controller area network (CAN) is a robust vehicle bus standard developed to allow devices and microm controllers to communicate with each other's applications without a host computer. The communication initialization process is composed of two parts. First, the socketCAN driver is executed, which loads the CAN drivers and networking stacks. Then, the CANopen control package is executed, which establishes the connection between the ROS and motor drivers via the CANopen interface. This package is based on the CiA402 standard (CAN in automation), uses the robot description file (URDF file), and defines the high-level controllers that drive the motors in a closed loop (joint trajectory controller and joint state controller). The motion planning of the robot is executed with the MoveIt software package. This package considers the structure of the robot and generates both the achievable poses and collision configurations. Moreover, MoveIt is used to (i) define the group of joints to be used in the painting process, (ii) save different joint combinations (e.g., home position), and (iii) incorporate the stream of the RGB-D camera in the motion planning process. A custom-made kinematics package was defined for forward and inverse kinematics-based manipulation, which is the base of the aforementioned MoveIt package. The RGB-D camera is started with the official realsense2_camera ROS package, which loads the driver and makes the real-time point cloud measurements available in the ROS environment. A static transformation is applied between the camera and robot coordinate systems. The RTAB-Map package is executed to merge the real-time camera stream. This algorithm is executed with default parameters and produces the concatenated point cloud data structure, which describes the environment in front of the robot. Our custom-made graphical user interface is used to control the whole painting process. The first step was to form a point cloud using the RTAB mapping procedure [23; 24; 25; 26; 27]. During the mapping process, the robot arm tracked the surface with the camera and slowly formed the point cloud. Then, voxel grid filtering was applied to merge overlapping surfaces in point clouds [23; 24; 25; 26; 27]. Figure 3a,b shows the procedure used to merge the point clouds. Finally, the resulting point cloud was observed to be suitable for further processing. Figure 4 shows the final merged point cloud of the wall surface with the obstacles (the blue colored stick, the white bottle, and the red colored boxes). The next step was plane detection, specifically the detection and extraction of the wall area to be painted. The goal of the wall detection process was to avoid the barriers on the wall that should not be colored. The plane extraction process was performed by executing the RANSAC algorithm, preceded by statistical filtering. The statistical filtering [30] removed unnecessary, noisy gross outlier data from the point cloud. This operation is important because the point cloud is cleaned of non-essential outliers, meaning it is more efficient and able to execute the RANSAC algorithm quicker [28; 29].