# Filtering Organized 3D Point Clouds for Bin Picking Applications
## 2 Related Work on Filtering 3D Point Clouds  
The quality of 3D point clouds depends greatly on the sensor hardware, software settings, and the choice of processing algorithms. For example, the accuracy of the camera calibration has an important effect on object localization [16]. Most commercial vision systems offer the user a set of adjustable parameters (such as lens aperture, exposure time, and sensor gain) and a list of built-in filtering algorithms. In this paper, we focus on the post-processing algorithms, such as those reviewed in [17]. These algorithms try to address two deficiencies in the acquired 3D point clouds: removing outliers and reducing noise. The distinction between these is subject to interpretation, but for point clouds obtained from objects for which computer-aided design (CAD) models are known, the distinction can be based on the distance between a datapoint and the closest vertex, or face, of the _correctly_ oriented CAD model [18]. For such cases, we consider a 3D point as noisy when its distance to the CAD model is smaller than \(3\times Err\), where \(Err\) is the residual error from fitting the CAD model to the 3D point cloud using, for example, the iterative closest point (ICP) or any other minimization technique. For such defined noisy points, the outlier may be defined as a point with a distance to the CAD model larger than \(3\times Err\). This paper focuses on detecting and filtering these outliers. The most commonly used filtering techniques belong to the family of statistical outlier removal (SOR) algorithms [19]. They share a similar strategy: neighboring points around each datapoint are selected, and a statistical parameter, such as the mean or standard deviation of the distances between the selected point and its nearest neighbors, is calculated. After the statistical parameter is calculated for each datapoint, a thresholding technique is applied to reject all datapoints that have their parameters above a certain threshold. Different implementations of SOR use different approaches to build a neighborhood or use different statistical parameters. For example, the \(L_{0}\) norm was used in [20], while box plots and quartiles were used to throw outliers in scans of a building in [21]. The distance of a point to the locally fitted plane and the robust Mahalanobis distance were used in [22]. The voxelization of 3D point clouds was used to build a neighborhood in [23]. Multiple scans of the same object acquired from different viewpoints were used to get a weighted average from distances between a checked point and intersecting points on nearby local surfaces [24]. Fast cluster SOR was designed specifically to speed up filtering in large datasets acquired by a sensor mounted on an unmanned ground vehicle (UGV) [25]. A median filter applied to a depth image acquired by a lidar in rainy or snowy weather conditions was demonstrated in [26]. In [27], outlier removal was performed in two steps: first, isolated clusters were removed based on local density and then, using local projection techniques, the remaining non-isolated outliers were removed. Another class of filtering techniques is available when a red-green-blue depth (RGB-D) camera is used. Such sensors provide not only the organized 3D point cloud but also the depth and RGB images. In [28], the filtering of 3D point clouds was performed by converting the RGB image to a hue saturation value (HSV) image and segmenting the grayscale V image using the Otsu method [29]: the calculated binary mask was then used to extract filtered 3D points. Another approach to filtering 3D point clouds is derived from machine or deep learning (ML/DL) techniques. The goal of these techniques is to correctly classify inliers and outliers from the acquired data. In [30], the 3D data were first preprocessed with an SOR-type procedure and then points marked as outliers were reclassified using random forest (RF) and support vector machine (SVM) techniques.