# Machine Learning-Based Garbage Detection and 3D Spatial Localization for Intelligent Robotic Grasp
## 1 Introduction  
The sorting and location of waste can increase waste disposal efficiency, whether the garbage is detected in outdoors or at a recycling station. It can be challenging to find trash in a cluttered environment. Several research studies have addressed this challenge by employing methodologies such as object detection or instance segmentation based on deep learning. These studies have also constructed diverse garbage datasets to overcome the associated difficulties. For example, [PERSON] et al. [1] trained an optimized YOLOv3 [2] using their own custom-made garbage dataset with four categories, resulting in garbage classification and localization with a \(mAP@50\) value of \(59.57\%\). With a custom-made bottle dataset based on transfer learning, [PERSON] et al. [3] fine-tuned the pre-trained Mask R-CNN [4] model, and the final model achieved a \(mAP\) of \(59.4\%\) on the test dataset. As collecting garbage images requires substantial effort, the datasets they utilize were collected from the internet and then manually labeled after pre-processing. For example,Carolis [1] utilized \"Google Images Download\" to download relevant images in bulk from Google based on keywords, then eliminate duplicate images before labeling the data with Labellmg. Similarly, [PERSON] [3] uses the Internet to download images of single and numerous bottles and combines them into a dataset. However, image annotation is a laborious and time-consuming procedure, meanwhile datasets produced contain only simple garbage targets. Moreover, the real-time performance of the models was not taken into account in these studies. In contrast, [PERSON] et al. [5] explicitly defined seven garbage categories and used two independently cascaded detector and classifier systems for garbage detection and classification. The approach achieved an average accuracy of 70% in garbage detection and approximately 75% in classification on the test dataset. To obtain the 3D spatial location of garbage, the authors of contribution [6] did the remarkable work of deploying YOLOv4 [7] with real-time detection performance on the NVIDIA Xavier NX, combined with a Pixhawk2 autopilot to control an unmanned aerial vehicle (UAV), and thus to locate garbage from images while flying at low altitude. This approach relied on the on-board sensors (Here2 GPS/altimeter) and camera imaging model to accurately translate the coordinates of the garbage into a global map, enabling automated path planning for subsequent pickups. However, the accuracy and real-time performance of YOLOv4 is limited in this approach. In order to improve the accuracy and efficiency of garbage detection and localization, this research focuses on detecting and calculating the spatial location of garbage instances under the scene point cloud reconstructed by the depth camera. Garbage detection is performed using the real-time instance segmentation model Yolactede to obtain the garbage mask from RGB images. The point cloud of garbage instances is then segmented from the overall point cloud using the mask. As shown in Figure 1a, the TACO-28 [8] training Yolactede families models, Mask R-CNN [4] and SegNet [9] achieved the highest _mPA_ of 37.4% for mask under real-time detection, and also achieved a 10% _mPA_ on the outdoor road garbage image data custom-collected with RealSenseD415, and then implemented point cloud segmentation of garbage instances. The assessed Oriented Bounding Boxes (OBB) is shown in Figure 1b. Figure 1: (**a**) On the TACO-28, there is a trade-off between speed and performance for various instance segmentation approaches. Where bbox _mAP_ and mask _mPA_ are represented by red and green shapes, respectively. (**b**) The RealSenseD415 camera captured and reconstructed the scene point cloud at (0, 0, 750, 0, 135, 90) in the world coordinate system, and the OBB of the garbage instances calculated after point cloud segmentation and DBSCAN clustering.