# Machine Learning-Based Garbage Detection and 3D Spatial Localization for Intelligent Robotic Grasp
## 3 Methodology
### Segmentation Models  
Mask R-CNN. It is two-stage detectors that uses FPN [16] and RPN [14] for instance segmentation and achieves exceptional performance. Mask R-CNN [4] consists of three parts: (1) a shared convolutional layer as the \"backbone\" for feature extraction. (2) The RPN is used to generate a large number of candidate regions to be identified from the feature map, and (3) the network head, which contains 3 branches for category classification, bbox regression, and mask prediction. The combination of transfer learning and pre-trained backbone parameters allows for the efficient application of Mask R-CNN to new garbage instance segmentation tasks. Yolact and YolactEdge. Pixels adjacent to each other in an image may belong to regions of the same instance, leading to spatial coherence in the corresponding masks. However, the spatial coherence can be lost when using fully connected layers, as they do not effectively preserve this information. As for one-stage detector, the fully connected layer is generally used as the output layer for classification and bbox regression. Two-stage detector, i.e., Mask R-CNN, uses ROI Align to preserve the spatial information of pixels,  
Figure 2: The algorithm schematic for garbage object detection and 3D spatial localization in this research. The segmentation model \(N\) predicts garbage’s classes and masks, which are then used to split the garbage instances point cloud from the entire scene point cloud. Then, using the DBSCAN [30] clustering algorithm, remove the interference point and calculate the OBB for garbage. and the mask prediction branch in Network head also needs to use FCN [15]. However, doing so increases the time consumption significantly. To address this issue, Yolact [18] introduces two parallel networks called Protonet and Prediction Head. Protonet generates prototype masks, while Prediction Head predicts mask coefficients. These components are then combined using a linear combination to produce the final masks. In addition, in [20], authors introduced a Fast NMS, which significantly improves speed compared to conventional NMS. All of these new structures make Yolact a high-performance and state-of-the-art single-stage detector for instance segmentation. Yolactedge [21] applies two main strategies to further accelerate the speed-accuracy trade-off. Firstly, different precision levels are assigned to the components of the deep-learning model. Yolact's backbone and FPN are converted to INT8 precision, while Protonet and Prediction Head are converted to FP16 precision, and ran them on Jetson AGX Xavier. The precision was tested on the MS COCO val2107 dataset [19], and the frame rate was improved by 21.3 with a slight reduction in \(mAP\). Secondly, FeatFlowNet is introduced to transform previous key frames into current non-key frames. By applying a partial feature transform design, the computational costs associated with backbone's convolutional layers are avoided, leading to additional speed improvements. Given that the Yolactedge families models achieve frame rates of up to 172 FPS and mAP values of 34.1% on MS COCO, their application to garbage instance segmentation in this study is highly practicable. SegNet. SegNet [9] is a deep convolutional encoder-decoder framework with a high level of detail for image sematic segmentation. The encoder-decoder structure has a total of 11 blocks. In the encoder part, SegNet can utilize the convolutional layers of VGG16 [31]. It consists of 5 convolutional blocks, each comprising 2 or 3 convolutional layers followed by a max pooling layer. The max pooling layer retains the index of the maximum feature value in a sparse matrix, preserving the position information for subsequent up sampling operations in the decoder. Accordingly, the decoder has 5 blocks corresponding to the encoder, each with an up sampling layer and 2 or 3 convolutional layers. After the output layer of the decoder is the SoftMax layer.