# Machine Learning-Based Garbage Detection and 3D Spatial Localization for Intelligent Robotic Grasp
## 4 Experiments and Results
### Implementation Details  
Dataset processing. The TACO dataset currently contains 1500 high-definition rubbish images taken with mobile phones, each with instance segmentation information saved as a json file in COCO format, and randomly divided into training, test, and validation sets in the proportion of 80%, 15%, and 5%, respectively. Due to limited computing resources (an RTX 2080 Ti GPU with 11 GB video memory), the input images needed to be scaled to dimensions of either 550 \(\times\) 550 \(\times\) 3 or 512 \(\times\) 512 \(\times\) 3, and since TACO has 60 categories(28 super categories), the code will apply a large amount of memory space when generating the training data. Therefore, it is necessary to keep the aspect-ratio scaling for the 1500 high-resolution images in TACO. Specifically, the width of all images was scaled to 640, the height was scaled by a ratio, and the instance labels in the image segmentation points and bounding boxes (x, y, w, h) were also scaled accordingly, with the json file being updated accordingly. On the other hand, this paper only classifies the garbage according to the 28 super categories defined by TACO, so we also need to process the defined categories in the json file as 28 super categories, and change the category id number of each instance so that it maps to the corresponding category. After doing these processes, this study named the dataset TACO-28. Finally, a software application was developed using the Intel RealSense SDK to capture real-time RGB and Depth streams of rubbish in various environments. The depth stream data was used to reconstruct the scene point cloud and calculate the OBB of the rubbish instances. The garbage instances in the RGB images were then manually labelled to test the accuracy of the segmentation model.  
Training details. The transfer learning scheme is applied to the fast training and optimization of the model. For the Yolactedge models, we conducted experiments using different backbones, including Res50, Res101 [32], and MobileNetV2 [33]. To initialize the model weights, we utilized the pretrained model parameters from ImageNet. and we fine-tuned these weights using the TACO [8] dataset based on the open source code of [PERSON] [21]. Learning-rate, momentum, decay and iteration are set to 0.003, 0.9, 0.0005, 80,000 respectively. For Mask R-CNN and SegNet, backbone uses VGG16 and its weights are initialized with ImageNet-pretrained weights.  
Model evaluation metrics. The evaluation of the instance segmentation model includes metrics for object detection precision, mask segmentation precision, as well as real-time performance and parametric quantities. The positioning precision of the model for each category is represented by the \(AP_{i}\) value, while the positioning accuracy for all categories in the dataset is represented by the \(mAP\) = _sum(APi)/n_ value. Comparatively, the mask segmentation precision of the instance segmentation model is evaluated by _mIoL_, \(mAP\), _PA_, etc. Since the TACO dataset contains 60 categories, some of which have insignificant feature differences or a small number of instances, it can be challenging for the instance segmentation model to achieve high precision in category detection, resulting in low values for the calculated evaluation metrics, and this is why we used TACO-28. In this study, the main purpose is the inference speed of the model and the accuracy of the mask segmentation. The higher the accuracy of the mask, the more accurate the segmented point cloud will be, which will simplify the computation of the point cloud OBB. Therefore, the main evaluation metrics used in this study are _mPA_ (mean Pixel Accuracy) and frame rate, as they provide insights into the accuracy and speed of the model's performance.