# RoboTwin Metaverse Platform for Robotic Random Bin Picking
## 2 Background
To address this limitation, an artificial intelligence system was introduced by the authors in reference [5]. This system aims to facilitate 2D object segmentation of workpieces, thereby enabling precise pixel-wise alignment of the bounding box. which could be used to determine whether the workpiece was pickable or not. Based on the segmentation, the model from [5] was able to estimate the normal direction of the targeted workpiece's surfaces in guiding the robot to conduct picking operations. Since the system does not require CAD models for matching, and high-resolution 3D cameras are not required, it significantly reduces the cost of common robotic random bin-picking systems. Additionally, with the use of AI to change and train the dataset, the aforementioned system can be easily transferred to detect another workpiece, which facilitates the need for rapid line change in factories. [PERSON] et al. [6] tackled the long-standing and challenging inverse projection problem in computer vision by proposing the depth-aware video panoptic segmentation method. This method combines monocular depth estimation and video panoptic segmentation as a step toward solving the inverse projection problem. The method demonstrated that instance segmentation, although challenging, is useful in image processing. In another study [7], [PERSON] et al. reported that a low-resolution grid is not sufficient to capture details, while a high-resolution grid would significantly increase the training. Discrete cosine transform (DCT) was proposed to encode the high-resolution binary grid mask into a compact vector. The DCT was demonstrated to be able to perform instance segmentation with high resolution and to achieve an AP50 score of 55.4, a higher score than with traditional Mask R-CNN. Few-shot instance segmentation methods are promising in the case when labeled training data for novel classes are scarce. Using stored embedding vectors rather than images can effectively solve memory overhead problems [8]. The results are promising and can narrow the gap between non-incremental and incremental few-shot instance segmentation. The work presented by [PERSON] et al. in [9] defines a new scale-aware search space where both image- and box-level augmentations are designed to maintain the scale invariance, resulting in higher AP scores for different models. Understanding the scene is a complex yet important task for a vision-based self-driving system. Researchers in [10] proposed a part-aware panoptic segmentation method that combines instance and semantic segmentation methods. In deep learning, collecting a vast quantity of data is time-consuming and labor-intensive. To alleviate this issue, [PERSON] and [PERSON] [11] suggested utilizing a 3D environment simulator to provide photorealistic simulations by using a view synthesis module to support a flexible configuration of multimodal sensors. Their results showed that vision-based algorithms developed in the simulation can be transferred to real physical platforms without domain adaptation. In machine learning, if the labeled datasets are inaccurate, the annotations generated by the ensemble may lead to performance degradation of the trained model. To address this, [PERSON] et al. [12] suggested a trained model that predicts the quality of the annotation from the degree of consensus between ensemble models. The results obtained in this study indicate that the process necessitates only 30% of the original data, and the rest can be replaced with automatically annotated data. The research from [4; 5; 6; 7; 8; 9; 10; 11; 12] offers valuable insights and contributions regarding the utilization of AI-based vision systems for object detection without the need for reprogramming. Instead, these studies emphasize the retraining of the AI model using different datasets to achieve robust detection capabilities.