Selfsupervised Transfer Learning for Instance Segmentation through Physical Interaction Instance segmentation of unknown objects from images is regarded as relevant for several robot skills including grasping, tracking and object sorting. Recent results from computer vision have shown that large handlabeled datasets enable high segmentation performance. To overcome the timeconsuming process of manually labeling data for new environments, we present a transfer learning approach for robots that learn to segment objects by interacting with their environment in a selfsupervised manner. Our robot pushes unknown objects on a table and uses information from optical flow to create training labels given by object masks. To achieve this, we finetune an existing DeepMask instance segmentation network on the selflabeled training data acquired by the robot. We evaluate our trained network (SelfDeepMask) on a set of real images showing challenging and cluttered scenes with novel objects. Here, SelfDeepMask outperforms the DeepMask network trained on the COCO dataset by 8.6% in average precision.

Joint Learning of Instance and Semantic Segmentation for Robotic PickandPlace with Heavy Occlusions in Clutter We present joint learning of instance and semantic segmentation for visible and occluded region masks. Sharing the feature extractor with instance occlusion segmentation, we introduce semantic occlusion segmentation into the instance segmentation model. This joint learning fuses the instance and imagelevel reasoning of the mask prediction on the different segmentation tasks, which was missing in the previous work of learning instance segmentation only (instanceonly). In the experiments, we evaluated the proposed joint learning comparing the instanceonly learning on the test dataset. We also applied the joint learning model to 2 different types of robotic pickandplace tasks (random and target picking) and evaluated its effectiveness to achieve realworld robotic tasks.