Moving object segmentation is a crucial task for autonomous vehicles as it can be used to segment objects in a class agnostic manner based on their motion cues. It enables the detection of unknown objects during training (e.g., moose or a construction truck) based on their motion  . Although pixel-wise motion segmentation has been studied in autonomous driving literature, it is not dealt with on the instance level, which would help separate connected segments of moving objects leading to better trajectory planning. Other generic video object segmentation tasks have dealt with instance-wise motion segmentation but on much simpler setting and have not dealt with the multi-task learning problem for both semantic and class agnostic instance segmentation. In this paper, we propose a motion-based instance segmentation task and provide a new annotated dataset based on KITTIMoSeg , which will be released publicly. Our datasetprovide extra class annotations which is crucial for studying class agnostic  segmentation. We further propose an efficient multi-task learning approach that learns an extra class agnostic instance segmentation head through sharing the prototype generation network with the semantic head . The model then learns separate prototype coefficients within the class agnostic and semantic heads  . To obtain real-time performance, we study different efficient encoders and obtain 39 fps on a Titan Xp GPU using MobileNetV2 with an improvement of 10\% mAP relative to the baseline. Our model outperforms state of the art motion segmentation methods with 3.3\% improvement. We summarize our work in a short video with qualitative results at URL
Moving object segmentation is a crucial task for autonomous vehicles as it can be used to segment objects in a class agnostic manner based on their motion cues. It enables the detection of unseen objects during training (e.g., moose or a construction truck) based on their motion and independent of their appearance . Although pixel-wise motion segmentation has been studied in autonomous driving literature, it has been rarely addressed at the instance level, which would help separate connected segments of moving objects leading to better trajectory planning. As the main issue is the lack of large public datasets, we create a new InstanceMotSeg dataset comprising of 12.9K samples improving upon our KITTIMoSeg dataset. In addition to providing instance level annotations, we have added 4 additional classes which is crucial for studying class agnostic motion segmentation. We adapt YOLACT and implement a motion-based class agnostic instance segmentation model which would act as a baseline for the dataset. We also extend it to an efficient multi-task model which additionally provides semantic instance segmentation sharing the encoder . The model then learns separate prototype coefficients within the class agnostic and semantic heads providing two independent paths of object detection for redundant safety . To obtain real-time performance, we study different efficient encoders and obtain 39 fps on a Titan Xp GPU using MobileNetV2 with an improvement of 10\% mAP relative to the baseline. Our model improves the previous state of the art motion segmentation method by 3.3\% . The dataset and qualitative results video are shared in our website at URL
R
unknown
unseen
A
and independent of their appearance
R
is not dealt with on
has been rarely addressed at
R
Other generic video object segmentation tasks have dealt with instance-wise motion segmentation but on much simpler setting and have not dealt with the multi-task learning problem for both semantic and class agnostic instance segmentation. In this paper, we propose a motion-based instance segmentation task and provide a new annotated dataset based on KITTIMoSeg , which will be released publicly. Our datasetprovide extra class annotations
As the main issue is the lack of large public datasets, we create a new InstanceMotSeg dataset comprising of 12.9K samples improving upon our KITTIMoSeg dataset. In addition to providing instance level annotations, we have added 4 additional classes
A
motion
R
further propose
adapt YOLACT and implement a motion-based class agnostic instance segmentation model which would act as a baseline for the dataset. We also extend it to
R
learning approach that learns an extra class agnostic instance segmentation head through sharing the prototype generation network with the semantic head
model which additionally provides semantic instance segmentation sharing the encoder
A
providing two independent paths of object detection for redundant safety
R
outperforms
improves the previous
R
methods with
method by
R
improvement. We summarize our work in a short video with qualitative results
. The dataset and qualitative results video are shared in our website