We present Adaptive Instance Selection network architecture for class-agnostic instance segmentation. Given an input image and a point (x, y), it generates a mask for the object located at (x, y). The network adapts to the input point with a help of AdaIN layers [13], thus producing different masks for different objects on the same image. Adap-tIS generates pixel-accurate object masks, therefore it accurately segments objects of complex shape or severely occluded ones. AdaptIS can be easily combined with standard semantic segmentation pipeline to perform panoptic segmentation. To illustrate the idea, we perform experiments on a challenging toy problem with difficult occlusions. Then we extensively evaluate the method on panoptic segmentation benchmarks. We obtain state-of-the-art results on Cityscapes and Mapillary even without pretraining on COCO, and show competitive results on a challenging COCO dataset. The source code of the method and the trained models are available at https://github.com/saicvul/adaptis.
More recently, AdaptIS REF uses point proposals to produce instance masks and jointly trains with a standard semantic segmentation pipeline to perform panoptic segmentation.
AdaptIS: Adaptive Instance Selection Network
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Computer Science