We investigate methods for combining multiple selfsupervised tasks-i.e., supervised tasks where data can be collected without manual labeling-in order to train a single visual representation. First, we provide an apples-toapples comparison of four different self-supervised tasks using the very deep ResNet-101 architecture. We then combine tasks to jointly train a network. We also explore lasso regularization to encourage the network to factorize the information in its representation, and methods for "harmonizing" network inputs in order to learn a more unified representation. We evaluate all methods on ImageNet classification, PASCAL VOC detection, and NYU depth prediction. Our results show that deeper networks work better, and that combining tasks-even via a naïve multihead architecture-always improves performance. Our best joint network nearly matches the PASCAL performance of a model pre-trained on ImageNet classification, and matches the ImageNet network on NYU depth prediction.
Doersch et al. REF investigated multiple selfsupervised methods to encourage the network to factorize the information in its representation.
Multi-task Self-Supervised Visual Learning
2017 IEEE International Conference on Computer Vision (ICCV)
2017 IEEE International Conference on Computer Vision (ICCV)
Computer Science