Watching the World Go By: Representation Learning from Unlabeled Videos
Authors: Daniel Gordon, Kiana Ehsani, Dieter Fox, Ali Farhadi
Keywords: Representation Learning, Unsupervised Learning, Video Analytics
Abstract: Recent unsupervised representation learning techniques show remarkable success on many single image tasks by using instance discrimination:  learning to differentiate between two augmented versions of the same image and a large batch of unrelated images.  Prior work uses artificial data augmentation techniques such as cropping, and color jitter which can only affect the image in superficial waysand are not aligned with how objects actually change e.g. occlusion, deformation,viewpoint change.  We argue that videos offer this natural augmentation for free.Videos can provide entirely new views of objects,  show deformation,  and even connect semantically similar but visually distinct concepts.We propose Video Noise Contrastive Estimation,  a method for using unlabeled video to learnstrong, transferable, single image representations.We demonstrate improvements over recent unsupervised single image techniques,as well as over fully supervised ImageNet pretraining, across temporal and non-temporal tasks.