Unsupervised Learning in Space and Time over Several Generations of Teacher and Student Networks

Unsupervised learningUnsupervised learning represents one of the most interesting challenges in computer vision today, as the material presented so far in the book has so often shown. The problem is essential for visual learning, coming in many different forms and tasks. Consequently, it has an immense practical value with many applications in artificial intelligence and emerging technologies, as large quantities of unlabeled images and videos can be collected at low cost. In this chapter, we address the unsupervised learningUnsupervised learning problem in the context of segmenting the main foreground objects in single images. We propose an unsupervised learning systemTeacher-student unsupervised learning system, which has two pathways, the teacher and the student, respectively. The system is designed to learn over several generations of teachers and students. At every generation, the teacher does unsupervised object discoveryUnsupervised object discovery in videos or collections of images and an automatic selection module picks up good frame segmentations and passes them to the student pathway for training. At every generation, multiple students are trained, with different deep network architectures to ensure a better diversity. The students at one iteration help in training a better selection module, forming together a more powerful teacher pathway at the next iteration. The approach presented in this chapter gathers together many ideas studied throughout the book. On one hand, the pathway used as teacher at the first iteration is an unsupervised video object discoverer that immediately relates to the VideoPCAVideoPCA method introduced in Chap. 5 and the space-time clustering GO-VOS method introduced in Chap. 6. Then, the idea of using agreements between multiple student nets at the current generation as teacher at the next generation relates directly to the principles for unsupervised learningPrinciples of unsupervised learning proposed in Chap. 1 . Such agreements constitute reliable HPP signal (Chap. 5), which becomes a robust teacher for training the next-generation students. Their agreements are also used to train in an unsupervised fashion a deep network that selects only high-quality HPP object masks as supervisory signal. The principles for unsupervised learning are exploited fully in the teacher-studentTeacher-student unsupervised learning system system presented here. The material in this chapter makes some original contributions, discussed next. They lay the foundation of our general concept of a universal unsupervised learning machineUniversal unsupervised learning machine, the Visual Story NetworkVisual Story Network (VSN), which we introduce in the final chapter of the book. In the extensive experimental validation of our proposed system, we show that the improvement in the selection power, the training of multiple students, and the increase in unlabeled data significantly improve segmentation accuracy from one generation to the next. Our method achieves top results on three challenging datasets for object discovery in video, unsupervised image segmentationUnsupervised image segmentation, and saliency detectionSaliency detection. At test time, our system is fast, being one to two orders of magnitude faster than published unsupervised methods. We also test the strength of our unsupervised features within a well-known transfer learningTransfer learning setup and achieve competitive performance, proving that our unsupervised approach can be reliably used in a variety of computer vision tasks.