Abstract. This work presents an unsupervised learning based approach to the ubiquitous computer vision problem of image matching. We start from the insight that the problem of frame-interpolation implicitly solves for inter-frame correspondences. This permits the application of analysisby-synthesis: we firstly train and apply a Convolutional Neural Network for frame-interpolation, then obtain correspondences by inverting the learned CNN. The key benefit behind this strategy is that the CNN for frame-interpolation can be trained in an unsupervised manner by exploiting the temporal coherency that is naturally contained in real-world video sequences. The present model therefore learns image matching by simply "watching videos". Besides a promise to be more generally applicable, the presented approach achieves surprising performance comparable to traditional empirically designed methods.
This idea is also explored in REF for image matching via training a CNN for frame interpolation through video sequences.
Learning Image Matching by Simply Watching Video
ArXiv
ArXiv
Computer Science