In this paper, we study the problem of answering visual analogy questions. These questions take the form of image A is to image B as image C is to what. Answering these questions entails discovering the mapping from image A to image B and then extending the mapping to image C and searching for the image D such that the relation from A to B holds for C to D. We pose this problem as learning an embedding that encourages pairs of analogous images with similar transformations to be close together using convolutional neural networks with a quadruple Siamese architecture. We introduce a dataset of visual analogy questions in natural images, and show first results of its kind on solving analogy questions on natural images.
Instead of synthesizing new images, VISALOGY REF solves visual analogies by discovering the mapping from image A to image B and searching for an image D such that A to B holds for C to D. VISALOGY conducts experiments on natural image datasets, where visual analogies involve different high-level semantic properties.
VISALOGY: Answering Visual Analogy Questions
ArXiv
ArXiv
Computer Science