  Abstract reasoning ability is fundamental to human intelligence. It enables
humans to uncover relations among abstract concepts and further deduce implicit
rules from the relations. As a well-known abstract visual reasoning task,
Raven's Progressive Matrices (RPM) are widely used in human IQ tests. Although
extensive research has been conducted on RPM solvers with machine intelligence,
few studies have considered further advancing the standard answer-selection
(classification) problem to a more challenging answer-painting (generating)
problem, which can verify whether the model has indeed understood the implicit
rules. In this paper we aim to solve the latter one by proposing a deep latent
variable model, in which multiple Gaussian processes are employed as priors of
latent variables to separately learn underlying abstract concepts from RPMs;
thus the proposed model is interpretable in terms of concept-specific latent
variables. The latent Gaussian process also provides an effective way of
extrapolation for answer painting based on the learned concept-changing rules.
We evaluate the proposed model on RPM-like datasets with multiple
continuously-changing visual concepts. Experimental results demonstrate that
our model requires only few training samples to paint high-quality answers,
generate novel RPM panels, and achieve interpretability through
concept-specific latent variables.
