  Machine intelligence is very successful at standard recognition tasks when
having high-quality training data. There is still a significant gap between
machine-level pattern recognition and human-level concept learning. Humans can
learn under uncertainty from only a few examples and generalize these concepts
to solve new problems. The growing interest in explainable machine
intelligence, requires experimental environments and diagnostic tests to
analyze weaknesses in existing approaches to drive progress in the field. In
this paper, we discuss existing diagnostic tests and test data sets such as
CLEVR, CLEVERER, CLOSURE, CURI, Bongard-LOGO, V-PROM, and present our own
experimental environment: The KANDINSKYPatterns, named after the Russian artist
Wassily Kandinksy, who made theoretical contributions to compositivity, i.e.
that all perceptions consist of geometrically elementary individual components.
This was experimentally proven by Hubel &Wiesel in the 1960s and became the
basis for machine learning approaches such as the Neocognitron and the even
later Deep Learning. While KANDINSKYPatterns have computationally controllable
properties on the one hand, bringing ground truth, they are also easily
distinguishable by human observers, i.e., controlled patterns can be described
by both humans and algorithms, making them another important contribution to
international research in machine intelligence.
