Pareidolia of AI
By: Vlad Alex, Merzmensch
Tags: Reality, Machine Learning, Artificial Intelligence, Art, Neural Networks

Survival Instinct as a pre-trained dataset.

It’s here to protect us. To make us cautious. Pareidolia. A well-known phenomenon, when we see faces where aren’t any. We see faces on Mars, we see Jesus in the toast, we see it everywhere. Well, it does our brain, which is trained on biometrical recognition: eyes, nose, mouth = when everything is in the right place, that’s a face.

It originated from our past — as we were strolling around the woods hunting mammoths. It was better to confound a bush with a tiger once more than to ignore an actual predator between the trees. Survival instinct.

Neural Networks are acting similarly. They recognize what they are trained on. The best example is StyleGAN2 Projection, where the latent image is juxtaposed with an uploaded one and is examined for its origins (more about this later).

Here is an example of human and artificial Pareidolia:

A small part of the Cydonia region, taken by the Viking 1 orbiter and released by NASA/JPL on July 25, 1976 (Source, Public Domain) // Mona Lisa by Leonardo Da Vinci, modified with Google Deep Dreams (Source: NixTown).

While our human eye sees a face in some random rock formations, Convolutional Neural Networks of an early version of Google Deep Dreams recognizes dogs. Everywhere. The explanation to the question “why dogs?” was obvious:

In Deep Dream’s case, that data set is from ImageNet, a database created by researchers at Stanford and Princeton who built a database of 14 million human-labeled images. But Google didn’t use the whole database. Instead, they used a smaller subset of the ImageNet database released in 2012 for use in a contest… a subset which contained “fine-grained classification of 120 dog sub-classes.”

(Source: FastCompany)

This was one of the first visualizations of biased Neural Networks, being trained on a limited dataset. It isn’t AI’s fault to be unable to recognize objects it isn’t trained on. It’s up to us.

The Beauty of a Fail

But this flaw can become an advantage — if used for experimental purposes.

You have already seen the StyleGAN2 Projection feature.

The main task of this function is to compare a given image with the seeds from the Latent Space (hidden layer of StyleGAN, where all unique image seeds are located before they are getting modified). Among others, the matching images can point out the Deep Fake (for criminologists’ Image Forensics).

At some point in my experiment, I tried other imagery than physiognomical pictures.

It delivered some surprising results:

Or even an unsettling one (speaking of Uncanny Valley):

I used for this experiment StyleGAN2-based Neural Network, trained on Faces from FlickrDataset, StyleGAN2 for FFHQ dataset at 1024×1024.

The same dataset is implemented in a web application ArtBreeder. Using the upload function, you can augment the latent space with new images. But they are aligned with StyleGAN2 trained network and modified in a compatible way using Projection feature.

Sometimes adding a particular non-latent image to the Latent Space causes interesting effects. In one case, StyleGAN2 “corrected” a photo of an artist, who playfully had covered his eye with a paper disc:

Experimenting with limited recognition skills (which are caused by limited training) can bring fascinating results.

It began, as I tried to upload a photo of an object. Without any facial expressions or biometric elements. My motivation was to examine, in what way StyleGAN2 can identify and categorize a random motif (instead of photo portrait).

So I uploaded my photo of a cup of coffee. Random as it is.