The reasons why faces are so messed up and lizards

#555
by yodude - opened

EDIT: I no longer believe that faces are deliberately distorted....

So human faces are extremely distorted:

evil.jpg

And so are cartoon faces:

face6.jpg

Swastikas are also highly distorted:

swastikas.jpeg

But lizards basically aren't:

lizards1.jpg

From:
https://twitter.com/iScienceLuvr/status/1536294746041114624

As a separate note, you might have noticed that many of the #dallemini artworks have messed up faces 😄

This is mainly since the VQGAN hasn't learned a good mapping to easily represent faces as a sequence of discrete values. (12/16)

The following shows that DALL-E mini is capable of generating undistorted images, it just has to be tricked into it like this:

https://huggingface.co/spaces/dalle-mini/dalle-mini/discussions/258

Artstation style = high detail : ( subject =highly detailed💃🏻🧕🏼+(anatomically correct facial features + (highly detailed = 👩🏼‍🦰🧝🏼‍♀️)+((highly detailed and anatomically correct (realistic and highly detailed + anatomically correct and accurately shaped eyes=👁👁),highly detailed and anatomically correct👃🏼,highly detailed and anatomically correct👄)))

Using a similar technique I was able to get DALL-E mini to generate nudity.

Anyway I was wondering if DALL-E mini is distorting faces deliberately or if it is just a limitation with VQGAN - and why does VQGAN have barely any issues with lizards?

https://labs.openai.com/policies/content-policy

OpenAI's content policy says "Do not upload images of people without their consent, including public figures" so OpenAI is against photorealistic human faces even though many people say that DALL-E mini has nothing to do with OpenAI

yodude changed discussion status to closed
yodude changed discussion status to open
DALLE mini org

Yeah as I mentioned this is a limitation of the VQGAN and it is one we saw from the very beginning. It is known in the literature too and some papers try to directly address this, like the Make-a-Scene paper that specifically had a loss for facial reconstruction.

It is not intentional and there is no censoring that is being done here.

It is not intentional and there is no censoring that is being done here.

It looks like swastikas are deliberately distorted though....

Sign up or log in to comment