Improving old 3D renders with AI and SD
Published: 2022-12-10 15:38:31

There are some really good images produced with [Stable Diffusion](https://stability.ai/blog/stable-diffusion-public-release) and the "new" methods that take in an image and/or a text prompt input. Since I've been dreaming with the day that we will have AI doing post rendering in games like in [some](https://www.youtube.com/watch?v=P1IcaBn3ej0) 'state of the art' [videos](https://www.youtube.com/watch?v=jl0XCslxwB0) I've seen on youtube, I decided to try out this techniques on my old architectural 3D renders from college.

So here's the image I've chosen, I've cropped it to a 512px square since most interfaces restrict the resolution.

![Bathroom original 3d render](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8zlhy41476sb5zcitcb2.png)

In order to generate the images I've used the following text prompt:

`bathroom photo, Canon EOS R3, f/1.4, ISO 200, 1/160s, 8K, RAW, unedited, balance, in-frame`

I've found the keywords to improve the results in [this repository](https://github.com/Maks-s/sd-akashic) along with a lot of usefull info.

I've used the following implementations to generate the images, all of them are hosted on [huggingface.co](https://huggingface.co/):

[Manjushri/SD-2.1-Img2Img](https://huggingface.co/spaces/Manjushri/SD-2.1-Img2Img)
[Manjushri/SD-IMG2IMG-CPU](https://huggingface.co/spaces/Manjushri/SD-IMG2IMG-CPU) * faster

I've also tried this other image generation implementation called Marco:

[AkiKagura/Marco-Generation-Img2img](https://huggingface.co/spaces/AkiKagura/Marco-Generation-Img2img)

Here are the parameters values I've used:

- Guidence Scale: 9 - 11
- Number of Iterations: 25
- Strength: 0.7 - 0.9

And here are some of the results of the SD img2img implementations:

![Bathroom SD AI render](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z1eqr2vklrzjbkdjrvny.png)

![Bathroom SD AI CPU render](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0s58hhmt8i5iq61dt7fy.png)

And here are the results for the Marco img2img generator, it's worth noting that this ones took about 3 times longer:

![Bathroom Marco render](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4o4dbv3xz28mnc3h5ecg.png)

![Bathroom Marco render number 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jlj08bc0ancmk8b8ue69.png)

As you can see in the first image, it's not rare to encounter watermark stamps on the generated images, certainlly a result of marked images on the training dataset.

![Bathroom SD render with watermark](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ne6ayjoeqhdx3spc0p4x.png)

Also pretty common to get weird toilets and other types of 'impossible designs'.

![Bathroom render with toilet blocking a door](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6t7vyg0iz848d013qqh2.png)

![Bathroom render with a weird double toilet](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7i1h8bom9l3wj8v6udol.png)

Considering that the time it took me to generate those images was close to the time it took to render the original images (in the old days, rendering reflection like this was a real pain), I think this is an amazing tool to quickly create "close to real" pictures to illustrate any kind of project.

What makes it amazing is that it accepts as input pretty raw images and the results can be as good if you calibrate the parameters correctly. (And it's free!)