How to use Stable Diffusion 3 to generate a similar image.
Tags: generativeai, javascript, ai
Published: 2024-04-22 12:37:33

Stable Diffusion 3 from Stability AI has now been released and is available for public use. However, they currently only offer a REST API to test these models. Therefore, I tried using this on my local machine using Node.js code.

```javascript
import fs from "node:fs";
import axios from "axios";
import FormData from "form-data";

const formData = {
  prompt: "prompt",
  output_format: "png",
  aspect_ratio: "16:9",
  seed: "645,508,968",
};

const response = await axios.postForm(
  `https://api.stability.ai/v2beta/stable-image/generate/sd3`,
  axios.toFormData(formData, new FormData()),
  {
    validateStatus: undefined,
    responseType: "arraybuffer",
    headers: {
      Authorization: `TOKEN`,
      Accept: "image/*",
    },
  },
);

if (response.status === 200) {
  fs.writeFileSync("./output.png", Buffer.from(response.data));
} else {
  throw new Error(`${response.status}: ${response.data.toString()}`);
}
```

If you prefer testing it without code, you can check and use your token wisely. [Fireworks Playground](https://fireworks.ai/models/stability/sd3)

## What is seed?

In Stable Diffusion, a `seed` is a number used to initialize the image generation process. It acts like a starting point for the AI to determine the random noise it uses to build the image. Here's a breakdown of how it works:

- **Random by default:** If you don't specify a `seed`, Stable Diffusion will automatically choose a random seed number for each image generation. This results in completely different images every time you run the program with the same prompt.


- **Replicability with specific seed:**  However, if you set a specific `seed` value, Stable Diffusion will always use that same `seed` number to generate an image whenever you provide the same prompt along with that seed. This allows you to reproduce the same image again.


- **Making minor adjustments:** You can use the same `seed` as a base and then tweak other settings in Stable Diffusion to see how they affect the image while keeping the overall content consistent.

- **Sharing results:** Sharing the `seed` value along with your prompt allows others to recreate the exact image you generated, which can be useful for collaboration or tutorials.

- **Seed and image similarity:** Interestingly, `seeds` that are close in value tend to generate images that share some similarities. This can be helpful if you're looking for variations on a theme. For example, using a `seed` value that's one number higher or lower than another seed might generate an image with a similar composition but with slight differences in details.

In summary, the `seed` value in Stable Diffusion acts like a recipe for generating random noise, which the AI then uses to build an image based on your text prompt. By controlling the `seed`, you can influence the reproducibility and explore variations of the generated images.

To generate a similar image, we need to utilize the `seed` parameter; using the same seed will produce similar images. 

Here, I'm trying to create a prompt for a countryside scene across four seasons and also a picture with a weather ambience vibe, where the result is not exactly the same but produces a similar output in terms of how the landscape looks.

## Generating Similar Image (4 different season)

seed -> 645,508,968

### Spring Prompt

A countryside village with a clear blue sky, surrounded by sakura trees, in an anime art style.

![A countryside village with a clear blue sky, surrounded by sakura trees, in an anime art style.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bkfzy8l6vg8e34r2lloz.png)

### Summer Prompt

A countryside village with a clear blue sky, surrounded by green trees, in an anime art style.

![A countryside village with a clear blue sky, surrounded by green trees, in an anime art style.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hk5khv2418eo8cen04tk.png)

### Autumn Prompt

A countryside village with a clear blue sky, surrounded by trees that are falling down in the autumn season, in an anime art style.

![A countryside village with a clear blue sky, surrounded by trees that are falling down in the autumn season, in an anime art style.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j0uqa8n8dzxtw99vsqky.png)

### Winter Prompt

A countryside village with a clear blue sky, surrounded by a snow, in an anime art style.

![A countryside village with a clear blue sky, surrounded by a snow, in an anime art style](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dstxw3g7xs0p5pqwgizu.png)

## Generating Similar Image (ambience vibes)

Seed -> 358,804,591

### Default Prompt

In a dark alley illuminated by a neon light, the scene exudes Tokyo city vibes. It's foggy and ultra-realistic in 4K resolution. A cat, not facing the camera, sits in the middle of the alley street, with a bokeh focus on the cat. A vending machine is mounted on the wall.

![In a dark alley illuminated by a neon light, the scene exudes Tokyo city vibes. It's foggy and ultra-realistic in 4K resolution. A cat, not facing the camera, sits in the middle of the alley street, with a bokeh focus on the cat. A vending machine is mounted on the wall.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vqgujoxyd2m8v2x2owio.png)

### Snow Ambience Prompt

In a dark alley illuminated by a neon light, the scene exudes Tokyo city vibes. It's foggy and ultra-realistic in 4K resolution, with a snowstorm and a thick layer of snow covering the street. A cat, not facing the camera, sits in the middle of the alley street, with a bokeh focus on the cat. A vending machine is mounted on the wall.

![In a dark alley illuminated by a neon light, the scene exudes Tokyo city vibes. It's foggy and ultra-realistic in 4K resolution, with a snowstorm and a thick layer of snow covering the street. A cat, not facing the camera, sits in the middle of the alley street, with a bokeh focus on the cat. A vending machine is mounted on the wall.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ufnqcnjqk7dfca57u7du.png)

### Rain Ambience Prompt

In a dark alley illuminated by a neon light, the scene exudes Tokyo city vibes. It's foggy and ultra-realistic in 4K resolution, with a heavy rainstorm creating a muddy environment. A cat, not facing the camera, sits in the middle of the alley street, sharply in focus with a bokeh effect. A vending machine is mounted on the wall.

![In a dark alley illuminated by a neon light, the scene exudes Tokyo city vibes. It's foggy and ultra-realistic in 4K resolution, with a heavy rainstorm creating a muddy environment. A cat, not facing the camera, sits in the middle of the alley street, sharply in focus with a bokeh effect. A vending machine is mounted on the wall.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nok7lwn79heiwrz9nb0e.png)


## Summary

While the output for each image is not exactly the same, it already has the same landscape structure if we use the same seeds and a similar prompt, with some customization added to create a new scene or ambience.

## Resources
- [Stable Diffusion](https://stability.ai/stable-image)
- [Stable Diffusion 3 API References](https://platform.stability.ai/docs/api-reference#tag/Generate/paths/~1v2beta~1stable-image~1generate~1sd3/post)
- [Stable Diffusion 3 Research Paper](https://stability.ai/news/stable-diffusion-3-research-paper)
- [Stable Diffusion in AWS Bedrock](https://aws.amazon.com/bedrock/stable-diffusion/)
- [Fireworks Playground](https://fireworks.ai/models/stability/sd3)
- [Seed Explanation by Segmind blog](https://blog.segmind.com/how-seed-parameter-influences-stable-diffusion-model-outputs/)
- [Seed Explanation by Wolfe](https://johnwolfecompton.com/the-seed-of-imagination-how-seed-numbers-influence-ai-image-generation/#:~:text=Consistency%20Across%20Iterations%3A,in%20a%20piece%20of%20music.)
- [Seed Explanation by getimg.ai](https://getimg.ai/guides/guide-to-seed-parameter-in-stable-diffusion)
- [Seed Explanation from springer](https://link.springer.com/article/10.1007/s43681-024-00440-7)


_All the images here were generated by Stable Diffusion 3, including the cover image._