Repository: huggingface/diffusers
Pull request: Add server example
# What does this PR do?

This is a followup to [9848](https://github.com/huggingface/diffusers/pull/9848). The aim of this is to serve as a fix for several issues around concurrent, multithreaded usage of diffusers: 

Fixes #3672, #5749, #9833, #7851, #4920, #2916 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/diffusers/blob/main/CONTRIBUTING.md)?
- [x] Did you read our [philosophy doc](https://github.com/huggingface/diffusers/blob/main/PHILOSOPHY.md) (important for complex PRs)?
- [x] Was this discussed/approved via a GitHub issue.  [Link to GH Issue](https://github.com/huggingface/diffusers/pull/9848)
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/diffusers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/diffusers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@yiyixuxu, anyone else that is interested

![](https://bukk.it/dubstep.gif)

Problem statement:
Concurrent Thread Failure in Image Inference: StableDiffusionPipeline
### Describe the bug

When I attempt to concurrently initiate two threads for image inference in one pipe, the process fails and the following information is displayed as logs during the inference. I intend to submit a pull request for thread-safety in one inference pipe if anyone else need.

### Reproduction
```python
from diffusers import StableDiffusionPipeline
import torch
import threading

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

def infer_one():
    pipe("a photo of an astronaut riding a horse on mars").images[0] 

x1 = threading.Thread(target=infer_one) 
x1.start()
    
x2 = threading.Thread(target=infer_one) 
x2.start()
```

### Logs

```shell
Exception in thread Thread-1 (infer_one):                                                                                        | 0/50 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/opt/miniconda3/envs/diffusers/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/opt/miniconda3/envs/diffusers/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/workdata/A40/main.py", line 10, in infer_one
    pipe("a photo of an astronaut riding a horse on mars").images[0] 
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/diffusers/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 710, in __call__
    latents = self.scheduler.step(noise_pred, t, latents, **extra_step_kwargs, return_dict=False)[0]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/schedulers/scheduling_pndm.py", line 221, in step
    return self.step_plms(model_output=model_output, timestep=timestep, sample=sample, return_dict=return_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/schedulers/scheduling_pndm.py", line 337, in step_plms
    prev_sample = self._get_prev_sample(sample, timestep, prev_timestep, model_output)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/schedulers/scheduling_pndm.py", line 371, in _get_prev_sample
    alpha_prod_t = self.alphas_cumprod[timestep]
                   ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
IndexError: index 1001 is out of bounds for dimension 0 with size 1000
  0%|                                                                                                                            | 0/50 [00:04<?, ?it/s]
Exception in thread Thread-2 (infer_one):
Traceback (most recent call last):
  File "/opt/miniconda3/envs/diffusers/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/opt/miniconda3/envs/diffusers/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/workdata/A40/main.py", line 10, in infer_one
    pipe("a photo of an astronaut riding a horse on mars").images[0] 
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/diffusers/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 710, in __call__
    latents = self.scheduler.step(noise_pred, t, latents, **extra_step_kwargs, return_dict=False)[0]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/schedulers/scheduling_pndm.py", line 221, in step
    return self.step_plms(model_output=model_output, timestep=timestep, sample=sample, return_dict=return_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/schedulers/scheduling_pndm.py", line 337, in step_plms
    prev_sample = self._get_prev_sample(sample, timestep, prev_timestep, model_output)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workdata/yehang/projects/diffusers/src/diffusers/schedulers/scheduling_pndm.py", line 396, in _get_prev_sample
    sample_coeff * sample - (alpha_prod_t_prev - alpha_prod_t) * model_output / model_output_denom_coeff
    ~~~~~~~~~~~~~^~~~~~~~
TypeError: unsupported operand type(s) for *: 'Tensor' and 'NoneType'
```


### System Info

- `diffusers` version: 0.17.0.dev0
- Platform: Linux-5.15.0-72-generic-x86_64-with-glibc2.31
- Python version: 3.11.3
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Huggingface_hub version: 0.14.1
- Transformers version: 4.29.1
- Accelerate version: 0.19.0
- xFormers version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

Hints from issue discussion:
Hey @Fantast416,

I have not yet tested SD with threads. Can you verify that the inference call works without threading to begin with? 
> Hey @Fantast416,
> 
> I have not yet tested SD with threads. Can you verify that the inference call works without threading to begin with?

Yes, I have verified it. if you remove the following code in my Reproduction,which means there will be only one thread run in one time, and the code will run correctly:
```python
x2 = threading.Thread(target=infer_one) 
x2.start()
```
So it seems like that SD can't run two or more threads in the same time.
Hey @patrickvonplaten,
I found that the internal implementation of the diffusers framework does not consider multi-threaded inference in one pipeline object. The reason for the mentioned error is primarily due to simultaneous modifications of scheduler object's member variables by multiple threads during sampling within the schedule. I envision that if each pipeline could use an independent scheduler during inference in different threads, it should resolve this problem. Taking the StableDiffusionPipeline as an example, I propose a workaround here #3685 

I would like to inquire if you have any better solution or if you believe such changes are necessary. I believe they are necessary for the deployment of open-source production services
Hey @Fantast416,

That's very interesting! We indeed don't assume schedulers to work in multi-threaded environments. I think the DDIMScheduler should work for multi-threading though. Could you try whether the following works:
```py
from diffusers import StableDiffusionPipeline, DDIMScheduler
import torch
import threading

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to("cuda")

def infer_one():
pipe("a photo of an astronaut riding a horse on mars").images[0]

x1 = threading.Thread(target=infer_one)
x1.start()

x2 = threading.Thread(target=infer_one)
x2.start()
```
> ```python
> DDIMScheduler
> ```

DDIMScheduler seems to work!
Hey,@liyi0206 @patrickvonplaten 
DDIMScheduler does work, **but please be careful to use it in multi-thread environments!** 

I have carefully examine the source code, and find that there are still thread-unsafe parts in DDIM. This is caused by the invocation of the scheduler in the 'call' function of StableDiffusionPipeline object. In the SD pipeline, 'set_timesteps' function is called first, followed by retrieving the timestep value. If there is a multi-threading conflict where timesteps are set and retrieved simultaneously, it can cause the program to deviate from the expected behavior though it may not cause exception. ( Specifically, if the following execution sequence occurs: Thread 1 calls 'set_timesteps', Thread 2 calls 'set_timesteps', Thread 1 retrieves timesteps, and Thread 2 retrieves timesteps, then Thread 1 will obtain the timestep set by Thread 2.) 

Currently, I am carefully reading the source code of different schedulers and the specific invocations within StableDiffusionPipeline to figure out how to address the thread-unsafe issue internally in the sampler. I will provide a more detailed plan later.
Hello, @patrickvonplaten 

- I have carefully read the code of `StableDiffusionPipeline`, `StableDiffusionImg2ImgPipeline`, and the code of three classic samplers (`PNDMScheduler`, `DDIMScheduler`, `LMSDiscreteScheduler`), and made the following research report.
- Within these two classic Pipelines, there are thread-unsafe calls that involve changing the member variables within self.scheduler, including the following functions and value retrievals:
    - .set_timesteps & .timesteps
    
    ```python
    # 4. Prepare timesteps
    self.scheduler.set_timesteps(num_inference_steps, device=device)
    timesteps = self.scheduler.timesteps
    ```
    
    - .step
    
    ```python
    with self.progress_bar(total=num_inference_steps) as progress_bar:
        for i, t in enumerate(timesteps):
            # ...
            latents = self.scheduler.step(noise_pred, t, latents, **extra_step_kwargs).prev_sample
            # ...
    ```
    
- Therefore, we need to pay attention to whether there are any thread-unsafe parts related to these three functions in different schedulers
    - **PNDMScheduler**
        - `.step` function=> `self.counter` => `self.counter` will change in `.step`
        - `self.counter` will be set to 0 in `.set_timesteps`
        - multi-thread visit will let `self.counter` be added multiple times
    - **DDIMScheduler**
        - `.set_timesteps` and `.timesteps` only be called once in pipeline，but multi-thread visit will cause `self.timesteps` unsafe. Specifically, imagine following timeline: it will
            - ————————————————————————————————> Timeline
            - Thread-1   `.set_timesteps`    &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp;  `.timesteps`
            - Thread-2    &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; &ensp; `.set_timesteps` &ensp; &ensp; &ensp; &ensp; &ensp; &ensp;  &ensp; &ensp;                         `.timesteps`
            - Thread-1 will read the variable `timesteps` set by Thread-2
        - `.step`  function will be called every iteration, but in DDIMScheduler, it will not share the `counter`  variable, so it is multi-thread safe.
    - **LMSDiscreteScheduler**
        - similar to DDIMScheduler
- Based on the above idea, if we want to solve this problem within the Scheduler as you said in PR, we need to avoid distinguishing all calculations that depend on the state of a single thread member variable within the Scheduler according to different threads. This may modify a lot of code. What do you think about it?

@patrickvonplaten  @Fantast416 
I am also experience another problems caused by multi thread infer.
When i call the StableDiffusionPipeline even no problems for the Scheduler, but the image generated is in disorder.
If not use multi thread, the image generated all normal
bad image
![image](https://github.com/huggingface/diffusers/assets/5241459/beec85eb-ac67-4c97-853c-7c5577c66239)

normal image
![image](https://github.com/huggingface/diffusers/assets/5241459/2a8a6ea3-1be7-4983-bdb1-0cbef6e335c0)

I think the easiest fix for now would be to just manually copy the scheduler in the `infer_one()` function, e.g. the following code snippet should work:

```py
from diffusers import StableDiffusionPipeline
import torch
import threading

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

def infer_one():
    # copy the scheduler for each thread to make it thread-safe
    pipe.scheduler = pipe.scheduler.from_config(pipe.scheduler.config)
    pipe("a photo of an astronaut riding a horse on mars").images[0] 

x1 = threading.Thread(target=infer_one) 
x1.start()
    
x2 = threading.Thread(target=infer_one) 
x2.start()
```
> I think the easiest fix for now would be to just manually copy the scheduler in the `infer_one()` function, e.g. the following code snippet should work:
> 
> ```python
> from diffusers import StableDiffusionPipeline
> import torch
> import threading
> 
> model_id = "runwayml/stable-diffusion-v1-5"
> pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
> pipe = pipe.to("cuda")
> 
> def infer_one():
>     # copy the scheduler for each thread to make it thread-safe
>     pipe.scheduler = pipe.scheduler.from_config(pipe.scheduler.config)
>     pipe("a photo of an astronaut riding a horse on mars").images[0] 
> 
> x1 = threading.Thread(target=infer_one) 
> x1.start()
>     
> x2 = threading.Thread(target=infer_one) 
> x2.start()
> ```

Thanks, that will work. So I think this issue should be closed.

Commit message: Add fixes after local testing.

Patch to review:
diff --git a/examples/server/README.md b/examples/server/README.md
new file mode 100644
index 000000000000..1bfdf2050f15
--- /dev/null
+++ b/examples/server/README.md
@@ -0,0 +1,35 @@
+
+## OpenAI Compatible `/v1/images/generations` Server
+
+This is a concurrent, multithreaded solution for running a server that can generate images using the `diffusers` library. This examples uses the Stable Diffusion 3 pipeline, but you can use any pipeline that you would like by swapping out the model and pipeline to be the ones that you want to use.
+
+### Installing Dependencies
+
+Start by going to the base of the repo and installing it with:
+``py
+pip install .
+```
+
+The pipeline can then have its dependencies installed with:
+```py
+pip install -f requirements.txt
+```
+
+### Running the server
+
+This server can be run with:
+```py
+python server.py
+```
+The server will be spun up at http://localhost:8000. You can `curl` this model with the following command:
+```
+curl -X POST -H "Content-Type: application/json" --data '{"model": "something", "prompt": "a kitten in front of a fireplace"}' http://localhost:8000/v1/images/generations
+```
+
+### Upgrading Dependencies
+
+If you need to upgrade some dependencies, you can do that with either [pip-tools](https://github.com/jazzband/pip-tools) or [uv](https://github.com/astral-sh/uv). With `uv`, this looks like:
+```
+uv pip compile requirements.in -o requirements.txt
+```
+
diff --git a/examples/server/requirements.in b/examples/server/requirements.in
new file mode 100644
index 000000000000..b49b285a8fc8
--- /dev/null
+++ b/examples/server/requirements.in
@@ -0,0 +1,9 @@
+torch~=2.4.0
+transformers==4.46.1
+sentencepiece
+aiohttp
+py-consul
+prometheus_client >= 0.18.0
+prometheus-fastapi-instrumentator >= 7.0.0
+fastapi
+uvicorn
\ No newline at end of file
diff --git a/examples/server/requirements.txt b/examples/server/requirements.txt
new file mode 100644
index 000000000000..065a381f0c9b
--- /dev/null
+++ b/examples/server/requirements.txt
@@ -0,0 +1,124 @@
+# This file was autogenerated by uv via the following command:
+#    uv pip compile requirements.in -o requirements.txt
+aiohappyeyeballs==2.4.3
+    # via aiohttp
+aiohttp==3.10.10
+    # via -r requirements.in
+aiosignal==1.3.1
+    # via aiohttp
+annotated-types==0.7.0
+    # via pydantic
+anyio==4.6.2.post1
+    # via starlette
+attrs==24.2.0
+    # via aiohttp
+certifi==2024.8.30
+    # via requests
+charset-normalizer==3.4.0
+    # via requests
+click==8.1.7
+    # via uvicorn
+fastapi==0.115.3
+    # via -r requirements.in
+filelock==3.16.1
+    # via
+    #   huggingface-hub
+    #   torch
+    #   transformers
+frozenlist==1.5.0
+    # via
+    #   aiohttp
+    #   aiosignal
+fsspec==2024.10.0
+    # via
+    #   huggingface-hub
+    #   torch
+h11==0.14.0
+    # via uvicorn
+huggingface-hub==0.26.1
+    # via
+    #   tokenizers
+    #   transformers
+idna==3.10
+    # via
+    #   anyio
+    #   requests
+    #   yarl
+jinja2==3.1.4
+    # via torch
+markupsafe==3.0.2
+    # via jinja2
+mpmath==1.3.0
+    # via sympy
+multidict==6.1.0
+    # via
+    #   aiohttp
+    #   yarl
+networkx==3.4.2
+    # via torch
+numpy==2.1.2
+    # via transformers
+packaging==24.1
+    # via
+    #   huggingface-hub
+    #   transformers
+prometheus-client==0.21.0
+    # via
+    #   -r requirements.in
+    #   prometheus-fastapi-instrumentator
+prometheus-fastapi-instrumentator==7.0.0
+    # via -r requirements.in
+propcache==0.2.0
+    # via yarl
+py-consul==1.5.3
+    # via -r requirements.in
+pydantic==2.9.2
+    # via fastapi
+pydantic-core==2.23.4
+    # via pydantic
+pyyaml==6.0.2
+    # via
+    #   huggingface-hub
+    #   transformers
+regex==2024.9.11
+    # via transformers
+requests==2.32.3
+    # via
+    #   huggingface-hub
+    #   py-consul
+    #   transformers
+safetensors==0.4.5
+    # via transformers
+sentencepiece==0.2.0
+    # via -r requirements.in
+sniffio==1.3.1
+    # via anyio
+starlette==0.41.0
+    # via
+    #   fastapi
+    #   prometheus-fastapi-instrumentator
+sympy==1.13.3
+    # via torch
+tokenizers==0.20.1
+    # via transformers
+torch==2.4.1
+    # via -r requirements.in
+tqdm==4.66.5
+    # via
+    #   huggingface-hub
+    #   transformers
+transformers==4.46.1
+    # via -r requirements.in
+typing-extensions==4.12.2
+    # via
+    #   fastapi
+    #   huggingface-hub
+    #   pydantic
+    #   pydantic-core
+    #   torch
+urllib3==2.2.3
+    # via requests
+uvicorn==0.32.0
+    # via -r requirements.in
+yarl==1.16.0
+    # via aiohttp
diff --git a/examples/server/server.py b/examples/server/server.py
new file mode 100644
index 000000000000..7e1968de58ac
--- /dev/null
+++ b/examples/server/server.py
@@ -0,0 +1,113 @@
+from fastapi import FastAPI, HTTPException
+from fastapi.staticfiles import StaticFiles
+from fastapi.middleware.cors import CORSMiddleware
+from pydantic import BaseModel
+import aiohttp, asyncio, logging, os, random, sys, tempfile, torch, traceback, uuid
+from diffusers.pipelines.stable_diffusion_3 import StableDiffusion3Pipeline
+
+logger = logging.getLogger(__name__)
+
+class TextToImageInput(BaseModel):
+    model: str
+    prompt: str
+    size: str | None = None
+    n: int | None = None
+
+class HttpClient:
+    session: aiohttp.ClientSession = None
+
+    def start(self):
+        self.session = aiohttp.ClientSession()
+
+    async def stop(self):
+        await self.session.close()
+        self.session = None
+
+    def __call__(self) -> aiohttp.ClientSession:
+        assert self.session is not None
+        return self.session
+    
+class TextToImagePipeline:
+    pipeline: StableDiffusion3Pipeline = None
+    device: str = None
+
+    def start(self):
+        if torch.cuda.is_available():
+            model_path = os.getenv("MODEL_PATH", "stabilityai/stable-diffusion-3.5-large")
+            logger.info("Loading CUDA")
+            self.device = "cuda"
+            self.pipeline = StableDiffusion3Pipeline.from_pretrained(
+                model_path, 
+                torch_dtype=torch.bfloat16,
+            ).to(device=self.device)
+        elif torch.backends.mps.is_available():
+            model_path = os.getenv("MODEL_PATH", "stabilityai/stable-diffusion-3.5-medium")
+            logger.info("Loading MPS for Mac M Series")
+            self.device = "mps"
+            self.pipeline = StableDiffusion3Pipeline.from_pretrained(
+                model_path,
+                torch_dtype=torch.bfloat16,
+            ).to(device=self.device)
+        else:
+            raise Exception("No CUDA or MPS device available")
+    
+app = FastAPI()
+service_url = os.getenv("SERVICE_URL", "http://localhost:8000")
+image_dir = os.path.join(tempfile.gettempdir(), "images")
+if not os.path.exists(image_dir):
+    os.makedirs(image_dir)
+app.mount("/images", StaticFiles(directory=image_dir), name="images")
+http_client = HttpClient()
+shared_pipeline = TextToImagePipeline()
+
+# Configure CORS settings
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],  # Allows all origins
+    allow_credentials=True,
+    allow_methods=["*"],  # Allows all methods, e.g., GET, POST, OPTIONS, etc.
+    allow_headers=["*"],  # Allows all headers
+)
+
+@app.on_event("startup")
+def startup():
+    http_client.start()
+    shared_pipeline.start()
+
+def save_image(image):
+    filename = "draw" + str(uuid.uuid4()).split('-')[0] + ".png"
+    image_path = os.path.join(image_dir, filename)
+    # write image to disk at image_path
+    logger.info(f"Saving image to {image_path}")
+    image.save(image_path)
+    return os.path.join(service_url, "images", filename)
+
+@app.get('/')
+@app.post('/')
+@app.options('/')
+async def base():
+    return "Welcome to Diffusers! Where you can use diffusion models to generate images"
+
+@app.post("/v1/images/generations")
+async def generate_image(image_input: TextToImageInput):
+    try:
+        loop = asyncio.get_event_loop()
+        scheduler = shared_pipeline.pipeline.scheduler.from_config(shared_pipeline.pipeline.scheduler.config)
+        pipeline = StableDiffusion3Pipeline.from_pipe(shared_pipeline.pipeline, scheduler=scheduler)
+        generator =torch.Generator(device=shared_pipeline.device)
+        generator.manual_seed(random.randint(0, 10000000))
+        output = await loop.run_in_executor(None, lambda: pipeline(image_input.prompt, generator = generator))
+        logger.info(f"output: {output}")
+        image_url = save_image(output.images[0])
+        return {"data": [{"url": image_url}]}
+    except Exception as e:
+        if isinstance(e, HTTPException):
+            raise e
+        elif hasattr(e, 'message'):
+            raise HTTPException(status_code=500, detail=e.message + traceback.format_exc())
+        raise HTTPException(status_code=500, detail=str(e) + traceback.format_exc())
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="0.0.0.0", port=8000)
\ No newline at end of file


Code review comments:
diff_hunk: @@ -0,0 +1,35 @@
+
+## OpenAI Compatible `/v1/images/generations` Server
+
+This is a concurrent, multithreaded solution for running a server that can generate images using the `diffusers` library. This examples uses the Stable Diffusion 3 pipeline, but you can use any pipeline that you would like by swapping out the model and pipeline to be the ones that you want to use.
+
+### Installing Dependencies
+
+Start by going to the base of the repo and installing it with:
+``py
+pip install .
+```
path: examples/server/README.md
text: @user1:
```suggestion
Start by navigating to the `examples/server` folder and installing all the dependencies.

``py
pip install .
pip install -f requirements.txt
```
```