╭──────────────────────────── 🔔 Update Available! ────────────────────────────╮
│ ✨ Newer version of comfy-cli is available: 1.3.5.                           │
│ Current version: 1.2.7                                                       │
│ Update by running: 'pip install --upgrade comfy-cli' ⬆                       │
╰──────────────────────────────────────────────────────────────────────────────╯

Launching ComfyUI from: /root/comfy/ComfyUI

[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2025-01-04 22:21:44.074899
** Platform: Linux
** Python version: 3.11.10 (main, Dec  3 2024, 02:25:00) [GCC 12.2.0]
** Python executable: /usr/local/bin/python
** ComfyUI Path: /root/comfy/ComfyUI
** Log path: /root/comfy/ComfyUI/user/comfyui.log

Prestartup times for custom nodes:
   1.7 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-Manager

Total VRAM 22716 MB, total RAM 928061 MB
pytorch version: 2.5.1+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA A10 : cudaMallocAsync
Using pytorch attention
[Prompt Server] web root: /root/comfy/ComfyUI/web
### Loading: ComfyUI-Manager (V3.3.9)
### ComfyUI Version: v0.3.10-29-gd45ebb6 | Released on '2025-01-04'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
Traceback (most recent call last):
  File "/root/comfy/ComfyUI/nodes.py", line 2092, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/comfy/ComfyUI/custom_nodes/comfyui-videohelpersuite/__init__.py", line 1, in <module>
    from .videohelpersuite.nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
  File "/root/comfy/ComfyUI/custom_nodes/comfyui-videohelpersuite/videohelpersuite/nodes.py", line 20, in <module>
    from .load_video_nodes import LoadVideoUpload, LoadVideoPath, LoadVideoFFmpegUpload, LoadVideoFFmpegPath, LoadImagePath
  File "/root/comfy/ComfyUI/custom_nodes/comfyui-videohelpersuite/videohelpersuite/load_video_nodes.py", line 6, in <module>
    import cv2
  File "/usr/local/lib/python3.11/site-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/usr/local/lib/python3.11/site-packages/cv2/__init__.py", line 153, in bootstrap
    native_module = importlib.import_module("cv2")
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

Cannot import /root/comfy/ComfyUI/custom_nodes/comfyui-videohelpersuite module for custom nodes: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
Traceback (most recent call last):
  File "/root/comfy/ComfyUI/nodes.py", line 2092, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/comfy/ComfyUI/custom_nodes/comfyui-cogvideoxwrapper/__init__.py", line 1, in <module>
    from .nodes import NODE_CLASS_MAPPINGS as NODES_CLASS, NODE_DISPLAY_NAME_MAPPINGS as NODES_DISPLAY
  File "/root/comfy/ComfyUI/custom_nodes/comfyui-cogvideoxwrapper/nodes.py", line 397, in <module>
    from .tora.traj_utils import process_traj, scale_traj_list_to_256
  File "/root/comfy/ComfyUI/custom_nodes/comfyui-cogvideoxwrapper/tora/traj_utils.py", line 2, in <module>
    import cv2
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

Cannot import /root/comfy/ComfyUI/custom_nodes/comfyui-cogvideoxwrapper module for custom nodes: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
[34mWAS Node Suite: [0mCreated default conf file at `/root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894/was_suite_config.json`.[0m
[34mWAS Node Suite [93mWarning: [0mOpenCV Python module cannot be found. Attempting install...[0m
Found existing installation: opencv-python 4.10.0.84
Uninstalling opencv-python-4.10.0.84:
  Would remove:
    /usr/local/lib/python3.11/site-packages/cv2/*
    /usr/local/lib/python3.11/site-packages/opencv_python-4.10.0.84.dist-info/*
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libQt5Core-e6d3f451.so.5.15.13
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libQt5Gui-5dea4132.so.5.15.13
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libQt5Test-d435aae7.so.5.15.13
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libQt5Widgets-e392eaf8.so.5.15.13
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libQt5XcbQpa-3bf8a55a.so.5.15.13
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libX11-xcb-0e257303.so.1.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libXau-00ec42fe.so.6.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libavcodec-9aae324f.so.59.37.100
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libavformat-3ff1be5b.so.59.27.100
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libavutil-a0a0531e.so.57.28.100
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libcrypto-337dac8b.so.1.1
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libgfortran-91cc3cb1.so.3.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libopenblas-r0-f650aae0.3.3.so
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libpng16-1bde1c40.so.16.43.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libquadmath-96973f99.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libssl-28bef1ac.so.1.1
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libswresample-2ec4394e.so.4.7.100
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libswscale-2c3c8be7.so.6.7.100
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libvpx-c3a7933e.so.9.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-icccm-413c9f41.so.4.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-image-e82a276d.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-keysyms-21015570.so.1.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-randr-a96a5a87.so.0.1.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-render-637b984a.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-render-util-43ce00f5.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-shape-25c2b258.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-shm-7a199f70.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-sync-89374f40.so.1.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-util-4d666913.so.1.0.0
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/uninstall.py", line 106, in run
    uninstall_pathset = req.uninstall(
                        ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 723, in uninstall
    uninstalled_pathset.remove(auto_confirm, verbose)
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/req/req_uninstall.py", line 364, in remove
    if auto_confirm or self._allowed_to_proceed(verbose):
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/req/req_uninstall.py", line 404, in _allowed_to_proceed
    return ask("Proceed (Y/n)? ", ("y", "n", "")) != "n"
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 228, in ask
    response = input(message)
               ^^^^^^^^^^^^^^
EOFError: EOF when reading a line
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-xfixes-9be3ba6f.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-xinerama-ae147f87.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxcb-xkb-9ba31ab3.so.1.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxkbcommon-71ae2972.so.0.0.0
    /usr/local/lib/python3.11/site-packages/opencv_python.libs/libxkbcommon-x11-c65ed502.so.0.0.0
Traceback (most recent call last):
  File "/root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894/WAS_Node_Suite.py", line 14663, in <module>
    import cv2
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/comfy/ComfyUI/nodes.py", line 2092, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894/__init__.py", line 1, in <module>
    from .WAS_Node_Suite import NODE_CLASS_MAPPINGS
  File "/root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894/WAS_Node_Suite.py", line 14680, in <module>
    install_package(
  File "/root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894/WAS_Node_Suite.py", line 353, in install_package
    subprocess.check_call([sys.executable, '-s', '-m', 'pip', 'uninstall', *uninstall_first])
  File "/usr/local/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/local/bin/python', '-s', '-m', 'pip', 'uninstall', 'opencv-python', 'opencv-python-headless[ffmpeg]']' returned non-zero exit status 2.

Cannot import /root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894 module for custom nodes: Command '['/usr/local/bin/python', '-s', '-m', 'pip', 'uninstall', 'opencv-python', 'opencv-python-headless[ffmpeg]']' returned non-zero exit status 2.

Import times for custom nodes:
   0.0 seconds: /root/comfy/ComfyUI/custom_nodes/websocket_image_save.py
   0.1 seconds (IMPORT FAILED): /root/comfy/ComfyUI/custom_nodes/comfyui-cogvideoxwrapper
   0.2 seconds (IMPORT FAILED): /root/comfy/ComfyUI/custom_nodes/comfyui-videohelpersuite
   1.0 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-Manager
   1.0 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-HunyuanVideoWrapper
   1.2 seconds (IMPORT FAILED): /root/comfy/ComfyUI/custom_nodes/pr-was-node-suite-comfyui-47064894

Starting server

To see the GUI go to: http://0.0.0.0:8000

# ---
# deploy: true
# cmd: ["modal", "serve", "06_gpu_and_ml/comfyui/comfyapp.py"]
# ---
#
# # Run Flux on ComfyUI interactively and as an API
#
# [ComfyUI](https://github.com/comfyanonymous/ComfyUI) is an open-source diffusion model platform with a graph/nodes interface that allows you to design and execute advanced image generation pipelines.

#
# In this example, we show you how to
#
# 1. run the [Flux](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux) diffusion model on ComfyUI interactively to develop workflows
#
# 2. serve a Flux ComfyUI workflow as an API
#
# ## Quickstart
#
# This example runs `workflow_api.json` in this directory, which is an adapation of [this simple FLUX.1-schnell workflow](https://openart.ai/workflows/reverentelusarca/flux-simple-workflow-schnell/40OkdaB23J2TMTXHmxxu) with an Image Resize custom node added at the end.
#
# For the prompt `"Surreal dreamscape with floating islands, upside-down waterfalls, and impossible geometric structures, all bathed in a soft, ethereal light"`
# we got this output:
#
# ![example comfyui image](./flux_gen_image.jpeg)
#
# To serve the workflow in this example as an API:
# 1. Download the Flux models to a Modal [Volume](/docs/guide/volumes):
# ```bash
# modal run 06_gpu_and_ml/comfyui/comfyapp.py::download_models
# ```
#
# 2. Stand up the ComfyUI server in development mode:
# ```bash
# modal serve 06_gpu_and_ml/comfyui/comfyapp.py
# ```
#
# 3. In another terminal, run inference:
# ```bash
# python 06_gpu_and_ml/comfyui/comfyclient.py --dev --modal-workspace $(modal profile current) --prompt "neon green sign that says Modal"
# ```
#
# The first inference will take ~1m since the container needs to launch the ComfyUI server and load Flux into memory. Successive inferences on a warm container should take a few seconds.
#
# ## Setup
#

import json
import subprocess
import uuid
from pathlib import Path
from typing import Dict, List, Tuple

import modal

# ### Building up the environment
#
# We start from a base image and specify all of our dependencies.
# We'll call out the interesting ones as they come up below.
#
# Note that these dependencies are not installed locally.
# They are only installed in the remote environment where our app runs.
# This happens the first time. On subsequent runs, the cached image will be reused.


image = (  # build up a Modal Image to run ComfyUI, step by step
    modal.Image.debian_slim(  # start from basic Linux with Python
        python_version="3.11"
    )
    .apt_install("git")  # install git to clone ComfyUI
    .apt_install("wget")  # Install wget for downloading
    .apt_install("curl")  # Install curl for downloading
    .apt_install("libgl1-mesa-dev")  # Install libGL.so.1 and related packages
    .pip_install("torchao")  # install torchao
    .pip_install("fastapi[standard]==0.115.4")  # install web dependencies
    .pip_install("comfy-cli==1.2.7")  # install comfy-cli
    .run_commands(  # use comfy-cli to install the ComfyUI repo and its dependencies
        "comfy --skip-prompt install --nvidia"
    )
    .add_local_file(
        Path(__file__).parent / "workflow_api.json",
        "/root/workflow_api.json",
        copy=True,
    )
)
# ### Downloading custom nodes
# We'll use `comfy-cli` to download custom nodes, in this case the popular WAS Node Suite pack.
image = image.run_commands(
    [
        "comfy node install https://github.com/WASasquatch/was-node-suite-comfyui",
        "comfy node install https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite",
        "comfy node install ComfyUI-HunyuanVideoWrapper",
        "comfy node install https://github.com/kijai/ComfyUI-CogVideoXWrapper",
        # Add .run_commands(...) calls for any other custom nodes you want to download
    ]
)

# See [this post](/blog/comfyui-custom-nodes) for more on how to install custom nodes on Modal.
# ### Downloading models

# You can also use comfy-cli to download models, but for this example we'll download the Flux models directly from Hugging Face into a Modal Volume.
# Then on container start, we'll mount our models Volume into the ComfyUI models directory.
# This allows us to avoid re-downloading the models every time you rebuild your image.

image = (
    # install huggingface_hub with hf_transfer support to speed up downloads
    image.pip_install("huggingface_hub[hf_transfer]==0.26.2")
    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1"})
    .run_commands(  # needs to be empty for Volume mount to work
        "rm -rf /root/comfy/ComfyUI/models"
    )
)

# We create the app and specify the image we built above.

app = modal.App(name="example-comfyui", image=image)


# First we need to run a function to download the Flux models to a Modal Volume.

vol = modal.Volume.from_name("comfyui-models", create_if_missing=True)

# Type alias for model download parameters
ModelDownloadParams = Tuple[str, str]  # (url/repo_id, destination_path)

@app.function(
    volumes={"/root/models": vol},
)
def hf_download_repo(repo_id: str, model_type: str, ignore_patterns: List[str] = None):
    from huggingface_hub import snapshot_download

    snapshot_download(
        repo_id=repo_id,
        local_dir=f"/root/models/{model_type}",
        local_dir_use_symlinks=False,  # Crucial for Modal Volumes
        ignore_patterns=ignore_patterns or [], # Handle ignore patterns
    )
    print(f"Repository {repo_id} downloaded to /root/models/{model_type}")

@app.function(
    volumes={"/root/models": vol},
)
def hf_download_file(repo_id: str, filename: str, model_type: str):
    from huggingface_hub import hf_hub_download

    hf_hub_download(
        repo_id=repo_id,
        filename=filename,
        local_dir=f"/root/models/{model_type}",
    )
    print(f"File {filename} from {repo_id} downloaded to /root/models/{model_type}")

@app.function(
    volumes={"/root/models": vol},
)
def download_from_url(url: str, destination_path: str):
    import os
    import subprocess

    print(f"Downloading {url} to {destination_path}...")
    # Create the destination directory if it doesn't exist
    os.makedirs(os.path.dirname(destination_path), exist_ok=True)

    # Use curl to download the file with progress bar and handle redirects
    try:
        subprocess.run(
            ["curl", "--progress-bar", "--location", "-o", destination_path, url],
            check=True,
        )
        print(f"Successfully downloaded {url} to {destination_path}")
    except subprocess.CalledProcessError as e:
        print(f"Error downloading {url}: {e}")
        # Optionally delete the partially downloaded file:
        # if os.path.exists(destination_path):
        #     os.remove(destination_path)

@app.local_entrypoint()
def download_models():
    models_to_download_from_hf_repos: List[Tuple[str, str, List[str]]] = [
        # Example: Download an entire repo
        # ("stabilityai/stable-diffusion-xl-base-1.0", "checkpoints"),
        # Add your repo_id and model_type here
        ("Kijai/llava-llama-3-8b-text-encoder-tokenizer", "LLM"),
        ("openai/clip-vit-large-patch14", "LLM", ["flax_model.msgpack", "pytorch_model.bin", "tf_model.h5"]), # Exclude these file patterns
    ]

    models_to_download_from_hf_files: List[Tuple[str, str, str]] = [
        # format is (huggingface repo_id, the model filename, comfyui models subdirectory)
        ("black-forest-labs/FLUX.1-schnell", "ae.safetensors", "vae"),
        ("black-forest-labs/FLUX.1-schnell", "flux1-schnell.safetensors", "unet"),
        ("comfyanonymous/flux_text_encoders", "t5xxl_fp8_e4m3fn.safetensors", "clip"),
        ("comfyanonymous/flux_text_encoders", "clip_l.safetensors", "clip"),
        ("Kijai/HunyuanVideo_comfy", "hunyuan_video_720_cfgdistill_fp8_e4m3fn.safetensors", "diffusion_models"),
        ("Kijai/HunyuanVideo_comfy", "hunyuan_video_vae_bf16.safetensors", "vae"),
        ("Laxhar/noobai-XL-1.1", "NoobAI-XL-v1.1.safetensors", "checkpoints"),
    ]

    models_to_download_from_urls: List[ModelDownloadParams] = [
        # Example:
        # ("https://civitai.com/api/download/models/12345", "/root/models/checkpoints/my_model.safetensors"),
        # Add your URLs and desired paths here
    ]

    # Download entire HF repos in parallel
    print("Downloading entire Hugging Face repositories...")
    list(hf_download_repo.starmap(models_to_download_from_hf_repos))

    # Download models from Hugging Face in parallel
    print("Downloading individual files from Hugging Face...")
    list(hf_download_file.starmap(models_to_download_from_hf_files))

    # Download models from URLs in parallel
    print("Downloading models from URLs...")
    list(download_from_url.starmap(models_to_download_from_urls))

# To run the download step, run `modal run 06_gpu_and_ml/comfyui/comfyapp.py::download_models`.
# By leveraging [hf_transfer](https://huggingface.co/docs/huggingface_hub/en/guides/download#faster-downloads), Modal starmap for parallelism, and Volumes, image build time drops from ~10 minutes to ~25 seconds.

# ## Running ComfyUI interactively and as an API on Modal
#
# To run ComfyUI interactively, simply wrap the `comfy launch` command in a Modal Function and serve it as a web server.


@app.function(
    allow_concurrent_inputs=10,
    concurrency_limit=1,
    container_idle_timeout=30,
    timeout=1800,
    gpu="L4",
    volumes={"/root/comfy/ComfyUI/models": vol},
)
@modal.web_server(8000, startup_timeout=60)
def ui():
    subprocess.Popen("comfy launch -- --listen 0.0.0.0 --port 8000", shell=True)


# Remember to **close your UI tab** when you are done developing to avoid accidental charges to your account.
# This will close the connection with the container serving ComfyUI, which will spin down based on your `container_idle_timeout` setting.
#
# To run an existing workflow as an API, we use Modal's class syntax to run our customized ComfyUI environment and workflow on Modal.
#
# Here's the basic breakdown of how we do it:
# 1. We stand up a "headless" ComfyUI server in the background when the app starts.
# 2. We define an `infer` method that takes in a workflow path and runs the workflow on the ComfyUI server.
# 3. We stand up an `api` with `web_endpoint`, so that we can run our workflows as a service.
#
# For more on how to run web services on Modal, check out [this guide](https://modal.com/docs/guide/webhooks).
@app.cls(
    allow_concurrent_inputs=10,
    container_idle_timeout=300,
    gpu="L4",
    volumes={"/root/comfy/ComfyUI/models": vol},
)
class ComfyUI:
    @modal.enter()
    def launch_comfy_background(self):
        cmd = "comfy launch --background"
        subprocess.run(cmd, shell=True, check=True)

    @modal.method()
    def infer(self, workflow_path: str = "/root/workflow_api.json"):
        # runs the comfy run --workflow command as a subprocess
        cmd = f"comfy run --workflow {workflow_path} --wait --timeout 1200"
        subprocess.run(cmd, shell=True, check=True)

        # completed workflows write output images to this directory
        output_dir = "/root/comfy/ComfyUI/output"
        # looks up the name of the output image file based on the workflow
        workflow = json.loads(Path(workflow_path).read_text())
        file_prefix = [
            node.get("inputs")
            for node in workflow.values()
            if node.get("class_type") == "SaveImage"
        ][0]["filename_prefix"]

        # returns the image as bytes
        for f in Path(output_dir).iterdir():
            if f.name.startswith(file_prefix):
                return f.read_bytes()

    @modal.web_endpoint(method="POST")
    def api(self, item: Dict):
        from fastapi import Response

        workflow_data = json.loads(
            (Path(__file__).parent / "workflow_api.json").read_text()
        )

        # insert the prompt
        workflow_data["6"]["inputs"]["text"] = item["prompt"]

        # give the output image a unique id per client request
        client_id = uuid.uuid4().hex
        workflow_data["9"]["inputs"]["filename_prefix"] = client_id

        # save this updated workflow to a new file
        new_workflow_file = f"{client_id}.json"
        json.dump(workflow_data, Path(new_workflow_file).open("w"))

        # run inference on the currently running container
        img_bytes = self.infer.local(new_workflow_file)

        return Response(img_bytes, media_type="image/jpeg")


# ### The workflow for developing workflows
#
# When you run this script with `modal deploy 06_gpu_and_ml/comfyui/comfyapp.py`, you'll see a link that includes `ui`.
# Head there to interactively develop your ComfyUI workflow. All of your models and custom nodes specified in the image build step will be loaded in.
#
# To serve the workflow after you've developed it, first export it as "API Format" JSON:
# 1. Click the gear icon in the top-right corner of the menu
# 2. Select "Enable Dev mode Options"
# 3. Go back to the menu and select "Save (API Format)"
#
# Save the exported JSON to the `workflow_api.json` file in this directory.
#
# Then, redeploy the app with this new workflow by running `modal deploy 06_gpu_and_ml/comfyui/comfyapp.py` again.
#
# ## Further optimizations
# - To decrease inference latency, you can process multiple inputs in parallel by setting `allow_concurrent_inputs=1`, which will run each input on its own container. See our [Scaling ComfyUI](https://modal.com/blog/scaling-comfyui) blog post for more details.
# - If you're noticing long startup times for the ComfyUI server (e.g. >30s), this is likely due to too many custom nodes being loaded in. Consider breaking out your deployments into one App per unique combination of models and custom nodes.
# - For those who prefer to run a ComfyUI workflow directly as a Python script, see [this blog post](https://modal.com/blog/comfyui-prototype-to-production).