Package: comfy-catapult
READMEWarning:Very raw and unmaintained code. Use at your own risk. Mainly
intended as a starting point.What is it?Comfy Catapult is a library for scheduling and running ComfyUI workflows from a
Python program, via the existing API endpoint. ComfyUI typically works by
hosting this API endpoint for its user interface.This makes it easier for you to make workflows via the UI, and then use it from
a program.Scheduling a jobFromexamples/sdxlturbo_example_catapulter.py:asyncdefRunExampleWorkflow(*,job_info:ExampleWorkflowInfo):# You have to write this function, to change the workflow_dict as you like.awaitPrepareWorkflow(job_info=job_info)# Here the magic happens, the job is submitted to the ComfyUI server.job_info.job_history_dict=awaitjob_info.catapult.Catapult(job_id=job_info.job_id,prepared_workflow=job_info.workflow_dict,important=job_info.important)# Now that the job is done, you have to write something that will go and get# the results you care about, if necessary.awaitDownloadResults(job_info=job_info)classExampleWorkflowInfo:# Direct wrapper around the ComfyUI API.client:ComfyAPIClientBase# Job scheduler (the main point of this library).catapult:ComfyCatapultBase# Something to help with retrieving files from the ComfyUI storage.remote:RemoteFileAPIBasecomfy_api_url:str# This should be the workflow json as a dict.workflow_template_dict:dict# This should begin as a deep copy of the template.workflow_dict:dict# This will hold the node ids that we must have results for.important:List[NodeID]# Make this any string unique to this job.job_id:str# When the job is complete, this will be the `/history` json/dictionary for# this job.job_history_dict:dict|None# These are inputs that modify this particular workflow.ckpt_name:str|Nonepositive_prompt:strnegative_prompt:str# For this particular workflow, this will define the path to the output image.output_path:PathRelated Projectscomfyui-deploy.ComfyUI script_examples.ComfyUI-to-Python-Extension.ComfyScripthordelibComfyUI_NetDistComfyUI-Serving-Toolkitcomfyui-python-apiGetting StartedExporting workflows in the API json formatIn ComfyUI web interface:Open settings (gear box in the corner).Enable the ability to export in the API format,Enable Dev mode Options.Click new menu itemSave (API format).Example workflow: Prepare ComfyUIIf you don't want to try the example workflow, you can skip this section.You need to getsd_xl_turbo_1.0_fp16.safetensorsinto the ComfyUI model
directory.Hugging Face page:huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors.Direct download link:huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors.Download the example workflow, and export it in the API formatThis is optional, you can use the example workflow intest_data/instead and
skip this step.# Download the workflow:wgethttps://github.com/comfyanonymous/ComfyUI_examples/raw/master/sdturbo/sdxlturbo_example.png# 1. Open the Workflow in ComfyUI and export it. AFAIK there isn't a nice way# to automated this right now.## 2, Save to `./sdxlturbo_example_api.json`.## Or just use `test_data/sdxlturbo_example_api.json`.Install as a library from git and run the examples# Inside your environment:pipinstallgit+https://github.com/realazthat/comfy-catapult.git# Or (inside your environment):gitclonehttps://github.com/realazthat/comfy-catapult.gitcdcomfy-catapult
pipinstall.# If you set this environment variable, you don't have to specify it as an# argument.exportCOMFY_API_URL=http://127.0.0.1:8188# Note, in WSL2 you may have to use the following if ComfyUI is running on the# Windows side:exportCOMFY_API_URL=http://host.docker.internal:8188


python-mcomfy_catapult.examples.sdxlturbo_example_catapulter\--api_workflow_json_path"$PWD/sdxlturbo_example_api.json"\--tmp_path"$PWD/.deleteme/tmp/"\--output_path"$PWD/.deleteme/output.png"\--positive_prompt"amazing cloudscape, towering clouds, thunderstorm, awe"\--negative_prompt"dull, blurry, nsfw"# Optional if you don't want to set the environment variable:#   --comfy_api_url "..."# Done! Now $PWD/.deleteme/output.png should contain the output image.# Some other examples:python-mcomfy_catapult.examples.add_a_node
python-mcomfy_catapult.examples.using_pydanticExamineexamples/sdxlturbo_example_catapulter.pyto see how to use the mainComfyCatapultlibrary.Examinetest_data/sdxlturbo_example_api.jsonto see the API format. This will be necessary in order to programmatically set
the proper inputs for the workflow.(Optional) Seeexamples/using_pydantic.pyfor how to parse the API format into the Pydantic models schema for easier
navigation.(Optional) Seeexamples/add_a_node.pyfor how to
add a new node to a workflow. This is useful when you need to add nodes at
runtime (such as adding a bunch of LoadImage nodes).Seecomfy_catapult/catapult_base.pyfor
the main library interface.(Optional) Seecomfy_catapult/catapult.pyfor the main library implementation.(Optional) Seecomfy_catapult/api_client_base.pyfor
the direct ComfyUI API endpoint client library interface; you don't need to
use this usually.(Optional) For those who want to do use the raw API themselves and learn how
it works: Examinecomfy_catapult/api_client.pyto see the API
client implementation if you want to directly interface with ComfyUI endpoints
yourself.(Optional) Also seeComfyUI/server.py(pinned to a specific commit) for the server@routesendpoint
implementations.Development; install dependencies and run the examplesThis is if you are intending on contributing or altering the library itself.gitclonehttps://github.com/realazthat/comfy-catapult.gitcdcomfy-catapult
pipinstall-rrequirements.txt# Run the example workflow:PYTHONPATH=$PYTHONPATH:$PWDpythonexamples/sdxlturbo_example_catapulter.py\--api_workflow_json_path"$PWD/sdxlturbo_example_api.json"--tmp_path"$PWD/.deleteme/tmp/"\--output_path"$PWD/.deleteme/output.png"\--positive_prompt"amazing cloudscape, towering clouds, thunderstorm, awe"\--negative_prompt"dull, blurry, nsfw"Parsing the API format into the Pydantic models schema for easier navigationFromexamples/using_pydantic.py:# -*- coding: utf-8 -*-# SPDX-License-Identifier: MIT## The Comfy Catapult project requires contributions made to this file be licensed# under the MIT license or a compatible open source license. See LICENSE.md for# the license text.fromcomfy_catapult.comfy_schemaimportAPIWorkflowapi_workflow_json_str:str="""{"1": {"inputs": {"image": "{remote_image_path}[input]","upload": "image"},"class_type": "LoadImage","_meta": {"title": "My Loader Title"}},"25": {"inputs": {"images": ["8",0]},"class_type": "PreviewImage","_meta": {"title": "Preview Image"}}}"""api_workflow:APIWorkflow=APIWorkflow.model_validate_json(api_workflow_json_str)# Or, if you have a APIWorkflow, and you want to deal with a dict instead:api_workflow_dict=api_workflow.model_dump()# Or, back to json:api_workflow_json=api_workflow.model_dump_json()# See comfy_catapult/comfyui_schema.py for the schema definition.print(api_workflow_json)Adding a new node to a workflowFromexamples/add_a_node.py:# -*- coding: utf-8 -*-# SPDX-License-Identifier: MIT## The Comfy Catapult project requires contributions made to this file be licensed# under the MIT license or a compatible open source license. See LICENSE.md for# the license text.frompathlibimportPathfromcomfy_catapult.comfy_schemaimport(APIWorkflow,APIWorkflowNodeInfo,APIWorkflowNodeMeta)fromcomfy_catapult.comfy_utilsimportGenerateNewNodeIDapi_workflow_json_str:str="""{"1": {"inputs": {"image": "{remote_image_path}[input]","upload": "image"},"class_type": "LoadImage","_meta": {"title": "My Loader Title"}},"25": {"inputs": {"images": ["8",0]},"class_type": "PreviewImage","_meta": {"title": "Preview Image"}}}"""api_workflow:APIWorkflow=APIWorkflow.model_validate_json(api_workflow_json_str)path_to_comfy_input=Path('/path/to/ComfyUI/input')path_to_image=path_to_comfy_input/'image.jpg'rel_path_to_image=path_to_image.relative_to(path_to_comfy_input)# Add a new LoadImage node to the workflow.new_node_id=GenerateNewNodeID(workflow=api_workflow)api_workflow.root[new_node_id]=APIWorkflowNodeInfo(inputs={'image':f'{rel_path_to_image}[input]','upload':'image',},class_type='LoadImage',_meta=APIWorkflowNodeMeta(title='My Loader Title'))print(api_workflow.model_dump_json())Known to work onPython 3.11.4, WSL2/Windows11, Ubuntu 22.04.2 LTSLimitationsETA estimator isn't workingSometimes the job ends early but no error is sent back from the server. Error
is detected because not all nodes have executed, but the error is opaque
(check the server logs for the error).TODOHelpers should support remote/cloud storage for ComfyUI input/output/model
directories. (Currently only supports local paths.)ETA Estimator.Make sure the schema can parse the formats even if the format adds new
fields.Comittingbash scripts/pre.sh.Check for modified files:git statusStage any modified:git add -uIf any modified files: Go to step 1.git commit -m "...".ReleasingBump version inREADME.template.md,setup.py,CHANGELOG.md.REL_VER=....git commit -nam "Release $REL_VER".git push.git tag $REL_VER.git push --tags.Create a GitHub release.