Package: comfy-script
ComfyScriptA Python front end and library forComfyUI.It has the following use cases:Serving as ahuman-readable formatfor ComfyUI's workflows.This makes it easy to compare and reuse different parts of one's workflows.It is also possible to train LLMs to generate workflows, since many LLMs can handle Python code relatively well. This approach can be more powerful than just asking LLMs for some hardcoded parameters.Scripts can be automatically translated from ComfyUI's workflows. Seetranspilerfor details.Directly running the script to generate images.The main advantage of doing this than using the web UI is being able to mix Python code with ComfyUI's nodes, such as doing loops, calling library functions, and easily encapsulating custom nodes. This also makes adding interaction easier since the UI and logic can be both written in Python. And, some people may feel more comfortable with simple Python code than a graph-based GUI.[^graph-gui]Seeruntimefor details. Scripts can be executed locally or remotely with a ComfyUI server.Using ComfyUI as a function library.With ComfyScript, ComfyUI's nodes can be used as functions to do ML research, reuse nodes in other projects, debug custom nodes, and optimize caching to run workflows faster.See runtime'sreal modefor details.Generating ComfyUI's workflows with scripts.Scripts can also be used to generate ComfyUI's workflows and then used in the web UI or elsewhere. This way, one can use loops and generate huge workflows where it would be time-consuming or impractical to create them manually. Seeworkflow generationfor details. It is also possible to load workflows from images generated by ComfyScript.Retrieving any wanted information by running the script with some stubs.Seeworkflow information retrievalfor details.Converting workflows from ComfyUI's web UI format to API format without the web UI.InstallationComfyScript can be installed in different ways.Package and nodes with ComfyUIInstallComfyUIfirst. And then:cdComfyUI/custom_nodes
gitclonehttps://github.com/Chaoses-Ib/ComfyScript.gitcdComfyScript
python-mpipinstall-e".[default]"(If you seeERROR: File "setup.py" or "setup.cfg" not found, runpython -m pip install -U pipfirst.)Update:cdComfyUI/custom_nodes/ComfyScript
gitpull
python-mpipinstall-e".[default]"Package and nodes with ComfyUI packageInstallComfyUI packagefirst:python-mpipinstallgit+https://github.com/hiddenswitch/ComfyUI.gitInstall/update ComfyScript:python-mpipinstall-U"comfy-script[default]"([default]is necessary to install common dependencies. Seepyproject.tomlfor other options. If no option is specified,comfy-scriptwill be installed without any dependencies.)Only packageInstall/update:python-mpipinstall-U"comfy-script[default]"Only nodes with ComfyUIInstallComfyUIfirst. And then:cdComfyUI/custom_nodes
gitclonehttps://github.com/Chaoses-Ib/ComfyScript.gitcdComfyScript
python-mpipinstall-rrequirements.txtUpdate:cdComfyUI/custom_nodes/ComfyScript
gitpull
python-mpipinstall-rrequirements.txtIf you want, you can still import the package with a hardcoded path:importsys# Or just '../src' if used in the examples directorysys.path.insert(0,r'D:\...\ComfyUI\custom_nodes\ComfyScript\src')importcomfy_scriptTranspilerThe transpiler can translate ComfyUI's workflows to ComfyScript.When ComfyScript is installed as custom nodes,SaveImageand similar nodes will be hooked to automatically save the script as images' metadata. And the script will also be output to the terminal.For example, here is a workflow in ComfyUI:ComfyScript translated from it:model,clip,vae=CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')conditioning=CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,',clip)conditioning2=CLIPTextEncode('text, watermark',clip)latent=EmptyLatentImage(512,512,1)latent=KSampler(model,156680208700286,20,8,'euler','normal',conditioning,conditioning2,latent,1)image=VAEDecode(latent,vae)SaveImage(image,'ComfyUI')If there two or moreSaveImagenodes in one workflow, only the necessary inputs of each node will be translated to scripts. For example, here is a 2 pass txt2img (hires fix) workflow:ComfyScript saved for each of the two saved image are respectively:model,clip,vae=CheckpointLoaderSimple('v2-1_768-ema-pruned.ckpt')conditioning=CLIPTextEncode('masterpiece HDR victorian portrait painting of woman, blonde hair, mountain nature, blue sky',clip)conditioning2=CLIPTextEncode('bad hands, text, watermark',clip)latent=EmptyLatentImage(768,768,1)latent=KSampler(model,89848141647836,12,8,'dpmpp_sde','normal',conditioning,conditioning2,latent,1)image=VAEDecode(latent,vae)SaveImage(image,'ComfyUI')model,clip,vae=CheckpointLoaderSimple('v2-1_768-ema-pruned.ckpt')conditioning=CLIPTextEncode('masterpiece HDR victorian portrait painting of woman, blonde hair, mountain nature, blue sky',clip)conditioning2=CLIPTextEncode('bad hands, text, watermark',clip)latent=EmptyLatentImage(768,768,1)latent=KSampler(model,89848141647836,12,8,'dpmpp_sde','normal',conditioning,conditioning2,latent,1)latent2=LatentUpscale(latent,'nearest-exact',1152,1152,'disabled')latent2=KSampler(model,469771404043268,14,8,'dpmpp_2m','simple',conditioning,conditioning2,latent2,0.5)image=VAEDecode(latent2,vae)SaveImage(image,'ComfyUI')Comparing scripts:You can also use the transpiler via theCLI.RuntimeWith the runtime, one can run ComfyScript like this:fromcomfy_script.runtimeimport*load()fromcomfy_script.runtime.nodesimport*withWorkflow():model,clip,vae=CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')conditioning=CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,',clip)conditioning2=CLIPTextEncode('text, watermark',clip)latent=EmptyLatentImage(512,512,1)latent=KSampler(model,156680208700286,20,8,'euler','normal',conditioning,conditioning2,latent,1)image=VAEDecode(latent,vae)SaveImage(image,'ComfyUI')A Jupyter Notebook example is available atexamples/runtime.ipynb. (Files underexamplesdirectory will be ignored by Git and you can put your personal notebooks there.)Type stubswill be generated atcomfy_script/runtime/nodes.pyiafter loading. Mainstream code editors (e.g.VS Code) can use them to help with coding:Python enumerationsare generated for all arguments provding the value list. So instead of copying and pasting strings like'v1-5-pruned-emaonly.ckpt', you can use:Checkpoints.v1_5_pruned_emaonly# orCheckpointLoaderSimple.ckpt_name.v1_5_pruned_emaonlyEmbeddings can also be referenced asEmbeddings.my_embedding, which is equivalent to'embedding:my-embedding'.Seeenumerationsfor details.The runtime is asynchronous by default. You can queue multiple tasks without waiting for the first one to finish. A daemon thread will watch and report the remaining tasks in the queue and the current progress, for example:Queue remaining: 1
Queue remaining: 2
100%|██████████████████████████████████████████████████| 20/20
Queue remaining: 1
100%|██████████████████████████████████████████████████| 20/20
Queue remaining: 0Some control functions are also available:# Interrupt the current taskqueue.cancel_current()# Clear the queuequeue.cancel_remaining()# Interrupt the current task and clear the queuequeue.cancel_all()# Call the callback when the queue is emptyqueue.when_empty(callback)# With Workflow:Workflow(cancel_remaining=True)Workflow(cancel_all=True)Seedifferences from ComfyUI's web UIif you are a previous user of ComfyUI's web UI, andruntimefor the details of runtime.ExamplesPlottingwithWorkflow():seed=0pos='sky, 1girl, smile'neg='embedding:easynegative'model,clip,vae=CheckpointLoaderSimple(Checkpoints.AOM3A1B_orangemixs)model2,clip2,vae2=CheckpointLoaderSimple(Checkpoints.CounterfeitV25_25)model2=TomePatchModel(model2,0.5)forcolorin'red','green','blue':latent=EmptyLatentImage(440,640)latent=KSampler(model,seed,steps=15,cfg=6,sampler_name='uni_pc',positive=CLIPTextEncode(f'{color},{pos}',clip),negative=CLIPTextEncode(neg,clip),latent_image=latent)SaveImage(VAEDecode(latent,vae2),f'{seed}{color}')latent=LatentUpscaleBy(latent,scale_by=2)latent=KSampler(model2,seed,steps=15,cfg=6,sampler_name='uni_pc',positive=CLIPTextEncode(f'{color},{pos}',clip2),negative=CLIPTextEncode(neg,clip2),latent_image=latent,denoise=0.6)SaveImage(VAEDecode(latent,vae2),f'{seed}{color}hires')Auto queueAutomatically queue new workflows when the queue becomes empty.For example, one can usecomfyui-photoshop(currently a bit buggy) to automatically do img2img with the image in Photoshop when it changes:deff(wf):seed=0pos='1girl, angry, middle finger'neg='embedding:easynegative'model,clip,vae=CheckpointLoaderSimple(Checkpoints.CounterfeitV25_25)image,width,height=PhotoshopToComfyUI(wait_for_photoshop_changes=True)latent=VAEEncode(image,vae)latent=LatentUpscaleBy(latent,scale_by=1.5)latent=KSampler(model,seed,steps=15,cfg=6,sampler_name='uni_pc',positive=CLIPTextEncode(pos,clip),negative=CLIPTextEncode(neg,clip),latent_image=latent,denoise=0.8)PreviewImage(VAEDecode(latent,vae))queue.when_empty(f)Screenshot:Select and processFor example, to generate 3 images at once, and then let the user decide which ones they want to hires fix:importipywidgetsaswidgetsqueue.watch_display(False,False)latents=[]image_batches=[]withWorkflow():seed=0pos='sky, 1girl, smile'neg='embedding:easynegative'model,clip,vae=CheckpointLoaderSimple(Checkpoints.AOM3A1B_orangemixs)model2,clip2,vae2=CheckpointLoaderSimple(Checkpoints.CounterfeitV25_25)forcolorin'red','green','blue':latent=EmptyLatentImage(440,640)latent=KSampler(model,seed,steps=15,cfg=6,sampler_name='uni_pc',positive=CLIPTextEncode(f'{color},{pos}',clip),negative=CLIPTextEncode(neg,clip),latent_image=latent)latents.append(latent)image_batches.append(SaveImage(VAEDecode(latent,vae),f'{seed}{color}'))grid=widgets.GridspecLayout(1,len(image_batches))fori,image_batchinenumerate(image_batches):image_batch=image_batch.wait()image=widgets.Image(value=image_batch[0]._repr_png_())button=widgets.Button(description=f'Hires fix{i}')defhiresfix(button,i=i):print(f'Image{i}is chosen')withWorkflow():latent=LatentUpscaleBy(latents[i],scale_by=2)latent=KSampler(model2,seed,steps=15,cfg=6,sampler_name='uni_pc',positive=CLIPTextEncode(pos,clip2),negative=CLIPTextEncode(neg,clip2),latent_image=latent,denoise=0.6)image_batch=SaveImage(VAEDecode(latent,vae2),f'{seed}hires')display(image_batch.wait())button.on_click(hiresfix)grid[0,i]=widgets.VBox(children=(image,button))display(grid)This example usesipywidgetsfor the GUI, but other GUI frameworks can be used as well.Screenshot:DocumentationRuntimeImagesAdditional NodesTranspiler[^graph-gui]:I hate nodes. (No offense comfyui) : StableDiffusion