Skill: nhuquangls/comfyui-automation (version 1.0.0)

--- name: comfyui description: Use when you need to automate ComfyUI tasks. This skill provides instructions and scripts for installing ComfyUI, parsing API-format workflow JSON files, checking and downloading missing models, and executing workflows. Use this for requests like "run this comfyui workflow", "set up comfyui", or "generate an image using this json". --- # ComfyUI Automation Skill This skill provides a reliable, standardized workflow for interacting with ComfyUI. It is specifically designed to be easily executed by automated agents. ## 1. Environment Setup & Verification ComfyUI should be installed in the primary workspace. **Never** install directly using root `pip` to avoid OS conflicts (PEP 668); always use a virtual environment (`venv`). **Check if installed:** ```bash ls /root/.openclaw/workspace/ComfyUI/venv/bin/activate ``` **If missing, install:** ```bash cd /root/.openclaw/workspace git clone https://github.com/comfyanonymous/ComfyUI.git cd ComfyUI python3 -m venv venv source venv/bin/activate pip install --upgrade pip pip install -r requirements.txt ``` ## 2. Workflow Format ComfyUI workflows must be in **API Format** (a flat JSON dictionary where keys are stringified Node IDs). If the user provides a standard UI format JSON (which contains `links`, `pos`, `groups` arrays), kindly request the API Format or use an appropriate converter. ## 3. Checking and Downloading Missing Models Workflows will fail if the required weights (UNet, Checkpoints, VAE, LoRA, CLIP) are missing from the `ComfyUI/models/` subdirectories. To avoid redownloading large files, **always check if the file exists first**. Use the provided script to scan the workflow JSON for model requirements and check them against the local directory: ```bash python3 scripts/analyze_models.py /path/to/workflow_api.json /root/.openclaw/workspace/ComfyUI ``` **Downloading:** If models are missing, formulate a download script. Always use `wget -nc` (no clobber) or `-c` (continue) to prevent overwriting or duplicate downloads. Example: ```bash cd /root/.openclaw/workspace/ComfyUI/models/checkpoints wget -nc https://huggingface.co/path/to/model.safetensors ``` ## 4. Execution via API Do not try to run ComfyUI workflows by modifying the UI state. Instead: 1. Ensure the ComfyUI server is running locally (usually port `8188`). 2. Write a lightweight Python wrapper that loads the API JSON, alters necessary parameters (like Prompts or Seeds), and posts it to the API. **Example Python Wrapper:** ```python import json, urllib.request, random # 1. Load the template with open("workflow_api.json", "r") as f: workflow = json.load(f) # 2. Inject user variables (Identify the correct Node IDs beforehand) # workflow["27"]["inputs"]["text"] = "A beautiful sunset..." # workflow["3"]["inputs"]["seed"] = random.randint(1, 9999999) # 3. Submit to API payload = json.dumps({"prompt": workflow}).encode('utf-8') req = urllib.request.Request("http://127.0.0.1:8188/prompt", data=payload) req.add_header('Content-Type', 'application/json') res = json.loads(urllib.request.urlopen(req).read()) print(f"Prompt queued with ID: {res['prompt_id']}") ``` ## 5. Error Handling & Reporting - **Missing Nodes:** If the API returns an error about a missing class/node, locate the corresponding Custom Node repository, clone it into `ComfyUI/custom_nodes/`, and `pip install -r requirements.txt` inside its folder using the `venv`. - **OOM (Out of Memory):** If the GPU runs out of VRAM, suggest adding optimization arguments like `--lowvram` to the ComfyUI startup command. - **Reporting:** Always report the explicit failure reason to the user, including which node failed and what is missing.

path: scripts/analyze_models.py
content: import json
import sys
import os
import re

def analyze_workflow(json_path, comfy_dir):
    try:
        with open(json_path, "r") as f:
            workflow = json.load(f)
    except Exception as e:
        print(f"Error reading JSON: {e}")
        sys.exit(1)

    # Common model loaders and their target directories
    loaders = {
        "UNETLoader": "diffusion_models",
        "CheckpointLoaderSimple": "checkpoints",
        "VAELoader": "vae",
        "CLIPLoader": "text_encoders",
        "LoraLoader": "loras",
        "ControlNetLoader": "controlnet"
    }

    missing_models = []

    # Iterate through the API-format JSON
    for node_id, node_data in workflow.items():
        class_type = node_data.get("class_type")
        if class_type in loaders:
            target_dir = loaders[class_type]
            
            # The model filename is usually in the inputs dictionary
            inputs = node_data.get("inputs", {})
            
            # Find the filename (could be named ckpt_name, vae_name, lora_name, unet_name, etc.)
            model_filename = None
            for key, val in inputs.items():
                if isinstance(val, str) and (val.endswith(".safetensors") or val.endswith(".ckpt") or val.endswith(".pt")):
                    model_filename = val
                    break
            
            if model_filename:
                # Resolve the absolute path
                model_path = os.path.join(comfy_dir, "models", target_dir, model_filename)
                
                if not os.path.exists(model_path):
                    missing_models.append((class_type, model_filename, target_dir))

    if missing_models:
        print("MISSING MODELS DETECTED:")
        for mt, fname, tdir in missing_models:
            print(f"  - [{mt}] requires '{fname}' in 'models/{tdir}/'")
        print("\nPlease download the missing models using wget -nc before running the workflow.")
        sys.exit(2)
    else:
        print("All required models are present.")
        sys.exit(0)

if __name__ == "__main__":
    if len(sys.argv) < 3:
        print("Usage: python3 analyze_models.py <workflow_api.json> <comfyui_base_dir>")
        sys.exit(1)
    
    json_file = sys.argv[1]
    comfy_base = sys.argv[2]
    analyze_workflow(json_file, comfy_base)

Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)