Skill: salmonrk/openclaw-comfyui (version 1.0.4)

# ComfyUI-OpenClaw Skill 🎨✨ A professional, token-saving agent skill for connecting and controlling ComfyUI via API. Designed for high efficiency, automatic asset handling, and seamless integration with OpenClaw. ## 🏗️ Skill Structure - **Host Address:** `192.168.1.38:8190` (Configured in `TOOLS.md`) - **Workflow Directory:** `skills/comfyui/workflows/` (Self-contained within the skill folder) - **Output Directory:** `outputs/comfy/` (Relative to workspace root) - **Core Script:** `skills/comfyui/comfy_client.py` (Handles prompt injection, image uploads, and result polling) ## 🛠️ Tools (CLI) Invoke via the `exec` command: `python3 skills/comfyui/comfy_client.py <template_id> "<prompt>" [input_image_path/orientation] [orientation]` ### Parameters: - **template_id:** 1. `gen_z`: Text-to-Image (uses `image_z_image_turbo.json`) 2. `qwen_edit`: Image-to-Image / Editing (uses `qwen_image_edit_2511.json`) - *Supports automatic image upload.* - **prompt:** The description of the image to generate or edits to perform. - **input_image_path:** (Optional) Local path for image-to-image tasks. - **orientation:** (Optional) Set to `portrait` (720x1280) or `landscape` (1280x720). Defaults to `portrait`. ## 💡 How to Add New Workflows You can expand this skill easily: 1. Place your new API-formatted JSON workflow in `skills/comfyui/workflows/`. 2. Update the `WORKFLOW_MAP` dictionary in `skills/comfyui/comfy_client.py` with a new ID and the file path. 3. (Optional) If the workflow uses unique node types, adjust the injection logic in the script's `main()` function. ## 🚀 Token-Saving Strategy - **Template Mapping:** Never send full workflow JSONs in the chat. Refer to them by `template_id`. - **Vision-Saving Strategy:** To minimize token usage, the agent should prioritize using the **file path from metadata** instead of analyzing image content via vision capabilities unless explicitly asked to describe or analyze the image. - **Direct Delivery:** Deliver images directly to users via messaging plugins (e.g., Telegram) or local file openers (`open`) to avoid bloating the LLM's context window with base64 data.

path: README.md
content: # 🦞 OpenClaw-ComfyUI Skill

A professional, token-saving agent skill for controlling ComfyUI through OpenClaw.

## ✨ Features
- **Token Optimization:** Uses Template IDs instead of sending full Workflow JSONs in prompts.
- **Auto-Asset Management:** 
    - Automatically **uploads** input images to ComfyUI remote host.
    - Automatically **polls and downloads** generated results to your local workspace.
- **Multi-Output Support:** Easily send results to Telegram, open on Mac, or display via Canvas.
- **Scalable Architecture:** Add new workflows by simply dropping JSON files into the `workflows/` folder.

## 🚀 Installation

1. Clone this repository into your OpenClaw `skills/` directory:
   ```bash
   cd ~/.openclaw/workspace/skills
   git clone https://github.com/SalmonRK/OpenClaw-ComfyUI comfyui
   ```

2. Install Python dependencies:
   ```bash
   pip3 install requests
   ```

3. Update your `TOOLS.md` in the workspace root:
   ```markdown
   ### ComfyUI
   - Host: <YOUR_COMFY_IP>
   - Port: <YOUR_COMFY_PORT>
   - Workflows Directory: skills/comfyui/workflows/
   - Output Directory: outputs/comfy/
   ```

## 🛠 Usage
The AI Agent can now use this skill by calling the `comfy_client.py` script. 

Example CLI command:
```bash
python3 skills/comfyui/comfy_client.py <template_id> "<prompt>" [input_image_path]
```

## 📂 Project Structure
- `comfy_client.py`: Core logic for API interaction.
- `SKILL.md`: Instructions for the AI Agent.
- `workflows/`: Collection of ComfyUI workflow JSON files.

---
Developed with ❤️ by **SalmonRK** & **Ava (เอวา)** 🎀


path: comfy_client.py
content: import json
import requests
import sys
import os
import time
import re

# --- DYNAMIC CONFIG ---
# Default values
COMFY_HOST = "127.0.0.1"
COMFY_PORT = "8188"

# Path logic
SKILL_ROOT = os.path.dirname(os.path.abspath(__file__))
WORKSPACE_ROOT = os.path.abspath(os.path.join(SKILL_ROOT, "..", ".."))
TOOLS_PATH = os.path.join(WORKSPACE_ROOT, "TOOLS.md")
OUTPUT_DIR = os.path.join(WORKSPACE_ROOT, "outputs", "comfy")
WORKFLOW_DIR = os.path.join(SKILL_ROOT, "workflows")

# Attempt to read from TOOLS.md to avoid hardcoding (Security Compliance)
if os.path.exists(TOOLS_PATH):
    with open(TOOLS_PATH, 'r') as f:
        content = f.read()
        host_match = re.search(r'Host:\s*([\d\.]+)', content)
        port_match = re.search(r'Port:\s*(\d+)', content)
        if host_match: COMFY_HOST = host_match.group(1)
        if port_match: COMFY_PORT = port_match.group(1)

COMFY_URL = f"http://{COMFY_HOST}:{COMFY_PORT}"

# ALLOWED EXTENSIONS (Security Check)
ALLOWED_EXT = {'.jpg', '.jpeg', '.png', '.webp', '.mp4', '.mov', '.wav', '.mp3'}

WORKFLOW_MAP = {
    "gen_z": os.path.join(WORKFLOW_DIR, "image_z_image_turbo.json"),
    "qwen_edit": os.path.join(WORKFLOW_DIR, "qwen_image_edit_2511.json")
}

os.makedirs(OUTPUT_DIR, exist_ok=True)

def upload_file(input_path):
    ext = os.path.splitext(input_path)[1].lower()
    if ext not in ALLOWED_EXT:
        raise ValueError(f"Security Error: File type {ext} is not allowed.")
    
    with open(input_path, 'rb') as f:
        files = {'image': f} # ComfyUI uses 'image' key for all uploads usually
        res = requests.post(f"{COMFY_URL}/upload/image", files=files)
        return res.json()

def send_prompt(workflow_data):
    p = {"prompt": workflow_data}
    data = json.dumps(p).encode('utf-8')
    res = requests.post(f"{COMFY_URL}/prompt", data=data)
    return res.json()

def check_history(prompt_id):
    res = requests.get(f"{COMFY_URL}/history/{prompt_id}")
    return res.json()

def download_file(filename, subfolder, folder_type):
    url = f"{COMFY_URL}/view?filename={filename}&subfolder={subfolder}&type={folder_type}"
    res = requests.get(url)
    file_path = os.path.join(OUTPUT_DIR, filename)
    with open(file_path, "wb") as f:
        f.write(res.content)
    return file_path

def main():
    if len(sys.argv) < 3:
        print(json.dumps({"error": "Usage: python3 comfy_client.py <template_id> <prompt_text> [input_path/orientation] [orientation]"}))
        return

    template_id = sys.argv[1]
    prompt_text = sys.argv[2]
    
    input_path = None
    orientation = "portrait" 

    for arg in sys.argv[3:]:
        if arg.lower() in ["portrait", "landscape"]:
            orientation = arg.lower()
        elif os.path.exists(arg):
            input_path = arg

    width, height = (720, 1280) if orientation == "portrait" else (1280, 720)

    if template_id not in WORKFLOW_MAP:
        print(json.dumps({"error": f"Unknown template: {template_id}"}))
        return

    with open(WORKFLOW_MAP[template_id], 'r') as f:
        workflow = json.load(f)

    uploaded_filename = None
    if input_path:
        try:
            upload_res = upload_file(input_path)
            uploaded_filename = upload_res.get("name")
        except Exception as e:
            print(json.dumps({"error": str(e)}))
            return

    for node_id in workflow:
        node = workflow[node_id]
        if node.get("class_type") in ["CLIPTextEncode", "TextEncodeQwenImageEditPlus"]:
            if "inputs" in node:
                if "prompt" in node["inputs"]: node["inputs"]["prompt"] = prompt_text
                elif "text" in node["inputs"]: node["inputs"]["text"] = prompt_text
        
        if uploaded_filename and node.get("class_type") == "LoadImage":
            node["inputs"]["image"] = uploaded_filename

        if node.get("class_type") in ["EmptyLatentImage", "LatentImage", "EmptyImage", "EmptySD3LatentImage"]:
            if "inputs" in node:
                node["inputs"]["width"] = width
                node["inputs"]["height"] = height

    prompt_res = send_prompt(workflow)
    prompt_id = prompt_res.get("prompt_id")
    
    if not prompt_id:
        print(json.dumps({"error": "Failed to get prompt_id", "response": prompt_res}))
        return

    print(f"Connected to {COMFY_HOST}:{COMFY_PORT}. Job: {prompt_id}. Waiting...", file=sys.stderr)
    while True:
        history = check_history(prompt_id)
        if prompt_id in history:
            outputs = history[prompt_id].get("outputs", {})
            results = []
            for node_id in outputs:
                if "images" in outputs[node_id]:
                    for img in outputs[node_id]["images"]:
                        local = download_file(img["filename"], img["subfolder"], img["type"])
                        results.append(local)
            
            print(json.dumps({"status": "success", "files": results, "resolution": f"{width}x{height}"}))
            return
        time.sleep(2)

if __name__ == "__main__":
    main()


path: manifest.json
content: {
  "name": "comfyui-skill",
  "version": "1.0.4",
  "description": "Professional ComfyUI control skill for OpenClaw with token-saving architecture, auto-asset management (Image/Video/Audio), and dynamic security validation.",
  "author": "SalmonRK",
  "repository": "https://github.com/SalmonRK/OpenClaw-ComfyUI",
  "keywords": ["comfyui", "ai-automation", "token-saving", "security-first"],
  "license": "MIT",
  "main": "comfy_client.py",
  "skill_file": "SKILL.md"
}


path: workflows/image_z_image_turbo.json
content: {
  "9": {
    "inputs": {
      "filename_prefix": "z-image",
      "images": [
        "43",
        0
      ]
    },
    "class_type": "SaveImage",
    "_meta": {
      "title": "Save Image"
    }
  },
  "39": {
    "inputs": {
      "clip_name": "qwen_3_4b.safetensors",
      "type": "lumina2",
      "device": "default"
    },
    "class_type": "CLIPLoader",
    "_meta": {
      "title": "Load CLIP"
    }
  },
  "40": {
    "inputs": {
      "vae_name": "ae.safetensors"
    },
    "class_type": "VAELoader",
    "_meta": {
      "title": "Load VAE"
    }
  },
  "41": {
    "inputs": {
      "width": 1280,
      "height": 720,
      "batch_size": 1
    },
    "class_type": "EmptySD3LatentImage",
    "_meta": {
      "title": "EmptySD3LatentImage"
    }
  },
  "42": {
    "inputs": {
      "conditioning": [
        "45",
        0
      ]
    },
    "class_type": "ConditioningZeroOut",
    "_meta": {
      "title": "ConditioningZeroOut"
    }
  },
  "43": {
    "inputs": {
      "samples": [
        "44",
        0
      ],
      "vae": [
        "40",
        0
      ]
    },
    "class_type": "VAEDecode",
    "_meta": {
      "title": "VAE Decode"
    }
  },
  "44": {
    "inputs": {
      "seed": 181294798325377,
      "steps": 9,
      "cfg": 1,
      "sampler_name": "res_multistep",
      "scheduler": "simple",
      "denoise": 1,
      "model": [
        "47",
        0
      ],
      "positive": [
        "45",
        0
      ],
      "negative": [
        "42",
        0
      ],
      "latent_image": [
        "41",
        0
      ]
    },
    "class_type": "KSampler",
    "_meta": {
      "title": "KSampler"
    }
  },
  "45": {
    "inputs": {
      "text": [
        "50",
        0
      ],
      "clip": [
        "39",
        0
      ]
    },
    "class_type": "CLIPTextEncode",
    "_meta": {
      "title": "CLIP Text Encode (Prompt)"
    }
  },
  "46": {
    "inputs": {
      "unet_name": "Z-image\\zimageTurboByStable_labRatV0FP8.safetensors",
      "weight_dtype": "default"
    },
    "class_type": "UNETLoader",
    "_meta": {
      "title": "Load Diffusion Model"
    }
  },
  "47": {
    "inputs": {
      "shift": 3,
      "model": [
        "46",
        0
      ]
    },
    "class_type": "ModelSamplingAuraFlow",
    "_meta": {
      "title": "ModelSamplingAuraFlow"
    }
  },
  "50": {
    "inputs": {
      "text": "alluring young thailand  idol girl, \nHer long, sandy-brown wavy hair flows in the ocean breeze \n,   slender waist, huge breasts and wide hips, \ntilting_head,Head gently angled ~ curious, playful, or just soaking in the moment.\n一位金发碧眼的年轻女性坐在一张古旧花纹布艺沙发上，她身穿一件白色改良式旗袍，领口系着黑色丝带，胸前和腰部有黑色绑带设计，裙摆开叉至大腿，露出修长双腿，脚踩一双黑色粗跟高跟鞋，姿态优雅而略带挑逗感，右手持一把绘有红梅图案的折扇半遮面，眼神直视镜头，表情冷静而深邃，她的金色长发自然垂落肩头，皮肤白皙细腻，整体造型融合了东方古典美学与现代性感元素，前景是人物主体占据画面中心偏左位置，身体微微侧转，右腿弯曲抬起，左腿伸展，形成优美的S型曲线，背景是一面带有木质格栅的窗户，窗外透入柔和的自然光，窗边摆放一株高大的翠绿竹子，叶片细长，生机盎然，室内光线温暖，主要光源来自右侧窗户，形成从右向左的斜射光，营造出明暗对比强烈的光影效果，左侧隐约可见一个暖黄色灯笼或灯罩，散发出柔和的橙色光晕，为画面增添温馨氛围，沙发上的花纹是复古风格的花卉图案，颜色以米黄和棕褐色为主，纹理清晰但不过于突兀，整体构图采用中景视角，人物占据画面大部分空间，背景元素合理分布，形成前后层次感，视觉引导线由人物的身体线条、折扇的展开方向以及竹子的垂直线条共同构成，将视线引向人物面部和眼神，构图平衡且富有张力，色彩以暖色调为主，白色旗袍与黑色绑带、高跟鞋形成强烈对比，竹叶的绿色成为画面中的点睛之笔，增强视觉活力，整体氛围兼具古典雅致与现代时尚感，仿佛一幅精心构思的艺术肖像作品，又像是用专业相机在安静室内拍摄的高质量人像照片，带有轻微的胶片质感和自然噪点，呈现出一种真实而富有故事性的影像风格，细节丰富，光影过渡自然，人物神态专注，场景布置考究，展现出浓郁的文化气息与个人魅力 \n"
    },
    "class_type": "Text Multiline",
    "_meta": {
      "title": "Text Multiline"
    }
  }
}

path: workflows/qwen_image_edit_2511.json
content: {
  "9": {
    "inputs": {
      "filename_prefix": "Qwen_Edit_2511",
      "images": [
        "89:8",
        0
      ]
    },
    "class_type": "SaveImage",
    "_meta": {
      "title": "Save Image"
    }
  },
  "41": {
    "inputs": {
      "image": "00039-2354001535.png"
    },
    "class_type": "LoadImage",
    "_meta": {
      "title": "Load Image"
    }
  },
  "89:67": {
    "inputs": {
      "shift": 3.1,
      "model": [
        "89:74",
        0
      ]
    },
    "class_type": "ModelSamplingAuraFlow",
    "_meta": {
      "title": "ModelSamplingAuraFlow"
    }
  },
  "89:71": {
    "inputs": {
      "reference_latents_method": "index_timestep_zero",
      "conditioning": [
        "89:69",
        0
      ]
    },
    "class_type": "FluxKontextMultiReferenceLatentMethod",
    "_meta": {
      "title": "Edit Model Reference Method"
    }
  },
  "89:70": {
    "inputs": {
      "reference_latents_method": "index_timestep_zero",
      "conditioning": [
        "89:68",
        0
      ]
    },
    "class_type": "FluxKontextMultiReferenceLatentMethod",
    "_meta": {
      "title": "Edit Model Reference Method"
    }
  },
  "89:64": {
    "inputs": {
      "strength": 1,
      "model": [
        "89:67",
        0
      ]
    },
    "class_type": "CFGNorm",
    "_meta": {
      "title": "CFGNorm"
    }
  },
  "89:69": {
    "inputs": {
      "prompt": "",
      "clip": [
        "89:61",
        0
      ],
      "vae": [
        "89:10",
        0
      ],
      "image1": [
        "89:88",
        0
      ]
    },
    "class_type": "TextEncodeQwenImageEditPlus",
    "_meta": {
      "title": "TextEncodeQwenImageEditPlus"
    }
  },
  "89:8": {
    "inputs": {
      "samples": [
        "89:65",
        0
      ],
      "vae": [
        "89:10",
        0
      ]
    },
    "class_type": "VAEDecode",
    "_meta": {
      "title": "VAE Decode"
    }
  },
  "89:68": {
    "inputs": {
      "prompt": "change the black shirt to jean jacket with white shirt",
      "clip": [
        "89:61",
        0
      ],
      "vae": [
        "89:10",
        0
      ],
      "image1": [
        "89:88",
        0
      ]
    },
    "class_type": "TextEncodeQwenImageEditPlus",
    "_meta": {
      "title": "TextEncodeQwenImageEditPlus (Positive)"
    }
  },
  "89:75": {
    "inputs": {
      "pixels": [
        "89:88",
        0
      ],
      "vae": [
        "89:10",
        0
      ]
    },
    "class_type": "VAEEncode",
    "_meta": {
      "title": "VAE Encode"
    }
  },
  "89:88": {
    "inputs": {
      "image": [
        "41",
        0
      ]
    },
    "class_type": "FluxKontextImageScale",
    "_meta": {
      "title": "FluxKontextImageScale"
    }
  },
  "89:12": {
    "inputs": {
      "unet_name": "qwen_image_edit_2511_bf16.safetensors",
      "weight_dtype": "default"
    },
    "class_type": "UNETLoader",
    "_meta": {
      "title": "Load Diffusion Model"
    }
  },
  "89:89": {
    "inputs": {
      "unet_name": "QWEN_Image\\qwenImageEdit2511_q50.gguf"
    },
    "class_type": "UnetLoaderGGUF",
    "_meta": {
      "title": "Unet Loader (GGUF)"
    }
  },
  "89:61": {
    "inputs": {
      "clip_name": "qwen_2.5_vl_7b_fp8_scaled.safetensors",
      "type": "qwen_image",
      "device": "default"
    },
    "class_type": "CLIPLoader",
    "_meta": {
      "title": "Load CLIP"
    }
  },
  "89:10": {
    "inputs": {
      "vae_name": "qwen_image_vae.safetensors"
    },
    "class_type": "VAELoader",
    "_meta": {
      "title": "Load VAE"
    }
  },
  "89:65": {
    "inputs": {
      "seed": 0,
      "steps": 4,
      "cfg": 4,
      "sampler_name": "euler",
      "scheduler": "simple",
      "denoise": 1,
      "model": [
        "89:64",
        0
      ],
      "positive": [
        "89:70",
        0
      ],
      "negative": [
        "89:71",
        0
      ],
      "latent_image": [
        "89:75",
        0
      ]
    },
    "class_type": "KSampler",
    "_meta": {
      "title": "KSampler"
    }
  },
  "89:74": {
    "inputs": {
      "lora_name": "QWEN_Lora\\Lightning\\Qwen-Image-Edit-2511-Lightning-4steps-V1.0-bf16.safetensors",
      "strength_model": 1,
      "model": [
        "89:89",
        0
      ]
    },
    "class_type": "LoraLoaderModelOnly",
    "_meta": {
      "title": "Load LoRA"
    }
  }
}

Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)