Skill: mwamedacen/n8n-evol-i (version 2.0.0)

--- name: n8n-evol-I description: A harness to help coding agents build, deploy, maintain, and debug multi-workflow n8n-powered automation systems. No lock-in — work from the agent, continue from the n8n UI, hand back to the agent at any time. --- # n8n-evol-I skill router This is the entry point. When the user asks anything n8n-related, route to the matching sub-skill. ## Mental model - The skill package (this directory) is **read-only**. You never edit files here. - The user's project state lives in `${PWD}/n8n-evol-I-workspace/` (created by `init.md`). - Helpers are Python scripts under `helpers/`. Invoke them with `python3 ${CLAUDE_PLUGIN_ROOT}/helpers/<name>.py [args]` (plugin mode) or `python3 <path-to-harness>/helpers/<name>.py [args]` (skill mode). - All helpers default to `--workspace ${PWD}/n8n-evol-I-workspace`. Pass `--workspace <path>` if the user runs from elsewhere. - **Agent memory**: read `N8N-WORKSPACE-MEMORY.md` in the workspace at the start of every session; append a dated entry whenever you learn something durable about this project. Full guidance in `AGENTS.md` (also in the workspace root). ## Lifecycle skills (use when the user wants to do X) | Skill | When | |---|---| | [init.md](skills/init.md) | First-time setup. Creates the workspace at `${PWD}/n8n-evol-I-workspace/`. | | [bootstrap-env.md](skills/bootstrap-env.md) | Configure an environment (`dev` / `staging` / `prod`). Creates env YAML + `.env`, validates, mints placeholder workflow IDs. | | [doctor.md](skills/doctor.md) | Health check. Run before/after major changes. | | [create-new-workflow.md](skills/create-new-workflow.md) | Author a brand-new workflow. | | [register-workflow-to-error-handler.md](skills/register-workflow-to-error-handler.md) | Wire `settings.errorWorkflow`. | | [create-lock.md](skills/create-lock.md) | First-time setup for distributed locking (Redis-backed primitives). | | [copy-primitive.md](skills/copy-primitive.md) | Copy a single primitive (any) into the workspace. General-purpose; doesn't register. | | [add-lock-to-workflow.md](skills/add-lock-to-workflow.md) | Wrap a workflow's main flow in lock acquire/release. | | [add-rate-limit-to-workflow.md](skills/add-rate-limit-to-workflow.md) | Gate a workflow's main flow with a Redis-backed fixed-window rate-limit check. | | [create-queue.md](skills/create-queue.md) | First-time setup for queue primitive (Redis Streams + atomic-INCR semaphore). | | [add-queue-publish-to-workflow.md](skills/add-queue-publish-to-workflow.md) | Wrap a workflow with a producer-side XADD call. | | [add-queue-consumer-to-workflow.md](skills/add-queue-consumer-to-workflow.md) | Turn a workflow into a schedule-polled queue consumer with bounded concurrency. | | [tidy-workflow.md](skills/tidy-workflow.md) | Apply n8n's canvas-layout algorithm to a workflow template to clean up node positions. | | [deploy.md](skills/deploy.md) | Deploy one workflow to one env. | | [activate-single-workflow-in-env.md](skills/activate-single-workflow-in-env.md) | Activate after deploy. | | [deactivate-single-workflow-in-env.md](skills/deactivate-single-workflow-in-env.md) | Pause triggers (commonly during dev). | | [archive-workflow.md](skills/archive-workflow.md) | Retire a deployed workflow (hidden + read-only on the live instance). | | [unarchive-workflow.md](skills/unarchive-workflow.md) | Restore a previously-archived workflow so it accepts updates again. | | [deploy_all.md](skills/deploy_all.md) | Roll out an entire env in tier order. | | [resync.md](skills/resync.md) | Pull live state of one workflow back into its template. | | [resync_all.md](skills/resync_all.md) | Snapshot a full env back to templates. | | [dehydrate-workflow.md](skills/dehydrate-workflow.md) | Convert raw exported JSON into a template. | | [validate.md](skills/validate.md) | Structural REST validation before deploy. | | [run.md](skills/run.md) | Fire a webhook + assert terminal status. | | [debug.md](skills/debug.md) | Investigate a failing or missing execution — from vague symptom to root-cause with evidence. | | [deploy-run-assert.md](skills/deploy-run-assert.md) | One-shot validate → deploy → run verify. | | [find-skills.md](skills/find-skills.md) | While authoring, find applicable patterns/integrations. | | [manage-credentials.md](skills/manage-credentials.md) | Create or link n8n credentials (Path A from `.env.<env>` / Path B from existing UI credential). | | [manage-variables.md](skills/manage-variables.md) | Lifecycle for n8n Variables (`$vars.*` — non-credential runtime values, or secret fallback when `$env` is blocked). | | [add-cloud-function.md](skills/add-cloud-function.md) | Scaffold a Python serverless function / cloud function / serverless API under `<workspace>/cloud-functions/`. | | [iterate-prompt.md](skills/iterate-prompt.md) | Optimize a prompt against a paired schema + dataset using DSPy. | | [test.md](skills/test.md) | Run unit tests over n8n Code-node JS and / or cloud-function Python. | ## Pattern skills (read-only knowledge) These are reference docs, not action triggers. Read them while authoring. - [skills/patterns/subworkflows.md](skills/patterns/subworkflows.md) - [skills/patterns/error-handling.md](skills/patterns/error-handling.md) - [skills/patterns/credential-refs.md](skills/patterns/credential-refs.md) - [skills/patterns/multi-env-uuid-collision.md](skills/patterns/multi-env-uuid-collision.md) - [skills/patterns/validate-deploy.md](skills/patterns/validate-deploy.md) - [skills/patterns/code-node-discipline.md](skills/patterns/code-node-discipline.md) - [skills/patterns/llm-providers.md](skills/patterns/llm-providers.md) - [skills/patterns/locking.md](skills/patterns/locking.md) - [skills/patterns/queues.md](skills/patterns/queues.md) - [skills/patterns/pindata-hygiene.md](skills/patterns/pindata-hygiene.md) - [skills/patterns/position-recalculation.md](skills/patterns/position-recalculation.md) - [skills/patterns/prompt-and-schema-conventions.md](skills/patterns/prompt-and-schema-conventions.md) - [skills/patterns/agent-api-discipline.md](skills/patterns/agent-api-discipline.md) - [skills/patterns/investigation-discipline.md](skills/patterns/investigation-discipline.md) ## Integration skills (per-service quirks) - [skills/integrations/microsoft-365/excel-and-sharepoint.md](skills/integrations/microsoft-365/excel-and-sharepoint.md) - [skills/integrations/gmail/sending-email.md](skills/integrations/gmail/sending-email.md) - [skills/integrations/redis/lock-pattern.md](skills/integrations/redis/lock-pattern.md) - [skills/integrations/sentry/README.md](skills/integrations/sentry/README.md) - [skills/integrations/datadog/README.md](skills/integrations/datadog/README.md) - [skills/integrations/slack/README.md](skills/integrations/slack/README.md) - [skills/integrations/google-drive/README.md](skills/integrations/google-drive/README.md) - [skills/integrations/notion/README.md](skills/integrations/notion/README.md) - [skills/integrations/airtable/README.md](skills/integrations/airtable/README.md) - [skills/integrations/webhooks/README.md](skills/integrations/webhooks/README.md) ## Placeholder syntax (workflow templates) Templates use `{{@type:path}}` (preferred form) or the canonical long form `{{INTERPOLATE_type:path}}`. The two are equivalent — `@` is an alias for `INTERPOLATE_`. Examples below use the `@` form. | Type | Syntax | Source | |---|---|---| | `env` | `{{@env:key.path}}` | YAML config value (dot notation) | | `txt` | `{{@txt:relative/path.txt}}` | Text file in workspace | | `json` | `{{@json:relative/path.json}}` | JSON file (stringified) | | `html` | `{{@html:relative/path.html}}` | HTML file | | `js` | `{{@js:relative/path.js}}` | JavaScript file | | `py` | `{{@py:relative/path.py}}` | Python file (Code-node `language: python`) | | `uuid` | `{{@uuid:identifier}}` | Fresh UUID v4 (consistent within one hydration) |

path: requirements.txt
content: pyyaml>=6.0


path: pyproject.toml
content: [build-system]
requires = ["setuptools>=69"]
build-backend = "setuptools.build_meta"

[project]
name = "n8n-evol-I"
version = "2.0.0"
description = "Read-only skill package for n8n workflow automation. Install via git clone into skills/."
requires-python = ">=3.11"
dependencies = [
    "pyyaml>=6.0",
    "requests>=2.31",
    "python-dotenv>=1.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=7",
    "pytest-httpserver>=1.0",
]
dspy = [
    "dspy>=2.0",
    "litellm>=1.0",
]

[tool.setuptools.packages.find]
where = ["."]
include = ["helpers*"]

[tool.pytest.ini_options]
pythonpath = ["."]
markers = [
    "gated_real: tests that hit the real n8n instance; skip if N8N_API_KEY unset",
]



path: install.md
content: ---
name: install
description: How to install n8n-evol-I as a skill package and what it expects on the host system.
---

# install

## Prerequisites

- Python ≥ 3.11 on PATH
- Pip-installable: `pyyaml`, `requests`, `python-dotenv`
- An n8n instance the agent can reach via REST + an API key with workflow + credential scopes

## Install (one-time)

### Skill mode (any agent runtime)

Clone into your agent's skills directory:

```bash
cd ~/.claude/skills   # or wherever your agent runtime reads skills from
git clone https://github.com/mwamedacen/n8n-evol-I.git
```

Install Python deps:

```bash
pip install pyyaml requests python-dotenv
```

(Optional, only if you'll use `iterate-prompt`:)

```bash
pip install dspy litellm
```

### Plugin mode (Claude Code only)

CLI form (run in your terminal):

```bash
claude plugin install https://github.com/mwamedacen/n8n-evol-I
```

In-session form (inside a Claude Code session):

```
/plugin install https://github.com/mwamedacen/n8n-evol-I
```

Local dev (loads from a local checkout):

```bash
claude --plugin-dir ./n8n-evol-I
```

#### Plugin extras

When installed as a plugin, n8n-evol-I ships two additional behaviors:

- **Slash commands** — 10 user-facing commands available as `/n8n-evol-I:deploy`, `/n8n-evol-I:tidyup`, etc. Hidden lifecycle skills remain agent-loadable via `SKILL.md` routing but do not appear in `/help`.

- **Auto-tidy hook** — a `PostToolUse` hook fires after every Write/Edit/MultiEdit tool call; the hook script filters to `*.template.json` files and runs `tidy_workflow.py --in-place` on matching files. This keeps node positions clean without any manual step.

  The hook runs asynchronously (`async: true`), so Claude Code does not wait for it before continuing. If the agent reads the template file immediately after writing it (e.g. to feed it into a deploy step), it may observe the un-tidied version. In that case, run `tidy-workflow` manually before deploying, or add a short pause between the write and the read.

  To disable the auto-tidy hook: remove or rename `hooks/hooks.json` in the plugin directory, or disable the plugin in Claude Code settings. Standalone-skill-mode users who want auto-tidy can configure the hook manually in `~/.claude/settings.json`.

## What the harness expects on disk

The skill package is a directory containing:

- `SKILL.md` — entry point
- `skills/*.md` and `skills/{patterns,integrations}/...md` — sub-skills
- `helpers/*.py` — the executable surface
- `primitives/` — seed templates copied into workspaces on demand

Helpers are invoked by absolute path (no console script, no PATH pollution):

```bash
python3 <harness>/helpers/<name>.py [args]
```

## Per-project setup

Per project, the agent runs `init.md` once to scaffold a workspace at `${PWD}/n8n-evol-I-workspace/`. From there, `bootstrap-env.md` configures envs, `create-new-workflow.md` authors workflows, etc. See [`SKILL.md`](SKILL.md) for the full skill catalogue.

## Updating

The harness's "version" is its git SHA. To upgrade:

```bash
cd ~/.claude/skills/n8n-evol-I
git pull
```

Breaking changes between versions are documented in [`CHANGELOG.md`](CHANGELOG.md). For the migration from the legacy single-repo shape (pre-`d6848fd`), see [`docs/migration-from-d6848fd.md`](docs/migration-from-d6848fd.md).

## Verifying the install

```bash
python3 <harness>/helpers/init.py --workspace /tmp/n8n-evol-I-smoke
ls /tmp/n8n-evol-I-smoke/n8n-config /tmp/n8n-evol-I-smoke/n8n-workflows-template
```

If both directories exist after the run, the install is good.


path: README.md
content: 
<img width="1376" height="768" alt="n8n_evol_I_hero" src="https://github.com/user-attachments/assets/0a28f1f6-106f-4f84-9ce1-cf7eebcf4cd6" />

# N8N EVOL I

A harness to help coding agents build, deploy, maintain, and debug multi-workflow n8n-powered automation systems.

No lock-in. Build with your agent and push upstream to n8n; tweak directly in the n8n UI and pull downstream back into your workspace. Round-trips are template-stable, so humans and agents can work the same project in parallel without stepping on each other's edits.

```mermaid
flowchart LR
    Agent <-->|edits / resync| Workspace
    Workspace <-->|deploy / resync| n8n[(n8n instance)]
    n8n <-->|UI edits| Human
```

## Features

- **Multi-environment workflows.** Per-env YAML in `n8n-config/<env>.yml` (instance URL, workflow IDs, credential refs); every helper accepts `--env`. See [`bootstrap-env.md`](skills/bootstrap-env.md).

  ```yaml
  # n8n-config/dev.yml
  instance: acme.dev.n8n.cloud
  envSuffix: " (dev)"          # appended to every workflow name
  workflows:
    lead_enrichment:
      id: wf_def456
      displayName: "Lead Enrichment"
    lead_persister:
      id: wf_abc123
      displayName: "Lead Persister"
  ```

- **Build-time substitution.** Code, prompts, schemas, templates, and env values live in workspace files; `{{@js|py|txt|json|html|env|uuid:...}}` placeholders are substituted at deploy time. Template-stable round-trips. See [`code-node-discipline.md`](skills/patterns/code-node-discipline.md).

  ```jsonc
  // n8n-workflows-template/lead_enrichment.template.json
  {
    "name": "{{@env:workflows.lead_enrichment.displayName}}{{@env:envSuffix}}",
    "nodes": [
      {
        "name": "Webhook",
        "type": "n8n-nodes-base.webhook",
        "parameters": { "path": "lead", "httpMethod": "POST" }
      },
      {
        "name": "Score Lead",
        "type": "@n8n/n8n-nodes-langchain.openAi",
        "parameters": {
          "messages": [
            { "role": "system", "content": "{{@txt:n8n-prompts/prompts/score_lead.txt}}" }
          ]
        }
      },
      {
        "name": "Normalize Fields",
        "type": "n8n-nodes-base.code",
        "parameters": {
          "jsCode": "{{@js:n8n-functions/js/normalize_lead.js}}\n\nreturn normalize(items);"
        }
      },
      {
        "name": "Persist via Sub-workflow",
        "type": "n8n-nodes-base.executeWorkflow",
        "parameters": {
          "workflowId": "{{@env:workflows.lead_persister.id}}"
        }
      }
    ]
  }
  ```

- **Dependency-ordered deployment.** `deploy_all.py` rolls out an env in tier order (set per-workflow in `deployment_order.yml`) so callees deploy before callers. See [`deploy_all.md`](skills/deploy_all.md).

  ```mermaid
  graph LR
    A[tier 0: leaves] --> B[tier 1: handlers] --> C[tier 2: callers]
  ```

- **Execution debugging.** Structured symptom → root-cause flow (dependency graph, candidate pre-screen, causal-linkage, sub-agent cross-check), backed by `list_executions.py`, `inspect_execution.py`, `dependency_graph.py`. See [`investigation-discipline.md`](skills/patterns/investigation-discipline.md).

  ```bash
  python3 $HARNESS/helpers/inspect_execution.py --env dev --id 91234
  ```

- **Distributed locking.** Redis-backed `lock_acquisition` / `lock_release` primitives with an identity-payload sidecar; crashes either self-heal via Redis TTL or are cleaned by the registered `error_handler_lock_cleanup` workflow. See [`locking.md`](skills/patterns/locking.md).

  ```
  ┌──────────┐   ┌──────┐   ┌──────────┐
  │ acquire  │ → │ work │ → │ release  │
  │ (scope)  │   │      │   │ (scope)  │
  └──────────┘   └──────┘   └──────────┘
  ```

- **Rate limiting.** Fixed-window Redis `INCR` primitive; configurable limit, window, and denied-branch behavior (passthrough / stop / error). See [`locking.md`](skills/patterns/locking.md).

  ```
                            ┌─→ work       (within N per window)
  request → check(scope) ───┤
                            └─→ denied     (over limit)
  ```

- **Error handling + observability.** Capture (Error Trigger) → log (sinks fan out in parallel) → process (lock cleanup, DB invalidation, compensating workflows). See [`error-handling.md`](skills/patterns/error-handling.md), [`datadog/`](skills/integrations/datadog/README.md).

  ```
                    ┌─ Sentry  ─┐
  Error Trigger ──→ ├─ Datadog ─┤ ──→ lock cleanup / invalidate / compensate
                    └─ Slack   ─┘
  ```

- **Serverless functions (a.k.a. cloud functions / serverless APIs).** Scaffolds a Python function into a FastAPI service in `cloud-functions/`, auto-registers it in the router, ships with Railway config, callable from n8n via HTTP Request nodes. See [`add-cloud-function.md`](skills/add-cloud-function.md).

  ```bash
  python3 $HARNESS/helpers/add_cloud_function.py --name parse_invoice
  ```

- **DSPy prompt optimization.** Optimize a workspace prompt against its paired schema + dataset (BootstrapFewShot or MIPROv2), eval on structural correctness, optionally write the result back to disk. See [`iterate-prompt.md`](skills/iterate-prompt.md).

  ```bash
  python3 $HARNESS/helpers/iterate_prompt.py --prompt extract_invoice --optimizer mipro_v2
  ```

## Install

| | Skill mode | Plugin mode |
|---|---|---|
| Slash commands | No | Yes (`/n8n-evol-I:*`) |
| Auto-tidy hook | Manual opt-in | Automatic |
| Discovery | Read `SKILL.md` | Claude Code `/help` |
| Other agent runtimes | Yes | Claude Code only |

**Skill mode** (any agent runtime):

```bash
git clone https://github.com/mwamedacen/n8n-evol-I.git ~/.claude/skills/n8n-evol-I
pip install pyyaml requests python-dotenv
```

**Plugin mode** (Claude Code) — adds 10 namespaced slash commands (`/n8n-evol-I:deploy`, `:deploy_all`, `:resync`, `:resync_all`, `:tidyup`, `:debug`, `:run`, `:doctor`, `:validate`, `:test`) plus an auto-tidy hook.

```bash
claude plugin install https://github.com/mwamedacen/n8n-evol-I
```

## Quick start

```bash
HARNESS=~/.claude/skills/n8n-evol-I   # or $CLAUDE_PLUGIN_ROOT in plugin mode
python3 $HARNESS/helpers/init.py
python3 $HARNESS/helpers/bootstrap_env.py --env dev --instance acme.app.n8n.cloud --api-key <key>
python3 $HARNESS/helpers/doctor.py --env dev
```

See [`install.md`](install.md) for prerequisites, optional extras, and update flow.

## Why

Operating n8n past the toy stage exposes the same gaps on every project. The UI editor is fine for a demo and tedious by the third workflow — every code change is click-paste-save, every refactor is manual, every diff is a 200KB JSON blob with code, prompts, schemas, and HTML inlined. A coding agent burns half its context window parsing one file. Two scheduled runs can quietly race on the same record. When something fails at 2am, the execution log tells you *what* broke, not *why*.

N8N EVOL I is the structure that takes those problems off the table.

## How it works

The harness directory is read-only at runtime. The harness operates against your existing project directory — referred to here as `{YOUR_N8N_PROJECT_DIR}` — which becomes the workspace; the agent reads and writes the opinionated subtree (`n8n-config/`, `n8n-workflows-template/`, …) directly inside it, and you version-control it however you want. The agent reads [`SKILL.md`](SKILL.md) to route any n8n request to the right sub-skill, which invokes a standalone Python helper in `helpers/`. No master CLI, no daemon. `validate.py` enforces segmentation discipline before any deploy.

```mermaid
graph LR
  A[agent] --> B[SKILL.md router]
  B --> C[skills/*.md]
  C --> D[helpers/*.py]
  D --> E[workspace files]
  D --> F[n8n REST API]
```

## Workspace layout

`init.py` scaffolds the opinionated default below. If your project already has its own layout, document the mapping in `N8N-WORKSPACE-MEMORY.md` — the agent reads that journal each session and reconciles, so most non-default shapes work fine as long as it can see where each thing actually lives:

```
{YOUR_N8N_PROJECT_DIR}/
├── AGENTS.md                # workspace orientation (read first every session)
├── N8N-WORKSPACE-MEMORY.md  # rolling journal — agent appends as it learns
├── n8n-config/              # env YAML (dev.yml, prod.yml, …) + .env.<env> secrets
├── n8n-workflows-template/  # *.template.json — canonical, version-controlled
├── n8n-build/               # hydrated outputs — gitignored, regenerated on deploy
├── n8n-functions/
│   ├── js/                  # pure JS injected via {{@js:...}}
│   └── py/                  # pure Python injected via {{@py:...}}
├── n8n-functions-tests/     # *.test.js / test_*.py — paired tests, validator-required
├── n8n-prompts/
│   ├── prompts/             # *_prompt.txt + *_schema.json
│   ├── datasets/            # *.json for iterate-prompt
│   └── evals/
├── n8n-assets/
│   ├── email-templates/     # *.html injected via {{@html:...}}
│   ├── images/
│   └── misc/
├── cloud-functions/         # FastAPI service scaffolded by add-cloud-function
│   └── functions/
└── cloud-functions-tests/
```

## Repository layout

| Path | Contents |
|---|---|
| [`SKILL.md`](SKILL.md) | Router — lists all lifecycle, pattern, and integration skills. |
| [`install.md`](install.md) | Prerequisites, install, smoke test, update flow. |
| `skills/` | 27 lifecycle + 13 pattern + 10 integration skills (50 total). |
| `helpers/` | 35 top-level Python helpers + 6 `placeholder/` resolvers. |
| `primitives/workflows/` | Seed templates: `_minimal`, `lock_acquisition`, `lock_release`, `error_handler_lock_cleanup`, `rate_limit_check`. |
| `primitives/cloud-functions/` | FastAPI app seed + Railway config (`app.py`, `registry.py`, `railpack.json`). |
| `primitives/prompts/` | Example prompt + schema for `iterate-prompt`. |
| `tests/` | 200+ offline tests (HTTP mocked) covering every helper, primitive, and pattern. |

## Changelog

See [CHANGELOG.md](CHANGELOG.md).

## License

MIT. See [LICENSE](LICENSE).


path: helpers/run.py
content: #!/usr/bin/env python3
"""Fire a workflow's webhook (or its paired error-source) and poll for terminal status."""
import argparse
import json
import os
import sys
import time
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

import requests

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, load_common, get_config_value
from helpers.n8n_client import ensure_client


def _find_webhook_path(workflow: dict) -> str | None:
    """Find the first webhook node in a workflow; return its path or None."""
    for node in workflow.get("nodes", []):
        if str(node.get("type", "")).endswith(".webhook"):
            params = node.get("parameters") or {}
            return params.get("path")
    return None


def _fire_webhook_and_poll(workspace: Path, env_name: str, workflow_key: str, payload: dict, timeout: int) -> dict:
    """POST to the workflow's webhook, then poll executions for the terminal result."""
    yaml_data = load_yaml(env_name, workspace)
    wf_id = str(get_config_value(yaml_data, f"workflows.{workflow_key}.id"))
    instance = yaml_data.get("n8n", {}).get("instanceName", "")
    base_url = instance if instance.startswith("http") else f"https://{instance.rstrip('/')}"

    client = ensure_client(env_name, workspace)
    workflow = client.get_workflow(wf_id)
    path = _find_webhook_path(workflow)
    if not path:
        raise SystemExit(f"Workflow '{workflow_key}' has no webhook node; cannot fire directly")

    fire_started_at = time.time()
    webhook_url = f"{base_url}/webhook/{path}"
    test_url = f"{base_url}/webhook-test/{path}"

    fired = False
    last_err = None
    for url in (webhook_url, test_url):
        try:
            r = requests.post(url, json=payload, timeout=timeout)
            if r.status_code < 500:
                fired = True
                break
            last_err = f"{url} → {r.status_code} {r.text[:200]}"
        except Exception as e:
            last_err = f"{url} → {e}"
    if not fired:
        raise SystemExit(f"Could not fire webhook for '{workflow_key}': {last_err}")

    return _poll_for_execution(client, wf_id, fire_started_at, timeout)


def _fire_via_error_source(workspace: Path, env_name: str, handler_key: str, payload: dict, timeout: int) -> dict:
    """Indirect dispatch: find the source workflow paired with the handler, fire it, poll the handler."""
    common = load_common(workspace)
    mapping = common.get("error_source_to_handler") or {}
    source_key = next((s for s, h in mapping.items() if h == handler_key), None)
    if not source_key:
        raise SystemExit(
            f"No source workflow paired with handler '{handler_key}' in n8n-config/common.yml.\n"
            f"Use register-workflow-to-error-handler to set up the pairing."
        )
    yaml_data = load_yaml(env_name, workspace)
    handler_id = str(get_config_value(yaml_data, f"workflows.{handler_key}.id"))

    fire_started_at = time.time()
    # Fire the source workflow (which is supposed to error and route to handler)
    try:
        _fire_webhook_and_poll(workspace, env_name, source_key, payload, timeout)
    except SystemExit:
        pass  # source workflow erroring is the expected dispatch path
    # Poll the handler's executions
    client = ensure_client(env_name, workspace)
    return _poll_for_execution(client, handler_id, fire_started_at, timeout)


def _poll_for_execution(client, workflow_id: str, started_at: float, timeout: int) -> dict:
    """Poll /executions for a terminal record with `workflowId == workflow_id` started after started_at."""
    deadline = started_at + timeout
    last_id = None
    while time.time() < deadline:
        try:
            execs = client.get("executions", params={"workflowId": workflow_id, "limit": 5})
            data = execs.get("data") or []
            for ex in data:
                # Match started after our fire
                started_at_str = ex.get("startedAt") or ""
                # Best-effort: parse ISO 8601
                try:
                    from datetime import datetime
                    ts = datetime.fromisoformat(started_at_str.replace("Z", "+00:00")).timestamp()
                except Exception:
                    ts = started_at  # fall back to "any execution"
                if ts >= started_at - 1:
                    if ex.get("finished"):
                        # Get full execution
                        full = client.get(f"executions/{ex['id']}", params={"includeData": "true"})
                        return full
                    last_id = ex.get("id")
        except Exception:
            pass
        time.sleep(1)
    raise SystemExit(f"Timeout waiting for terminal execution of workflow {workflow_id} (last seen={last_id})")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--payload", default="{}", help="JSON payload to POST to the webhook")
    parser.add_argument("--expect-status", default=None, dest="expect_status",
                        choices=("success", "error", "canceled", None))
    parser.add_argument("--timeout", type=int, default=30)
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    payload = json.loads(args.payload)

    common = load_common(ws)
    mapping = common.get("error_source_to_handler") or {}

    if args.workflow_key in mapping.values():
        # Indirect dispatch — workflow_key is a handler
        ex = _fire_via_error_source(ws, args.env, args.workflow_key, payload, args.timeout)
    else:
        ex = _fire_webhook_and_poll(ws, args.env, args.workflow_key, payload, args.timeout)

    status = ex.get("status") or ("error" if ex.get("stoppedAt") and not ex.get("finished") else "success")
    print(json.dumps({"id": ex.get("id"), "status": status, "finished": ex.get("finished")}, indent=2))

    if args.expect_status and status != args.expect_status:
        print(f"FAIL: expected status='{args.expect_status}', got '{status}'", file=sys.stderr)
        sys.exit(1)


if __name__ == "__main__":
    main()


path: helpers/config.py
content: import os
from pathlib import Path
from typing import Any

import yaml


def load_env(env_name: str, workspace: Path) -> dict:
    """Load .env.<env> from workspace/n8n-config/ into os.environ. Returns loaded dict."""
    env_file = workspace / "n8n-config" / f".env.{env_name}"
    loaded = {}
    if env_file.exists():
        with open(env_file) as f:
            for line in f:
                line = line.strip()
                if not line or line.startswith("#") or "=" not in line:
                    continue
                key, _, val = line.partition("=")
                key = key.strip()
                val = val.strip()
                if len(val) >= 2 and val[0] == val[-1] and val[0] in ('"', "'"):
                    val = val[1:-1]
                os.environ[key] = val
                loaded[key] = val
    return loaded


def load_yaml(env_name: str, workspace: Path) -> dict:
    """Read n8n-config/<env>.yml and validate required keys."""
    yaml_file = workspace / "n8n-config" / f"{env_name}.yml"
    if not yaml_file.exists():
        raise FileNotFoundError(
            f"No environment config at {yaml_file}. "
            f"Run `python3 <harness>/helpers/bootstrap_env.py --env {env_name}` first."
        )
    with open(yaml_file) as f:
        data = yaml.safe_load(f) or {}
    _validate_env_yaml(data, yaml_file)
    return data


def _validate_env_yaml(data: dict, path: Path) -> None:
    for key in ("name", "displayName", "n8n"):
        if key not in data:
            raise ValueError(f"Missing required key '{key}' in {path}")
    if "instanceName" not in data.get("n8n", {}):
        raise ValueError(f"Missing required key 'n8n.instanceName' in {path}")


def load_common(workspace: Path) -> dict:
    """Read n8n-config/common.yml. Returns {} if absent; raises only on YAML parse error."""
    common_file = workspace / "n8n-config" / "common.yml"
    if not common_file.exists():
        return {}
    with open(common_file) as f:
        return yaml.safe_load(f) or {}


def get_config_value(config: dict, dot_path: str) -> Any:
    """Look up a dot-notation path in a nested dict. Raises KeyError if not found."""
    parts = dot_path.split(".")
    current = config
    for part in parts:
        if not isinstance(current, dict) or part not in current:
            raise KeyError(f"Path '{dot_path}' not found in config (missing key '{part}')")
        current = current[part]
    return current


def flatten_config(config: dict, prefix: str = "") -> dict:
    """Flatten a nested dict to dot-path keys."""
    result = {}
    for key, val in config.items():
        full_key = f"{prefix}.{key}" if prefix else key
        if isinstance(val, dict):
            result.update(flatten_config(val, full_key))
        else:
            result[full_key] = val
    return result


path: helpers/deploy.py
content: #!/usr/bin/env python3
"""Hydrate, PUT to n8n, and (by default) activate a workflow."""
import argparse
import json
import os
import subprocess
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client, redact_for_debug
from helpers.validate import validate_workflow_json


_PUT_FIELDS = ("name", "nodes", "connections", "settings", "staticData")

# Triggers that can stand alone (and thus support the /activate endpoint).
# Sub-workflow triggers (executeWorkflowTrigger, errorTrigger) cannot be activated alone.
_ACTIVATABLE_TRIGGER_TYPES = (
    "n8n-nodes-base.webhook",
    "n8n-nodes-base.scheduleTrigger",
    "n8n-nodes-base.cron",
    "n8n-nodes-base.formTrigger",
    "n8n-nodes-base.emailReadImap",
    "n8n-nodes-base.manualTrigger",
    "n8n-nodes-base.errorTrigger",  # Error Trigger workflows DO need activation
)


def _filter_for_put(data: dict) -> dict:
    """n8n's PUT /workflows/{id} accepts only certain fields; drop the rest."""
    return {k: data[k] for k in _PUT_FIELDS if k in data}


def _has_activatable_trigger(workflow: dict) -> bool:
    """A workflow with only ExecuteWorkflowTrigger cannot be activated."""
    for node in workflow.get("nodes", []):
        ntype = node.get("type", "")
        if ntype in _ACTIVATABLE_TRIGGER_TYPES:
            return True
        # Heuristic: any node whose type ends in 'Trigger' BUT isn't a sub-workflow trigger
        if ntype.endswith("Trigger") and ntype != "n8n-nodes-base.executeWorkflowTrigger":
            return True
    return False


def _resolve_workflow_id(env_name: str, workflow_key: str, workspace: Path) -> str:
    data = load_yaml(env_name, workspace)
    try:
        return str(get_config_value(data, f"workflows.{workflow_key}.id"))
    except KeyError:
        raise SystemExit(f"No workflow id for key '{workflow_key}' in env '{env_name}'.")


def _write_debug(env_name: str, workflow_key: str, payload: dict, response, stage: str) -> None:
    debug_dir = Path.home() / ".cache" / "n8n-evol-I" / "debug" / str(os.getpid())
    debug_dir.mkdir(parents=True, exist_ok=True)
    seq = len(list(debug_dir.glob(f"deploy-*.json"))) + 1
    out = debug_dir / f"deploy-{seq:03d}.json"
    blob = {
        "env": env_name,
        "workflow_key": workflow_key,
        "stage": stage,
        "payload": redact_for_debug(payload),
        "response": redact_for_debug(response if isinstance(response, (dict, list)) else str(response)),
    }
    out.write_text(json.dumps(blob, indent=2))
    out.chmod(0o600)
    print(f"  Debug artifact: {out}", file=sys.stderr)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--no-activate", action="store_true")
    parser.add_argument("--rehydrate", action="store_true", help="Re-hydrate before deploy even if generated JSON exists")
    parser.add_argument("--debug", action="store_true", help="Dump redacted pre/post hydration + request/response")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)

    template_path = ws / "n8n-workflows-template" / f"{args.workflow_key}.template.json"
    if not template_path.exists():
        print(f"ERROR: template not found: {template_path}", file=sys.stderr)
        sys.exit(1)
    template_text = template_path.read_text(encoding="utf-8")
    valid, errors = validate_workflow_json(template_text, source="template", workspace=ws)
    if not valid:
        print("ERROR: template validation failed (deploy aborted):", file=sys.stderr)
        for e in errors:
            print(f"  - {e}", file=sys.stderr)
        sys.exit(1)

    generated = ws / "n8n-build" / args.env / f"{args.workflow_key}.generated.json"

    if args.rehydrate or not generated.exists():
        # Compose with hydrate.py as a subprocess
        cmd = [
            sys.executable,
            str(Path(__file__).parent / "hydrate.py"),
            "--workspace", str(ws),
            "--env", args.env,
            "--workflow-key", args.workflow_key,
        ]
        r = subprocess.run(cmd, capture_output=True, text=True)
        if r.returncode != 0:
            print(r.stdout, file=sys.stderr)
            print(r.stderr, file=sys.stderr)
            sys.exit(r.returncode)

    if not generated.exists():
        print(f"ERROR: generated file missing after hydrate: {generated}", file=sys.stderr)
        sys.exit(1)

    payload_full = json.loads(generated.read_text())
    payload = _filter_for_put(payload_full)

    load_env(args.env, ws)
    wf_id = _resolve_workflow_id(args.env, args.workflow_key, ws)
    client = ensure_client(args.env, ws)

    resp = client.put(f"workflows/{wf_id}", payload)
    if args.debug:
        _write_debug(args.env, args.workflow_key, payload, resp, "put")
    print(f"Deployed workflow '{args.workflow_key}' (id={wf_id}) on env '{args.env}'")

    if not args.no_activate:
        try:
            act_resp = client.post(f"workflows/{wf_id}/activate")
            if args.debug:
                _write_debug(args.env, args.workflow_key, {}, act_resp, "activate")
            print(f"Activated workflow '{args.workflow_key}'")
        except Exception as e:
            # The PUT (deploy) succeeded — only the trigger flip failed. Common
            # causes: a sub-workflow target isn't yet activated (n8n Cloud
            # rejects with "Cannot publish workflow: Node X references workflow
            # Y which is not published"), invalid credential ref, etc.
            #
            # Severity policy (per task #9 deploy_all decision): exit code 2
            # signals "deployed-but-not-activated" — distinct from PUT failure
            # (1). deploy_all treats 2 as warn-and-continue by default and 1
            # as tier-stop. Use --strict-activate on deploy_all to escalate 2
            # to fail-fast.
            print(f"WARNING: activate failed for '{args.workflow_key}': {e}", file=sys.stderr)
            sys.exit(2)


if __name__ == "__main__":
    main()


path: helpers/tidy_workflow.py
content: #!/usr/bin/env python3
"""
Tidy workflow node positions using @n8n/workflow-sdk's layoutWorkflowJSON.

Usage:
    python3 helpers/tidy_workflow.py --workspace <ws> --workflow-key <key> [--in-place]

If Node.js is unavailable or the shim subprocess fails for any reason, falls back
to a pure-Python BFS layout that approximates left-to-right depth assignment.

SDK pin: @n8n/workflow-sdk@stable (0.10.2) — licensed under n8n Sustainable Use License.
"""
import argparse
import json
import shutil
import subprocess
import sys
from collections import deque
from pathlib import Path

_HARNESS_ROOT = Path(__file__).resolve().parent.parent
_SHIM = _HARNESS_ROOT / "helpers" / "tidy_shim.mjs"
_SDK_VERSION = "0.10.2"
_STICKY_TYPE = "n8n-nodes-base.stickyNote"
_START_X = 240
_START_Y = 300
_H_GAP = 220
_V_GAP = 120


def _ensure_sdk() -> bool:
    """Install @n8n/workflow-sdk into helpers/node_modules if not already present. Returns True on success."""
    helpers_dir = _HARNESS_ROOT / "helpers"
    sdk_dir = helpers_dir / "node_modules" / "@n8n" / "workflow-sdk"
    if sdk_dir.exists():
        return True
    if not shutil.which("node") or not shutil.which("npm"):
        return False
    result = subprocess.run(
        ["npm", "install", "--prefix", str(helpers_dir),
         f"@n8n/workflow-sdk@{_SDK_VERSION}",
         "--silent", "--no-fund", "--no-audit"],
        capture_output=True,
        text=True,
    )
    if result.returncode != 0:
        print(f"[tidy_workflow] npm install failed:\n{result.stderr}", file=sys.stderr)
        return False
    return sdk_dir.exists()


def _layout_via_shim(workflow: dict) -> dict | None:
    """Run the Node.js shim. Returns the laid-out workflow dict, or None on any failure."""
    if not shutil.which("node"):
        return None
    if not _ensure_sdk():
        return None
    try:
        result = subprocess.run(
            ["node", str(_SHIM)],
            input=json.dumps(workflow),
            capture_output=True,
            text=True,
            timeout=60,
        )
    except Exception as exc:
        print(f"[tidy_workflow] shim subprocess error: {exc}", file=sys.stderr)
        return None
    if result.returncode != 0:
        print(f"[tidy_workflow] shim exited {result.returncode}: {result.stderr}", file=sys.stderr)
        return None
    try:
        return json.loads(result.stdout)
    except json.JSONDecodeError as exc:
        print(f"[tidy_workflow] shim output not valid JSON: {exc}", file=sys.stderr)
        return None


def _bfs_layout(workflow: dict) -> dict:
    """
    Pure-Python BFS layout fallback.

    Sticky notes are left untouched. All other nodes are assigned
    [start_x + depth * H_GAP, start_y + branch * V_GAP].

    Pure-cycle graphs (no root nodes) fall back to insertion-order layout.
    Disconnected components each get their own row offset.
    """
    nodes = workflow.get("nodes", [])
    connections = workflow.get("connections", {})

    non_sticky = [n for n in nodes if n.get("type") != _STICKY_TYPE]

    if not non_sticky:
        return workflow

    # Build adjacency: name -> set of child names
    children: dict[str, set[str]] = {n["name"]: set() for n in non_sticky}
    in_degree: dict[str, int] = {n["name"]: 0 for n in non_sticky}
    name_set = {n["name"] for n in non_sticky}

    for src_name, output_types in connections.items():
        if src_name not in name_set:
            continue
        for _output_type, output_branches in output_types.items():
            if not isinstance(output_branches, list):
                continue
            for branch in output_branches:
                if not isinstance(branch, list):
                    continue
                for edge in branch:
                    dst = edge.get("node", "")
                    if dst in name_set and dst != src_name:
                        children[src_name].add(dst)
                        in_degree[dst] = in_degree.get(dst, 0) + 1

    # Assign positions
    positions: dict[str, list[int]] = {}
    visited: set[str] = set()
    component_row = 0  # row offset per disconnected component

    def _bfs_from_roots(roots: list[str]) -> None:
        nonlocal component_row
        queue: deque[tuple[str, int]] = deque()
        depth_count: dict[int, int] = {}
        local_visited: set[str] = set()

        for r in roots:
            queue.append((r, 0))
            local_visited.add(r)

        while queue:
            name, depth = queue.popleft()
            if name in visited:
                continue
            visited.add(name)
            branch = depth_count.get(depth, 0)
            depth_count[depth] = branch + 1
            positions[name] = [
                _START_X + depth * _H_GAP,
                _START_Y + (component_row + branch) * _V_GAP,
            ]
            for child in sorted(children.get(name, [])):
                if child not in local_visited:
                    local_visited.add(child)
                    queue.append((child, depth + 1))

        if depth_count:
            component_row += max(depth_count.values())

    insertion_order = [n["name"] for n in non_sticky]

    # Collect roots (in-degree 0)
    roots = [n for n in insertion_order if in_degree.get(n, 0) == 0]

    if roots:
        _bfs_from_roots(roots)
        # Handle any remaining unvisited nodes (disconnected components)
        remaining = [n for n in insertion_order if n not in visited]
        while remaining:
            component_row += 1
            next_root = remaining[0]
            reachable = {next_root}
            q: deque[str] = deque([next_root])
            while q:
                cur = q.popleft()
                for child in children.get(cur, []):
                    if child not in reachable:
                        reachable.add(child)
                        q.append(child)
            comp_roots = [n for n in remaining if in_degree.get(n, 0) == 0 and n in reachable]
            if not comp_roots:
                comp_roots = [n for n in remaining if n in reachable]
            _bfs_from_roots(comp_roots)
            remaining = [n for n in insertion_order if n not in visited]
    else:
        # Pure-cycle graph: assign insertion-order layout, no crash
        for i, name in enumerate(insertion_order):
            positions[name] = [_START_X + i * _H_GAP, _START_Y]
            visited.add(name)

    # Apply positions back
    result_nodes = []
    for node in nodes:
        n = dict(node)
        if n.get("type") != _STICKY_TYPE and n["name"] in positions:
            n["position"] = positions[n["name"]]
        result_nodes.append(n)

    return {**workflow, "nodes": result_nodes}


def tidy(workflow: dict) -> dict:
    """Apply tidy layout: try SDK shim first, fall back to BFS.

    Sticky note positions are always preserved — the SDK moves them, so we
    snapshot before layout and restore after.
    """
    sticky_positions = {
        n["name"]: n["position"]
        for n in workflow.get("nodes", [])
        if n.get("type") == _STICKY_TYPE
    }
    laid = _layout_via_shim(workflow)
    if laid is None:
        print("[tidy_workflow] using Python BFS fallback", file=sys.stderr)
        laid = _bfs_layout(workflow)
    if sticky_positions:
        restored_nodes = []
        for n in laid.get("nodes", []):
            if n.get("type") == _STICKY_TYPE and n["name"] in sticky_positions:
                n = {**n, "position": sticky_positions[n["name"]]}
            restored_nodes.append(n)
        laid = {**laid, "nodes": restored_nodes}
    return laid


def _load_workflow(workspace: Path, key: str) -> tuple[Path, dict]:
    template_dir = workspace / "n8n-workflows-template"
    path = template_dir / f"{key}.template.json"
    if not path.exists():
        sys.exit(f"[tidy_workflow] template not found: {path}")
    with open(path) as f:
        return path, json.load(f)


def main() -> None:
    parser = argparse.ArgumentParser(description="Tidy n8n workflow node positions.")
    parser.add_argument("--workspace", required=True, help="Path to n8n-evol-I workspace directory")
    parser.add_argument("--workflow-key", required=True, help="Workflow key (filename stem without .template.json)")
    parser.add_argument("--in-place", action="store_true", help="Write result back to template file (default: print to stdout)")
    args = parser.parse_args()

    workspace = Path(args.workspace).expanduser().resolve()
    path, workflow = _load_workflow(workspace, args.workflow_key)
    result = tidy(workflow)

    if args.in_place:
        with open(path, "w") as f:
            json.dump(result, f, indent=2)
            f.write("\n")
        print(f"[tidy_workflow] wrote {path}", file=sys.stderr)
    else:
        print(json.dumps(result, indent=2))


if __name__ == "__main__":
    main()


path: helpers/deploy_all.py
content: #!/usr/bin/env python3
"""Walk deployment_order.yml tiers and deploy each workflow per tier."""
import argparse
import json
import subprocess
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

import yaml

from helpers.workspace import workspace_root


_EXTERNAL_TRIGGER_TYPES = frozenset({
    "n8n-nodes-base.webhook",
    "n8n-nodes-base.scheduleTrigger",
    "n8n-nodes-base.cron",
    "n8n-nodes-base.formTrigger",
    "n8n-nodes-base.emailReadImap",
})


def _has_external_trigger(template: dict) -> bool:
    """True if the workflow has a Webhook / Schedule / Cron-style trigger.

    Sub-workflow triggers (executeWorkflowTrigger) and Error Trigger don't fire on their own,
    so they stay active in dev (parent workflows need them published to activate).
    """
    for node in template.get("nodes", []):
        if node.get("type") in _EXTERNAL_TRIGGER_TYPES:
            return True
    return False


def _load_order(workspace: Path) -> dict:
    order_file = workspace / "n8n-config" / "deployment_order.yml"
    if not order_file.exists():
        return {"tiers": {}}
    return yaml.safe_load(order_file.read_text()) or {"tiers": {}}


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--keep-active", action="store_true", dest="keep_active")
    parser.add_argument("--continue-on-failure", action="store_true", dest="continue_on_failure",
                        help="Continue past PUT failures (exit=1). Activate-only failures (exit=2) "
                             "are warned-and-continued by default; pass --strict-activate to escalate.")
    parser.add_argument("--strict-activate", action="store_true", dest="strict_activate",
                        help="Treat 'deployed-but-not-activated' (deploy.py exit=2) as a hard tier-stop "
                             "failure. Without this flag, activate failures are warned and the rollout "
                             "continues — a workflow whose PUT succeeded is fine to leave inactive while "
                             "the operator follows up (e.g. activate sub-workflows manually first).")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    order = _load_order(ws)
    tiers = order.get("tiers") or {}
    helpers = Path(__file__).parent

    failures: list[tuple[str, int]] = []
    activate_warnings: list[str] = []
    for tier in sorted(tiers.keys()):
        keys = tiers[tier] or []
        if not keys:
            continue
        print(f"=== Tier: {tier} ===")
        for key in keys:
            cmd = [sys.executable, str(helpers / "deploy.py"),
                   "--workspace", str(ws), "--env", args.env, "--workflow-key", key]
            r = subprocess.run(cmd)
            if r.returncode == 2 and not args.strict_activate:
                # PUT succeeded; only activate failed. Warn and continue —
                # downstream tiers may not need this workflow to be active
                # (e.g. it's a sub-workflow whose parent will trigger a retry
                # at activate time once dependencies are publish/active).
                activate_warnings.append(key)
                print(f"WARN: '{key}' deployed but not activated (continuing; pass "
                      f"--strict-activate to fail-fast).", file=sys.stderr)
                continue
            if r.returncode != 0:
                failures.append((key, r.returncode))
                if not args.continue_on_failure:
                    print(f"FAIL: deploy '{key}' exit={r.returncode}", file=sys.stderr)
                    sys.exit(r.returncode)

    # Auto-deactivate after dev deploys unless --keep-active.
    # Only deactivate workflows with external triggers (webhook/schedule/cron) — those fire
    # on their own. Sub-workflows and error handlers must stay active so parent workflows
    # remain valid; deactivating them would invalidate any source workflow that references them.
    if args.env == "dev" and not args.keep_active:
        all_keys: list[str] = []
        for keys in tiers.values():
            all_keys.extend(keys or [])
        template_dir = ws / "n8n-workflows-template"
        for key in all_keys:
            template_path = template_dir / f"{key}.template.json"
            if not template_path.exists():
                continue
            try:
                template = json.loads(template_path.read_text())
            except json.JSONDecodeError:
                continue
            if not _has_external_trigger(template):
                continue
            cmd = [sys.executable, str(helpers / "deactivate.py"),
                   "--workspace", str(ws), "--env", args.env, "--workflow-key", key]
            subprocess.run(cmd)

    if activate_warnings:
        print(f"deploy_all: {len(activate_warnings)} workflow(s) deployed but not activated: "
              f"{activate_warnings}", file=sys.stderr)
    if failures:
        print(f"deploy_all complete with {len(failures)} failure(s): {failures}", file=sys.stderr)
        sys.exit(1)
    print("deploy_all complete.")


if __name__ == "__main__":
    main()


path: helpers/add_rate_limit_to_workflow.py
content: #!/usr/bin/env python3
"""Insert a rate-limit gate around a workflow's main flow."""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root


_RATE_LIMIT_NODE_NAME = "Rate Limit"
_IF_NODE_NAME = "Rate Limit Allowed?"
_DENIED_PASSTHROUGH_NAME = "Rate Limit Denied"
_DENIED_STOP_NAME = "Rate Limit Stop"

_VALID_ON_DENIED = ("passthrough", "stop", "error")


def _make_rate_limit_node(
    name: str,
    target_workflow_placeholder: str,
    position: list,
    scope_expr: str,
    limit: int,
    window_seconds: int,
) -> dict:
    return {
        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "}}",
        "name": name,
        "type": "n8n-nodes-base.executeWorkflow",
        "typeVersion": 1.2,
        "position": position,
        "parameters": {
            "source": "database",
            "workflowId": {
                "__rl": True,
                "value": target_workflow_placeholder,
                "mode": "id",
            },
            "workflowInputs": {
                "mappingMode": "defineBelow",
                "value": {
                    "scope": scope_expr,
                    "limit": limit,
                    "windowSeconds": window_seconds,
                },
                "matchingColumns": [],
                "schema": [],
            },
            "options": {},
        },
    }


def _make_if_node(name: str, position: list) -> dict:
    return {
        "id": "{{@uuid:" + name.lower().replace(" ", "-").replace("?", "") + "}}",
        "name": name,
        "type": "n8n-nodes-base.if",
        "typeVersion": 2,
        "position": position,
        "parameters": {
            "conditions": {
                "options": {
                    "caseSensitive": True,
                    "leftValue": "",
                    "typeValidation": "strict",
                },
                "conditions": [
                    {
                        "id": "{{@uuid:" + name.lower().replace(" ", "-").replace("?", "") + "-cond}}",
                        "leftValue": "={{ $json.allowed === true }}",
                        "rightValue": True,
                        "operator": {
                            "type": "boolean",
                            "operation": "true",
                            "singleValue": True,
                        },
                    },
                ],
                "combinator": "and",
            },
            "options": {},
        },
    }


def _make_denied_passthrough_node(name: str, position: list, scope_expr: str) -> dict:
    return {
        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "}}",
        "name": name,
        "type": "n8n-nodes-base.set",
        "typeVersion": 3.4,
        "position": position,
        "parameters": {
            "assignments": {
                "assignments": [
                    {
                        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "-allowed}}",
                        "name": "allowed",
                        "value": False,
                        "type": "boolean",
                    },
                    {
                        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "-scope}}",
                        "name": "scope",
                        "value": "={{ $json.scope }}",
                        "type": "string",
                    },
                    {
                        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "-count}}",
                        "name": "count",
                        "value": "={{ $json.count }}",
                        "type": "number",
                    },
                    {
                        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "-limit}}",
                        "name": "limit",
                        "value": "={{ $json.limit }}",
                        "type": "number",
                    },
                ],
            },
            "options": {},
        },
    }


def _make_denied_stop_node(name: str, position: list) -> dict:
    return {
        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "}}",
        "name": name,
        "type": "n8n-nodes-base.stopAndError",
        "typeVersion": 1,
        "position": position,
        "parameters": {
            "errorMessage": "Rate limit exceeded for scope={{ $json.scope }} (count={{ $json.count }}, limit={{ $json.limit }})",
            "options": {},
        },
    }


def _insert_rate_limit(
    template: dict,
    scope_expr: str,
    limit: int,
    window_seconds: int,
    on_denied: str = "passthrough",
) -> dict:
    if on_denied not in _VALID_ON_DENIED:
        raise SystemExit(f"--on-denied must be one of {_VALID_ON_DENIED}; got {on_denied!r}")

    # Reuse the lock helper's normalizer — same n8n expression-form constraints
    # apply to executeWorkflow defineBelow inputs in this primitive.
    from helpers.add_lock_to_workflow import _normalize_n8n_expression
    scope_expr, was_normalized = _normalize_n8n_expression(scope_expr)
    if was_normalized:
        print(
            f"WARNING: --scope-expression normalized to canonical form: {scope_expr!r}. "
            "Bare '=<expr>' or literal scopes are auto-wrapped to '={{{{ <expr> }}}}'.",
            file=sys.stderr,
        )

    nodes = template.setdefault("nodes", [])
    connections = template.setdefault("connections", {})

    if any(n.get("name") == _RATE_LIMIT_NODE_NAME for n in nodes):
        raise SystemExit(f"Workflow already has a '{_RATE_LIMIT_NODE_NAME}' node; refusing to add again")

    # Find the trigger (a node with no inbound connection)
    referenced: set[str] = set()
    for src_name, by_type in connections.items():
        for type_branches in by_type.values():
            for branch in type_branches:
                for c in branch:
                    referenced.add(c.get("node"))
    triggers = [n for n in nodes if n.get("name") not in referenced]
    if not triggers:
        raise SystemExit("Could not detect a trigger node (no node lacks inbound connections)")
    trigger = triggers[0]
    trigger_name = trigger["name"]

    trigger_out = connections.get(trigger_name, {}).get("main", [[]])
    first_branch = trigger_out[0] if trigger_out else []

    trigger_pos = trigger.get("position") or [240, 300]
    rl_pos = [trigger_pos[0] + 220, trigger_pos[1]]
    if_pos = [trigger_pos[0] + 440, trigger_pos[1]]
    denied_pos = [trigger_pos[0] + 660, trigger_pos[1] + 160]

    # Push every other node 660 right (220 for rate-limit + 220 for if + 220 spacing)
    for n in nodes:
        if n is trigger:
            continue
        pos = n.get("position") or [0, 0]
        n["position"] = [pos[0] + 660, pos[1]]

    rl_node = _make_rate_limit_node(
        _RATE_LIMIT_NODE_NAME,
        "{{@env:workflows.rate_limit_check.id}}",
        rl_pos,
        scope_expr,
        limit,
        window_seconds,
    )
    if_node = _make_if_node(_IF_NODE_NAME, if_pos)
    nodes.append(rl_node)
    nodes.append(if_node)

    if on_denied == "passthrough":
        denied_node = _make_denied_passthrough_node(_DENIED_PASSTHROUGH_NAME, denied_pos, scope_expr)
        denied_name = _DENIED_PASSTHROUGH_NAME
    else:
        # `stop` and `error` both map to stopAndError (errors trigger errorWorkflow if configured)
        denied_node = _make_denied_stop_node(_DENIED_STOP_NAME, denied_pos)
        denied_name = _DENIED_STOP_NAME
    nodes.append(denied_node)

    # Wire: trigger → rate_limit → if → (allowed: original first_branch | denied: denied_node)
    connections[trigger_name] = {"main": [[{"node": _RATE_LIMIT_NODE_NAME, "type": "main", "index": 0}]]}
    connections[_RATE_LIMIT_NODE_NAME] = {"main": [[{"node": _IF_NODE_NAME, "type": "main", "index": 0}]]}
    connections[_IF_NODE_NAME] = {
        "main": [
            first_branch,
            [{"node": denied_name, "type": "main", "index": 0}],
        ],
    }

    return template


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--limit", type=int, required=True)
    parser.add_argument("--window-seconds", type=int, required=True, dest="window_seconds")
    parser.add_argument(
        "--scope-expression",
        default="={{ $execution.id }}",
        dest="scope_expression",
    )
    parser.add_argument(
        "--on-denied",
        choices=_VALID_ON_DENIED,
        default="passthrough",
        dest="on_denied",
    )
    args = parser.parse_args()

    ws = workspace_root(args.workspace)

    if not (ws / "n8n-workflows-template" / "rate_limit_check.template.json").exists():
        print(
            "ERROR: primitive 'rate_limit_check' not found in workspace. "
            "Run create-lock --include-rate-limit first.",
            file=sys.stderr,
        )
        sys.exit(1)

    template_path = ws / "n8n-workflows-template" / f"{args.workflow_key}.template.json"
    if not template_path.exists():
        print(f"ERROR: workflow template not found: {template_path}", file=sys.stderr)
        sys.exit(1)

    template = json.loads(template_path.read_text())
    template = _insert_rate_limit(
        template,
        args.scope_expression,
        args.limit,
        args.window_seconds,
        on_denied=args.on_denied,
    )
    template_path.write_text(json.dumps(template, indent=2))
    print(f"  Inserted rate-limit gate in {template_path} (on-denied={args.on_denied})")
    print("add-rate-limit-to-workflow complete.")


if __name__ == "__main__":
    main()


path: helpers/manage_variables.py
content: #!/usr/bin/env python3
"""Manage n8n environment variables. Subcommands:

  list    — GET /api/v1/variables (optionally filtered by --name).
  create  — POST /api/v1/variables with {key, value}.
  update  — PUT /api/v1/variables/{id} with {key, value}.
  delete  — DELETE /api/v1/variables/{id}; requires --force unless dry-listing.

CLI surface uses `--name` (NOT `--key`) for the n8n variable's name. Reason: in
the rest of the harness, `--key` means a YAML config slot under
`credentials.<key>` or `workflows.<key>`. n8n variables are not version-
controlled and have no YAML representation, so we avoid the semantic overload.

The helper translates `--name` into the n8n request body's `key` field at the
HTTP boundary; that's where n8n's REST contract lives.

JSON output to stdout. Mutations warn on stdout that the change is immediate
and not reflected in n8n-config/<env>.yml.
"""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_env
from helpers.n8n_client import ensure_client


_NOT_VERSIONED_WARNING = (
    "Note: variables are not version-controlled. The change is immediate; "
    "not reflected in n8n-config/<env>.yml."
)


def _list_variables(client) -> list[dict]:
    resp = client.get("variables")
    if isinstance(resp, dict):
        return resp.get("data") or []
    if isinstance(resp, list):
        return resp
    return []


def cmd_list(args, client) -> None:
    rows = _list_variables(client)
    if args.name:
        rows = [r for r in rows if r.get("key") == args.name]
    print(json.dumps(rows, indent=2))


def cmd_create(args, client) -> None:
    if not args.name or args.value is None:
        raise SystemExit("create requires --name and --value")
    body = {"key": args.name, "value": args.value}
    resp = client.post("variables", body)
    print(json.dumps(resp, indent=2))
    print(_NOT_VERSIONED_WARNING)


def cmd_update(args, client) -> None:
    if not args.id or not args.name or args.value is None:
        raise SystemExit("update requires --id, --name, and --value")
    body = {"key": args.name, "value": args.value}
    resp = client.put(f"variables/{args.id}", body)
    print(json.dumps(resp, indent=2))
    print(_NOT_VERSIONED_WARNING)


def cmd_delete(args, client) -> None:
    if not args.id:
        raise SystemExit("delete requires --id")

    if not args.force:
        # Look up the variable so the dry-run message is specific.
        rows = _list_variables(client)
        match = next((r for r in rows if str(r.get("id")) == str(args.id)), None)
        name = match.get("key") if match else "<not found>"
        print(json.dumps({
            "dry_run": True,
            "id": args.id,
            "name": name,
            "message": f"Variable {args.id} ({name}) will be deleted. Rerun with --force to confirm.",
        }, indent=2))
        return

    resp = client.delete(f"variables/{args.id}")
    print(json.dumps(resp, indent=2))
    print(_NOT_VERSIONED_WARNING)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    sub = parser.add_subparsers(dest="cmd", required=True)

    p_list = sub.add_parser("list", help="GET /variables")
    p_list.add_argument("--env", required=True)
    p_list.add_argument("--name", default=None,
                        help="Filter list to a single variable by name (n8n's `key`).")

    p_create = sub.add_parser("create", help="POST /variables")
    p_create.add_argument("--env", required=True)
    p_create.add_argument("--name", required=True,
                          help="The n8n variable's name (sent as `key` in the request body).")
    p_create.add_argument("--value", required=True)

    p_update = sub.add_parser("update", help="PUT /variables/{id}")
    p_update.add_argument("--env", required=True)
    p_update.add_argument("--id", required=True)
    p_update.add_argument("--name", required=True,
                          help="The n8n variable's name (sent as `key` in the request body).")
    p_update.add_argument("--value", required=True)

    p_delete = sub.add_parser("delete", help="DELETE /variables/{id}")
    p_delete.add_argument("--env", required=True)
    p_delete.add_argument("--id", required=True)
    p_delete.add_argument("--force", action="store_true",
                          help="Actually issue DELETE. Without --force, exit 0 with a confirmation prompt message.")

    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    client = ensure_client(args.env, ws)

    handlers = {
        "list": cmd_list,
        "create": cmd_create,
        "update": cmd_update,
        "delete": cmd_delete,
    }
    # Defaults so every handler can read the same arg namespace.
    if not hasattr(args, "name"):
        args.name = None
    if not hasattr(args, "value"):
        args.value = None
    if not hasattr(args, "id"):
        args.id = None
    if not hasattr(args, "force"):
        args.force = False
    handlers[args.cmd](args, client)


if __name__ == "__main__":
    main()


path: helpers/dependency_graph.py
content: #!/usr/bin/env python3
"""Build a dependency graph across the workflows in an env (templates and/or live).

Three adjacency outputs:
  - calls            — Execute Workflow edges: caller_key → target_key (or live id when target unknown).
  - error_handlers   — source_key → handler_key (from common.yml.error_source_to_handler + live settings.errorWorkflow).
  - credential_groups — credential_id → list of workflows referencing it.

`--source template` reads workspace templates only.
`--source live` reads `GET /api/v1/workflows` only.
`--source both` (default) reads both and merges; live state takes precedence on disagreement.

Output is human-readable text by default; `--json` emits a single JSON document instead.
"""
import argparse
import json
import re
import sys
from pathlib import Path
from typing import Optional

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, load_common
from helpers.n8n_client import ensure_client


_PLACEHOLDER_RE = re.compile(r"^\{\{(?:INTERPOLATE_|@)env:workflows\.([A-Za-z0-9_\-]+)\.id\}\}$")


def _read_templates(workspace: Path) -> dict[str, dict]:
    """Return {workflow_key: parsed_template_dict}."""
    out: dict[str, dict] = {}
    template_dir = workspace / "n8n-workflows-template"
    if not template_dir.is_dir():
        return out
    for path in sorted(template_dir.glob("*.template.json")):
        key = path.name.removesuffix(".template.json")
        try:
            out[key] = json.loads(path.read_text(encoding="utf-8"))
        except Exception as e:
            print(f"[dependency_graph] WARN: skip {path.name}: {e}", file=sys.stderr)
    return out


def _id_to_key_map(env_yaml: dict) -> dict[str, str]:
    """Map live workflow id → yaml key, from <env>.yml workflows.<key>.id rows."""
    out: dict[str, str] = {}
    workflows = env_yaml.get("workflows") or {}
    for key, row in workflows.items():
        if not isinstance(row, dict):
            continue
        wid = row.get("id")
        if wid:
            out[str(wid)] = key
    return out


def _resolve_workflow_id_to_key(value: str, id_to_key: dict[str, str]) -> str:
    """Resolve a workflowId reference (placeholder or live id) to a yaml key, falling back to the raw value."""
    if not value:
        return ""
    m = _PLACEHOLDER_RE.match(value.strip())
    if m:
        return m.group(1)
    return id_to_key.get(value, value)


def _extract_calls(nodes: list, id_to_key: dict[str, str]) -> list[str]:
    """Pull every Execute Workflow target out of a node list and resolve to keys/ids."""
    targets: list[str] = []
    for node in nodes or []:
        if node.get("type") != "n8n-nodes-base.executeWorkflow":
            continue
        params = node.get("parameters") or {}
        wf = params.get("workflowId")
        # Newer (resource-locator) shape: { __rl, value, mode }
        if isinstance(wf, dict):
            value = wf.get("value", "")
        else:
            value = wf
        target = _resolve_workflow_id_to_key(str(value or ""), id_to_key)
        if target:
            targets.append(target)
    return targets


def _extract_credentials(nodes: list) -> list[str]:
    """Pull every credential id referenced by any node in this workflow."""
    out: list[str] = []
    for node in nodes or []:
        creds = node.get("credentials") or {}
        if not isinstance(creds, dict):
            continue
        for entry in creds.values():
            if not isinstance(entry, dict):
                continue
            cid = entry.get("id")
            if cid:
                out.append(str(cid))
    return out


def _extract_error_handler(workflow: dict, id_to_key: dict[str, str]) -> Optional[str]:
    """Read settings.errorWorkflow and resolve to a yaml key (or raw id)."""
    settings = workflow.get("settings") or {}
    raw = settings.get("errorWorkflow")
    if not raw:
        return None
    return _resolve_workflow_id_to_key(str(raw), id_to_key)


def build_graph(env_name: str, workspace: Path, source: str, workflow_key: Optional[str] = None) -> dict:
    """Return adjacency dicts. Read templates and/or live workflows per `source`."""
    env_yaml = load_yaml(env_name, workspace)
    common = load_common(workspace)
    id_to_key = _id_to_key_map(env_yaml)

    workflows: dict[str, dict] = {}
    if source in ("template", "both"):
        workflows.update(_read_templates(workspace))
    if source in ("live", "both"):
        load_env(env_name, workspace)
        client = ensure_client(env_name, workspace)
        for wf in client.list_workflows():
            wid = str(wf.get("id", ""))
            key = id_to_key.get(wid)
            # Live takes precedence on overlap; if the live workflow has no yaml row, key falls back to id.
            workflows[key or wid] = wf

    if workflow_key:
        if workflow_key in workflows:
            workflows = {workflow_key: workflows[workflow_key]}
        else:
            workflows = {}

    calls: dict[str, list[str]] = {}
    error_handlers: dict[str, str] = {}
    credential_groups: dict[str, list[str]] = {}

    for key, wf in workflows.items():
        nodes = wf.get("nodes") or []
        targets = _extract_calls(nodes, id_to_key)
        if targets:
            calls[key] = sorted(set(targets))
        eh = _extract_error_handler(wf, id_to_key)
        if eh:
            error_handlers[key] = eh
        for cid in _extract_credentials(nodes):
            credential_groups.setdefault(cid, []).append(key)

    # Merge in common.yml.error_source_to_handler (template-side authoritative pairing).
    for src, handler in (common.get("error_source_to_handler") or {}).items():
        # Only include pairs whose source is in scope (key match or single-key filter).
        if workflow_key and src != workflow_key:
            continue
        error_handlers.setdefault(src, handler)

    # Stable sort credential consumers; drop dupes per credential.
    for cid in list(credential_groups):
        credential_groups[cid] = sorted(set(credential_groups[cid]))

    return {
        "source": source,
        "env": env_name,
        "workflow_key": workflow_key,
        "calls": calls,
        "error_handlers": error_handlers,
        "credential_groups": credential_groups,
    }


def _format_text(graph: dict) -> str:
    lines: list[str] = []
    lines.append(f"dependency_graph: env={graph['env']} source={graph['source']}"
                 + (f" workflow_key={graph['workflow_key']}" if graph["workflow_key"] else ""))

    lines.append("")
    lines.append("calls (Execute Workflow):")
    if not graph["calls"]:
        lines.append("  (none)")
    else:
        for caller, targets in sorted(graph["calls"].items()):
            lines.append(f"  {caller} → {', '.join(targets)}")

    lines.append("")
    lines.append("error_handlers (settings.errorWorkflow + common.yml):")
    if not graph["error_handlers"]:
        lines.append("  (none)")
    else:
        for src, handler in sorted(graph["error_handlers"].items()):
            lines.append(f"  {src} → {handler}")

    lines.append("")
    lines.append("credential_groups (credential_id → workflows):")
    if not graph["credential_groups"]:
        lines.append("  (none)")
    else:
        for cid, refs in sorted(graph["credential_groups"].items()):
            lines.append(f"  {cid}: {', '.join(refs)}")
    return "\n".join(lines)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", default=None, dest="workflow_key",
                        help="Restrict the graph to one workflow as the focal point.")
    parser.add_argument("--source", choices=("template", "live", "both"), default="both",
                        help="Which side to read. Default 'both' merges templates + live.")
    parser.add_argument("--json", action="store_true", help="Emit a single JSON document on stdout.")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    graph = build_graph(args.env, ws, args.source, args.workflow_key)

    if args.json:
        print(json.dumps(graph, indent=2, sort_keys=True))
    else:
        print(_format_text(graph))


if __name__ == "__main__":
    main()


path: helpers/cleanup_queue_state.py
content: #!/usr/bin/env python3
"""Wipe a queue's Redis state (semaphore counter, permit sidecars, consumer
group, main stream, DLQ stream). Operator script for debugging, draining a
deprecated queue, or post-incident reset.

Dry-run by default — shows current state and the commands that would be issued.
Pass --force to actually mutate.

Usage:

    python3 helpers/cleanup_queue_state.py --env <env> --stream <name> [--group <name>] [--keep-dlq] [--force]

Reads UPSTASH_REDIS_REST_URL + UPSTASH_REDIS_REST_TOKEN from <workspace>/n8n-config/.env.<env>.
"""
import argparse
import json
import os
import sys
from pathlib import Path

import requests

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_env


def _upstash_call(base_url: str, token: str, cmd: list) -> dict:
    resp = requests.post(
        base_url.rstrip("/") + "/",
        headers={"Authorization": f"Bearer {token}"},
        json=cmd,
    )
    resp.raise_for_status()
    return resp.json()


def _probe_state(base_url: str, token: str, stream: str, group: str, dlq_stream: str) -> dict:
    """Best-effort snapshot of current state. Tolerates errors per-command."""
    state = {"stream": stream, "group": group, "dlq_stream": dlq_stream}
    def safe(cmd):
        try:
            return _upstash_call(base_url, token, cmd).get("result")
        except Exception as e:
            return f"<error: {e}>"
    state["inflight"] = safe(["GET", f"q:{stream}:inflight"])
    state["permit_keys"] = safe(["KEYS", f"q:{stream}:permits:*"])
    state["xlen_main"] = safe(["XLEN", stream])
    state["xlen_dlq"] = safe(["XLEN", dlq_stream])
    # XINFO GROUPS — may return error if stream/group doesn't exist
    state["xinfo_groups"] = safe(["XINFO", "GROUPS", stream])
    return state


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__,
                                     formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True, help="Env name (e.g. dev, prod, dod)")
    parser.add_argument("--stream", required=True, help="Stream name (e.g. orders, test-stream)")
    parser.add_argument("--group", default=None,
                        help="Consumer group name. Defaults to '<stream>-cg' (the queue_pop default).")
    parser.add_argument("--keep-dlq", action="store_true",
                        help="Do not delete the DLQ stream (<stream>-dlq). Use when you want to preserve "
                             "forensic data for inspection after cleanup.")
    parser.add_argument("--force", action="store_true",
                        help="Actually mutate. Without --force, prints the snapshot + planned commands and exits.")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)

    base_url = os.environ.get("UPSTASH_REDIS_REST_URL")
    [REDACTED_SECRET]"UPSTASH_REDIS_REST_TOKEN")
    if not base_url or not token:
        raise SystemExit(
            f"UPSTASH_REDIS_REST_URL or UPSTASH_REDIS_REST_TOKEN missing from "
            f".env.{args.env}. Set both before running."
        )

    stream = args.stream
    group = args.group or f"{stream}-cg"
    dlq_stream = f"{stream}-dlq"

    print(f"Probing current state for stream='{stream}' (group='{group}')…")
    snapshot = _probe_state(base_url, token, stream, group, dlq_stream)
    print(json.dumps(snapshot, indent=2))

    permit_keys = snapshot.get("permit_keys") or []
    if not isinstance(permit_keys, list):
        permit_keys = []

    planned = [
        ["DEL", f"q:{stream}:inflight"],
        *[["DEL", k] for k in permit_keys],
        ["XGROUP", "DESTROY", stream, group],
        ["DEL", stream],
    ]
    if not args.keep_dlq:
        planned.append(["DEL", dlq_stream])

    print(f"\nPlanned commands ({len(planned)}):")
    for cmd in planned:
        print(f"  {' '.join(cmd)}")

    if not args.force:
        print("\nDRY-RUN. Rerun with --force to execute.")
        return

    print("\nExecuting…")
    results = []
    for cmd in planned:
        try:
            r = _upstash_call(base_url, token, cmd)
            results.append({"cmd": cmd, "result": r.get("result"), "error": r.get("error")})
        except Exception as e:
            results.append({"cmd": cmd, "result": None, "error": str(e)})
    print(json.dumps(results, indent=2))

    n_ok = sum(1 for r in results if not r.get("error"))
    print(f"\nDone. {n_ok}/{len(results)} commands succeeded.")
    if n_ok != len(results):
        sys.exit(1)


if __name__ == "__main__":
    main()


path: helpers/stop_executions.py
content: #!/usr/bin/env python3
"""Stop n8n executions matching a status filter (and optional --workflow-key scope).

Two-phase:
  1. GET /api/v1/executions per workflow (list candidates) so the caller sees what would be stopped.
  2. POST /api/v1/executions/stop with the matching status set (and workflowId if scoped).

Without `--force`, the helper prints the candidate list and exits 0 with
"Rerun with --force to stop these executions." — never prompts interactively
(agents run non-TTY).

`--status running,waiting,queued` is a comma-separated list. n8n's POST stop endpoint
accepts a status array; the helper passes it through verbatim.
"""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client, N8nClient


_STATUS_CHOICES = ("error", "success", "running", "canceled", "waiting", "crashed", "queued")
_DEFAULT_STATUSES = "running,waiting,queued"


def _parse_status_list(raw: str) -> list[str]:
    """Split a comma-separated status list and validate each entry."""
    parts = [p.strip() for p in (raw or "").split(",") if p.strip()]
    bad = [p for p in parts if p not in _STATUS_CHOICES]
    if bad:
        raise SystemExit(f"Unknown status value(s): {bad}. Choices: {list(_STATUS_CHOICES)}")
    return parts


def _resolve_workflow_id(yaml_data: dict, workflow_key: str) -> str:
    return str(get_config_value(yaml_data, f"workflows.{workflow_key}.id"))


def _list_workflow_ids(client: N8nClient) -> list[str]:
    return [str(wf.get("id")) for wf in client.list_workflows() if wf.get("id")]


def _list_candidates(client: N8nClient, workflow_ids: list[str], statuses: list[str]) -> list[dict]:
    """Per-workflow GET /executions for each requested status; flatten and return rows."""
    out: list[dict] = []
    for wid in workflow_ids:
        for status in statuses:
            cursor = None
            while True:
                params: dict = {"workflowId": wid, "status": status, "limit": 250}
                if cursor:
                    params["cursor"] = cursor
                resp = client.get("executions", params=params)
                rows = resp.get("data") or []
                out.extend(rows)
                cursor = resp.get("nextCursor")
                if not cursor:
                    break
    return out


def _summarize(candidates: list[dict]) -> dict:
    """Group candidates by status for the dry-run output."""
    by_status: dict = {}
    for row in candidates:
        st = row.get("status", "unknown")
        by_status.setdefault(st, []).append(row.get("id"))
    return {"total": len(candidates), "by_status": {k: sorted(v) for k, v in sorted(by_status.items())}}


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", default=None, dest="workflow_key",
                        help="Yaml key; if omitted, scope is the whole env (every workflow).")
    parser.add_argument("--status", default=_DEFAULT_STATUSES,
                        help=f"Comma-separated statuses to stop. Default: {_DEFAULT_STATUSES}")
    parser.add_argument("--force", action="store_true",
                        help="Actually issue the POST /executions/stop. Without --force, exit 0 with the candidate list.")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    yaml_data = load_yaml(args.env, ws)
    load_env(args.env, ws)
    client = ensure_client(args.env, ws)

    statuses = _parse_status_list(args.status)
    if args.workflow_key:
        workflow_ids = [_resolve_workflow_id(yaml_data, args.workflow_key)]
    else:
        workflow_ids = _list_workflow_ids(client)

    candidates = _list_candidates(client, workflow_ids, statuses)
    summary = _summarize(candidates)

    if not args.force:
        result = {
            "dry_run": True,
            "candidates": summary,
            "message": "Rerun with --force to stop these executions.",
        }
        print(json.dumps(result, indent=2))
        return

    if not candidates:
        print(json.dumps({"stopped": [], "message": "No candidates matched."}, indent=2))
        return

    body: dict = {"status": statuses}
    if args.workflow_key:
        body["workflowId"] = workflow_ids[0]

    resp = client.post("executions/stop", body)
    result = {
        "stopped": summary,
        "n8n_response": resp,
    }
    print(json.dumps(result, indent=2))


if __name__ == "__main__":
    main()


path: helpers/hydrate.py
content: #!/usr/bin/env python3
"""Resolve all {{INTERPOLATE_...}} / {{@...}} placeholders in a workflow template for one env."""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root, ensure_workspace
from helpers.placeholder import env_resolver, file_resolver, js_resolver, py_resolver, uuid_resolver, validator


def hydrate(env_name: str, workflow_key: str, workspace: Path, strict: bool = False) -> Path:
    """Hydrate a template and return the path to the generated JSON."""
    ensure_workspace(workspace)

    template_dir = workspace / "n8n-workflows-template"
    template_file = template_dir / f"{workflow_key}.template.json"
    if not template_file.exists():
        raise FileNotFoundError(f"Template not found: {template_file}")

    text = template_file.read_text(encoding="utf-8")

    # Run resolvers in order
    text = env_resolver.resolve(text, env_name, workspace)
    text = file_resolver.resolve(text, workspace)
    text = js_resolver.resolve(text, workspace)
    text = py_resolver.resolve(text, workspace)
    text = uuid_resolver.resolve(text)

    # Validate no residuals
    validator.validate_no_absolute_paths(text)
    residuals = validator.check_residuals(text)
    if residuals:
        if strict:
            raise ValueError(
                f"Residual placeholders after hydration in '{workflow_key}':\n"
                + "\n".join(f"  {r}" for r in residuals)
            )
        else:
            print(f"WARNING: {len(residuals)} residual placeholder(s) in '{workflow_key}': {residuals}", file=sys.stderr)

    # Validate JSON parses
    try:
        data = json.loads(text)
    except json.JSONDecodeError as e:
        raise ValueError(f"Generated JSON is invalid for '{workflow_key}': {e}")

    out_dir = workspace / "n8n-build" / env_name
    out_dir.mkdir(parents=True, exist_ok=True)
    out_file = out_dir / f"{workflow_key}.generated.json"
    out_file.write_text(json.dumps(data, indent=2))
    return out_file


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--strict", action="store_true", help="Error (not warn) on residual placeholders")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    out = hydrate(args.env, args.workflow_key, ws, strict=args.strict)
    print(f"Hydrated: {out}")


if __name__ == "__main__":
    main()


path: helpers/dehydrate.py
content: #!/usr/bin/env python3
"""Convert raw n8n workflow JSON into a template (inverse of hydrate)."""
import argparse
import json
import re
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, flatten_config
from helpers.placeholder import js_resolver, py_resolver

# Volatile / runtime-only fields that must not appear in templates.
# (Live n8n GET /workflows/:id returns these alongside the canonical template
# fields. NEVER add `name`, `nodes`, `connections`, or `settings` here — those
# are the workflow's actual content.)
_METADATA_FIELDS = frozenset({
    "id", "active", "versionId", "createdAt", "updatedAt",
    "tags", "shared", "isArchived", "triggerCount", "homeProject",
    "scopes", "meta", "usedCredentials", "sharedWithProjects",
    "pinData",
    # Added 2026-04: n8n's response shape grew with the publish/version model.
    "activeVersion", "activeVersionId", "versionCounter",
    "description", "staticData",
})


def _strip_metadata(data: dict) -> dict:
    """Remove volatile / runtime fields recursively from the top-level."""
    return {k: v for k, v in data.items() if k not in _METADATA_FIELDS}


def _restore_uuids_by_name(data: dict, existing_template: dict) -> dict:
    """For each node, if the existing template had a UUID placeholder for the same name, restore it."""
    uuid_pattern = re.compile(r"\{\{(?:INTERPOLATE_|@)uuid:[^}]+\}\}")
    name_to_uuid_placeholder: dict[str, str] = {}
    for node in existing_template.get("nodes", []):
        name = node.get("name")
        node_id = str(node.get("id", ""))
        if name and uuid_pattern.fullmatch(node_id):
            name_to_uuid_placeholder[name] = node_id

    for node in data.get("nodes", []):
        name = node.get("name")
        if name in name_to_uuid_placeholder:
            node["id"] = name_to_uuid_placeholder[name]
    return data


def _reverse_env_values(text: str, env_data: dict) -> str:
    """Reverse-substitute env values back into {{@env:...}} placeholders.

    Skips workflows.* and credentials.* blocks (those are n8n internal IDs/names, not
    values that should appear in template body content). Only substitutes strings of
    length >= 4 to avoid clobbering tiny matches.
    """
    flat = flatten_config(env_data)
    sortable = sorted(
        (
            (k, v) for k, v in flat.items()
            if isinstance(v, str)
            and len(v) >= 4
            and not k.startswith("workflows.")
            and not k.startswith("credentials.")
        ),
        key=lambda kv: -len(kv[1]),  # longest first to avoid partial overshadowing
    )
    for key, value in sortable:
        encoded = json.dumps(value)[1:-1]  # strip outer quotes for substring match in JSON text
        placeholder = "{{@env:" + key + "}}"
        text = text.replace(encoded, placeholder)
    return text


def dehydrate_data(
    raw: dict,
    env_name: str,
    workspace: Path,
    output_key: str,
    remove_triggers: bool = False,
) -> str:
    """Run the full dehydrate pipeline on raw workflow JSON. Return template JSON text."""
    cleaned = _strip_metadata(raw)

    # If a previous template exists, use it as a guide for UUID restoration
    existing_path = workspace / "n8n-workflows-template" / f"{output_key}.template.json"
    if existing_path.exists():
        try:
            existing = json.loads(existing_path.read_text())
            cleaned = _restore_uuids_by_name(cleaned, existing)
        except json.JSONDecodeError:
            pass  # ignore corrupt previous template

    if remove_triggers:
        cleaned["nodes"] = [
            n for n in cleaned.get("nodes", []) if "trigger" not in str(n.get("type", "")).lower()
        ]

    text = json.dumps(cleaned, indent=2)

    # Reverse env values (e.g. instance names, display names, credential ids)
    try:
        env_data = load_yaml(env_name, workspace)
        text = _reverse_env_values(text, env_data)
    except Exception:
        pass  # if env YAML missing, skip reverse-substitution

    # Restore JS / Python placeholders from DEHYDRATE markers
    text = js_resolver.dehydrate(text)
    text = py_resolver.dehydrate(text)

    return text


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--input", required=True, help="Path to raw workflow JSON")
    parser.add_argument("--output-key", required=True, dest="output_key")
    parser.add_argument("--remove-triggers", action="store_true")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    raw = json.loads(Path(args.input).read_text())
    text = dehydrate_data(raw, args.env, ws, args.output_key, args.remove_triggers)
    out_dir = ws / "n8n-workflows-template"
    out_dir.mkdir(parents=True, exist_ok=True)
    out_file = out_dir / f"{args.output_key}.template.json"
    out_file.write_text(text)
    print(f"Wrote {out_file}")


if __name__ == "__main__":
    main()


path: helpers/create_queue.py
content: #!/usr/bin/env python3
"""Copy queue primitive templates from harness/primitives into the workspace and register them."""
import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
# Import the lock helpers' generic primitive-copy + create-workflow registration
# functions directly. They are key-agnostic (each takes a `key` argument), so
# duplicating them into a queue-specific module would buy nothing. Marked as
# leading-underscore for cohesion within create_lock.py's module, but Python
# doesn't enforce privacy and these are the canonical implementations.
# Refactor target: extract `_copy_primitive` + `_register_via_create_workflow`
# into `helpers/_primitive_install.py` so create_lock + create_queue both import
# from a shared module instead of one importing from the other.
from helpers.create_lock import _copy_primitive, _register_via_create_workflow


_PRIMITIVES = {
    "queue_publish": "Queue Publish",
    "queue_pop": "Queue Pop",
    "queue_ack": "Queue Ack",
}
_ERROR_HANDLER = ("error_handler_queue_cleanup", "Error Handler Queue Cleanup")
# Sample-test pair: a webhook-triggered producer that publishes 5 tagged messages
# (3 happy / 1 transient_fail_once / 1 poison) and a schedule-polled consumer
# that simulates per-tag behaviour. Together they exercise XAUTOCLAIM retry,
# the semaphore cap, and DLQ routing — without the operator hand-writing test
# scaffolding. Tier 1 (callers) since they invoke the queue primitives.
_SAMPLE_TEST = {
    "queue_sample_producer": "Queue Sample Producer",
    "queue_sample_consumer": "Queue Sample Consumer",
}


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--include-error-handler", action="store_true", dest="include_error_handler")
    parser.add_argument("--with-sample-test", action="store_true", dest="with_sample_test",
                        help="Also copy a paired producer+consumer test that exercises happy / "
                             "transient-retry / poison-DLQ paths against a 'test-stream'. Useful for "
                             "first-time end-to-end validation after install.")
    parser.add_argument("--force-overwrite", action="store_true", dest="force_overwrite")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)

    primitives = dict(_PRIMITIVES)
    if args.include_error_handler:
        key, name = _ERROR_HANDLER
        primitives[key] = name

    # Sample-test workflows register at Tier 1 (callers) not Tier 0a, since
    # they depend on the queue primitives being deployed first.
    sample_test = dict(_SAMPLE_TEST) if args.with_sample_test else {}

    # Two-pass copy → register, mirroring create_lock.py. If a registration
    # fails (transient n8n API hiccup, expired key) the workspace still has
    # all template files on disk so a retry can resume cleanly.
    for key in (*primitives.keys(), *sample_test.keys()):
        _copy_primitive(ws, key, force_overwrite=args.force_overwrite)

    failures: list[tuple[str, Exception]] = []
    for key, name in primitives.items():
        try:
            _register_via_create_workflow(ws, key, name, "Tier 0a: leaves")
        except SystemExit as e:
            failures.append((key, e))
            print(f"  WARNING: registration failed for '{key}'; continuing.", file=sys.stderr)
    for key, name in sample_test.items():
        try:
            _register_via_create_workflow(ws, key, name, "Tier 1")
        except SystemExit as e:
            failures.append((key, e))
            print(f"  WARNING: registration failed for '{key}'; continuing.", file=sys.stderr)

    total = len(primitives) + len(sample_test)
    if failures:
        print(
            f"create-queue partial: {total - len(failures)}/{total} registered. "
            f"Re-run after fixing the underlying issue (commonly: invalid N8N_API_KEY).",
            file=sys.stderr,
        )
        sys.exit(1)
    if args.with_sample_test:
        print(
            "create-queue complete. Sample-test workflows installed:\n"
            "  - queue_sample_producer (webhook /queue-sample-producer; publishes 5 tagged messages)\n"
            "  - queue_sample_consumer (schedule 10s; pops + simulates per-tag behaviour)\n"
            "Deploy with deploy_all, then POST to the producer's webhook and watch the consumer drain."
        )
    else:
        print("create-queue complete.")


if __name__ == "__main__":
    main()


path: helpers/manage_credentials.py
content: #!/usr/bin/env python3
"""Manage n8n credentials. Subcommands:

  create     — POST /api/v1/credentials with values from .env.<env> (Path A).
  list-link  — GET /api/v1/credentials and link an existing credential into the YAML (Path B).

Path A flow (agent-mediated creation):
  1. Agent appends required secret env-var names to <workspace>/n8n-config/.env.example
     and tells the user to copy those entries into .env.<env> with real values.
  2. Agent runs:
        manage_credentials.py create --env <env> --key <yaml-key> \\
          --type <n8n-credential-type> --name "<display name>" \\
          --env-vars KEY1,KEY2,...
  3. Helper loads .env.<env> via config.py:load_env(), builds the credential `data`
     payload, POSTs to /api/v1/credentials, and writes the returned id+name into
     <workspace>/n8n-config/<env>.yml under credentials.<key>.

Path B flow (user-mediated):
  1. User creates the credential in the n8n UI.
  2. Agent runs:
        manage_credentials.py list-link --env <env> --key <yaml-key> \\
          --type <n8n-credential-type> [--from-name "<existing display name>"]
  3. Helper GETs /credentials, filters by type (and name if --from-name), writes
     the matching id+name into the YAML.

POLICY (encoded here AND in skills/manage-credentials.md):
  - The agent NEVER reads .env* files itself.
  - The agent NEVER collects secrets in chat.
  - This helper is the SOLE code path that loads .env.<env> for credential creation.
"""
import argparse
import json
import os
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

import yaml

from helpers.workspace import workspace_root
from helpers.config import load_env, load_yaml
from helpers.n8n_client import N8nClient


def _client_for(env_name: str, workspace: Path) -> N8nClient:
    data = load_yaml(env_name, workspace)
    load_env(env_name, workspace)
    [REDACTED_SECRET]"N8N_API_KEY", "")
    instance = data.get("n8n", {}).get("instanceName", "")
    return N8nClient(base_url=instance, [REDACTED_SECRET])


def _save_yaml(yaml_file: Path, data: dict) -> None:
    yaml_file.write_text(yaml.dump(data, default_flow_style=False, sort_keys=False))


def _get_schema(client: N8nClient, cred_type: str) -> dict:
    """Best-effort: GET /credentials/schema/{type}. Returns empty dict on failure."""
    try:
        return client.get(f"credentials/schema/{cred_type}")
    except Exception:
        return {}


def _build_data_payload(env_vars: list[str]) -> dict:
    """Map declared env-var names to their values. The keys in the returned dict
    are the n8n field names — convention: same casing as the env var (e.g. CLIENT_ID
    becomes the field 'CLIENT_ID' OR 'clientId' depending on the cred type schema).
    For simplicity, we pass the env var name as-is and let the schema's `data` shape
    drive the mapping. The agent typically passes camelCased var names.
    """
    data = {}
    for raw in env_vars:
        var = raw.strip()
        if not var:
            continue
        if "=" in var:
            field, _, env_var = var.partition("=")
            data[field.strip()] = os.environ.get(env_var.strip(), "")
        else:
            data[var] = os.environ.get(var, "")
    return data


def cmd_create(args) -> None:
    ws = workspace_root(args.workspace)
    yaml_file = ws / "n8n-config" / f"{args.env}.yml"
    yaml_data = load_yaml(args.env, ws)
    client = _client_for(args.env, ws)

    env_var_names = [v.strip() for v in (args.env_vars or "").split(",") if v.strip()]
    data_payload = _build_data_payload(env_var_names)
    # Detect-by-env-var-presence: a token like `field=ENV_VAR` is "missing" iff
    # ENV_VAR isn't set in os.environ. Avoids false positives on legitimately
    # falsy values (empty string, "0", "false") and reports the env-var name
    # the user needs to set, not the raw `field=ENV_VAR` token.
    missing = []
    for raw in env_var_names:
        env_name = raw.split("=", 1)[1].strip() if "=" in raw else raw.strip()
        if env_name not in os.environ:
            missing.append(env_name)
    if missing:
        print(f"WARNING: missing env vars: {missing}. Add them to .env.{args.env}.", file=sys.stderr)

    body = {
        "name": args.name,
        "type": args.type,
        "data": data_payload,
    }

    if args.dry_run:
        redacted = dict(body)
        redacted["data"] = {k: "[REDACTED]" for k in body["data"]}
        print("[dry-run] would POST /credentials:", json.dumps(redacted, indent=2))
        return

    resp = client.post("credentials", body)
    cred_id = resp.get("id")
    cred_name = resp.get("name", args.name)

    creds = yaml_data.setdefault("credentials", {}) or {}
    creds[args.key] = {"id": cred_id, "name": cred_name, "type": args.type}
    yaml_data["credentials"] = creds
    _save_yaml(yaml_file, yaml_data)
    print(f"Created credential '{cred_name}' (id={cred_id}, type={args.type}) → wrote to {yaml_file} under credentials.{args.key}")


def cmd_list_link(args) -> None:
    ws = workspace_root(args.workspace)
    yaml_file = ws / "n8n-config" / f"{args.env}.yml"
    yaml_data = load_yaml(args.env, ws)
    client = _client_for(args.env, ws)

    try:
        all_creds = client.get("credentials")
        if isinstance(all_creds, dict):
            all_creds = all_creds.get("data", [])
    except Exception as e:
        print(f"ERROR: list /credentials failed: {e}", file=sys.stderr)
        sys.exit(1)

    matches = [c for c in all_creds if c.get("type") == args.type]
    if args.from_name:
        matches = [c for c in matches if c.get("name") == args.from_name]
    if not matches:
        print(f"No credentials match type='{args.type}'" + (f" name='{args.from_name}'" if args.from_name else ""), file=sys.stderr)
        sys.exit(1)
    if len(matches) > 1 and not args.from_name:
        print("Multiple credentials match. Use --from-name to disambiguate:", file=sys.stderr)
        for c in matches:
            print(f"  - id={c.get('id')} name='{c.get('name')}'", file=sys.stderr)
        sys.exit(1)

    chosen = matches[0]
    creds = yaml_data.setdefault("credentials", {}) or {}
    creds[args.key] = {"id": chosen.get("id"), "name": chosen.get("name"), "type": args.type}
    yaml_data["credentials"] = creds
    _save_yaml(yaml_file, yaml_data)
    print(f"Linked credential '{chosen.get('name')}' (id={chosen.get('id')}) → {yaml_file} under credentials.{args.key}")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    sub = parser.add_subparsers(dest="cmd", required=True)

    p_create = sub.add_parser("create", help="POST /credentials from .env.<env> (Path A)")
    p_create.add_argument("--env", required=True)
    p_create.add_argument("--key", required=True, help="YAML key to write under credentials.<key>")
    p_create.add_argument("--type", required=True, help="n8n credential type (e.g. microsoftOAuth2Api)")
    p_create.add_argument("--name", required=True, help="Display name for the credential")
    p_create.add_argument("--env-vars", default=None, dest="env_vars",
                          help="Comma-separated env-var names that map to credential data fields. "
                               "Use 'fieldName=ENV_VAR' for explicit mapping.")
    p_create.add_argument("--dry-run", action="store_true")

    p_link = sub.add_parser("list-link", help="GET /credentials and link an existing one (Path B)")
    p_link.add_argument("--env", required=True)
    p_link.add_argument("--key", required=True)
    p_link.add_argument("--type", required=True)
    p_link.add_argument("--from-name", default=None, dest="from_name")
    p_link.add_argument("--dry-run", action="store_true")

    args = parser.parse_args()
    if args.cmd == "create":
        cmd_create(args)
    else:
        cmd_list_link(args)


if __name__ == "__main__":
    main()


path: helpers/deploy_run_assert.py
content: #!/usr/bin/env python3
"""Composite verify: validate (built) → deploy → run --expect-status success."""
import argparse
import subprocess
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root


def _run(cmd: list[str], stage: str) -> None:
    r = subprocess.run(cmd)
    if r.returncode != 0:
        print(f"FAIL: stage={stage} exit={r.returncode}", file=sys.stderr)
        sys.exit(r.returncode)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--payload", default="{}")
    parser.add_argument("--timeout", type=int, default=30)
    parser.add_argument("--no-activate", action="store_true")
    parser.add_argument(
        "--expect-status",
        default="success",
        dest="expect_status",
        choices=("success", "error"),
        help="Expected terminal status of the run (default: success).",
    )
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    helpers = Path(__file__).parent
    ws_args = ["--workspace", str(ws)]

    # 1. hydrate (so the built JSON exists for validation)
    _run(
        [sys.executable, str(helpers / "hydrate.py"), *ws_args, "--env", args.env, "--workflow-key", args.workflow_key],
        stage="hydrate",
    )

    # 2. validate built
    _run(
        [sys.executable, str(helpers / "validate.py"), *ws_args, "--env", args.env, "--workflow-key", args.workflow_key, "--source", "built"],
        stage="validate",
    )

    # 3. deploy
    deploy_cmd = [sys.executable, str(helpers / "deploy.py"), *ws_args, "--env", args.env, "--workflow-key", args.workflow_key]
    if args.no_activate:
        deploy_cmd.append("--no-activate")
    _run(deploy_cmd, stage="deploy")

    # 4. run with expectation
    run_cmd = [
        sys.executable, str(helpers / "run.py"), *ws_args,
        "--env", args.env, "--workflow-key", args.workflow_key,
        "--payload", args.payload,
        "--timeout", str(args.timeout),
        "--expect-status", args.expect_status,
    ]
    _run(run_cmd, stage="run")

    print("deploy-run-assert OK")


if __name__ == "__main__":
    main()


path: helpers/deactivate.py
content: #!/usr/bin/env python3
"""Deactivate a workflow on its env's n8n instance."""
import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client


def _resolve_workflow_id(env_name: str, workflow_key: str, workspace: Path) -> str:
    data = load_yaml(env_name, workspace)
    try:
        return str(get_config_value(data, f"workflows.{workflow_key}.id"))
    except KeyError:
        raise SystemExit(
            f"No workflow id for key '{workflow_key}' in env '{env_name}'."
        )


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    wf_id = _resolve_workflow_id(args.env, args.workflow_key, ws)
    client = ensure_client(args.env, ws)
    client.post(f"workflows/{wf_id}/deactivate")
    print(f"Deactivated workflow '{args.workflow_key}' (id={wf_id}) on env '{args.env}'")


if __name__ == "__main__":
    main()


path: helpers/bootstrap_env.py
content: #!/usr/bin/env python3
"""Bootstrap an n8n environment: create YAML + .env, validate, mint placeholder workflow IDs."""
import argparse
import os
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

import yaml

from helpers.workspace import workspace_root, ensure_workspace


def _is_placeholder_id(id_val) -> bool:
    if not id_val:
        return True
    s = str(id_val)
    return s.startswith("your-") or s == "placeholder"


def _write_env_yaml(yaml_file: Path, env_name: str, instance: str, postfix: str, display_name: str) -> None:
    data = {
        "name": env_name,
        "displayName": display_name,
        "workflowNamePostfix": postfix,
        "n8n": {"instanceName": instance},
        "credentials": {},
        "workflows": {},
    }
    yaml_file.write_text(yaml.dump(data, default_flow_style=False, sort_keys=False))


def _write_env_file(env_file: Path, api_key: str) -> None:
    env_file.write_text(f"N8N_API_KEY={api_key}\n")
    env_file.chmod(0o600)


def _load_yaml(yaml_file: Path) -> dict:
    with open(yaml_file) as f:
        return yaml.safe_load(f) or {}


def _save_yaml(yaml_file: Path, data: dict) -> None:
    yaml_file.write_text(yaml.dump(data, default_flow_style=False, sort_keys=False))


def _validate_instance(instance: str, api_key: str) -> None:
    """Attempt GET /api/v1/workflows?limit=1. Raises on failure."""
    from helpers.n8n_client import N8nClient
    client = N8nClient(base_url=instance, [REDACTED_SECRET])
    client.get("workflows", params={"limit": 1})


def _mint_placeholder_workflows(
    yaml_file: Path,
    data: dict,
    instance: str,
    api_key: str,
    dry_run: bool,
) -> None:
    from helpers.n8n_client import N8nClient
    client = N8nClient(base_url=instance, [REDACTED_SECRET])
    workflows = data.get("workflows") or {}
    changed = False
    for key, wf in workflows.items():
        if not isinstance(wf, dict):
            continue
        if _is_placeholder_id(wf.get("id")):
            wf_name = wf.get("name", key)
            display_name = data.get("displayName", "")
            postfix = data.get("workflowNamePostfix", "")
            full_name = f"{display_name} {wf_name}{postfix}".strip()
            if dry_run:
                print(f"  [dry-run] would mint workflow '{full_name}' for key '{key}'")
            else:
                resp = client.post("workflows", {"name": full_name, "nodes": [], "connections": {}, "settings": {}})
                new_id = resp.get("id", "")
                print(f"  Minted workflow '{full_name}' → id={new_id} (key={key})")
                data["workflows"][key]["id"] = new_id
                changed = True
    if changed:
        _save_yaml(yaml_file, data)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None, help="Workspace path. Default: ${PWD}/n8n-evol-I-workspace, or ${PWD} if its basename is already n8n-evol-I-workspace.")
    parser.add_argument("--env", required=True, help="Environment name (e.g. dev, prod)")
    parser.add_argument("--instance", default=None, help="n8n instance URL or hostname")
    parser.add_argument("--api-key", default=None, dest="api_key", help="n8n API key")
    parser.add_argument("--postfix", default=" [DEV]", help="Workflow name postfix (default: ' [DEV]')")
    parser.add_argument("--display-name", default="Development", dest="display_name", help="Human-readable env name")
    parser.add_argument("--dry-run", action="store_true", help="Print what would be done without writing/POSTing")
    parser.add_argument("--force-update-instance", action="store_true", help="Overwrite instance URL in existing YAML")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    config_dir = ws / "n8n-config"
    config_dir.mkdir(parents=True, exist_ok=True)

    yaml_file = config_dir / f"{args.env}.yml"
    env_file = config_dir / f".env.{args.env}"

    # ── Stage 1: create YAML + .env if absent ──────────────────────────────
    stage1_wrote_yaml = False
    stage1_wrote_env = False

    if not yaml_file.exists():
        instance = args.instance
        [REDACTED_SECRET] or os.environ.get("N8N_API_KEY", "")
        if not instance:
            instance = input(f"n8n instance URL for env '{args.env}': ").strip()
        if not [REDACTED_SECRET] = input(f"API key for env '{args.env}': ").strip()
        if not args.dry_run:
            _write_env_yaml(yaml_file, args.env, instance, args.postfix, args.display_name)
            stage1_wrote_yaml = True
            print(f"  Wrote {yaml_file}")
        else:
            print(f"  [dry-run] would write {yaml_file}")
    else:
        data = _load_yaml(yaml_file)
        if args.force_update_instance and args.instance:
            data.setdefault("n8n", {})["instanceName"] = args.instance
            if not args.dry_run:
                _save_yaml(yaml_file, data)

    if not env_file.exists():
        [REDACTED_SECRET] or os.environ.get("N8N_API_KEY", "")
        if not [REDACTED_SECRET] = input(f"API key for env '{args.env}': ").strip()
        if not args.dry_run:
            _write_env_file(env_file, api_key)
            stage1_wrote_env = True
            print(f"  Wrote {env_file} (mode 0600)")
        else:
            print(f"  [dry-run] would write {env_file}")
    else:
        [REDACTED_SECRET] or None

    if args.dry_run:
        print("Dry-run complete. No files written.")
        return

    # ── Stage 2: live validation ────────────────────────────────────────────
    data = _load_yaml(yaml_file)
    instance = data.get("n8n", {}).get("instanceName", "")
    # Load API key: prefer flag, then .env file, then os.environ
    if not [REDACTED_SECRET] dict = {}
        if env_file.exists():
            with open(env_file) as f:
                for line in f:
                    line = line.strip()
                    if "=" in line and not line.startswith("#"):
                        k, _, v = line.partition("=")
                        env_vars[k.strip()] = v.strip()
        [REDACTED_SECRET]"N8N_API_KEY", os.environ.get("N8N_API_KEY", ""))

    try:
        _validate_instance(instance, api_key)
        print(f"  n8n instance at {instance} is reachable.")
    except Exception as e:
        # Rollback stage 1 writes
        if stage1_wrote_yaml:
            yaml_file.unlink(missing_ok=True)
        if stage1_wrote_env:
            env_file.unlink(missing_ok=True)
        print(f"ERROR: Could not reach n8n at {instance}: {e}", file=sys.stderr)
        print("Rolled back any files written in this run.", file=sys.stderr)
        sys.exit(1)

    # ── Stage 3: mint placeholder workflow IDs ─────────────────────────────
    _mint_placeholder_workflows(yaml_file, data, instance, api_key, dry_run=False)
    print("bootstrap-env complete.")


if __name__ == "__main__":
    main()


path: helpers/register_error_handler.py
content: #!/usr/bin/env python3
"""Wire a workflow's settings.errorWorkflow to point at an existing error-handler workflow."""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

import yaml

from helpers.workspace import workspace_root


def _handler_exists(workspace: Path, handler_key: str) -> bool:
    cfg_dir = workspace / "n8n-config"
    if not cfg_dir.is_dir():
        return False
    for yml in cfg_dir.glob("*.yml"):
        if yml.stem in ("common", "deployment_order"):
            continue
        try:
            data = yaml.safe_load(yml.read_text()) or {}
            workflows = data.get("workflows") or {}
            if handler_key in workflows:
                return True
        except Exception:
            continue
    return False


def _update_error_source_map(workspace: Path, source_key: str, handler_key: str) -> None:
    """Append an entry to common.yml.error_source_to_handler if not present."""
    common = workspace / "n8n-config" / "common.yml"
    data: dict = {}
    if common.exists():
        data = yaml.safe_load(common.read_text()) or {}
    mapping = data.setdefault("error_source_to_handler", {}) or {}
    if mapping.get(source_key) != handler_key:
        mapping[source_key] = handler_key
        data["error_source_to_handler"] = mapping
        common.write_text(yaml.dump(data, default_flow_style=False, sort_keys=False))
        print(f"  Updated common.yml: error_source_to_handler.{source_key} = {handler_key}")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--handler-key", required=True, dest="handler_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    if not _handler_exists(ws, args.handler_key):
        print(f"ERROR: handler '{args.handler_key}' not registered in any env YAML", file=sys.stderr)
        sys.exit(1)

    template = ws / "n8n-workflows-template" / f"{args.workflow_key}.template.json"
    if not template.exists():
        print(f"ERROR: template not found: {template}", file=sys.stderr)
        sys.exit(1)

    data = json.loads(template.read_text())
    settings = data.setdefault("settings", {})
    placeholder = "{{@env:workflows." + args.handler_key + ".id}}"
    settings["errorWorkflow"] = placeholder
    template.write_text(json.dumps(data, indent=2))
    print(f"  Wired {args.workflow_key}.settings.errorWorkflow → {placeholder}")

    _update_error_source_map(ws, args.workflow_key, args.handler_key)
    print(f"register-error-handler complete.")


if __name__ == "__main__":
    main()


path: helpers/add_lock_to_workflow.py
content: #!/usr/bin/env python3
"""Insert lock acquire / release Execute Workflow nodes around a workflow's main flow."""
import argparse
import json
import re
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root


_LOCK_ACQUIRE_NODE_NAME = "Lock Acquire"
_LOCK_RELEASE_NODE_NAME = "Lock Release"
_DEFAULT_TTL_SECONDS = 86400  # 24h — Redis-side EXPIRE on the lock key
_DEFAULT_MAX_WAIT_SECONDS = 86400  # 24h — wait-loop deadline; effectively unbounded by default


def _normalize_n8n_expression(expr) -> tuple[str, bool]:
    """Normalize a user-supplied n8n expression for executeWorkflow defineBelow inputs.

    n8n's `executeWorkflow@1.2` in `defineBelow` mode requires the canonical
    `={{ <js> }}` form for expression evaluation. The bare `=<js>` prefix is
    treated as a literal string and silently degrades — e.g. `--scope-expression
    "='foo-' + $json.x"` becomes the literal scope `'foo-' + $json.x`, so every
    caller hits the same lock regardless of payload. Skill MD historically
    documented both forms inconsistently; finding #15 from the deep-dive.

    Input shapes:
      (a) `={{ <js> }}`   — already canonical; pass through, no warning.
      (b) `=<js>` (bare)  — wrap into `={{ <js> }}`; emit a one-shot deprecation
                           warning the first time normalization happens.
      (c) `<literal>`     — no leading `=`; treat as a static scope and wrap as
                           `={{ "<literal>" }}` (escaped) so the deployed
                           workflow consistently uses the canonical form.
      (d) `""` / `None`   — reject with a clear error pointing to the canonical
                           form. Avoids silent empty-scope-string bugs from
                           unset env vars.

    Returns `(normalized, was_normalized)`. `was_normalized=True` only for (b)
    and (c), so the caller can emit the deprecation/auto-wrap notice once.
    """
    if expr is None or (isinstance(expr, str) and expr.strip() == ""):
        raise ValueError(
            "--scope-expression must be a non-empty string. Use "
            "'={{ $execution.id }}' for a per-execution scope, "
            "'={{ \"foo-\" + $json.fileId }}' for a per-resource scope, "
            "or pass a literal scope like 'global'."
        )
    s = str(expr)
    if s.startswith("={{") and s.rstrip().endswith("}}"):
        return (s, False)
    if s.startswith("="):
        return ("={{ " + s[1:].strip() + " }}", True)
    # Pure literal — wrap as a JS string expression so the field always uses
    # the canonical `={{ ... }}` form on the wire.
    return ('={{ ' + json.dumps(s) + ' }}', True)


def _make_execute_workflow_node(
    name: str,
    target_workflow_placeholder: str,
    position: list,
    inputs: dict,
) -> dict:
    """Build an Execute Workflow node with `inputs` mapped into workflowInputs.value."""
    return {
        "id": "{{@uuid:" + name.lower().replace(" ", "-") + "}}",
        "name": name,
        "type": "n8n-nodes-base.executeWorkflow",
        "typeVersion": 1.2,
        "position": position,
        "parameters": {
            "source": "database",
            "workflowId": {
                "__rl": True,
                "value": target_workflow_placeholder,
                "mode": "id",
            },
            "workflowInputs": {
                "mappingMode": "defineBelow",
                "value": inputs,
                "matchingColumns": [],
                "schema": [],
            },
            "options": {},
        },
    }


def _insert_lock(
    template: dict,
    scope_expr: str,
    ttl_seconds: int = _DEFAULT_TTL_SECONDS,
    max_wait_seconds: int = _DEFAULT_MAX_WAIT_SECONDS,
    fail_fast: bool = False,
) -> dict:
    """Splice in two Execute Workflow nodes, one before the main flow, one after."""
    nodes = template.setdefault("nodes", [])
    connections = template.setdefault("connections", {})

    if any(n.get("name") == _LOCK_ACQUIRE_NODE_NAME for n in nodes):
        raise SystemExit(f"Workflow already has a '{_LOCK_ACQUIRE_NODE_NAME}' node; refusing to add again")

    # Find the trigger (a node with no inbound connection — i.e. nobody references it)
    referenced: set[str] = set()
    for src_name, by_type in connections.items():
        for type_branches in by_type.values():
            for branch in type_branches:
                for c in branch:
                    referenced.add(c.get("node"))
    triggers = [n for n in nodes if n.get("name") not in referenced]
    if not triggers:
        raise SystemExit("Could not detect a trigger node (no node lacks inbound connections)")
    trigger = triggers[0]
    trigger_name = trigger["name"]

    # Existing trigger's outbound connections
    trigger_out = connections.get(trigger_name, {}).get("main", [[]])
    first_branch = trigger_out[0] if trigger_out else []

    trigger_pos = trigger.get("position") or [240, 300]
    acquire_pos = [trigger_pos[0] + 220, trigger_pos[1]]

    # Push every other node 440 right (220 for acquire + 220 for spacing) so we have room
    for n in nodes:
        if n is trigger:
            continue
        pos = n.get("position") or [0, 0]
        n["position"] = [pos[0] + 440, pos[1]]

    # Normalize the user's scope expression to the canonical n8n form.
    normalized_scope, was_normalized = _normalize_n8n_expression(scope_expr)
    if was_normalized:
        print(
            f"WARNING: --scope-expression normalized to canonical form: {normalized_scope!r}. "
            "Bare '=<expr>' or literal scopes are auto-wrapped to '={{{{ <expr> }}}}'. "
            "Update calls to pass the canonical form directly to silence this warning.",
            file=sys.stderr,
        )

    # Acquire input contract — task-13 INCR + sidecar-meta primitive takes 8
    # fields. lock_id is a per-acquire UUID minted in Build Lock Context if not
    # supplied; we let the primitive default it. workflow_id/name flow through
    # so the meta sidecar carries identity for ownership-checked release and
    # active error-handler cleanup.
    acquire_inputs = {
        "scope": normalized_scope,
        "ttl_seconds": ttl_seconds,
        "execution_id": "={{ $execution.id }}",
        "wait_till_lock_released": not fail_fast,
        "max_wait_seconds": max_wait_seconds,
        "lock_id": "={{ $execution.id }}",
        "workflow_id": "={{ $workflow.id }}",
        "workflow_name": "={{ $workflow.name }}",
    }
    acquire = _make_execute_workflow_node(
        _LOCK_ACQUIRE_NODE_NAME,
        "{{@env:workflows.lock_acquisition.id}}",
        acquire_pos,
        acquire_inputs,
    )

    # Release input contract — task-13 ownership-checked: pass scope + lock_id.
    # The release primitive GETs the meta sidecar, compares lock_id, and DELs
    # only on match. lock_id matches what acquire used ($execution.id by default).
    release_pos = [trigger_pos[0] + 880, trigger_pos[1]]
    release_inputs = {
        "scope": normalized_scope,
        "lock_id": "={{ $execution.id }}",
    }
    release = _make_execute_workflow_node(
        _LOCK_RELEASE_NODE_NAME,
        "{{@env:workflows.lock_release.id}}",
        release_pos,
        release_inputs,
    )

    nodes.append(acquire)
    nodes.append(release)

    # Wire: trigger → acquire → (whatever was first) ... → release
    connections[trigger_name] = {"main": [[{"node": _LOCK_ACQUIRE_NODE_NAME, "type": "main", "index": 0}]]}
    connections[_LOCK_ACQUIRE_NODE_NAME] = {"main": [first_branch]} if first_branch else {"main": [[]]}

    # Find terminal node(s): nodes that have no outbound connection
    outbound_names = set(connections.keys())
    terminals = [n["name"] for n in nodes if n["name"] not in outbound_names and n["name"] not in (_LOCK_ACQUIRE_NODE_NAME, _LOCK_RELEASE_NODE_NAME, trigger_name)]
    if terminals:
        for t in terminals:
            connections[t] = {"main": [[{"node": _LOCK_RELEASE_NODE_NAME, "type": "main", "index": 0}]]}
    else:
        # No real downstream content — wire acquire directly to release
        connections[_LOCK_ACQUIRE_NODE_NAME] = {"main": [[{"node": _LOCK_RELEASE_NODE_NAME, "type": "main", "index": 0}]]}

    return template


def _extract_static_scope(scope_expr: str) -> str | None:
    """Return the static literal scope from a normalized expression, or None.

    Recognized static patterns:
      - Pure literals (no leading `=`): `"global"` → `"global"`.
      - Quoted literals inside `={{ "..." }}` or `={{ '...' }}`: `={{ "foo" }}` → `"foo"`.
    Anything else (template literals with `${...}`, `+ $json...`, function calls)
    is treated as dynamic and returns None.
    """
    if not scope_expr:
        return None
    s = scope_expr.strip()
    if not s.startswith("="):
        # pure literal (post-normalize this gets wrapped, but we handle pre-wrap too)
        return s
    # ={{ <expr> }} canonical form
    inner_match = re.match(r"^=\{\{\s*(.+?)\s*\}\}$", s, re.DOTALL)
    if not inner_match:
        return None
    inner = inner_match.group(1).strip()
    # Match a single-quoted or double-quoted string with NO concatenation / interpolation
    quote_match = re.match(r'^([\'"])(.*)\1$', inner, re.DOTALL)
    if not quote_match:
        return None
    literal = quote_match.group(2)
    # Reject if backticks/template-literals appear — those imply interpolation
    if "${" in literal:
        return None
    return literal


def _auto_register_lock_scopes(workspace: Path, scope_expr: str) -> None:
    """Append the static literal scope to every <env>.yml.lockScopes (idempotent)."""
    import re as _re  # local alias to keep this fn dependency-tight
    static = _extract_static_scope(scope_expr)
    if static is None:
        print(
            "  NOTE: --scope-expression is dynamic (depends on $json or runtime data). "
            "Active error-handler cleanup requires manual lockScopes maintenance — "
            "add the resolved scope strings to <env>.yml.lockScopes by hand.",
            file=sys.stderr,
        )
        return
    config_dir = workspace / "n8n-config"
    if not config_dir.is_dir():
        return
    import yaml as _yaml
    for yml in sorted(config_dir.glob("*.yml")):
        if yml.stem in ("common", "deployment_order"):
            continue
        try:
            data = _yaml.safe_load(yml.read_text()) or {}
        except Exception:
            continue
        scopes = data.setdefault("lockScopes", [])
        if not isinstance(scopes, list):
            scopes = []
            data["lockScopes"] = scopes
        if static not in scopes:
            scopes.append(static)
            yml.write_text(_yaml.dump(data, default_flow_style=False, sort_keys=False))
            print(f"  Registered lockScopes += {static!r} in {yml.name}")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--lock-on-error", action="store_true", dest="lock_on_error")
    parser.add_argument("--scope-expression", default="={{ $execution.id }}", dest="scope_expression")
    parser.add_argument(
        "--ttl-seconds",
        type=int,
        default=_DEFAULT_TTL_SECONDS,
        dest="ttl_seconds",
        help="Lock TTL in seconds (default 86400 = 24h). Set on Redis at acquire; bounds leak on crash.",
    )
    parser.add_argument(
        "--max-wait-seconds",
        type=int,
        default=_DEFAULT_MAX_WAIT_SECONDS,
        dest="max_wait_seconds",
        help="Wait-loop deadline in seconds (default 86400 = 24h). Override per-workflow if you need fail-fast-ish semantics for time-sensitive callers (e.g. webhooks).",
    )
    parser.add_argument(
        "--fail-fast",
        action="store_true",
        dest="fail_fast",
        help="Pass wait_till_lock_released=false; Stop and Error immediately if lock is held instead of waiting.",
    )
    args = parser.parse_args()

    ws = workspace_root(args.workspace)

    # Sanity: lock primitives must already exist in the workspace
    for prim in ("lock_acquisition", "lock_release"):
        if not (ws / "n8n-workflows-template" / f"{prim}.template.json").exists():
            print(f"ERROR: primitive '{prim}' not found in workspace. Run create-lock first.", file=sys.stderr)
            sys.exit(1)

    template_path = ws / "n8n-workflows-template" / f"{args.workflow_key}.template.json"
    if not template_path.exists():
        print(f"ERROR: workflow template not found: {template_path}", file=sys.stderr)
        sys.exit(1)

    template = json.loads(template_path.read_text())
    template = _insert_lock(
        template,
        args.scope_expression,
        ttl_seconds=args.ttl_seconds,
        max_wait_seconds=args.max_wait_seconds,
        fail_fast=args.fail_fast,
    )
    template_path.write_text(json.dumps(template, indent=2))
    print(f"  Inserted lock acquire/release in {template_path}")

    # Auto-register the static portion of the scope in <env>.yml.lockScopes so
    # the active error-handler cleanup can iterate it and DEL only the failed
    # execution's lock keys. Only static literals are appended — dynamic
    # expressions (anything containing $json or other runtime refs) are skipped
    # because there's no way to enumerate them from a static config.
    _auto_register_lock_scopes(ws, args.scope_expression)

    if args.lock_on_error:
        import subprocess
        cmd = [
            sys.executable,
            str(Path(__file__).parent / "register_error_handler.py"),
            "--workspace", str(ws),
            "--workflow-key", args.workflow_key,
            "--handler-key", "error_handler_lock_cleanup",
        ]
        r = subprocess.run(cmd, capture_output=True, text=True)
        print(r.stdout)
        if r.returncode != 0:
            print(r.stderr, file=sys.stderr)
            sys.exit(r.returncode)
    print("add-lock-to-workflow complete.")


if __name__ == "__main__":
    main()


path: helpers/validate.py
content: #!/usr/bin/env python3
"""Structural validation for a workflow template or generated JSON."""
import argparse
import json
import re
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.placeholder import validator as placeholder_validator


_JS_PLACEHOLDER_RE = re.compile(r"\{\{(?:INTERPOLATE_|@)js:([^}]+)\}\}")
_PY_PLACEHOLDER_RE = re.compile(r"\{\{(?:INTERPOLATE_|@)py:([^}]+)\}\}")
_JS_TRAILER_REQUIRED = 'if (typeof module !== "undefined")'
_PRIMITIVE_MARKER = "// @n8n-evol-I:primitive"

_JS_BLOCK_COMMENT_RE = re.compile(r"/\*.*?\*/", re.DOTALL)
_JS_TOP_LEVEL_KEYWORDS = ("function", "async function", "module.exports", "exports.")
_JS_TRAILER_PREFIX = 'if (typeof module !== "undefined")'

_PY_TOP_LEVEL_KEYWORDS = ("def ", "async def ", "import ", "from ")


def _walk_js_line(line: str, start_depth: int) -> int:
    """Walk a JS line char-by-char, tracking brace depth while ignoring strings + // comments."""
    depth = start_depth
    i = 0
    n = len(line)
    in_string: str | None = None  # quote char that opened the active string
    while i < n:
        c = line[i]
        if in_string:
            if c == "\\":
                i += 2
                continue
            if c == in_string:
                in_string = None
        else:
            if c in ('"', "'", "`"):
                in_string = c
            elif c == "/" and i + 1 < n and line[i + 1] == "/":
                break  # rest of the line is a // comment
            elif c == "{":
                depth += 1
            elif c == "}":
                depth -= 1
        i += 1
    return depth


def _js_top_level_allowed(stripped: str) -> bool:
    """Is this stripped, non-blank, depth-0 JS line one of the allowed top-level shapes?"""
    if stripped.startswith("//"):
        return True
    if stripped.startswith("*") or stripped.startswith("/*"):
        # Block-comment fragments (after stripping); treat residual fragments as benign.
        return True
    if stripped.startswith(_JS_TRAILER_PREFIX):
        return True
    if stripped.startswith("function ") or stripped.startswith("function("):
        return True
    if stripped.startswith("async function "):
        return True
    if stripped.startswith("module.exports") or stripped.startswith("exports."):
        return True
    return False


def _js_top_level_violations(text: str) -> list[tuple[int, str]]:
    """Return [(line_no, excerpt), ...] for every JS line that introduces non-function code at depth 0."""
    text = _JS_BLOCK_COMMENT_RE.sub("", text)
    violations: list[tuple[int, str]] = []
    depth = 0
    for line_no, line in enumerate(text.splitlines(), start=1):
        stripped = line.strip()
        if depth == 0 and stripped and not _js_top_level_allowed(stripped):
            violations.append((line_no, stripped[:80]))
        depth = _walk_js_line(line, depth)
    return violations


def _py_top_level_allowed(stripped: str) -> bool:
    """Is this stripped, non-blank, column-0 Python line one of the allowed top-level shapes?"""
    if stripped.startswith("#"):
        return True
    for kw in _PY_TOP_LEVEL_KEYWORDS:
        if stripped.startswith(kw):
            return True
    return False


def _py_top_level_violations(text: str) -> list[tuple[int, str]]:
    """Return [(line_no, excerpt), ...] for every column-0 Python line that introduces code outside def/import.

    The first non-blank, non-comment line may be a module-level docstring (a triple-quoted string
    literal); it is consumed without violation. Bare strings anywhere else at top-level are
    violations.
    """
    lines = text.splitlines()
    skip_until = _consume_module_docstring(lines)

    violations: list[tuple[int, str]] = []
    in_triple: str | None = None  # '"""' or "'''" if currently inside a multi-line string, else None
    for idx, line in enumerate(lines):
        line_no = idx + 1
        if idx < skip_until:
            continue
        if in_triple is not None:
            if in_triple in line:
                in_triple = None
            continue
        stripped = line.strip()
        if not stripped:
            continue
        # Detect a triple-quote opener that doesn't close on the same line.
        for quote in ('"""', "'''"):
            if stripped.startswith(quote):
                rest = stripped[len(quote):]
                if quote not in rest:
                    in_triple = quote
                # Either way, this line is a string expression at top level — reject it.
                indent = len(line) - len(line.lstrip(" \t"))
                if indent == 0:
                    violations.append((line_no, stripped[:80]))
                break
        else:
            indent = len(line) - len(line.lstrip(" \t"))
            if indent == 0 and not _py_top_level_allowed(stripped):
                violations.append((line_no, stripped[:80]))
    return violations


def _consume_module_docstring(lines: list[str]) -> int:
    """Return the index of the first line AFTER any leading module docstring.

    Skips leading blank/comment lines. If the first real line opens with a triple-quoted string,
    consumes through the matching closer and returns the index past it. Otherwise returns 0.
    """
    i = 0
    while i < len(lines):
        stripped = lines[i].strip()
        if not stripped or stripped.startswith("#"):
            i += 1
            continue
        break
    if i >= len(lines):
        return 0
    first = lines[i].strip()
    for quote in ('"""', "'''"):
        if first.startswith(quote):
            rest = first[len(quote):]
            if quote in rest:
                return i + 1
            j = i + 1
            while j < len(lines):
                if quote in lines[j]:
                    return j + 1
                j += 1
            return j
    return 0


def _validate_code_node(
    node: dict,
    index: int,
    workspace: Path | None,
) -> list[str]:
    """Return discipline errors for a single n8n-nodes-base.code node (template only)."""
    errors: list[str] = []
    name = node.get("name", f"node[{index}]")
    params = node.get("parameters", {}) or {}

    lang = params.get("language", "javascript")
    if lang == "python":
        code_field_name = "pythonCode"
        ext = "py"
        placeholder_re = _PY_PLACEHOLDER_RE
        test_filename = lambda stem: f"test_{stem}.py"
    else:
        code_field_name = "jsCode"
        ext = "js"
        placeholder_re = _JS_PLACEHOLDER_RE
        test_filename = lambda stem: f"{stem}.test.js"

    code_field = params.get(code_field_name, "")
    if code_field.lstrip().startswith(_PRIMITIVE_MARKER):
        return []  # primitive — exempt from pure-function discipline
    if not code_field:
        errors.append(f"node '{name}': {code_field_name} is empty or missing")
        return errors

    m = placeholder_re.search(code_field)
    if not m:
        errors.append(
            f"node '{name}': no {{{{@{ext}:...}}}} placeholder found in {code_field_name}. "
            f"Extract the pure function to n8n-functions/{ext}/<name>.{ext} and add the placeholder."
        )
        return errors

    if workspace is None:
        return errors

    rel_path = m.group(1).strip()
    fn_file = workspace / rel_path
    if not fn_file.exists():
        errors.append(f"node '{name}': referenced file not found: {rel_path}")
        return errors

    fn_stem = fn_file.stem

    fn_text = fn_file.read_text(encoding="utf-8")

    if ext == "js":
        if _JS_TRAILER_REQUIRED not in fn_text:
            errors.append(
                f"node '{name}': {rel_path} missing required export trailer "
                f'`if (typeof module !== "undefined") module.exports = {{ <fnName> }};` — '
                "without it, n8n-functions-tests/<name>.test.js cannot import the function."
            )
        for line_no, excerpt in _js_top_level_violations(fn_text):
            errors.append(
                f"node '{name}': {rel_path} contains top-level code outside function declarations "
                f"(line {line_no}: {excerpt!r}). Pure-function files must declare functions only — "
                "n8n-glue belongs in the Code-node body, not the file."
            )
    else:
        for line_no, excerpt in _py_top_level_violations(fn_text):
            errors.append(
                f"node '{name}': {rel_path} contains top-level code outside function declarations "
                f"(line {line_no}: {excerpt!r}). Pure-function files must declare functions only — "
                "n8n-glue belongs in the Code-node body, not the file."
            )

    test_file = workspace / "n8n-functions-tests" / test_filename(fn_stem)
    if not test_file.exists():
        errors.append(
            f"node '{name}': missing test file {test_file.relative_to(workspace)}. "
            "Create it before deploying."
        )

    return errors


def validate_workflow_json(
    text: str,
    source: str = "template",
    workspace: Path | None = None,
) -> tuple[bool, list[str]]:
    """Run structural REST validation. Returns (is_valid, errors)."""
    errors: list[str] = []
    try:
        data = json.loads(text)
    except json.JSONDecodeError as e:
        return (False, [f"JSON parse error: {e}"])

    if not isinstance(data, dict):
        return (False, ["top-level value is not a JSON object"])

    if "nodes" not in data:
        errors.append("missing top-level 'nodes' key")
    elif not isinstance(data["nodes"], list):
        errors.append("'nodes' must be a list")
    else:
        for i, node in enumerate(data["nodes"]):
            if not isinstance(node, dict):
                errors.append(f"node {i} is not an object")
                continue
            for required in ("name", "type", "parameters"):
                if required not in node:
                    errors.append(f"node {i} missing '{required}'")

    if "connections" not in data:
        errors.append("missing top-level 'connections' key")
    elif not isinstance(data["connections"], dict):
        errors.append("'connections' must be an object keyed by node name")

    if source == "template":
        if "pinData" in data and data["pinData"]:
            errors.append("template contains pinData (forbidden in templates)")

        for i, node in enumerate(data.get("nodes", [])):
            if not isinstance(node, dict):
                continue
            ntype = node.get("type", "")
            name = node.get("name", f"node[{i}]")
            if ntype == "n8n-nodes-base.function":
                errors.append(
                    f"node '{name}': n8n-nodes-base.function is deprecated and forbidden; "
                    "use n8n-nodes-base.code"
                )
                continue
            if ntype == "n8n-nodes-base.code":
                errors.extend(_validate_code_node(node, i, workspace))

    if source == "built":
        residuals = placeholder_validator.check_residuals(text)
        if residuals:
            errors.append(f"residual placeholders in built JSON: {residuals}")

    return (len(errors) == 0, errors)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    parser.add_argument("--env", default=None)
    parser.add_argument("--source", choices=("built", "template"), default=None,
                        help="Default: 'built' if --env given, else 'template'")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    source = args.source or ("built" if args.env else "template")

    if source == "built":
        if not args.env:
            print("ERROR: --source built requires --env", file=sys.stderr)
            sys.exit(2)
        path = ws / "n8n-build" / args.env / f"{args.workflow_key}.generated.json"
    else:
        path = ws / "n8n-workflows-template" / f"{args.workflow_key}.template.json"

    if not path.exists():
        print(f"ERROR: file not found: {path}", file=sys.stderr)
        sys.exit(1)

    text = path.read_text(encoding="utf-8")
    valid, errors = validate_workflow_json(text, source=source, workspace=ws)
    print(json.dumps({"valid": valid, "source": source, "path": str(path), "errors": errors}, indent=2))
    sys.exit(0 if valid else 1)


if __name__ == "__main__":
    main()


path: helpers/create_workflow.py
content: #!/usr/bin/env python3
"""Scaffold a brand-new workflow: write template + register IDs in env YAML(s) + mint placeholder n8n workflow."""
import argparse
import json
import re
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

import yaml

from helpers.workspace import workspace_root, harness_root
from helpers.config import load_yaml, load_env
from helpers.n8n_client import N8nClient


def _seed_minimal_template(workflow_key: str, name: str) -> str:
    seed = (harness_root() / "primitives" / "workflows" / "_minimal.template.json").read_text()
    seed = seed.replace("__NAME__", name)
    safe_path = re.sub(r"[^a-zA-Z0-9_-]+", "-", workflow_key).strip("-").lower() or "workflow"
    seed = seed.replace("__PATH__", safe_path)
    return seed


def _list_envs(workspace: Path) -> list[str]:
    cfg = workspace / "n8n-config"
    if not cfg.is_dir():
        return []
    return [p.stem for p in sorted(cfg.glob("*.yml")) if p.stem not in ("common", "deployment_order")]


def _ensure_workflow_row(workspace: Path, env_name: str, key: str, name: str, mint: bool) -> None:
    yaml_file = workspace / "n8n-config" / f"{env_name}.yml"
    data = load_yaml(env_name, workspace)
    workflows = data.setdefault("workflows", {}) or {}
    data["workflows"] = workflows  # ensure not None
    if key not in workflows or not isinstance(workflows[key], dict):
        workflows[key] = {"id": "", "name": name}

    wf = workflows[key]
    needs_mint = mint and (not wf.get("id") or str(wf.get("id", "")).startswith("your-") or wf.get("id") == "placeholder")

    if needs_mint:
        load_env(env_name, workspace)
        import os
        [REDACTED_SECRET]"N8N_API_KEY", "")
        instance = data.get("n8n", {}).get("instanceName", "")
        client = N8nClient(base_url=instance, [REDACTED_SECRET])
        full_name = f"{data.get('displayName', '')} {name}{data.get('workflowNamePostfix', '')}".strip()
        resp = client.post("workflows", {"name": full_name, "nodes": [], "connections": {}, "settings": {}})
        wf["id"] = resp.get("id", "")
        print(f"  Minted '{full_name}' → id={wf['id']} on env '{env_name}'")
    yaml_file.write_text(yaml.dump(data, default_flow_style=False, sort_keys=False))


def _add_to_deployment_order(workspace: Path, key: str, tier: str) -> None:
    order_file = workspace / "n8n-config" / "deployment_order.yml"
    data: dict = {}
    if order_file.exists():
        data = yaml.safe_load(order_file.read_text()) or {}
    tiers = data.setdefault("tiers", {})
    members = tiers.setdefault(tier, []) or []
    if key not in members:
        members.append(key)
    tiers[tier] = members
    order_file.write_text(yaml.dump(data, default_flow_style=False, sort_keys=False))


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--key", required=True, help="Workflow key (e.g. report_v2)")
    parser.add_argument("--name", required=True, help="Display name (e.g. 'Daily Report')")
    parser.add_argument("--register-in", default=None, dest="register_in",
                        help="Comma-separated env names (default: all envs with a YAML present)")
    parser.add_argument("--with-error-handler", default=None, dest="with_error_handler",
                        help="Workflow key of an existing error handler to wire as settings.errorWorkflow")
    parser.add_argument("--tier", default="Tier 1",
                        help="Tier name in deployment_order.yml. Defaults to 'Tier 1' so deploy_all.py "
                             "picks the workflow up out of the box. Pass an explicit tier name "
                             "('Tier 0a: leaves' for primitives, 'Tier 2' for downstream callers, etc.) "
                             "or 'none' to skip deployment_order registration entirely.")
    parser.add_argument("--no-mint", action="store_true", help="Skip the n8n POST step")
    parser.add_argument("--no-template", action="store_true", help="Skip the template-write step")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)

    # Step 1: write template
    template_dir = ws / "n8n-workflows-template"
    template_dir.mkdir(parents=True, exist_ok=True)
    template_path = template_dir / f"{args.key}.template.json"
    if not args.no_template:
        if template_path.exists():
            print(f"  Template exists at {template_path}, leaving in place")
        else:
            template_path.write_text(_seed_minimal_template(args.key, args.name))
            print(f"  Wrote {template_path}")

    # Step 2: register in env YAML(s) and (optionally) mint
    if args.register_in:
        envs = [e.strip() for e in args.register_in.split(",") if e.strip()]
    else:
        envs = _list_envs(ws)
    if not envs:
        print("WARNING: no env YAMLs to register in; create one with bootstrap-env first")
    for env in envs:
        try:
            _ensure_workflow_row(ws, env, args.key, args.name, mint=not args.no_mint)
        except Exception as e:
            print(f"  ERROR registering in env '{env}': {e}", file=sys.stderr)
            sys.exit(1)

    # Step 3: deployment order. `--tier none` opts out (e.g. workflows you
    # deploy manually outside deploy_all.py); any other value registers.
    if args.tier and args.tier.lower() != "none":
        _add_to_deployment_order(ws, args.key, args.tier)
        print(f"  Added '{args.key}' to deployment_order.yml under '{args.tier}'")

    # Step 4: error-handler wiring
    if args.with_error_handler:
        import subprocess
        cmd = [
            sys.executable,
            str(Path(__file__).parent / "register_error_handler.py"),
            "--workspace", str(ws),
            "--workflow-key", args.key,
            "--handler-key", args.with_error_handler,
        ]
        subprocess.run(cmd, check=True)

    print(f"create-workflow complete for '{args.key}'.")


if __name__ == "__main__":
    main()


path: helpers/list_executions.py
content: #!/usr/bin/env python3
"""List n8n executions for an env, with cursor pagination, status filter, and tally mode.

Default mode: emit a JSON array of execution rows on stdout.
`--tally` mode: walk every page and emit a status histogram (plus `hung_count` and
`crash_count` aggregates) — used by the inspect-execution Step 1.5 baseline.
`--limit` is ignored in `--tally` mode (single source of truth).

Status enum (from n8n's documented `/executions` schema):
    error | success | running | canceled | waiting | crashed | queued

Pagination follows the documented cursor-based scheme: `nextCursor` from the previous
response is passed as `cursor` on the next request; absence of `nextCursor` means
last page.

`workflowId` is `required: false` per n8n's official OpenAPI spec
(packages/cli/src/public-api/v1/handlers/executions/spec/paths/executions.yml).
When `--workflow-key` is omitted, the helper makes a single env-scoped call without
workflowId; when provided, it scopes the call to that one workflow's id.
"""
import argparse
import json
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client, N8nClient


_STATUS_CHOICES = ("error", "success", "running", "canceled", "waiting", "crashed", "queued")
_PAGE_SIZE = 250  # max documented per-page; we cap our own --limit downstream
_RUNNING_HUNG_THRESHOLD_SECONDS = 60 * 60  # "running with startedAt > 1h ago" is the hung signal


def _parse_iso(ts: Optional[str]) -> Optional[datetime]:
    if not ts:
        return None
    try:
        return datetime.fromisoformat(ts.replace("Z", "+00:00"))
    except Exception:
        return None


def _resolve_workflow_id(yaml_data: dict, workflow_key: str) -> str:
    return str(get_config_value(yaml_data, f"workflows.{workflow_key}.id"))


def _fetch_executions(
    client: N8nClient,
    *,
    workflow_id: Optional[str] = None,
    status: Optional[str] = None,
    started_after: Optional[datetime] = None,
    started_before: Optional[datetime] = None,
    limit: Optional[int] = None,
) -> list[dict]:
    """Walk pages of /executions, applying client-side time-window filtering.

    `workflow_id=None` issues an env-scoped call (no workflowId param). `limit` caps
    the post-filter row count returned (None = no cap). Pagination is cursor-based:
    pass `cursor` from `nextCursor` of the prior page until absent.
    """
    out: list[dict] = []
    cursor: Optional[str] = None
    while True:
        params: dict = {"limit": _PAGE_SIZE}
        if workflow_id:
            params["workflowId"] = workflow_id
        if status:
            params["status"] = status
        if cursor:
            params["cursor"] = cursor

        resp = client.get("executions", params=params)
        rows = resp.get("data") or []
        for row in rows:
            started = _parse_iso(row.get("startedAt"))
            if started_after and started and started < started_after:
                continue
            if started_before and started and started > started_before:
                continue
            out.append(row)
            if limit is not None and len(out) >= limit:
                return out

        cursor = resp.get("nextCursor")
        if not cursor:
            break
    return out


def _is_hung_running(row: dict, now: datetime) -> bool:
    """A row counts as "hung" running if its startedAt is older than the threshold."""
    started = _parse_iso(row.get("startedAt"))
    if not started:
        return False
    age = (now - started).total_seconds()
    return age > _RUNNING_HUNG_THRESHOLD_SECONDS


def _tally_executions(rows: list[dict]) -> dict:
    """Status histogram + derived `hung_count` and `crash_count`."""
    hist: dict = {s: 0 for s in _STATUS_CHOICES}
    now = datetime.now(timezone.utc)
    hung_running = 0
    for row in rows:
        st = row.get("status")
        if st in hist:
            hist[st] += 1
        if st == "running" and _is_hung_running(row, now):
            hung_running += 1
    hung_count = hist["waiting"] + hist["queued"] + hung_running
    crash_count = hist["crashed"]
    total = sum(hist.values())
    return {
        "total": total,
        "by_status": hist,
        "hung_count": hung_count,
        "crash_count": crash_count,
        "running_hung_count": hung_running,
    }


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", default=None, dest="workflow_key",
                        help="Yaml key; if omitted, the call is env-scoped (no workflowId filter).")
    parser.add_argument("--status", choices=_STATUS_CHOICES, default=None,
                        help="Filter by execution status server-side.")
    parser.add_argument("--started-after", default=None, dest="started_after",
                        help="ISO 8601 UTC; filter to executions whose startedAt >= this (client-side).")
    parser.add_argument("--started-before", default=None, dest="started_before",
                        help="ISO 8601 UTC; filter to executions whose startedAt <= this (client-side).")
    parser.add_argument("--limit", type=int, default=100,
                        help="Cap on rows returned (post-filter). Default 100. Ignored in --tally.")
    parser.add_argument("--tally", action="store_true",
                        help="Walk every page; emit a status histogram only.")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    yaml_data = load_yaml(args.env, ws)
    load_env(args.env, ws)
    client = ensure_client(args.env, ws)

    started_after = _parse_iso(args.started_after) if args.started_after else None
    started_before = _parse_iso(args.started_before) if args.started_before else None

    workflow_id = _resolve_workflow_id(yaml_data, args.workflow_key) if args.workflow_key else None

    # `--limit` is a post-filter cap; in tally mode we ignore it (walk everything).
    per_call_cap = None if args.tally else args.limit

    rows = _fetch_executions(
        client,
        workflow_id=workflow_id,
        status=args.status,
        started_after=started_after,
        started_before=started_before,
        limit=per_call_cap,
    )

    if args.tally:
        result = _tally_executions(rows)
        print(json.dumps(result, indent=2))
        return

    print(json.dumps(rows, indent=2))


if __name__ == "__main__":
    main()


path: helpers/package-lock.json
content: {
  "name": "n8n-harness-helpers",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {
    "": {
      "name": "n8n-harness-helpers",
      "dependencies": {
        "@n8n/workflow-sdk": "^0.10.2"
      }
    },
    "node_modules/@dagrejs/dagre": {
      "version": "1.1.8",
      "resolved": "https://registry.npmjs.org/@dagrejs/dagre/-/dagre-1.1.8.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "@dagrejs/graphlib": "2.2.4"
      }
    },
    "node_modules/@dagrejs/graphlib": {
      "version": "2.2.4",
      "resolved": "https://registry.npmjs.org/@dagrejs/graphlib/-/graphlib-2.2.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">17.0.0"
      }
    },
    "node_modules/@n8n_io/riot-tmpl": {
      "version": "4.0.1",
      "resolved": "https://registry.npmjs.org/@n8n_io/riot-tmpl/-/riot-tmpl-4.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "eslint-config-riot": "^1.0.0"
      }
    },
    "node_modules/@n8n/errors": {
      "version": "0.7.0",
      "resolved": "https://registry.npmjs.org/@n8n/errors/-/errors-0.7.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "SEE LICENSE IN LICENSE.md",
      "dependencies": {
        "callsites": "3.1.0"
      }
    },
    "node_modules/@n8n/expression-runtime": {
      "version": "0.9.1",
      "resolved": "https://registry.npmjs.org/@n8n/expression-runtime/-/expression-runtime-0.9.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "SEE LICENSE IN LICENSE.md",
      "dependencies": {
        "@n8n/errors": "0.7.0",
        "@n8n/tournament": "1.0.6",
        "isolated-vm": "^6.0.2",
        "js-base64": "3.7.8",
        "jssha": "3.3.1",
        "lodash": "4.18.1",
        "luxon": "3.7.2",
        "md5": "2.3.0",
        "title-case": "3.0.3",
        "transliteration": "2.3.5"
      }
    },
    "node_modules/@n8n/tournament": {
      "version": "1.0.6",
      "resolved": "https://registry.npmjs.org/@n8n/tournament/-/tournament-1.0.6.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "Apache-2.0",
      "dependencies": {
        "@n8n_io/riot-tmpl": "^4.0.1",
        "ast-types": "^0.16.1",
        "esprima-next": "^5.8.4",
        "recast": "^0.22.0"
      },
      "engines": {
        "node": ">=20.15",
        "pnpm": ">=9.5"
      }
    },
    "node_modules/@n8n/workflow-sdk": {
      "version": "0.10.2",
      "resolved": "https://registry.npmjs.org/@n8n/workflow-sdk/-/workflow-sdk-0.10.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "SEE LICENSE IN LICENSE.md",
      "dependencies": {
        "@dagrejs/dagre": "^1.1.4",
        "acorn": "8.14.0",
        "n8n-workflow": "2.17.2",
        "uuid": "10.0.0",
        "zod": "3.25.67"
      }
    },
    "node_modules/acorn": {
      "version": "8.14.0",
      "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.14.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "bin": {
        "acorn": "bin/acorn"
      },
      "engines": {
        "node": ">=0.4.0"
      }
    },
    "node_modules/ansi-regex": {
      "version": "5.0.1",
      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=8"
      }
    },
    "node_modules/ansi-styles": {
      "version": "4.3.0",
      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "color-convert": "^2.0.1"
      },
      "engines": {
        "node": ">=8"
      },
      "funding": {
        "url": "https://github.com/chalk/ansi-styles?sponsor=1"
      }
    },
    "node_modules/assert": {
      "version": "2.1.0",
      "resolved": "https://registry.npmjs.org/assert/-/assert-2.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind": "^1.0.2",
        "is-nan": "^1.3.2",
        "object-is": "^1.1.5",
        "object.assign": "^4.1.4",
        "util": "^0.12.5"
      }
    },
    "node_modules/ast-types": {
      "version": "0.16.1",
      "resolved": "https://registry.npmjs.org/ast-types/-/ast-types-0.16.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "tslib": "^2.0.1"
      },
      "engines": {
        "node": ">=4"
      }
    },
    "node_modules/asynckit": {
      "version": "0.4.0",
      "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT"
    },
    "node_modules/available-typed-arrays": {
      "version": "1.0.7",
      "resolved": "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "possible-typed-array-names": "^1.0.0"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/call-bind": {
      "version": "1.0.9",
      "resolved": "https://registry.npmjs.org/call-bind/-/call-bind-1.0.9.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind-apply-helpers": "^1.0.2",
        "es-define-property": "^1.0.1",
        "get-intrinsic": "^1.3.0",
        "set-function-length": "^1.2.2"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/call-bind-apply-helpers": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "es-errors": "^1.3.0",
        "function-bind": "^1.1.2"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/call-bound": {
      "version": "1.0.4",
      "resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind-apply-helpers": "^1.0.2",
        "get-intrinsic": "^1.3.0"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/callsites": {
      "version": "3.1.0",
      "resolved": "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=6"
      }
    },
    "node_modules/charenc": {
      "version": "0.0.2",
      "resolved": "https://registry.npmjs.org/charenc/-/charenc-0.0.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-3-Clause",
      "engines": {
        "node": "*"
      }
    },
    "node_modules/cliui": {
      "version": "8.0.1",
      "resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "ISC",
      "dependencies": {
        "string-width": "^4.2.0",
        "strip-ansi": "^6.0.1",
        "wrap-ansi": "^7.0.0"
      },
      "engines": {
        "node": ">=12"
      }
    },
    "node_modules/color-convert": {
      "version": "2.0.1",
      "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "color-name": "~1.1.4"
      },
      "engines": {
        "node": ">=7.0.0"
      }
    },
    "node_modules/color-name": {
      "version": "1.1.4",
      "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT"
    },
    "node_modules/combined-stream": {
      "version": "1.0.8",
      "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "delayed-stream": "~1.0.0"
      },
      "engines": {
        "node": ">= 0.8"
      }
    },
    "node_modules/crypt": {
      "version": "0.0.2",
      "resolved": "https://registry.npmjs.org/crypt/-/crypt-0.0.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-3-Clause",
      "engines": {
        "node": "*"
      }
    },
    "node_modules/define-data-property": {
      "version": "1.1.4",
      "resolved": "https://registry.npmjs.org/define-data-property/-/define-data-property-1.1.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "es-define-property": "^1.0.0",
        "es-errors": "^1.3.0",
        "gopd": "^1.0.1"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/define-properties": {
      "version": "1.2.1",
      "resolved": "https://registry.npmjs.org/define-properties/-/define-properties-1.2.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "define-data-property": "^1.0.1",
        "has-property-descriptors": "^1.0.0",
        "object-keys": "^1.1.1"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/delayed-stream": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=0.4.0"
      }
    },
    "node_modules/dunder-proto": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind-apply-helpers": "^1.0.1",
        "es-errors": "^1.3.0",
        "gopd": "^1.2.0"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/emoji-regex": {
      "version": "8.0.0",
      "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT"
    },
    "node_modules/es-define-property": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/es-errors": {
      "version": "1.3.0",
      "resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/es-object-atoms": {
      "version": "1.1.1",
      "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "es-errors": "^1.3.0"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/es-set-tostringtag": {
      "version": "2.1.0",
      "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "es-errors": "^1.3.0",
        "get-intrinsic": "^1.2.6",
        "has-tostringtag": "^1.0.2",
        "hasown": "^2.0.2"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/escalade": {
      "version": "3.2.0",
      "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=6"
      }
    },
    "node_modules/eslint-config-riot": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/eslint-config-riot/-/eslint-config-riot-1.0.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT"
    },
    "node_modules/esprima": {
      "version": "4.0.1",
      "resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-2-Clause",
      "bin": {
        "esparse": "bin/esparse.js",
        "esvalidate": "bin/esvalidate.js"
      },
      "engines": {
        "node": ">=4"
      }
    },
    "node_modules/esprima-next": {
      "version": "5.8.4",
      "resolved": "https://registry.npmjs.org/esprima-next/-/esprima-next-5.8.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-2-Clause",
      "bin": {
        "esparse": "bin/esparse.js",
        "esvalidate": "bin/esvalidate.js"
      },
      "engines": {
        "node": ">=12"
      }
    },
    "node_modules/for-each": {
      "version": "0.3.5",
      "resolved": "https://registry.npmjs.org/for-each/-/for-each-0.3.5.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "is-callable": "^1.2.7"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/form-data": {
      "version": "4.0.4",
      "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "asynckit": "^0.4.0",
        "combined-stream": "^1.0.8",
        "es-set-tostringtag": "^2.1.0",
        "hasown": "^2.0.2",
        "mime-types": "^2.1.12"
      },
      "engines": {
        "node": ">= 6"
      }
    },
    "node_modules/function-bind": {
      "version": "1.1.2",
      "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/generator-function": {
      "version": "2.0.1",
      "resolved": "https://registry.npmjs.org/generator-function/-/generator-function-2.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/get-caller-file": {
      "version": "2.0.5",
      "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "ISC",
      "engines": {
        "node": "6.* || 8.* || >= 10.*"
      }
    },
    "node_modules/get-intrinsic": {
      "version": "1.3.0",
      "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind-apply-helpers": "^1.0.2",
        "es-define-property": "^1.0.1",
        "es-errors": "^1.3.0",
        "es-object-atoms": "^1.1.1",
        "function-bind": "^1.1.2",
        "get-proto": "^1.0.1",
        "gopd": "^1.2.0",
        "has-symbols": "^1.1.0",
        "hasown": "^2.0.2",
        "math-intrinsics": "^1.1.0"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/get-proto": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "dunder-proto": "^1.0.1",
        "es-object-atoms": "^1.0.0"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/gopd": {
      "version": "1.2.0",
      "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/has-property-descriptors": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/has-property-descriptors/-/has-property-descriptors-1.0.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "es-define-property": "^1.0.0"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/has-symbols": {
      "version": "1.1.0",
      "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/has-tostringtag": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "has-symbols": "^1.0.3"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/hasown": {
      "version": "2.0.3",
      "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.3.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "function-bind": "^1.1.2"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/inherits": {
      "version": "2.0.4",
      "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "ISC"
    },
    "node_modules/is-arguments": {
      "version": "1.2.0",
      "resolved": "https://registry.npmjs.org/is-arguments/-/is-arguments-1.2.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bound": "^1.0.2",
        "has-tostringtag": "^1.0.2"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/is-buffer": {
      "version": "1.1.6",
      "resolved": "https://registry.npmjs.org/is-buffer/-/is-buffer-1.1.6.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT"
    },
    "node_modules/is-callable": {
      "version": "1.2.7",
      "resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.2.7.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/is-fullwidth-code-point": {
      "version": "3.0.0",
      "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=8"
      }
    },
    "node_modules/is-generator-function": {
      "version": "1.1.2",
      "resolved": "https://registry.npmjs.org/is-generator-function/-/is-generator-function-1.1.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bound": "^1.0.4",
        "generator-function": "^2.0.0",
        "get-proto": "^1.0.1",
        "has-tostringtag": "^1.0.2",
        "safe-regex-test": "^1.1.0"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/is-nan": {
      "version": "1.3.2",
      "resolved": "https://registry.npmjs.org/is-nan/-/is-nan-1.3.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind": "^1.0.0",
        "define-properties": "^1.1.3"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/is-regex": {
      "version": "1.2.1",
      "resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.2.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bound": "^1.0.2",
        "gopd": "^1.2.0",
        "has-tostringtag": "^1.0.2",
        "hasown": "^2.0.2"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/is-typed-array": {
      "version": "1.1.15",
      "resolved": "https://registry.npmjs.org/is-typed-array/-/is-typed-array-1.1.15.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "which-typed-array": "^1.1.16"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/isolated-vm": {
      "version": "6.1.2",
      "resolved": "https://registry.npmjs.org/isolated-vm/-/isolated-vm-6.1.2.tgz",
      "integrity": [REDACTED_SECRET],
      "hasInstallScript": true,
      "license": "ISC",
      "dependencies": {
        "node-gyp-build": "^4.8.4"
      },
      "engines": {
        "node": ">=22.0.0"
      }
    },
    "node_modules/jmespath": {
      "version": "0.16.0",
      "resolved": "https://registry.npmjs.org/jmespath/-/jmespath-0.16.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "Apache-2.0",
      "engines": {
        "node": ">= 0.6.0"
      }
    },
    "node_modules/js-base64": {
      "version": "3.7.8",
      "resolved": "https://registry.npmjs.org/js-base64/-/js-base64-3.7.8.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-3-Clause"
    },
    "node_modules/jsonrepair": {
      "version": "3.13.2",
      "resolved": "https://registry.npmjs.org/jsonrepair/-/jsonrepair-3.13.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "ISC",
      "bin": {
        "jsonrepair": "bin/cli.js"
      }
    },
    "node_modules/jssha": {
      "version": "3.3.1",
      "resolved": "https://registry.npmjs.org/jssha/-/jssha-3.3.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-3-Clause",
      "engines": {
        "node": "*"
      }
    },
    "node_modules/lodash": {
      "version": "4.18.1",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.18.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT"
    },
    "node_modules/luxon": {
      "version": "3.7.2",
      "resolved": "https://registry.npmjs.org/luxon/-/luxon-3.7.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=12"
      }
    },
    "node_modules/math-intrinsics": {
      "version": "1.1.0",
      "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/md5": {
      "version": "2.3.0",
      "resolved": "https://registry.npmjs.org/md5/-/md5-2.3.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-3-Clause",
      "dependencies": {
        "charenc": "0.0.2",
        "crypt": "0.0.2",
        "is-buffer": "~1.1.6"
      }
    },
    "node_modules/mime-db": {
      "version": "1.52.0",
      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.6"
      }
    },
    "node_modules/mime-types": {
      "version": "2.1.35",
      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "mime-db": "1.52.0"
      },
      "engines": {
        "node": ">= 0.6"
      }
    },
    "node_modules/n8n-workflow": {
      "version": "2.17.2",
      "resolved": "https://registry.npmjs.org/n8n-workflow/-/n8n-workflow-2.17.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "SEE LICENSE IN LICENSE.md",
      "dependencies": {
        "@n8n/errors": "0.7.0",
        "@n8n/expression-runtime": "0.9.1",
        "@n8n/tournament": "1.0.6",
        "ast-types": "0.16.1",
        "callsites": "3.1.0",
        "esprima-next": "5.8.4",
        "form-data": "4.0.4",
        "jmespath": "0.16.0",
        "js-base64": "3.7.8",
        "jsonrepair": "3.13.2",
        "jssha": "3.3.1",
        "lodash": "4.18.1",
        "luxon": "3.7.2",
        "md5": "2.3.0",
        "recast": "0.22.0",
        "title-case": "3.0.3",
        "transliteration": "2.3.5",
        "uuid": "10.0.0",
        "xml2js": "0.6.2",
        "zod": "3.25.67"
      }
    },
    "node_modules/node-gyp-build": {
      "version": "4.8.4",
      "resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.4.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "bin": {
        "node-gyp-build": "bin.js",
        "node-gyp-build-optional": "optional.js",
        "node-gyp-build-test": "build-test.js"
      }
    },
    "node_modules/object-is": {
      "version": "1.1.6",
      "resolved": "https://registry.npmjs.org/object-is/-/object-is-1.1.6.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind": "^1.0.7",
        "define-properties": "^1.2.1"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/object-keys": {
      "version": "1.1.1",
      "resolved": "https://registry.npmjs.org/object-keys/-/object-keys-1.1.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/object.assign": {
      "version": "4.1.7",
      "resolved": "https://registry.npmjs.org/object.assign/-/object.assign-4.1.7.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bind": "^1.0.8",
        "call-bound": "^1.0.3",
        "define-properties": "^1.2.1",
        "es-object-atoms": "^1.0.0",
        "has-symbols": "^1.1.0",
        "object-keys": "^1.1.1"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/possible-typed-array-names": {
      "version": "1.1.0",
      "resolved": "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/recast": {
      "version": "0.22.0",
      "resolved": "https://registry.npmjs.org/recast/-/recast-0.22.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "assert": "^2.0.0",
        "ast-types": "0.15.2",
        "esprima": "~4.0.0",
        "source-map": "~0.6.1",
        "tslib": "^2.0.1"
      },
      "engines": {
        "node": ">= 4"
      }
    },
    "node_modules/recast/node_modules/ast-types": {
      "version": "0.15.2",
      "resolved": "https://registry.npmjs.org/ast-types/-/ast-types-0.15.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "tslib": "^2.0.1"
      },
      "engines": {
        "node": ">=4"
      }
    },
    "node_modules/require-directory": {
      "version": "2.1.1",
      "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=0.10.0"
      }
    },
    "node_modules/safe-regex-test": {
      "version": "1.1.0",
      "resolved": "https://registry.npmjs.org/safe-regex-test/-/safe-regex-test-1.1.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "call-bound": "^1.0.2",
        "es-errors": "^1.3.0",
        "is-regex": "^1.2.1"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/sax": {
      "version": "1.6.0",
      "resolved": "https://registry.npmjs.org/sax/-/sax-1.6.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BlueOak-1.0.0",
      "engines": {
        "node": ">=11.0.0"
      }
    },
    "node_modules/set-function-length": {
      "version": "1.2.2",
      "resolved": "https://registry.npmjs.org/set-function-length/-/set-function-length-1.2.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "define-data-property": "^1.1.4",
        "es-errors": "^1.3.0",
        "function-bind": "^1.1.2",
        "get-intrinsic": "^1.2.4",
        "gopd": "^1.0.1",
        "has-property-descriptors": "^1.0.2"
      },
      "engines": {
        "node": ">= 0.4"
      }
    },
    "node_modules/source-map": {
      "version": "0.6.1",
      "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "BSD-3-Clause",
      "engines": {
        "node": ">=0.10.0"
      }
    },
    "node_modules/string-width": {
      "version": "4.2.3",
      "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "emoji-regex": "^8.0.0",
        "is-fullwidth-code-point": "^3.0.0",
        "strip-ansi": "^6.0.1"
      },
      "engines": {
        "node": ">=8"
      }
    },
    "node_modules/strip-ansi": {
      "version": "6.0.1",
      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "ansi-regex": "^5.0.1"
      },
      "engines": {
        "node": ">=8"
      }
    },
    "node_modules/title-case": {
      "version": "3.0.3",
      "resolved": "https://registry.npmjs.org/title-case/-/title-case-3.0.3.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "tslib": "^2.0.3"
      }
    },
    "node_modules/transliteration": {
      "version": "2.3.5",
      "resolved": "https://registry.npmjs.org/transliteration/-/transliteration-2.3.5.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "yargs": "^17.5.1"
      },
      "bin": {
        "slugify": "dist/bin/slugify",
        "transliterate": "dist/bin/transliterate"
      },
      "engines": {
        "node": ">=6.0.0"
      }
    },
    "node_modules/tslib": {
      "version": "2.8.1",
      "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "0BSD"
    },
    "node_modules/util": {
      "version": "0.12.5",
      "resolved": "https://registry.npmjs.org/util/-/util-0.12.5.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "inherits": "^2.0.3",
        "is-arguments": "^1.0.4",
        "is-generator-function": "^1.0.7",
        "is-typed-array": "^1.1.3",
        "which-typed-array": "^1.1.2"
      }
    },
    "node_modules/uuid": {
      "version": "10.0.0",
      "resolved": "https://registry.npmjs.org/uuid/-/uuid-10.0.0.tgz",
      "integrity": [REDACTED_SECRET],
      "funding": [
        "https://github.com/sponsors/broofa",
        "https://github.com/sponsors/ctavan"
      ],
      "license": "MIT",
      "bin": {
        "uuid": "dist/bin/uuid"
      }
    },
    "node_modules/which-typed-array": {
      "version": "1.1.20",
      "resolved": "https://registry.npmjs.org/which-typed-array/-/which-typed-array-1.1.20.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "available-typed-arrays": "^1.0.7",
        "call-bind": "^1.0.8",
        "call-bound": "^1.0.4",
        "for-each": "^0.3.5",
        "get-proto": "^1.0.1",
        "gopd": "^1.2.0",
        "has-tostringtag": "^1.0.2"
      },
      "engines": {
        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
      }
    },
    "node_modules/wrap-ansi": {
      "version": "7.0.0",
      "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "ansi-styles": "^4.0.0",
        "string-width": "^4.1.0",
        "strip-ansi": "^6.0.0"
      },
      "engines": {
        "node": ">=10"
      },
      "funding": {
        "url": "https://github.com/chalk/wrap-ansi?sponsor=1"
      }
    },
    "node_modules/xml2js": {
      "version": "0.6.2",
      "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.6.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "sax": ">=0.6.0",
        "xmlbuilder": "~11.0.0"
      },
      "engines": {
        "node": ">=4.0.0"
      }
    },
    "node_modules/xmlbuilder": {
      "version": "11.0.1",
      "resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-11.0.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "engines": {
        "node": ">=4.0"
      }
    },
    "node_modules/y18n": {
      "version": "5.0.8",
      "resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "ISC",
      "engines": {
        "node": ">=10"
      }
    },
    "node_modules/yargs": {
      "version": "17.7.2",
      "resolved": "https://registry.npmjs.org/yargs/-/yargs-17.7.2.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "dependencies": {
        "cliui": "^8.0.1",
        "escalade": "^3.1.1",
        "get-caller-file": "^2.0.5",
        "require-directory": "^2.1.1",
        "string-width": "^4.2.3",
        "y18n": "^5.0.5",
        "yargs-parser": "^21.1.1"
      },
      "engines": {
        "node": ">=12"
      }
    },
    "node_modules/yargs-parser": {
      "version": "21.1.1",
      "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "ISC",
      "engines": {
        "node": ">=12"
      }
    },
    "node_modules/zod": {
      "version": "3.25.67",
      "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.67.tgz",
      "integrity": [REDACTED_SECRET],
      "license": "MIT",
      "funding": {
        "url": "https://github.com/sponsors/colinhacks"
      }
    }
  }
}


path: helpers/package.json
content: {"private":true,"name":"n8n-evol-I-helpers","dependencies":{"@n8n/workflow-sdk":"0.10.2"}}

path: helpers/test_functions.py
content: #!/usr/bin/env python3
"""Run unit tests over JS used in n8n Code nodes and/or Python used in cloud functions."""
import argparse
import shutil
import subprocess
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_common


def _layout(workspace: Path) -> dict:
    common = load_common(workspace)
    return common.get("workspace_layout", {}) or {}


def _resolve(workspace: Path, key: str, default: str) -> Path:
    layout = _layout(workspace)
    rel = layout.get(key, default).rstrip("/")
    return workspace / rel


def _run_node_tests(tests_dir: Path, name_filter: str | None) -> tuple[int, str]:
    if not tests_dir.is_dir():
        return (0, "n8n: no tests dir")
    tests = sorted(tests_dir.glob("*.test.js"))
    if name_filter:
        tests = [t for t in tests if name_filter in t.stem]
    if not tests:
        return (0, "n8n: no tests")

    if shutil.which("node") is None:
        return (1, "n8n: node binary not found on PATH")

    pkg_json = tests_dir / "package.json"
    if pkg_json.exists():
        # Defer to project's npm test runner
        cmd = ["npm", "test", "--silent"]
        cwd = tests_dir
    else:
        cmd = ["node", "--test", *[str(t) for t in tests]]
        cwd = tests_dir

    r = subprocess.run(cmd, cwd=cwd)
    return (r.returncode, f"n8n: ran {len(tests)} test file(s) (exit={r.returncode})")


def _run_pytest_tests(tests_dir: Path, name_filter: str | None) -> tuple[int, str]:
    if not tests_dir.is_dir():
        return (0, "cloud: no tests dir")
    tests = sorted(tests_dir.glob("test_*.py"))
    if name_filter:
        tests = [t for t in tests if name_filter in t.stem]
    if not tests:
        return (0, "cloud: no tests")

    cmd = [sys.executable, "-m", "pytest", "-v", *[str(t) for t in tests]]
    r = subprocess.run(cmd, cwd=tests_dir)
    return (r.returncode, f"cloud: ran {len(tests)} test file(s) (exit={r.returncode})")


def _run_pytest_n8n_tests(tests_dir: Path, name_filter: str | None) -> tuple[int, str]:
    """Run pytest over test_*.py files in n8n-functions-tests/ (Python pure-function tests)."""
    if not tests_dir.is_dir():
        return (0, "n8n-py: no tests dir")
    tests = sorted(tests_dir.glob("test_*.py"))
    if name_filter:
        tests = [t for t in tests if name_filter in t.stem]
    if not tests:
        return (0, "n8n-py: no tests")

    cmd = [sys.executable, "-m", "pytest", "-v", *[str(t) for t in tests]]
    r = subprocess.run(cmd, cwd=tests_dir)
    return (r.returncode, f"n8n-py: ran {len(tests)} test file(s) (exit={r.returncode})")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--target", choices=("n8n", "cloud", "all"), default="all")
    parser.add_argument("--filter", default=None, dest="name_filter")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    n8n_tests = _resolve(ws, "n8n_functions_tests_dir", "n8n-functions-tests")
    cloud_tests = _resolve(ws, "cloud_functions_tests_dir", "cloud-functions-tests")

    summaries: list[tuple[int, str]] = []
    if args.target in ("n8n", "all"):
        summaries.append(_run_node_tests(n8n_tests, args.name_filter))
        summaries.append(_run_pytest_n8n_tests(n8n_tests, args.name_filter))
    if args.target in ("cloud", "all"):
        summaries.append(_run_pytest_tests(cloud_tests, args.name_filter))

    print("\nTest summary:")
    for code, line in summaries:
        print(f"  [{'OK' if code == 0 else 'FAIL'}] {line}")

    sys.exit(0 if all(c == 0 for c, _ in summaries) else 1)


if __name__ == "__main__":
    main()


path: helpers/inspect_execution.py
content: #!/usr/bin/env python3
"""Fetch a single n8n execution by id and emit its JSON to stdout.

`--include-data` defaults FALSE (matches the lighter-weight metadata default of
GET /executions/{id}). With it on, the helper truncates the `data` payload at
`--max-size-kb` and prints a TRUNCATED warning to stderr; pass `--no-truncate`
to get the full payload (warns on stderr if > 100KB so the agent can route to
targeted node inspection instead of consuming the full blob).

Status-enum semantics, retryOf chain, and the truncation-escalation rule are
documented in skills/patterns/investigation-discipline.md (Step 3 / Step 3b).
"""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_env
from helpers.n8n_client import ensure_client


_DEFAULT_MAX_SIZE_KB = 50
_NO_TRUNCATE_WARN_THRESHOLD_KB = 100


def _truncate_data_field(execution: dict, max_size_kb: int) -> tuple[dict, bool]:
    """If execution['data'] serializes to more than max_size_kb, replace with a marker.

    Returns (possibly-truncated dict, was_truncated_bool).
    """
    if "data" not in execution or execution["data"] is None:
        return execution, False
    serialized = json.dumps(execution["data"])
    size_bytes = len(serialized.encode("utf-8"))
    if size_bytes <= max_size_kb * 1024:
        return execution, False

    truncated = dict(execution)
    truncated["data"] = {
        "__truncated__": True,
        "original_size_bytes": size_bytes,
        "max_size_kb": max_size_kb,
        "preview": serialized[: max_size_kb * 1024],
    }
    return truncated, True


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--execution-id", required=True, dest="execution_id")
    parser.add_argument("--include-data", action="store_true", dest="include_data",
                        help="Include the per-node data field. Default off.")
    parser.add_argument("--max-size-kb", type=int, default=_DEFAULT_MAX_SIZE_KB, dest="max_size_kb",
                        help=f"Truncate `data` if it exceeds this many KB. Default {_DEFAULT_MAX_SIZE_KB}.")
    parser.add_argument("--no-truncate", action="store_true", dest="no_truncate",
                        help="Return the full payload regardless of size; warns to stderr if >100KB.")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    client = ensure_client(args.env, ws)

    params: dict = {}
    if args.include_data:
        params["includeData"] = "true"

    execution = client.get(f"executions/{args.execution_id}", params=params)

    if args.include_data and not args.no_truncate:
        execution, was_truncated = _truncate_data_field(execution, args.max_size_kb)
        if was_truncated:
            print(
                f"TRUNCATED — execution {args.execution_id} `data` exceeded "
                f"{args.max_size_kb}KB; pass --no-truncate for full payload",
                file=sys.stderr,
            )

    if args.include_data and args.no_truncate and execution.get("data") is not None:
        size_bytes = len(json.dumps(execution["data"]).encode("utf-8"))
        if size_bytes > _NO_TRUNCATE_WARN_THRESHOLD_KB * 1024:
            print(
                f"WARN — full payload is {size_bytes // 1024}KB; consider node-targeted inspection",
                file=sys.stderr,
            )

    print(json.dumps(execution, indent=2))


if __name__ == "__main__":
    main()


path: helpers/create_lock.py
content: #!/usr/bin/env python3
"""Copy lock primitive templates from harness/primitives into the workspace and register them."""
import argparse
import shutil
import subprocess
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root, harness_root


_PRIMITIVES = {
    "lock_acquisition": "Lock Acquisition",
    "lock_release": "Lock Release",
}
_ERROR_HANDLER = ("error_handler_lock_cleanup", "Error Handler Lock Cleanup")
_RATE_LIMIT = ("rate_limit_check", "Rate Limit Check")


def _copy_primitive(workspace: Path, key: str, force_overwrite: bool = False) -> Path:
    src = harness_root() / "primitives" / "workflows" / f"{key}.template.json"
    if not src.exists():
        raise FileNotFoundError(f"Primitive missing: {src}")
    dst_dir = workspace / "n8n-workflows-template"
    dst_dir.mkdir(parents=True, exist_ok=True)
    dst = dst_dir / f"{key}.template.json"
    if dst.exists() and not force_overwrite:
        print(
            f"  WARNING: {key}.template.json already exists — re-run with "
            f"--force-overwrite to update to the real Redis implementation."
        )
    else:
        existed = dst.exists()
        shutil.copyfile(src, dst)
        action = "Overwrote" if existed else "Copied"
        print(f"  {action} {src.name} → {dst}")
    return dst


def _register_via_create_workflow(workspace: Path, key: str, name: str, tier: str) -> None:
    """Register the workflow in env YAMLs (and mint placeholder IDs) without re-writing the template."""
    cmd = [
        sys.executable,
        str(Path(__file__).parent / "create_workflow.py"),
        "--workspace", str(workspace),
        "--key", key,
        "--name", name,
        "--no-template",
        "--tier", tier,
    ]
    r = subprocess.run(cmd, capture_output=True, text=True)
    if r.returncode != 0:
        print(r.stdout, file=sys.stderr)
        print(r.stderr, file=sys.stderr)
        raise SystemExit(r.returncode)
    print(r.stdout)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--include-error-handler", action="store_true", dest="include_error_handler")
    parser.add_argument("--include-rate-limit", action="store_true", dest="include_rate_limit")
    parser.add_argument("--force-overwrite", action="store_true", dest="force_overwrite")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)

    primitives = dict(_PRIMITIVES)
    if args.include_error_handler:
        key, name = _ERROR_HANDLER
        primitives[key] = name
    if args.include_rate_limit:
        key, name = _RATE_LIMIT
        primitives[key] = name

    # Two-pass: copy all primitives first, then attempt registrations. If a
    # registration fails (e.g. transient n8n API hiccup or expired API key),
    # the workspace is at least left with all template files on disk so a
    # later re-run can resume by registering only the un-registered keys.
    for key, _ in primitives.items():
        _copy_primitive(ws, key, force_overwrite=args.force_overwrite)

    failures: list[tuple[str, Exception]] = []
    for key, name in primitives.items():
        try:
            _register_via_create_workflow(ws, key, name, "Tier 0a: leaves")
        except SystemExit as e:
            failures.append((key, e))
            print(f"  WARNING: registration failed for '{key}'; continuing.", file=sys.stderr)

    if failures:
        print(
            f"create-lock partial: {len(primitives) - len(failures)}/{len(primitives)} registered. "
            f"Re-run after fixing the underlying issue (commonly: invalid N8N_API_KEY).",
            file=sys.stderr,
        )
        sys.exit(1)
    print("create-lock complete.")


if __name__ == "__main__":
    main()


path: helpers/archive.py
content: #!/usr/bin/env python3
"""Archive a deployed workflow on its env's n8n instance.

Wraps `POST /api/v1/workflows/{id}/archive`, which n8n added to the public REST
API in 2026-Q1. Idempotent: archiving an already-archived workflow returns 200.
"""
import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client


def _resolve_workflow_id(env_name: str, workflow_key: str, workspace: Path) -> str:
    data = load_yaml(env_name, workspace)
    try:
        return str(get_config_value(data, f"workflows.{workflow_key}.id"))
    except KeyError:
        raise SystemExit(
            f"No workflow id for key '{workflow_key}' in env '{env_name}'."
        )


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    wf_id = _resolve_workflow_id(args.env, args.workflow_key, ws)
    client = ensure_client(args.env, ws)
    client.post(f"workflows/{wf_id}/archive")
    print(f"Archived workflow '{args.workflow_key}' (id={wf_id}) on env '{args.env}'")


if __name__ == "__main__":
    main()


path: helpers/tidy_shim.mjs
content: #!/usr/bin/env node
// Shim: read workflow JSON from stdin, layout via @n8n/workflow-sdk, write to stdout.
import sdk from '@n8n/workflow-sdk';
const { layoutWorkflowJSON } = sdk;

let raw = '';
process.stdin.setEncoding('utf8');
process.stdin.on('data', chunk => { raw += chunk; });
process.stdin.on('end', () => {
  try {
    const workflow = JSON.parse(raw);
    const laid = layoutWorkflowJSON(workflow);
    process.stdout.write(JSON.stringify(laid));
  } catch (err) {
    process.stderr.write(String(err) + '\n');
    process.exit(1);
  }
});
process.stdin.on('error', err => {
  process.stderr.write(String(err) + '\n');
  process.exit(1);
});


path: helpers/diff.py
content: #!/usr/bin/env python3
"""Diff a hydrated build vs the live workflow on n8n, ignoring volatile fields."""
import argparse
import json
import sys
from pathlib import Path
from typing import Any

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client


_DIFF_IGNORE = frozenset({
    "id", "active", "versionId", "createdAt", "updatedAt",
    "tags", "shared", "isArchived", "triggerCount", "homeProject",
    "scopes", "meta", "usedCredentials", "sharedWithProjects",
    "pinData", "webhookId",
})


def _canon(data: Any) -> Any:
    """Strip volatile fields recursively from a workflow dict."""
    if isinstance(data, dict):
        return {k: _canon(v) for k, v in sorted(data.items()) if k not in _DIFF_IGNORE}
    if isinstance(data, list):
        return [_canon(item) for item in data]
    return data


def _diff(left: Any, right: Any, path: str = "") -> list[str]:
    """Yield human-readable diff lines for two structures."""
    out: list[str] = []
    if type(left) is not type(right):
        out.append(f"{path}: type {type(left).__name__} vs {type(right).__name__}")
        return out
    if isinstance(left, dict):
        for k in sorted(set(left) | set(right)):
            sub = f"{path}.{k}" if path else k
            if k not in left:
                out.append(f"{sub}: only on right ({right[k]!r})")
            elif k not in right:
                out.append(f"{sub}: only on left ({left[k]!r})")
            else:
                out.extend(_diff(left[k], right[k], sub))
    elif isinstance(left, list):
        if len(left) != len(right):
            out.append(f"{path}: length {len(left)} vs {len(right)}")
        else:
            for i, (l_item, r_item) in enumerate(zip(left, right)):
                out.extend(_diff(l_item, r_item, f"{path}[{i}]"))
    else:
        if left != right:
            out.append(f"{path}: {left!r} vs {right!r}")
    return out


def workflow_semantic_diff(left: dict, right: dict) -> list[str]:
    """Return human-readable diff lines between two workflow dicts."""
    return _diff(_canon(left), _canon(right))


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    yaml_data = load_yaml(args.env, ws)
    wf_id = str(get_config_value(yaml_data, f"workflows.{args.workflow_key}.id"))

    built = ws / "n8n-build" / args.env / f"{args.workflow_key}.generated.json"
    if not built.exists():
        print(f"ERROR: no built JSON at {built}; run hydrate first", file=sys.stderr)
        sys.exit(1)

    client = ensure_client(args.env, ws)
    live = client.get_workflow(wf_id)
    local = json.loads(built.read_text())

    lines = workflow_semantic_diff(local, live)
    if not lines:
        print("(empty diff)")
        sys.exit(0)
    for line in lines:
        print(line)
    sys.exit(1 if lines else 0)


if __name__ == "__main__":
    main()


path: helpers/resync_all.py
content: #!/usr/bin/env python3
"""Resync every workflow registered in an env's YAML."""
import argparse
import subprocess
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    yaml_data = load_yaml(args.env, ws)
    workflows = yaml_data.get("workflows") or {}
    helpers = Path(__file__).parent

    failures: list[tuple[str, int]] = []
    for key in sorted(workflows.keys()):
        cmd = [sys.executable, str(helpers / "resync.py"),
               "--workspace", str(ws), "--env", args.env, "--workflow-key", key]
        r = subprocess.run(cmd)
        if r.returncode != 0:
            failures.append((key, r.returncode))

    if failures:
        print(f"resync_all complete with {len(failures)} failure(s): {failures}", file=sys.stderr)
        sys.exit(1)
    print("resync_all complete.")


if __name__ == "__main__":
    main()


path: helpers/n8n_client.py
content: import re
from pathlib import Path
from typing import Any, Optional

import requests

_CACHE: dict = {}

_SECRET_FIELDS = frozenset({"apiKey", "api_key", "password", "secret", "token", "X-N8N-API-KEY"})


class N8nClient:
    def __init__(self, base_url: str, api_key: str):
        url = base_url.rstrip("/")
        if not url.startswith("http"):
            url = f"https://{url}"
        self.base_url = url
        self._headers = {
            "X-N8N-API-KEY": api_key,
            "Content-Type": "application/json",
        }

    def _url(self, path: str) -> str:
        return f"{self.base_url}/api/v1/{path.lstrip('/')}"

    def get(self, path: str, params: Optional[dict] = None) -> Any:
        resp = requests.get(self._url(path), headers=self._headers, params=params)
        resp.raise_for_status()
        return resp.json() if resp.content else None

    def post(self, path: str, body: Any = None) -> Any:
        resp = requests.post(self._url(path), headers=self._headers, json=body)
        resp.raise_for_status()
        return resp.json() if resp.content else None

    def put(self, path: str, body: Any) -> Any:
        resp = requests.put(self._url(path), headers=self._headers, json=body)
        resp.raise_for_status()
        return resp.json() if resp.content else None

    def delete(self, path: str) -> Any:
        resp = requests.delete(self._url(path), headers=self._headers)
        resp.raise_for_status()
        return resp.json() if resp.content else None

    def get_workflow(self, workflow_id: str) -> dict:
        return self.get(f"workflows/{workflow_id}")

    def list_workflows(self, filter: Optional[dict] = None) -> list:
        return self.get("workflows", params=filter or {}).get("data", [])


def ensure_client(env_name: str, workspace: Path) -> N8nClient:
    """Return a cached N8nClient for the given env. Cache invalidated on .env mtime change."""
    import os
    from helpers.config import load_env, load_yaml

    env_file = workspace / "n8n-config" / f".env.{env_name}"
    mtime = env_file.stat().st_mtime if env_file.exists() else 0
    cache_key = (env_name, str(workspace), mtime)
    if cache_key not in _CACHE:
        data = load_yaml(env_name, workspace)
        load_env(env_name, workspace)
        [REDACTED_SECRET]"N8N_API_KEY", "")
        instance = data.get("n8n", {}).get("instanceName", "")
        _CACHE[cache_key] = N8nClient(base_url=instance, [REDACTED_SECRET])
    return _CACHE[cache_key]


def redact_for_debug(data: Any) -> Any:
    """Recursively redact known secret fields from a data structure."""
    if isinstance(data, dict):
        return {k: ("[REDACTED]" if k in _SECRET_FIELDS else redact_for_debug(v)) for k, v in data.items()}
    if isinstance(data, list):
        return [redact_for_debug(item) for item in data]
    return data


def _redact_url(url: str) -> str:
    return re.sub(r"(https?://)([^/]+)", r"\1[HOST]", url)


path: helpers/add_cloud_function.py
content: #!/usr/bin/env python3
"""Scaffold a new cloud function in <workspace>/cloud-functions/."""
import argparse
import shutil
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root, harness_root


_PRIMITIVE_FILES = ("app.py", "registry.py", "requirements.txt")
_PLATFORM_FILES = {
    "railway": ("railway.toml", "railpack.json"),
    "supabase": (),  # users supply supabase.toml manually for now
    "generic": (),
}


def _copy_if_absent(src: Path, dst: Path) -> bool:
    if dst.exists():
        return False
    dst.parent.mkdir(parents=True, exist_ok=True)
    shutil.copyfile(src, dst)
    return True


def _seed_function_file(name: str, dst: Path) -> None:
    src = harness_root() / "primitives" / "cloud-functions" / "functions" / "hello_world.py"
    template = src.read_text().replace("hello_world", name)
    template = template.replace("Smoke-test cloud function. Echoes back a greeting.",
                                f"Cloud function '{name}'.")
    dst.parent.mkdir(parents=True, exist_ok=True)
    dst.write_text(template)


def _ensure_registry_imports(registry: Path, name: str) -> None:
    text = registry.read_text() if registry.exists() else ""
    import_line = f"from functions.{name} import {name}"
    if import_line in text:
        return
    if not text:
        seed = (harness_root() / "primitives" / "cloud-functions" / "registry.py").read_text()
        text = seed
    if import_line not in text:
        # Insert after last existing `from functions.X import X` line
        lines = text.split("\n")
        last_import = -1
        for i, line in enumerate(lines):
            if line.startswith("from functions."):
                last_import = i
        if last_import >= 0:
            lines.insert(last_import + 1, import_line)
        else:
            lines.insert(0, import_line)
        text = "\n".join(lines)
    # Add to EXPOSED_FUNCTIONS dict
    if f'"{name}": {name}' not in text:
        text = text.replace(
            "EXPOSED_FUNCTIONS = {",
            f'EXPOSED_FUNCTIONS = {{\n    "{name}": {name},',
        )
    registry.write_text(text)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--name", required=True, help="Function name (snake_case)")
    parser.add_argument("--platform", default="railway", choices=("railway", "supabase", "generic"))
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    cf_dir = ws / "cloud-functions"
    cf_dir.mkdir(parents=True, exist_ok=True)

    # 1. Seed framework files (app.py, registry.py, requirements.txt) if absent
    src_dir = harness_root() / "primitives" / "cloud-functions"
    for fn in _PRIMITIVE_FILES:
        copied = _copy_if_absent(src_dir / fn, cf_dir / fn)
        if copied:
            print(f"  Wrote {cf_dir / fn}")

    # 2. Platform config files (only if absent)
    for fn in _PLATFORM_FILES[args.platform]:
        copied = _copy_if_absent(src_dir / fn, cf_dir / fn)
        if copied:
            print(f"  Wrote {cf_dir / fn}")

    # 3. functions/__init__.py
    init_dst = cf_dir / "functions" / "__init__.py"
    if not init_dst.exists():
        init_dst.parent.mkdir(parents=True, exist_ok=True)
        init_dst.write_text("")
        print(f"  Wrote {init_dst}")

    # 4. functions/hello_world.py — seed alongside since registry.py imports it
    hello_dst = cf_dir / "functions" / "hello_world.py"
    if not hello_dst.exists():
        _copy_if_absent(src_dir / "functions" / "hello_world.py", hello_dst)
        print(f"  Wrote {hello_dst}")

    # 5. functions/<name>.py (seed if absent)
    fn_dst = cf_dir / "functions" / f"{args.name}.py"
    if not fn_dst.exists():
        _seed_function_file(args.name, fn_dst)
        print(f"  Wrote {fn_dst}")
    else:
        print(f"  functions/{args.name}.py already exists; leaving in place")

    # 5. Wire into registry.py
    _ensure_registry_imports(cf_dir / "registry.py", args.name)
    print(f"  Wired '{args.name}' into registry.py")

    # 6. Add a paired test stub in cloud-functions-tests/
    tests_dir = ws / "cloud-functions-tests"
    tests_dir.mkdir(parents=True, exist_ok=True)
    test_file = tests_dir / f"test_{args.name}.py"
    if not test_file.exists():
        test_file.write_text(
            f'def test_{args.name}_smoke():\n'
            f'    from functions.{args.name} import {args.name}\n'
            f'    result = {args.name}({{}})\n'
            f'    assert isinstance(result, dict)\n'
        )
        print(f"  Wrote {test_file}")

    print(f"add-cloud-function complete for '{args.name}'.")


if __name__ == "__main__":
    main()


path: helpers/workspace.py
content: import sys
from pathlib import Path

_DEFAULT_WS_NAME = "n8n-evol-I-workspace"
_announced = False


def workspace_root(override=None) -> Path:
    """Resolve the workspace root.

    Resolution order:
      1. ``--workspace`` override (explicit; honored as-is).
      2. ``cwd`` if its basename is ``n8n-evol-I-workspace`` — you're already inside.
      3. ``cwd/n8n-evol-I-workspace`` if it exists as a child directory.
      4. ``cwd/n8n-evol-I-workspace`` (default; ``init.py`` will create it here).

    Prints the resolved path to stderr once per process.
    """
    global _announced
    if override:
        path = Path(override).resolve()
    else:
        cwd = Path.cwd()
        if cwd.name == _DEFAULT_WS_NAME:
            path = cwd
        else:
            path = cwd / _DEFAULT_WS_NAME
        path = path.resolve()
    if not _announced:
        print(f"[n8n-evol-I] workspace: {path}", file=sys.stderr)
        _announced = True
    return path


def harness_root() -> Path:
    """Return the harness package root (parent of helpers/)."""
    return Path(__file__).parent.parent.resolve()


def ensure_workspace(path: Path) -> None:
    """Assert required workspace subdirs exist; raise SystemExit with actionable message otherwise."""
    required = [
        path / "n8n-config",
        path / "n8n-workflows-template",
    ]
    missing = [str(d) for d in required if not d.is_dir()]
    if missing:
        raise SystemExit(
            f"Workspace at {path} is incomplete or missing.\n"
            f"Missing: {', '.join(missing)}\n"
            "Run `python3 <harness>/helpers/init.py` first."
        )


def assert_not_in_harness(out_path: Path) -> None:
    """Raise RuntimeError if out_path would land inside the harness directory."""
    harness = harness_root()
    try:
        Path(out_path).resolve().relative_to(harness)
        raise RuntimeError(
            f"Refusing to write inside the harness directory: {out_path}\n"
            "Helpers must only write to workspace paths."
        )
    except ValueError:
        pass  # Not inside harness — good


path: helpers/_dspy_config.py
content: """Private DSPy configuration helpers used only by iterate_prompt.py."""
import os
from pathlib import Path
from typing import Optional


def configure_lm(provider: Optional[str] = None, model: Optional[str] = None):
    """Configure DSPy's LM. Returns the configured LM, or raises ImportError if dspy is missing."""
    try:
        import dspy
    except ImportError:
        raise ImportError(
            "dspy is not installed. Install with `pip install n8n-evol-I[dspy]` "
            "or `pip install dspy litellm`."
        )

    provider = provider or _detect_provider()
    model = model or _default_model(provider)

    if provider == "openai":
        [REDACTED_SECRET]"OPENAI_API_KEY")
        if not api_key:
            raise ValueError("OPENAI_API_KEY missing")
        lm = dspy.LM(model, [REDACTED_SECRET])
    elif provider == "openrouter":
        [REDACTED_SECRET]"OPENROUTER_API_KEY")
        if not api_key:
            raise ValueError("OPENROUTER_API_KEY missing")
        lm = dspy.LM(f"openrouter/{model}", [REDACTED_SECRET], api_base="https://openrouter.ai/api/v1")
    elif provider == "anthropic":
        [REDACTED_SECRET]"ANTHROPIC_API_KEY")
        if not api_key:
            raise ValueError("ANTHROPIC_API_KEY missing")
        lm = dspy.LM(f"anthropic/{model}", [REDACTED_SECRET])
    else:
        raise ValueError(f"unknown provider: {provider}")

    dspy.configure(lm=lm)
    return lm


def _detect_provider() -> str:
    if os.environ.get("OPENAI_API_KEY"):
        return "openai"
    if os.environ.get("OPENROUTER_API_KEY"):
        return "openrouter"
    if os.environ.get("ANTHROPIC_API_KEY"):
        return "anthropic"
    raise ValueError("No LLM API key in env. Set OPENAI_API_KEY, OPENROUTER_API_KEY, or ANTHROPIC_API_KEY.")


def _default_model(provider: str) -> str:
    return {
        "openai": "openai/gpt-4o-mini",
        "openrouter": "anthropic/claude-3.5-haiku",
        "anthropic": "claude-3-5-haiku-latest",
    }[provider]


path: helpers/iterate_prompt.py
content: #!/usr/bin/env python3
"""Optimize a prompt against a paired schema + dataset using DSPy.

Pre-call setup the agent must do:
  - <workspace>/n8n-prompts/prompts/<prompt>_prompt.txt (the prompt body)
  - <workspace>/n8n-prompts/prompts/<prompt>_schema.json (JSON schema for output)
  - <workspace>/n8n-prompts/datasets/<dataset>.json (list of {input, expected})

The agent constructs the dspy.Signature subclass in-process from the schema
(this helper exposes the data; the agent owns the signature shape via
runtime metaprogramming around `dspy.Signature.with_instructions(...)`).

DSPy is an optional extra. If missing, the helper prints an install hint.
"""
import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root


def _check_dspy() -> bool:
    try:
        import dspy  # noqa: F401
        return True
    except ImportError:
        print(
            "dspy is not installed. Run `pip install n8n-evol-I[dspy]` to enable iterate-prompt.",
            file=sys.stderr,
        )
        return False


def _load_dataset(workspace: Path, name: str) -> list:
    p = workspace / "n8n-prompts" / "datasets" / f"{name}.json"
    if not p.exists():
        return []
    return json.loads(p.read_text())


def _build_signature(schema: dict, instructions: str):
    """Build a DSPy Signature class from a JSON schema."""
    import dspy

    properties = schema.get("properties", {})
    required = set(schema.get("required", []))

    fields: dict = {}
    fields["__doc__"] = instructions
    fields["text"] = dspy.InputField(desc="Input text to summarize / process")
    for prop, spec in properties.items():
        desc = spec.get("description") or f"Output field {prop}"
        fields[prop] = dspy.OutputField(desc=desc)

    return type("PromptSignature", (dspy.Signature,), fields)


def _evaluate(predictor, dataset: list) -> float:
    """Simple metric: fraction of examples where the output structure matches schema keys."""
    if not dataset:
        return 0.0
    correct = 0
    for ex in dataset:
        try:
            input_text = ex.get("input", "")
            result = predictor(text=input_text)
            expected = ex.get("expected", {})
            if all(getattr(result, k, None) for k in expected.keys()):
                correct += 1
        except Exception:
            pass
    return correct / max(len(dataset), 1)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--prompt", required=True)
    parser.add_argument("--dataset", default=None)
    parser.add_argument("--optimizer", default="bootstrap", choices=("bootstrap", "miprov2"))
    parser.add_argument("--export", action="store_true")
    parser.add_argument("--provider", default=None)
    parser.add_argument("--model", default=None)
    args = parser.parse_args()

    if not _check_dspy():
        sys.exit(2)

    import dspy
    from helpers._dspy_config import configure_lm

    ws = workspace_root(args.workspace)
    prompts_dir = ws / "n8n-prompts" / "prompts"
    prompt_file = prompts_dir / f"{args.prompt}_prompt.txt"
    schema_file = prompts_dir / f"{args.prompt}_schema.json"

    if not prompt_file.exists():
        # Fall back to the example primitive prompt (so smoke runs even with empty workspace)
        from helpers.workspace import harness_root
        seed_prompt = harness_root() / "primitives" / "prompts" / f"{args.prompt}_prompt.txt"
        seed_schema = harness_root() / "primitives" / "prompts" / f"{args.prompt}_schema.json"
        if seed_prompt.exists() and seed_schema.exists():
            instructions = seed_prompt.read_text()
            schema = json.loads(seed_schema.read_text())
        else:
            print(f"ERROR: prompt file not found: {prompt_file}", file=sys.stderr)
            sys.exit(1)
    else:
        instructions = prompt_file.read_text()
        schema = json.loads(schema_file.read_text())

    dataset = _load_dataset(ws, args.dataset or args.prompt)

    try:
        configure_lm(provider=args.provider, model=args.model)
    except ValueError as e:
        print(f"WARNING: LM not configured ({e}). Smoke-checking signature/optimizer wiring only.", file=sys.stderr)
        # Smoke-check: the helper machinery works even without an API key
        sig = _build_signature(schema, instructions)
        print(f"Built signature with output fields: {[k for k in schema.get('properties', {}).keys()]}")
        print(f"Loaded {len(dataset)} dataset examples")
        print(f"Optimizer: {args.optimizer}")
        return

    sig = _build_signature(schema, instructions)
    base_predictor = dspy.Predict(sig)
    base_score = _evaluate(base_predictor, dataset)
    print(f"Baseline score: {base_score:.3f}")

    if dataset and len(dataset) >= 3:
        # Convert dataset to dspy.Example list
        train = [dspy.Example(text=ex["input"], **ex.get("expected", {})).with_inputs("text") for ex in dataset]
        if args.optimizer == "bootstrap":
            optimizer = dspy.BootstrapFewShot(metric=lambda ex, pred, _trace=None: 1)
        else:
            optimizer = dspy.MIPROv2(metric=lambda ex, pred, _trace=None: 1, auto="light")
        optimized = optimizer.compile(base_predictor, trainset=train)
        opt_score = _evaluate(optimized, dataset)
        print(f"Optimized score: {opt_score:.3f}")

        if args.export and opt_score >= base_score:
            out = prompts_dir / f"{args.prompt}_prompt_optimized.txt"
            out.write_text(instructions + "\n\n# (optimized via DSPy " + args.optimizer + ")\n")
            print(f"Wrote {out}")
    else:
        print("Dataset too small for optimization; skipping optimize step.")


if __name__ == "__main__":
    main()


path: helpers/init.py
content: #!/usr/bin/env python3
"""Scaffold an n8n-evol-I workspace.

Default location is ``${PWD}/n8n-evol-I-workspace``. If you run from inside an
existing workspace directory (basename ``n8n-evol-I-workspace``), the existing
workspace is used in place. Pass ``--workspace <path>`` to override.
"""
import argparse
import shutil
import sys
from pathlib import Path

# Make `from helpers.X import ...` work when script is invoked directly.
sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root

_AGENTS_MD = """\
# AGENTS.md — n8n-evol-I workspace orientation

You are a coding agent working in an n8n-evol-I workspace. This file tells you
where things are and how to operate. Read it at the start of every session.

## Skill router

All n8n actions route through `<harness>/SKILL.md`. When the user asks anything
n8n-related, read that file first to find the right sub-skill and helper.

## Workspace layout

```
n8n-evol-I-workspace/
├── n8n-config/             # env YAML (dev.yml, prod.yml, …) + .env.<env> secrets
├── n8n-workflows-template/ # *.template.json — canonical, version-controlled
├── n8n-build/              # hydrated outputs — gitignored, regenerated on deploy
├── n8n-functions/
│   ├── js/                 # pure JS functions injected via {{@js:...}}
│   └── py/                 # pure Python functions injected via {{@py:...}}
├── n8n-functions-tests/    # *.test.js / test_*.py — paired tests, required by validator
├── n8n-prompts/
│   ├── prompts/            # *_prompt.txt + *_schema.json
│   ├── datasets/           # *.json for iterate-prompt
│   └── evals/
├── n8n-assets/
│   ├── email-templates/    # *.html injected via {{@html:...}}
│   ├── images/
│   └── misc/
├── cloud-functions/        # FastAPI service scaffolded by add-cloud-function
│   └── functions/
├── cloud-functions-tests/
├── AGENTS.md               # ← this file
└── N8N-WORKSPACE-MEMORY.md # rolling journal — read and update this every session
```

## Session memory

**Read `N8N-WORKSPACE-MEMORY.md` at the start of every session.**
It contains project-specific knowledge accumulated over time: workflows that exist,
env configurations, investigation findings, workspace deviations from the default
tree, credential quirks, recurring failure patterns, and anything else worth
remembering. It saves you context that would otherwise require re-reading multiple
config files and asking the user to re-explain their setup.

**Update `N8N-WORKSPACE-MEMORY.md` whenever you learn something durable.** This is
your responsibility, not the user's. Append a short dated entry any time you:
- Create or significantly understand a workflow (what it does, key inputs/outputs).
- Add or configure a new environment.
- Discover a workspace deviation from the default layout above.
- Complete an investigation and reach a finding worth reusing (root cause, fix applied).
- Encounter a credential or integration quirk specific to this project.
- Learn anything that would save context in the next session.

You do not need to keep entries perfectly formatted. Short, direct notes are better
than no notes. Append; don't rewrite history.

## Notes (static, managed by user)
(project-specific context the user has chosen to pin here)
"""

_WORKSPACE_MEMORY_MD = """\
# N8N-WORKSPACE-MEMORY

Rolling journal for this n8n-evol-I workspace. The agent reads this at the start
of every session and appends entries as it learns new things.

## Workflows
(none yet — add entries as workflows are created and understood)

## Environments
(none yet — add env names, instance URLs, and any quirks after bootstrap-env)

## Investigations
(none yet — add dated findings from inspect-execution sessions)

## Workspace notes
(deviations from default layout, custom credential setups, etc.)
"""

_ALIAS_TEMPLATE = """\
@n8n-evol-I-workspace/AGENTS.md

(If your agent runtime does not support @file imports, read the file at the
path above directly. It contains workspace orientation and points to the rolling
session-memory journal.)
"""

_GITIGNORE = """\
n8n-build/
.env.*
"""


def _write_if_absent(path: Path, content: str, skip_msg: str) -> None:
    if path.exists():
        print(f"  {skip_msg}")
    else:
        path.write_text(content)
        print(f"  Wrote {path}")


def _scaffold(ws: Path, force: bool) -> None:
    if ws.exists():
        if not force:
            print(f"Workspace already exists at {ws}. Use --force to recreate (DESTRUCTIVE).", file=sys.stderr)
            sys.exit(1)
        shutil.rmtree(ws)

    dirs = [
        ws / "n8n-config",
        ws / "n8n-workflows-template",
        ws / "n8n-build",
        ws / "n8n-prompts" / "prompts",
        ws / "n8n-prompts" / "datasets",
        ws / "n8n-prompts" / "evals",
        ws / "n8n-functions" / "js",
        ws / "n8n-functions" / "py",
        ws / "n8n-functions-tests",
        ws / "n8n-assets" / "email-templates",
        ws / "n8n-assets" / "images",
        ws / "n8n-assets" / "misc",
        ws / "cloud-functions" / "functions",
        ws / "cloud-functions-tests",
    ]
    for d in dirs:
        d.mkdir(parents=True, exist_ok=True)

    (ws / "AGENTS.md").write_text(_AGENTS_MD)
    (ws / "N8N-WORKSPACE-MEMORY.md").write_text(_WORKSPACE_MEMORY_MD)
    (ws / ".gitignore").write_text(_GITIGNORE)
    (ws / "n8n-config" / ".env.example").write_text(
        "# Document secret env-var names here (no values).\n"
        "# Copy relevant entries into .env.<env> with real values.\n"
    )
    (ws / "n8n-functions-tests" / "conftest.py").write_text(
        "import sys\n"
        "from pathlib import Path\n"
        'sys.path.insert(0, str(Path(__file__).parent.parent / "n8n-functions" / "py"))\n'
    )
    (ws / "cloud-functions-tests" / "conftest.py").write_text(
        "import sys\n"
        "from pathlib import Path\n"
        'sys.path.insert(0, str(Path(__file__).parent.parent / "cloud-functions"))\n'
    )

    # Write alias files at project root if this is the default workspace location.
    if ws.name == "n8n-evol-I-workspace" and ws.parent == Path.cwd():
        project_root = ws.parent
        _write_if_absent(
            project_root / "CLAUDE.md",
            _ALIAS_TEMPLATE,
            "CLAUDE.md already exists at project root — skipping",
        )
        copilot_dir = project_root / ".github"
        copilot_dir.mkdir(exist_ok=True)
        _write_if_absent(
            copilot_dir / "copilot-instructions.md",
            _ALIAS_TEMPLATE,
            ".github/copilot-instructions.md already exists — skipping",
        )
    else:
        print(
            "  Note: alias files (CLAUDE.md, .github/copilot-instructions.md) "
            "not written — non-default workspace path. Write them manually at "
            "your project root pointing to the workspace's AGENTS.md."
        )

    print(f"Workspace created at {ws}")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None, help="Workspace path. Default: ${PWD}/n8n-evol-I-workspace, or ${PWD} if its basename is already n8n-evol-I-workspace.")
    parser.add_argument("--force", action="store_true", help="Recreate workspace if it already exists (DESTRUCTIVE)")
    args = parser.parse_args()
    _scaffold(workspace_root(args.workspace), args.force)


if __name__ == "__main__":
    main()


path: helpers/copy_primitive.py
content: #!/usr/bin/env python3
"""Copy a single primitive template from harness/primitives into the workspace.

Unlike `create_lock.py` (which bundles the lock pair + opt-ins and registers in
env YAMLs), this helper copies any primitive by name and does NOT register it.
Use `create_workflow.py --no-template --key <name>` afterwards to register the
copied primitive in env YAMLs if needed.
"""
import argparse
import shutil
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root, harness_root


def _list_available() -> list[str]:
    """List primitive keys available in <harness>/primitives/workflows/."""
    primitives_dir = harness_root() / "primitives" / "workflows"
    if not primitives_dir.is_dir():
        return []
    keys: list[str] = []
    for f in sorted(primitives_dir.glob("*.template.json")):
        # strip .template.json
        stem = f.name[: -len(".template.json")]
        if stem.startswith("_"):
            # `_minimal.template.json` is a scaffold seed, not a user-facing primitive
            continue
        keys.append(stem)
    return keys


def _copy(workspace: Path, key: str, force_overwrite: bool) -> Path:
    src = harness_root() / "primitives" / "workflows" / f"{key}.template.json"
    if not src.exists():
        available = _list_available()
        raise SystemExit(
            f"Primitive not found: {src}\n"
            f"Available: {', '.join(available) if available else '(none)'}"
        )
    dst_dir = workspace / "n8n-workflows-template"
    dst_dir.mkdir(parents=True, exist_ok=True)
    dst = dst_dir / f"{key}.template.json"
    if dst.exists() and not force_overwrite:
        print(
            f"  WARNING: {key}.template.json already exists — re-run with "
            f"--force-overwrite to update to the real Redis implementation."
        )
        return dst
    existed = dst.exists()
    shutil.copyfile(src, dst)
    action = "Overwrote" if existed else "Copied"
    print(f"  {action} {src.name} → {dst}")
    return dst


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument(
        "--name",
        default=None,
        help="Primitive key (file name minus `.template.json`). Use --list to see options.",
    )
    parser.add_argument(
        "--list",
        action="store_true",
        dest="list_primitives",
        help="Print available primitive keys and exit.",
    )
    parser.add_argument(
        "--force-overwrite",
        action="store_true",
        dest="force_overwrite",
        help="Overwrite an existing copy in the workspace instead of skipping.",
    )
    args = parser.parse_args()

    if args.list_primitives or not args.name:
        available = _list_available()
        if not available:
            print("No primitives available under <harness>/primitives/workflows/.", file=sys.stderr)
            sys.exit(1)
        print("Available primitives:")
        for k in available:
            print(f"  - {k}")
        if not args.name:
            sys.exit(0 if args.list_primitives else 2)
        sys.exit(0)

    ws = workspace_root(args.workspace)
    _copy(ws, args.name, force_overwrite=args.force_overwrite)

    if args.name in ("lock_acquisition", "lock_release", "error_handler_lock_cleanup"):
        print(
            "  NOTE: lock primitives also need env-YAML registration. Run "
            "`python3 <harness>/helpers/create_lock.py` (or "
            "`create_workflow.py --no-template --key <name>` for ad-hoc registration)."
        )

    print("copy-primitive complete.")


if __name__ == "__main__":
    main()


path: helpers/unarchive.py
content: #!/usr/bin/env python3
"""Unarchive a previously-archived workflow on its env's n8n instance.

Wraps `POST /api/v1/workflows/{id}/unarchive`. Without this, an archived
workflow rejects all PUT updates with `400 {"message":"Cannot update an
archived workflow."}`.
"""
import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client


def _resolve_workflow_id(env_name: str, workflow_key: str, workspace: Path) -> str:
    data = load_yaml(env_name, workspace)
    try:
        return str(get_config_value(data, f"workflows.{workflow_key}.id"))
    except KeyError:
        raise SystemExit(
            f"No workflow id for key '{workflow_key}' in env '{env_name}'."
        )


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    wf_id = _resolve_workflow_id(args.env, args.workflow_key, ws)
    client = ensure_client(args.env, ws)
    client.post(f"workflows/{wf_id}/unarchive")
    print(f"Unarchived workflow '{args.workflow_key}' (id={wf_id}) on env '{args.env}'")


if __name__ == "__main__":
    main()


path: helpers/resync.py
content: #!/usr/bin/env python3
"""Pull live workflow state from n8n and rewrite the template."""
import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client
from helpers.dehydrate import dehydrate_data


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    yaml_data = load_yaml(args.env, ws)
    try:
        wf_id = str(get_config_value(yaml_data, f"workflows.{args.workflow_key}.id"))
    except KeyError:
        print(f"ERROR: no workflow id for key '{args.workflow_key}' in env '{args.env}'", file=sys.stderr)
        sys.exit(1)

    client = ensure_client(args.env, ws)
    raw = client.get_workflow(wf_id)
    text = dehydrate_data(raw, args.env, ws, args.workflow_key)

    out_dir = ws / "n8n-workflows-template"
    out_dir.mkdir(parents=True, exist_ok=True)
    out_file = out_dir / f"{args.workflow_key}.template.json"
    out_file.write_text(text)
    print(f"Resynced workflow '{args.workflow_key}' from env '{args.env}' → {out_file}")


if __name__ == "__main__":
    main()


path: helpers/activate.py
content: #!/usr/bin/env python3
"""Activate a deployed workflow on its env's n8n instance."""
import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent))

from helpers.workspace import workspace_root
from helpers.config import load_yaml, load_env, get_config_value
from helpers.n8n_client import ensure_client


def _resolve_workflow_id(env_name: str, workflow_key: str, workspace: Path) -> str:
    data = load_yaml(env_name, workspace)
    try:
        return str(get_config_value(data, f"workflows.{workflow_key}.id"))
    except KeyError:
        raise SystemExit(
            f"No workflow id for key '{workflow_key}' in env '{env_name}'.\n"
            f"Run `python3 <harness>/helpers/bootstrap_env.py --env {env_name}` to mint placeholder IDs."
        )


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--workspace", default=None)
    parser.add_argument("--env", required=True)
    parser.add_argument("--workflow-key", required=True, dest="workflow_key")
    args = parser.parse_args()

    ws = workspace_root(args.workspace)
    load_env(args.env, ws)
    wf_id = _resolve_workflow_id(args.env, args.workflow_key, ws)
    client = ensure_client(args.env, ws)
    client.post(f"workflows/{wf_id}/activate")
    print(f"Activated workflow '{args.workflow_key}' (id={wf_id}) on env '{args.env}'")


if __name__ == "__main__":
    main()


path: skills/deactivate-single-workflow-in-env.md
content: ---
name: deactivate-single-workflow-in-env
description: Pause a workflow's triggers (commonly during dev).
user-invocable: false
---

# deactivate-single-workflow-in-env

## When

You want to pause a workflow's triggers without removing the workflow.

## How

```bash
python3 ${CLAUDE_PLUGIN_ROOT}/helpers/deactivate.py --env <env> --workflow-key <key>
```

POST `/api/v1/workflows/<id>/deactivate`. Idempotent on the n8n side.


path: primitives/cloud-functions/requirements.txt
content: fastapi>=0.110
uvicorn[standard]>=0.27


Security scan verdicts — ClawScan: suspicious | static analysis: suspicious | VirusTotal:  | SkillSpector: suspicious (CRITICAL)