Skill: sdk-team/alibabacloud-compute-provision (version 0.0.1-beta.1)

--- name: alibabacloud-compute-provision description: "Alibaba Cloud Compute Provision - Automatically selects an Alibaba Cloud compute resource (ECS, FC, ACK, PAI) based on user intent, then creates instances and executes scripts. Use this skill when the user needs to run compute jobs, execute scripts, train models, or deploy containerized applications on Alibaba Cloud, or mentions keywords such as cpu_bound, gpu, vCPU, budget, training, A100, or qwen. Provides a full loop of resource selection, pricing, budget control, instance creation, and script execution." license: MIT metadata: author: Aliyun version: "0.1.0" --- # Alibaba Cloud Compute Provision Automatically selects an Alibaba Cloud compute resource based on user intent, then creates instances and executes scripts. ## How this skill works This skill operates by **writing and executing Python code that calls Alibaba Cloud APIs**. The `scripts/` directory contains ready-made Python modules (ECS, FC, ACK, PAI, VPC, etc.) that wrap the Alibaba Cloud OpenAPI. To accomplish any task in this skill, you write Python code snippets that import and call functions from these modules — you do NOT use CLI tools, Terraform, or the web console. Typical workflow: 1. **Read the reference doc** for the product you're about to use (see Reference Index below). 2. Write a Python code block that imports from the skill's `scripts/` modules. 3. Execute the code to call Alibaba Cloud APIs (query instance types, check pricing, create resources, run scripts, etc.). 4. Read the output and decide the next step. > **⛔ MUST-READ RULE**: Before calling ANY function from `scripts/`, you MUST first read its reference doc (e.g. `references/ecs.md` for ECS functions, `references/fc.md` for FC functions). The reference docs contain exact function signatures, parameter names, constraints, and usage examples. **Do NOT guess parameter names** — incorrect parameters waste tool calls and may create/leak cloud resources. Use the defaults when in doubt. ## Prerequisites ### Step 0: Environment bootstrap (MUST run first) **Before doing anything else**, execute the following code block to set up the Python path and ensure all dependencies are installed. This MUST be the very first code you run in every session — do NOT skip it or defer it. ```python import sys sys.path.insert(0, "${SKILL_DIR}/scripts") from bootstrap import ensure_dependencies ensure_dependencies() ``` `bootstrap.py` is a standalone module with **zero third-party dependencies** (stdlib only), so it can always be imported even before any pip packages are installed. `ensure_dependencies()` automatically: - Checks that the Python version is >= 3.8 (exits with a clear error if not). - Detects missing pip packages (`alibabacloud_credentials`, `alibabacloud_tea_openapi`, `darabonba-core`) and installs them. > If this step fails, fix the reported issue (e.g. install a newer Python) before proceeding — all subsequent steps depend on it. ### Credentials Credentials are resolved via the Alibaba Cloud default credential provider chain (environment variables, `~/.alibabacloud/credentials`, `~/.aliyun/config.json`, ECS RAM role, etc.). Do NOT hardcode AK/SK or read them explicitly. ``` ALIBABA_CLOUD_REGION # optional, defaults to cn-hangzhou ``` ## Step 1: Intent Parsing and Resource Selection ### 1.1 Parse user intent Extract the following elements from the user's input: | Element | Description | Example | |---------|-------------|---------| | Task type | One-shot script / Long-running service / AI training | "deploy nginx" → long-running service | | Compute requirement | CPU / GPU / memory | "8 vCPU, 16 GB" | | Budget | Cost cap | "$50" | | Script / intent | Explicit script or task description | "a.sh" or "deploy an nginx site" | ### 1.2 Script generation (when no explicit script is provided) When the user provides intent rather than a script (e.g. "deploy an nginx site"), generate the script automatically. Key rules: - **Script-image coupling**: package managers depend on the OS — Ubuntu uses `apt-get`, CentOS/Alinux uses `yum`. Finalize the script only after the image is decided; if the image changes later, re-check script compatibility. - **Long-running service scripts** must use background/systemd commands (e.g. `systemctl start nginx`), not foreground-blocking ones. - **One-shot task scripts** simply exit when finished. ### 1.3 Resource selection **If the user explicitly specifies a product, use that product directly and skip selection comparison.** > **⛔ PRODUCT-LOCK RULE**: When the user explicitly specifies a product (e.g. "用 ECS", "use FC"), you are **locked to that product for the entire task**. If you encounter errors (out of stock, quota limits, etc.), you MUST retry within the same product — try different availability zones, regions, or instance types. **NEVER silently switch to a different product.** If all retries within the specified product are exhausted, report the failure to the user and ask for guidance — do NOT auto-switch. > > For ECS, use `ecs.find_available_instance_type()` to search across zones/regions for available stock and pricing, then after cost confirmation use `ecs.create_instance_with_infra()` to create the instance. When unspecified, follow the decision tree in [references/select-resource.md](references/select-resource.md): ``` User specified a product? → use it directly Long-running service? → ECS or ACK (FC / PAI-DLC are not suitable for long-running) AI / ML training? → PAI or FC (GPU) → if both viable, MUST compare in Step 1.5 K8s / containers? → ACK Multiple products viable? → MUST compare in Step 1.5 Default (single match) → ECS ``` > **⛔ ANTI-BIAS**: The decision tree only narrows candidates. When 2+ products remain, you **MUST proceed to Step 1.5** for real API-based comparison — never assume one is "obviously cheaper" from general knowledge. ### 1.4 Region selection — MANDATORY BEFORE resource creation > **⛔ HARD RULE**: Region selection MUST be performed explicitly as a documented step — not deferred to or assumed during resource creation. The chosen region directly affects network connectivity, package installation success, and end-to-end reliability. **Decision flow (execute in order):** 1. **Detect external dependency requirements** — scan the script (user-provided or agent-generated) and the task intent for signals that the workload will access overseas sources at runtime: - Package managers pulling from default mirrors: `pip install`, `npm install`, `apt-get install`, `yum install`, `go get`, `cargo build`, `gem install`, `composer install` - Downloads from GitHub, Docker Hub, PyPI, npmjs.com, Hugging Face, or other overseas hosts - `curl` / `wget` to non-Chinese URLs - Git clone from github.com / gitlab.com 2. **Apply region rule:** | Condition | Region | Rationale | |-----------|--------|-----------| | Script installs external dependencies from overseas sources (pip, npm, apt, GitHub, etc.) | **Overseas region** (prefer `ap-southeast-1` Singapore) | Domestic regions have poor/unstable connectivity to overseas package registries, causing timeouts and failures | | Task deploys a website/service with no overseas dependencies | Domestic region (e.g. `cn-hangzhou`, `cn-shanghai`) | Lower latency for end users | | AI training downloading models/datasets from Hugging Face, GitHub, etc. | **Overseas region** | Model downloads from China often timeout | | No external network access needed (pure compute, local data) | Domestic region (e.g. `cn-hangzhou`) | Default, lowest latency | | User explicitly specified a region | User's specified region | Respect user choice | > **Pitfall**: deploying a website seems "domestic", but if the setup script runs `npm install` / `pip install`, the packages come from overseas — choose an overseas region. **Always check the script's dependency commands, not just the service purpose.** 3. **Output the chosen region and reason** to the user before proceeding: ``` Region: ap-southeast-1 (Singapore) Reason: The task requires installing packages via pip/npm from overseas sources. Domestic regions may cause installation timeouts. ``` ### 1.5 Multi-option parallel comparison — MANDATORY SUB-AGENT DISPATCH > **⛔ HARD RULE**: When the user has NOT explicitly specified a product AND the decision tree yields more than one candidate, you **MUST** launch parallel sub-agents — one per candidate product. It is strictly forbidden to compare in the main thread using documentation knowledge or heuristics alone. **Dispatch rules:** 1. **One sub-agent per candidate product** — launch them in parallel (e.g. one for ECS, one for FC). 2. **Each sub-agent MUST call real APIs** — instance-type queries (`DescribeInstanceTypes`), inventory checks (`DescribeAvailableResource`), and pricing queries (`DescribePrice` or product-specific formulas). Memorized prices are NOT acceptable. 3. **Return structured results** — format defined in the "Sub-agent task template" in [references/select-resource.md](references/select-resource.md). 4. **Main agent aggregates and presents** — build the comparison table (template in [references/select-resource.md](references/select-resource.md)), recommend the best option, and wait for user confirmation. **Comparison dimensions** (all required): end-to-end time, estimated cost (from API), complexity, resource cleanup. When uncertain about API usage, search the docs with `scripts/doc_search.py`: ```python from doc_search import search_and_format print(search_and_format("DescribeInstanceTypes", product="ecs")) ``` ## Step 2: Create Compute Resources After selecting a product, **read its reference doc** (linked below) for full API usage — especially function signatures and parameter constraints — then create resources. Use the region from Step 1.4; if Step 1.4 is not yet done, go back and complete it first. | Product | Reference | Workflow summary | |---------|-----------|------------------| | ECS | [references/ecs.md](references/ecs.md) | `find_available_instance_type()` → **cost confirmation** → `create_instance_with_infra()` (VPC/SG/image handled internally) | | FC | [references/fc.md](references/fc.md) | choose spec → **cost confirmation** → create function → invoke function | | ACK | [references/ack.md](references/ack.md) | choose node spec → **cost confirmation** → VPC/SG → create cluster → submit K8s Job | | PAI | [references/pai.md](references/pai.md) | list_ecs_specs → choose CPU/GPU → **cost confirmation** → create_training_job | **Network preparation** (ACK only; ECS is handled by `create_instance_with_infra`): see [references/vpc.md](references/vpc.md) ### MANDATORY RULE: Cost confirmation > **⛔ HARD BLOCK**: Before calling ANY resource-creation API (`RunInstances`, `CreateFunction`, `CreateCluster`, `CreateTrainingJob`), you MUST estimate cost and get user confirmation. The agent may NOT self-approve — regardless of how low the cost is. **Flow:** 1. **Estimate cost** — use the product's pricing API or formula (see each product's reference doc). 2. **Output the cost estimate** using the template below — do not omit or summarize it. 3. **Wait for user confirmation** — stop and do nothing further until the user replies affirmatively (e.g. "yes", "ok", "确认"). Silence or implied consent do NOT count. 4. **Proceed** only after receiving confirmation. 5. If over budget — recommend a cheaper alternative, re-estimate, and repeat from step 2. > **Skip-confirmation exception**: if the user has **explicitly stated** in the current conversation that no confirmation is needed (e.g. "直接执行不用确认", "skip confirmation", "just do it, no need to ask"), then still output the cost estimate (step 2) for the record, but proceed immediately without waiting — skip steps 3-4. Cost display template: ``` Cost estimate: Spec: ecs.t6-c1m2.large (2 vCPU, 4 GB) Unit price: CNY 0.017 / hour Duration: ~5 minutes Total: CNY 0.002 Billing: PostPaid (pay-as-you-go) Proceed with creation? ``` > Exchange-rate reference: $1 ≈ CNY 7.2 ## Step 3: Execute the Script ### MANDATORY PRE-EXECUTION CHECK: Script & Resource Validation > **⛔ HARD BLOCK**: Before executing any script, the following validation steps are **required and non-skippable**. If validation fails, you MUST stop the flow and report the error to the user. **It is strictly forbidden to generate a placeholder/stub script, fabricate execution output, or silently proceed when a required file is missing.** **Validation flow (apply before every execution):** 1. **Determine script source type:** - **(A) User-provided script path** — the user referenced a specific file (e.g. `/home/user/train.py`, `./scripts/run.sh`). - **(B) User-provided script content** — the user pasted the script inline or its content is already in the conversation. - **(C) Agent-generated script** — no explicit script was provided; the agent generated one from intent (per Step 1.2). In this case, the agent already holds the full content — skip to step 3. 2. **For type (A) — verify file existence and content:** - **Local path**: use `Read` tool or `ls` / `cat` to confirm the file exists at the given path and is non-empty. If the file is on a remote instance (ECS), run the check via Cloud Assistant (`test -f <path> && wc -l <path>`). - **If the file does NOT exist or is empty**: immediately stop and report to the user: ``` ❌ Script not found: <path> The specified script file does not exist or is empty. Please verify the path and try again. ``` **Do NOT** create a replacement script, guess the content, or continue execution. - **If the file exists**: read its content to confirm it is a valid, complete script (not a stub or template with only comments/placeholders). 3. **Content completeness check** (for all source types): - The script must contain actual executable logic — not just comments, empty functions, or `pass`/`TODO` placeholders. - For training scripts (PAI / GPU tasks): verify the script references the expected framework entry points (e.g. `model.fit()`, `trainer.train()`, `torch.distributed.launch`). - If the content appears incomplete, ask the user for clarification before proceeding. 4. **Dependency & environment pre-check** (best effort): - If the script imports packages or references external data paths, note them so the execution environment can be prepared accordingly (e.g. `pip install` in the startup command, mount data volumes). > **Rationale**: creating compute resources costs money. Running a missing or placeholder script wastes that cost and misleads the user into thinking the task succeeded. ### Execution methods | Product | Task type | Call | |---------|-----------|------| | ECS | **One-shot** (run and release) | `ecs.run_command_and_cleanup(instance_id, script, infra=infra)` | | ECS | **Long-running** (keep alive) | `ecs.run_command_and_wait(instance_id, script)` | | FC | One-shot | `fc.create_and_invoke(script_path=path)` or `fc.create_and_invoke(script_content=code, script_type="shell")` | | ACK | K8s Job | `ack.run_script_as_job(cluster_id, script)` | | PAI | Training job | script is set at `create_training_job` time | > **⛔ ECS cleanup rule**: For **one-shot tasks**, you **MUST** use `run_command_and_cleanup()` with the `infra` parameter (from `create_instance_with_infra()`). This releases the instance + security group, and only deletes VSwitch/VPC if they were freshly created (shared resources are preserved). Forgetting to release ECS instances causes ongoing charges. > > Use `run_command_and_wait()` (without cleanup) **only** when the user explicitly needs the instance to stay running (e.g. "deploy a website", "keep the service online"). ## Error Handling The whole flow uses retry-with-adjustment: | Error | Strategy | |-------|----------| | Out of stock | Try in order: switch availability zone → switch region → downgrade instance type. For ECS use `find_available_instance_type(regions=[...])` which searches across regions automatically. **NEVER switch to a different product.** | | Quota exceeded | Prompt user to raise quota | | Over budget | Downgrade spec or shrink scale | | Script execution failed | Analyze the error, adjust environment / dependencies, then retry | | Unknown error | Search docs with `doc_search.search(error_message, product)` | Keep adjusting and retrying until the instance is created and the script is running. ## Reference Index | Document | Content | |----------|---------| | [references/select-resource.md](references/select-resource.md) | Comparison of the four products and selection decision tree | | [references/vpc.md](references/vpc.md) | VPC / VSwitch API quick reference | | [references/ecs.md](references/ecs.md) | Full ECS API quick reference (specs / inventory / pricing / creation / execution) | | [references/fc.md](references/fc.md) | FC API quick reference + script-packaging method | | [references/ack.md](references/ack.md) | ACK cluster API quick reference + K8s Job execution | | [references/pai.md](references/pai.md) | PAI-DLC training-job API quick reference + GPU spec table | | [references/ram-policies.md](references/ram-policies.md) | RAM 最小权限清单与 Policy JSON |

path: references/ack.md
content: # ACK (Container Service for Kubernetes) Reference

Script: `scripts/ack.py` | API style: ROA | Version: CS/2015-12-15

ACK is suited for containerized workloads and Kubernetes orchestration scenarios. Cluster creation takes a long time (5-15 minutes), so **prefer reusing existing clusters**.

Dependency: `pip install kubernetes` (Job execution uses the K8s Python SDK; no local kubectl install required).

## End-to-end workflow

```
1. Decide node specs
→ 2. Cost confirmation (ECS price query + Pro edition management fee)
→ 3. Network preparation (VPC)
→ 4. ensure_cluster (name match → reuse any available cluster → create + wait_nodes_ready)
→ 5. run_script_as_job (submit Job via K8s SDK, dynamic image resolution)
→ 6. cleanup_resources (reverse order: cluster → wait → SG)
```

> `create_and_run` wraps steps 3-6 to complete the full flow in a single call.
>
> **Key**: step 4 not only waits for the cluster to be running, it also waits for nodes to become Ready, so Jobs can actually be scheduled.

## Cost confirmation (mandatory)

**Before creating a cluster you must estimate the cost and confirm with the user. Do not call create_cluster without confirmation.**

ACK itself **has no price-query API**. ACK cost = cluster management fee + Worker node ECS cost:
- Managed basic edition (ManagedKubernetes): cluster management fee is **free**
- Managed Pro edition: cluster management fee ~CNY 0.64/hour
- Worker node cost: estimate via ECS `describe_price`

```python
# Use the ECS pricing helper (see SKILL.md)

result = describe_price(
    instance_type="ecs.g7.xlarge",       # Worker node spec
    price_unit="Hour",
    instance_charge_type="PostPaid",
    system_disk_category="cloud_essd",
    system_disk_size=120,
)
hourly_per_node = result["PriceInfo"]["Price"]["TradePrice"]
total_hourly = hourly_per_node * num_of_nodes
```

**Determining cluster edition**: when reusing an existing cluster, check the `profile` field returned by `describe_cluster_detail`:
- `profile="Default"` → basic edition (management fee is free)
- `profile="XEnhance"` or user explicitly requests Pro → Pro edition (CNY 0.64/hour)

Cost display template (basic edition):
```
ACK cost estimate:
  Cluster type: ManagedKubernetes (managed basic edition, management fee free)
  Worker spec: ecs.g7.xlarge (4vCPU, 16GB) x 2 nodes
  Per-node price: CNY 0.84/hour
  Total node cost: CNY 1.68/hour
  System disk: cloud_essd 120GB x 2
  Estimated runtime: ~1 hour
  Estimated total cost: CNY 1.68
  Billing method: pay-as-you-go (PostPaid)

Proceed with creation?
```

Cost display template (Pro edition):
```
ACK cost estimate:
  Cluster type: ManagedKubernetes Pro (managed Pro edition)
  Cluster management fee: CNY 0.64/hour
  Worker spec: ecs.g7.xlarge (4vCPU, 16GB) x 2 nodes
  Per-node price: CNY 0.84/hour
  Total node cost: CNY 1.68/hour
  System disk: cloud_essd 120GB x 2
  Estimated runtime: ~1 hour
  Estimated total cost: CNY 2.32 (management CNY 0.64 + nodes CNY 1.68)
  Billing method: pay-as-you-go (PostPaid)

Proceed with creation?
```

## API quick reference

### ensure_cluster(...) -> str

**Recommended entry point**. Three-tier lookup: name match → `acf-` prefix cluster → create new. Returns cluster_id and **guarantees nodes are Ready**.

```python
from ack import ensure_cluster

cluster_id = ensure_cluster(
    cluster_name="acf-cluster",                   # used for name matching
    vpcid="vpc-xxx",                               # VPC ID (used when creating)
    vswitch_ids=["vsw-xxx"],                       # VSwitch ID list (used when creating)
    worker_instance_types=["ecs.g7.xlarge"],       # Worker node ECS spec
    num_of_nodes=1,                                # Worker node count
    reuse_any=False,                               # whether to search for available acf- prefixed clusters (default False)
    region=None,
)
# cluster_id -> "c-xxx" (may be a reused existing cluster)
```

Logic:
1. `describe_clusters(name=cluster_name)` looks up clusters with the same name
2. Same-name + running -> reuse directly after RBAC check passes
3. Same-name + initial -> `wait_cluster_running` + `wait_nodes_ready`
4. No same-name + `reuse_any=True` -> search running clusters with `acf-` prefix (concurrent probing, 3s timeout); reuse any cluster that has Ready nodes and RBAC permission via the K8s API
5. None of the above -> `create_cluster` + `wait_cluster_running` + `wait_nodes_ready`

> **Note**: `reuse_any` defaults to `False`, only same-name clusters are reused. Other `acf-` prefixed clusters are only searched when the user explicitly sets `reuse_any=True`. Clusters whose names do not start with `acf-` are never reused.

### wait_nodes_ready(cluster_id, min_nodes, region, timeout) -> int

Polls node Ready status via the K8s API. Returns the count of Ready nodes.

```python
from ack import wait_nodes_ready

ready_count = wait_nodes_ready(
    cluster_id="c-xxx",
    min_nodes=1,                                   # minimum number of Ready nodes required
    timeout=300,                                   # timeout in seconds
)
```

> `ensure_cluster` calls `wait_nodes_ready` internally, so manual invocation is usually unnecessary.

### describe_clusters(name, cluster_type, region) -> list[dict]

Lists clusters. Calls the `DescribeClustersV1` API (`GET /api/v1/clusters`).

```python
from ack import describe_clusters

clusters = describe_clusters(name="acf-cluster")
for c in clusters:
    print(f"{c['cluster_id']}: {c['name']} ({c['state']})")
```

### create_cluster(...) -> dict

Creates a managed ACK cluster. Returns `{cluster_id, task_id, request_id}`.

```python
from ack import create_cluster

result = create_cluster(
    cluster_name="acf-cluster",                   # required
    cluster_type="ManagedKubernetes",             # managed edition (recommended)
    vpcid="vpc-xxx",                               # required
    vswitch_ids=["vsw-xxx"],                       # required
    container_cidr="172.20.0.0/16",                # Pod CIDR
    service_cidr="172.21.0.0/20",                  # Service CIDR
    worker_instance_types=["ecs.g7.xlarge"],       # Worker ECS spec
    num_of_nodes=2,                                # node count
    worker_system_disk_category="cloud_essd",
    worker_system_disk_size=120,
    # security_group_id=None -> auto-create an enterprise security group, released with the cluster
    # login_password=None    -> auto-generate a random password
    region=None,
)
```

> **Security group**: when `security_group_id` is omitted an enterprise security group is auto-created (`is_enterprise_security_group=True`) and released automatically when the cluster is deleted, no manual cleanup needed.

> **Login credentials**: when neither `login_password` nor `key_pair` is provided, a 16-character random password is generated.

### wait_cluster_running(cluster_id, region, timeout=900) -> dict

Waits for the cluster to become ready. Default timeout is 15 minutes.

### describe_cluster_detail(cluster_id, region) -> dict

Returns cluster details. State values: `initial`, `running`, `failed`, `deleted`, `deleting`.

### run_script_as_job(...) -> str

**Runs a script in the cluster via the Kubernetes Python SDK**. Automatically resolves the in-cluster image registry, polls Pod status, and raises on errors.

```python
from ack import run_script_as_job

output = run_script_as_job(
    cluster_id="c-xxx",                            # required
    script_content="echo hello",                   # script content
    job_name="acf-job",                            # Job name
    script_type="shell",                           # "shell" or "python"
    # image=None -> dynamically resolve the image registry prefix from kube-system pods
    poll_interval=10,                              # status polling interval (seconds)
    timeout=600,                                   # maximum wait time (seconds)
    region=None,
)
# output -> "hello\n" (Job stdout)
```

Key features:
- **Dynamic image resolution**: extracts the in-cluster registry prefix from kube-system pods (e.g. `registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/`), guaranteeing the image is pullable from inside the cluster
- **Polling-based status checks**: checks every `poll_interval` seconds; raises immediately on fatal errors such as `ImagePullBackOff` or `CrashLoopBackOff`
- **Auto cleanup of stale Jobs**: if a Job with the same name exists, it is deleted and recreated
- **Auto Job reclamation**: sets `ttlSecondsAfterFinished=300`, so completed Jobs are cleaned up 5 minutes after completion

### delete_cluster(cluster_id, region) -> dict

Submits a cluster delete request (asynchronous). Use together with `wait_cluster_deleted`.

### wait_cluster_deleted(cluster_id, region, timeout=600) -> None

Waits for the cluster to be fully deleted. Polls until the state is `deleted` or the API returns 404.

### cleanup_resources(cluster_id, security_group_id, region) -> None

**Reverse-order resource cleanup**: delete cluster -> wait for deletion -> optionally delete security group.

```python
from ack import cleanup_resources

# Auto-created security groups need no manual cleanup (released with the cluster)
cleanup_resources(cluster_id="c-xxx")

# Externally provided security groups must be specified explicitly
cleanup_resources(cluster_id="c-xxx", security_group_id="sg-xxx")
```

### create_and_run(...) -> dict

**One-stop convenience entry point**. Runs the full flow automatically: VPC preparation -> cluster reuse/creation -> script execution -> auto cleanup.

```python
from ack import create_and_run

# Option 1: auto-create network (specify zone_id)
result = create_and_run(
    script_content="echo hello",
    cluster_name="acf-cluster",
    zone_id="cn-hangzhou-h",                       # auto-create VPC/VSwitch
    worker_instance_types=["ecs.t6-c1m2.large"],
    num_of_nodes=1,
    script_type="shell",
    auto_cleanup=True,                             # delete the cluster automatically after execution
)

# Option 2: use existing network
result = create_and_run(
    script_content="print('hello')",
    vpcid="vpc-xxx",
    vswitch_ids=["vsw-xxx"],
    script_type="python",
    auto_cleanup=False,                            # keep the cluster for later use
)
# result -> {"cluster_id": "c-xxx", "job_output": "hello\n"}
```

Key features:
- **Cluster reuse**: looks up an existing running cluster by name to avoid redundant creation
- **Automatic network preparation**: when `zone_id` is provided, the network is created automatically via the network preparation helpers (see SKILL.md)
- **Hands-off security group**: when no SG is provided, ACK auto-creates an enterprise security group that is released with the cluster
- **try/finally auto cleanup**: cleans up the cluster even if script execution fails (when `auto_cleanup=True`)

### Other APIs

- `describe_regions(region)` -> dict — query available regions
- `get_cluster_kubeconfig(cluster_id, region)` -> str — fetch kubeconfig YAML
- `create_cluster_node_pool(cluster_id, ...)` -> dict — create a node pool
- `describe_cluster_node_pools(cluster_id, region)` -> dict — list node pools

## Documentation search

When uncertain about parameters, encountering unknown error codes, or needing the latest API docs:

```python
from doc_search import search_and_format

print(search_and_format("CreateCluster managed cluster parameters", product="ack"))
print(search_and_format("node pool GPU scheduling", product="ack"))
```

ACK API reference: https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/developer-reference/api-cs-2015-12-15-overview

## Notes

- Cluster creation takes 5-15 minutes; prefer `ensure_cluster` to reuse same-name clusters; pass `reuse_any=True` explicitly to search other clusters
- When `reuse_any=True`, only `acf-` prefixed clusters are searched; clusters whose names do not start with `acf-` are never auto-reused
- Cluster probing uses a 3-second timeout, concurrency of 10, and inspects up to 100 candidates while validating RBAC permissions (batch/jobs)
- After the cluster is running, `ensure_cluster` calls `wait_nodes_ready` automatically and only returns once nodes are schedulable
- container_cidr and service_cidr must not overlap with the VPC CIDR
- When `security_group_id` is omitted, ACK auto-creates an enterprise security group that is released with the cluster
- Cluster cost mainly comes from Worker node ECS instances and is billed continuously even with no workload; the Pro edition adds a CNY 0.64/hour management fee
- Images are resolved dynamically from kube-system pods, no manual registry configuration needed
- When reusing an existing cluster, set `auto_cleanup=False` to avoid deleting a cluster that was not created by this run


path: references/ecs.md
content: # ECS Reference

Script: `scripts/ecs.py` | API style: RPC | Version: ecs/2014-05-26

ECS is the most general-purpose compute resource, suitable for all scenarios, and is the default choice.

## Region Selection Guidance

**If the task involves installing external dependencies (pip install, apt-get install, npm install, downloading GitHub assets via curl, etc.), prefer overseas regions** to avoid connectivity issues when accessing overseas sources from mainland China.

| Scenario | Recommended Region | Notes |
|----------|--------------------|-------|
| External dependencies required | `ap-southeast-1` (Singapore) | Smooth network; access to PyPI/npm/GitHub/Docker Hub works without issue |
| No external dependencies | `cn-hangzhou` (Hangzhou) and other mainland regions | Low latency, lower price |
| User-specified region | Per user request | — |

> When using an overseas region, the VPC/VSwitch must also be created in the same region (`ensure_vpc_and_vswitch(region="ap-southeast-1", zone_id=...)`).
> A public IP must also be allocated (`internet_max_bandwidth_out > 0`); otherwise the instance cannot reach external networks.

## Full Workflow

```
1. Spec recommendation/lookup -> 2. Inventory check -> 3. Price quote -> 4. Network preparation -> 5. Security group -> 6. Image lookup -> 7. Create instance -> 8. Run script -> 9. Resource cleanup
```

## API Quick Reference

### 1. Spec Recommendation / Lookup

#### describe_recommend_instance_type(...) -> dict

**Recommended entry point**. Recommends instance specs based on requirements (vCPU/memory), with results sorted by inventory, price, or newest products. Avoids manually filtering through a large set of specs.

```python
from ecs import describe_recommend_instance_type

result = describe_recommend_instance_type(
    cores=2,                              # vCPU count (used together with memory)
    memory=4.0,                           # Memory in GiB (used together with cores)
    # instance_type="ecs.g7.large",       # Or specify a spec to find alternatives (mutually exclusive with cores/memory)
    instance_family_level=None,           # EntryLevel | EnterpriseLevel | CreditEntryLevel
    instance_charge_type="PostPaid",      # PostPaid | PrePaid
    priority_strategy="InventoryFirst",   # InventoryFirst | PriceFirst | NewProductFirst
    zone_id=None,                         # Restrict to a specific zone
    system_disk_category="cloud_essd",    # Disk type filter
    max_price=None,                       # Maximum hourly unit price (spot only)
    instance_type_family=None,            # Restrict to instance families, e.g. ["ecs.g7", "ecs.c7"]
    region=None,
)
# result["Data"]["RecommendInstanceType"] -> [{InstanceType: {InstanceType, Cores, Memory, MemoryGB, ...}, ZoneId, Priority, ...}]
# Note: Memory is originally in MB; the function automatically adds a MemoryGB field (converted to GB). Use MemoryGB when displaying.
```

#### describe_instance_types(...) -> dict

Filters instance spec details by CPU/memory/GPU. Use this when you need to inspect CPU, memory, and GPU information for specific specs.

```python
from ecs import describe_instance_types

result = describe_instance_types(
    instance_type_family=None,   # Instance family, e.g. "ecs.g7"
    instance_types=None,         # Exact spec list, e.g. ["ecs.g7.xlarge"]
    min_cpu=8,                   # Minimum vCPU
    max_cpu=32,                  # Maximum vCPU
    min_memory=None,             # Minimum memory (GB)
    max_memory=None,             # Maximum memory (GB)
    gpu_amount=None,             # Number of GPU cards, e.g. 1
    region=None,
)
# result["InstanceTypes"]["InstanceType"] -> [{InstanceTypeId, CpuCoreCount, MemorySize, GPUAmount, ...}]
```

#### describe_instance_type_families(generation, region) -> dict

Lists instance families (e.g. ecs.g7 general-purpose, ecs.c7 compute-optimized). Returns `InstanceTypeFamilies.InstanceTypeFamily[]`.

### 2. Inventory Check

#### describe_regions(region) -> dict

Returns the list of all available regions.

#### describe_zones(region) -> dict

Returns the list of all available zones in the current region.

#### describe_available_resource(...) -> dict

Queries purchasable resources (specs/disk types/etc.) within a zone. **After picking a spec, you must call this to confirm inventory.**

```python
from ecs import describe_available_resource

result = describe_available_resource(
    destination_resource="InstanceType",  # InstanceType | SystemDisk | DataDisk | Network
    zone_id="cn-hangzhou-h",              # Optional, specifies a zone
    instance_type="ecs.g7.2xlarge",       # Optional, filters a specific spec
    instance_charge_type="PostPaid",      # PostPaid (pay-as-you-go) | PrePaid (subscription)
    io_optimized="optimized",             # Defaults to optimized
    region=None,
)
# result["AvailableZones"]["AvailableZone"] -> available resources grouped by zone
```

### 3. Price Quote and Cost Confirmation

#### describe_price(...) -> dict

Queries the real-time price of an instance configuration. **Must be called before creating an instance; show the cost to the user and obtain confirmation before proceeding.**

```python
from ecs import describe_price

result = describe_price(
    instance_type="ecs.g7.2xlarge",       # Required
    period=1,                             # Billing period
    price_unit="Hour",                    # Hour | Month | Year
    instance_charge_type="PostPaid",      # PostPaid | PrePaid
    system_disk_category="cloud_essd",    # System disk type
    system_disk_size=40,                  # System disk size (GB)
    internet_max_bandwidth_out=0,         # Public bandwidth (Mbps); 0 = no public IP, >0 = allocate public IP
    amount=1,                             # Number of instances
    region=None,
    zone_id=None,
)
# result["PriceInfo"]["Price"] -> {TradePrice, OriginalPrice, DiscountPrice, Currency}
```

#### Cost Confirmation Flow (Mandatory)

After getting a quote, follow the steps below; **do not call run_instances without user confirmation**:

1. Call `describe_price` to obtain the real-time unit price
2. Estimate the runtime based on task type
3. Compute estimated total cost = unit price * estimated runtime * number of instances
4. Present the cost breakdown to the user and wait for confirmation

```
ECS Cost Estimate:
  Spec: ecs.t6-c1m2.large (2 vCPU, 4 GB)
  Image: Ubuntu 24.04
  System disk: cloud_essd 40 GB
  Unit price: CNY 0.017/hour (trade price)
  Estimated runtime: ~5 minutes
  Estimated total cost: CNY 0.002
  Billing: pay-as-you-go (PostPaid), released after completion

Proceed with creation?
```

Runtime estimation reference:
- Simple shell commands (echo/ls/curl): < 1 minute
- Data processing/compilation: 5-30 minutes
- Model training: 1-24 hours
- Long-running services (Web/API): runs continuously, quoted hourly

### 4. Network Preparation

You must prepare a VPC and VSwitch first (see the network preparation steps in SKILL.md). Once you have `vpc_id` and `vsw_id`, proceed to the next step.

### 5. Security Group

#### setup_security_group(vpc_id, sg_name, ingress_rules, region) -> str

**Recommended entry point**. Creates a security group, opens inbound ports per application requirements, and applies a full-egress rule in one step. Returns `security_group_id`.

> WARNING: **Inbound rules must be set based on the application actually being deployed**: open 80/443 for websites; for management ports such as SSH, restrict `source_cidr_ip` to the operator's public IP (e.g. `1.2.3.4/32`); never expose database ports to `0.0.0.0/0`. When external access is not needed, pass `None` or `[]`.

```python
from ecs import setup_security_group

# Public website
sg_id = setup_security_group(vpc_id="vpc-xxx", ingress_rules=[
    {"port_range": "80/80"},
    {"port_range": "443/443"},
])

# Website + SSH only from the operator's IP
sg_id = setup_security_group(vpc_id="vpc-xxx", ingress_rules=[
    {"port_range": "80/80"},
    {"port_range": "443/443"},
    {"port_range": "22/22", "source_cidr_ip": "203.0.113.5/32"},
])

# Internal-only worker, no inbound traffic
sg_id = setup_security_group(vpc_id="vpc-xxx", ingress_rules=None)
```

Per-rule fields: `port_range` (required, e.g. `"80/80"`), `source_cidr_ip` (defaults to `0.0.0.0/0`), `ip_protocol` (defaults to `tcp`).

For finer-grained control, call the steps separately:
- `create_security_group(vpc_id, sg_name, description, region)` -> str (sg_id)
- `authorize_security_group(sg_id, ip_protocol, port_range, source_cidr_ip, region)` - inbound rule
- `authorize_security_group_egress(sg_id, ip_protocol, port_range, dest_cidr_ip, region)` - outbound rule

### 6. Image Lookup

#### describe_images(...) -> dict

```python
from ecs import describe_images

result = describe_images(
    image_name=None,                      # Exact image name
    os_type=None,                         # linux | windows
    image_family=None,                    # e.g. "acs:ubuntu_24_04_x64"
    instance_type=None,                   # Filter compatible images by spec
    region=None,
    page_size=20,
)
# result["Images"]["Image"] -> [{ImageId, ImageName, OSName, ...}]
```

**Prefer using image_family to retrieve the latest image precisely** - it returns a single image that is the latest version in that family.

Common image families (official names):

| OS | image_family |
|----|--------------|
| Alibaba Cloud Linux 3 LTS 64-bit (recommended) | `acs:alibaba_cloud_linux_3_2104_lts_x64` |
| Ubuntu 24.04 LTS 64-bit | `acs:ubuntu_24_04_x64` |
| Ubuntu 22.04 LTS 64-bit | `acs:ubuntu_22_04_x64` |
| Ubuntu 20.04 LTS 64-bit | `acs:ubuntu_20_04_x64` |
| CentOS 7.9 64-bit | `acs:centos_7_9_x64` |
| Anolis OS 8.10 ANCK 64-bit | `acs:anolis_8_10_anck_x64` |
| Debian 12 64-bit | `acs:debian_12_x64` |

> Note: image_family follows the format `acs:<os>_<version>_<arch>[_<variant>]`, and does not include suffixes such as `20G_alibase`.

### 7. Create Instance

#### Recommended two-step flow: find → confirm → create

**Step A: find_available_instance_type(...)** — searches across regions/zones to find an instance type with stock AND returns its pricing. Call this BEFORE cost confirmation.

```python
from ecs import find_available_instance_type

spec = find_available_instance_type(
    cores=2,
    memory=4,
    regions=["cn-hangzhou", "cn-shanghai", "cn-beijing"],
    # instance_type="ecs.c7.large",   # Optional: check a specific type
)
# spec = {
#   "instance_type": "ecs.c7.large",
#   "region": "cn-shanghai",
#   "zone_id": "cn-shanghai-b",
#   "cores": 2, "memory": 4.0,
#   "price_per_hour": 0.17, "currency": "CNY"
# }
```

**→ Now present the cost to the user and wait for confirmation (see Cost Confirmation rule).**

**Step B: create_instance_with_infra(...)** — creates VPC/VSwitch/SecurityGroup/Image and the instance. Call this AFTER user confirms the cost.

```python
from ecs import create_instance_with_infra

result = create_instance_with_infra(
    instance_type=spec["instance_type"],   # From step A
    zone_id=spec["zone_id"],               # From step A
    region=spec["region"],                 # From step A
    image_family="acs:alibaba_cloud_linux_3_2104_lts_x64",
    internet_max_bandwidth_out=5,          # 0=no public IP, >0=with public IP
)
# result keys: instance_id, region, zone_id, instance_type,
#              vpc_id, vswitch_id, security_group_id, image_id,
#              vpc_is_new, vswitch_is_new
```

> When using these two functions, you do NOT need to manually call `ensure_vpc_and_vswitch`, `create_security_group`, `describe_images`, or `run_instances` — they are all handled internally by `create_instance_with_infra`.
>
> **⛔ Do NOT pass `image_id`/`ImageId` or `UserData`** to `create_instance_with_infra()` — these are managed internally and will raise `ValueError` if passed. Use `image_family` to select the OS, and `run_command()`/`run_command_and_wait()` for script execution after creation.

#### run_instances(...) -> dict (low-level)

Creates an ECS instance and waits until it reaches Running. Requires VPC, VSwitch, SecurityGroup, and ImageId to be prepared in advance. Use the two-step flow above instead unless you need fine-grained control.

> **⛔ UserData is forbidden**: `run_instances()` will raise `ValueError` if `UserData` is passed. To execute scripts on the instance, create the instance first, then use `run_command()` or `run_command_and_wait()` (see Section 8). This ensures script execution is observable and retriable.

```python
from ecs import run_instances

result = run_instances(
    instance_type="ecs.g7.2xlarge",       # Required
    vswitch_id="vsw-xxx",                 # Required
    security_group_id="sg-xxx",           # Required
    image_id="aliyun_3_x64_20G_alibase_xxx",  # Required
    instance_name="acf-instance",         # Instance name
    system_disk_category="cloud_essd",    # System disk type
    system_disk_size=40,                  # System disk size (GB)
    internet_max_bandwidth_out=0,         # Public bandwidth (Mbps); see note below
    instance_charge_type="PostPaid",      # PostPaid | PrePaid
    amount=1,                             # Number of instances
    region=None,
    # **extra_params accepts any additional RunInstances parameters (except UserData)
)
# result["InstanceIdSets"]["InstanceIdSet"] -> ["i-xxx"]
```

**Public IP allocation rules** (`internet_max_bandwidth_out` parameter):

| Value | Effect | Use case |
|-------|--------|----------|
| `0` (default) | **No public IP allocated** | Pure internal tasks, no external access required |
| `1~100` | **Public IP allocated automatically**, with the value as outbound bandwidth (Mbps) | Scenarios that require public network access |

Typical scenarios that require a public IP:
- Deploying a website / Web service (external users need access)
- Scripts that need `apt-get install`, `pip install`, or `curl` to download external resources
- The user wants to SSH directly to the instance

> Once a public IP is allocated, bandwidth is billed by traffic (`PayByTraffic`), based on actual usage.

#### describe_instances(instance_ids, instance_name, status, region, page_size) -> dict

Queries instance information. Returns `Instances.Instance[]`.

### 8. Run Script

#### run_command_and_cleanup(...) -> dict (ONE-SHOT TASKS)

**For one-shot tasks (run script then release).** Runs the script, collects output, then automatically releases the instance + security group. VSwitch and VPC are only deleted if they were freshly created (not shared/reused). Cleanup errors are logged but do not raise — the command result is always returned.

**Preferred usage** — pass the `infra` dict returned by `create_instance_with_infra()`:

```python
from ecs import run_command_and_cleanup

result = run_command_and_cleanup(
    instance_id=infra["instance_id"],
    command_content="#!/bin/bash\necho hello && date",
    # script_path="/path/to/a.sh",       # Alternative: read from local file
    timeout=600,
    infra=infra,                          # From create_instance_with_infra(); handles cleanup scope automatically
)
# result -> {ExitCode, DecodedOutput, ...}
# Instance + SG released; VPC/VSwitch released only if they were newly created
```

When `infra` is provided, the function reads `vpc_is_new` / `vswitch_is_new` flags to decide cleanup scope — reused VPC/VSwitch are preserved. You can also pass individual IDs manually if needed.

> **⛔ You MUST use this function for one-shot ECS tasks.** Using `run_command_and_wait()` for one-shot tasks will leave the instance running and incurring charges.

#### run_command_and_wait(instance_id, ...) -> dict (LONG-RUNNING SERVICES)

**For long-running services (keep the instance alive).** Runs the script and returns output, but does NOT release the instance. Use this only when the user needs the instance to keep running (e.g. web server, API service).

```python
from ecs import run_command_and_wait

result = run_command_and_wait(
    instance_id="i-xxx",
    script_path="/path/to/a.sh",          # Local script file path
    command_type="RunShellScript",
    timeout=600,
    region=None,
)
# result -> {ExitCode, Output(Base64), DecodedOutput, ErrorInfo, ...}
```

> When the user provides a script file (e.g. `a.sh`), prefer `script_path` to read the file content; do not manually hard-code the file content into `command_content`.

Cloud Assistant InvocationStatus state transitions: `Pending -> Running -> Success/Failed/Stopped`

For asynchronous execution, call the steps separately:
- `run_command(instance_id, command_content, command_type, timeout, region, script_path)` -> str (invoke_id)
- `describe_invocations(invoke_id, region)` -> dict - query execution status
- `describe_invocation_results(invoke_id, instance_id, region)` -> dict - retrieve output

### 9. Resource Cleanup

#### Cleanup Strategy (Decide by Scenario)

| Scenario | Strategy | Examples |
|----------|----------|----------|
| One-shot script | **Automatically release all resources** after execution | `echo hello`, data processing scripts, batch compute |
| Long-running service | **Do not release**; report the resource list and cost to the user | Web servers, API services, databases |
| Unclear scenario | **Ask the user** whether to release | Dev/test environments, debugging tasks |

Decision logic:
1. Does the script start a persistent process (e.g. `nohup`, `systemctl`, `docker run -d`, a Web framework `serve`)? -> Long-running, do not release
2. The user explicitly says "release after execution" or the script is a one-shot command -> Auto-release
3. Otherwise -> Ask the user

#### Cleanup Order (Reverse of Creation Order)

```
ECS instance -> Security group -> VSwitch -> VPC
```

For each step, describe to confirm resource state, delete, then describe again to verify deletion:

```python
from ecs import describe_instances, delete_instances, delete_security_group

# 1. Confirm the instance exists and check its state, then release
instances = describe_instances(instance_ids=["i-xxx"])
# Release (built-in retry; automatically handles states such as IncorrectInstanceStatus.Initializing)
delete_instances(instance_ids=["i-xxx"], force=True)
# Verify the release completed (instance disappears or state becomes Deleted)
describe_instances(instance_ids=["i-xxx"])

# 2. Delete the security group (only possible after instances are released)
delete_security_group(security_group_id="sg-xxx")

# 3. Delete the VSwitch / VPC (see the network cleanup steps in SKILL.md)
```

> **Do not delete reused resources**: `create_instance_with_infra()` returns `vpc_is_new` / `vswitch_is_new` flags. Only delete resources where the flag is `True`. If using `run_command_and_cleanup()` with the `infra` parameter, this is handled automatically.
> **Handling delete failures**: if a delete call fails (e.g. resources still have dependencies), use describe to inspect the relationships, resolve the dependencies, then retry.

#### Resource Summary

Regardless of whether resources are released, output a resource list to the user after the script finishes:

```
Created resources:
  - ECS: i-xxx (ecs.t6-c1m2.large, PostPaid ~CNY 0.017/h) [released/running]
  - SecurityGroup: sg-xxx [released/retained]
  - VSwitch: vsw-xxx [released/retained (reused)]
  - VPC: vpc-xxx [released/retained (reused)]
```

## Spec Lookup Strategy Selection

| Scenario | Recommended API | Notes |
|----------|-----------------|-------|
| Known requirements (vCPU + memory) | `describe_recommend_instance_type(cores, memory)` | Auto-recommends and sorts |
| Pick the cheapest by price | `describe_recommend_instance_type(priority_strategy="PriceFirst")` | Sorted by per-vCPU hourly price |
| Find which zones have stock for a given spec | `describe_available_resource(instance_type=...)` | Inventory grouped by zone |
| Inspect detailed spec parameters | `describe_instance_types(instance_types=[...])` | View CPU/memory/GPU details |

## Instance Family Quick Reference

| Family | Ratio | Suitable For |
|--------|-------|--------------|
| g (general) | 1:4 (vCPU:memory) | Web, middleware |
| c (compute) | 1:2 | Batch compute, high concurrency |
| r (memory) | 1:8 | Databases, big data |
| t (burstable) | Baseline + credits | Dev/test |
| gn (GPU) | Includes GPU | AI inference/training |

## Documentation Search

When you encounter uncertain parameters, unknown error codes, or need to consult the latest API docs, use `scripts/doc_search.py` to search the official Alibaba Cloud documentation:

```python
from doc_search import search_and_format

# Search ECS-related docs
print(search_and_format("RunInstances parameter reference", product="ecs"))
print(search_and_format("DescribeImages ImageFamily", product="ecs"))
print(search_and_format("Cloud Assistant InvocationStatus states", product="ecs"))

# Search by error code
print(search_and_format("OperationDenied.NoStock", product="ecs"))
```

Search results return titles, summaries, and links; use the web_fetch tool to inspect the full document content.

## Common Error Handling

| Error | Cause | Resolution |
|-------|-------|------------|
| OperationDenied.NoStock | Insufficient inventory | Switch to another zone or spec; use describe_recommend_instance_type to find alternatives |
| InvalidInstanceType.NotSupported | The zone does not support this spec | Confirm with describe_available_resource |
| QuotaExceed | Quota exceeded | Prompt the user to request a quota increase |
| InvalidSystemDiskCategory.NotSupported | The disk type is unavailable | Switch to cloud_essd or cloud_ssd |


path: references/fc.md
content: # FC (Function Compute) Reference

Script: `scripts/fc.py` | API style: ROA | Version: FC/2023-03-30

FC is suited for short-lived, Serverless compute tasks. Billed by invocation count plus execution duration; short tasks are the most cost-effective.

The Account ID is fetched automatically via STS GetCallerIdentity, so no manual configuration is required.

## End-to-end Workflow

```
1. Cost confirmation -> 2. create_and_invoke(script_path/script_content) -> automatic steps:
   wrap script (base64) -> create function (with timestamped name) -> [enable asyncTask config when async]
   -> invoke (auto Sync/Async) -> wait for result -> auto cleanup (async config + function)
```

> **Invocation strategy**: timeout <= 60s triggers a sync invocation automatically; timeout > 60s triggers an async invocation plus GetAsyncTask polling.
> Using `create_and_invoke` or `invoke_function_and_wait` hides the sync/async details: asyncTask config, polling fault tolerance, and resource cleanup are all handled for you.

## API Quick Reference

### create_function(...) -> dict

Create a function. Code is supplied through `code_inline` (most common), `code_dir`, or `code_zip_path`.

```python
from fc import create_function

result = create_function(
    function_name="acf-task",             # required, function name
    runtime="python3.10",                 # python3.10 | python3.9 | nodejs18 | custom | ...
    handler="index.handler",              # entry point: module.function_name
    code_inline=script_content,           # inline code string (auto-zipped)
    code_dir=None,                        # local directory path (auto-zipped)
    code_zip_path=None,                   # path to a pre-built zip file
    memory_size=512,                      # memory in MB (128~32768); see ratio constraint below
    cpu=0.35,                             # vCPU (0.05~16); see ratio constraint below
    timeout=300,                          # timeout in seconds (max 86400 for async)
    disk_size=512,                        # disk in MB (512 | 10240)
    gpu_memory_size=None,                 # GPU memory in MB (requires a GPU instance)
    environment_variables=None,           # dict of environment variables
    description="Created by Alibaba Cloud Compute Provision skill",
    region=None,
    account_id=None,
)
```

> **⛔ Memory/CPU ratio constraint**: FC requires `Memory(GB) / CPU(core)` to be between **1 and 4**. The defaults (`memory_size=512`, `cpu=0.35` → ratio 1.43) are safe. If you change one, check the ratio. Common safe combos: `128MB/0.05cpu`, `512MB/0.35cpu`, `1024MB/0.5cpu`, `2048MB/1cpu`. Using `memory_size=128` with `cpu=0.35` will fail (ratio 0.36 < 1).

### invoke_function(...) -> dict

Invoke a function. **When timeout > 60 seconds the call is automatically switched to async.**

```python
from fc import invoke_function

# Short task (<=60s) -> auto sync invocation, returns the result directly
result = invoke_function(
    function_name="acf-task",
    payload={"key": "value"},             # event data (str or dict)
    timeout=30,                           # <=60 -> Sync, >60 -> Async (auto-detected)
    # invocation_type="Sync",             # may also be set manually to override auto-detection
    log_type="Tail",                      # None | Tail (returns execution log, Sync only)
    qualifier=None,
    region=None,
    account_id=None,
)

# Long task (>60s) -> auto async invocation, returns {taskId, invocationType: "Async"}
result = invoke_function(
    function_name="acf-task",
    payload={"key": "value"},
    timeout=600,                          # >60 -> auto async
    async_task_id="my-custom-id",         # optional, custom task ID (auto-generated if omitted)
)
# result["taskId"] -> used for get_async_task lookups
```

### invoke_function_and_wait(...) -> dict

**Recommended entry point.** Invokes a function and waits for the result. Short tasks return synchronously; long tasks switch to async automatically and poll GetAsyncTask.

For async invocations the following are handled automatically:
1. `put_async_invoke_config` is called automatically to enable asyncTask (a precondition for GetAsyncTask).
2. A 3-second wait before polling (to absorb task registration latency).
3. `AsyncTaskNotFound` errors are retried automatically without breaking the polling loop.

```python
from fc import invoke_function_and_wait

result = invoke_function_and_wait(
    function_name="acf-task",
    payload={"input": "data"},
    timeout=600,                          # function timeout in seconds (>60 -> auto async)
    poll_interval=5,                      # async polling interval in seconds
    region=None,
    account_id=None,
)
```

### get_async_task(function_name, task_id, ...) -> dict

Query the status of an async task. Status values: `Running` | `Succeeded` | `Failed` | `Stopped` | `Expired` | `Retrying`.

```python
from fc import get_async_task

task = get_async_task("acf-task", "my-task-id")
# task -> {taskId, status, startedTime, endTime, durationMs, taskErrorMessage, returnPayload, ...}
```

### create_and_invoke(...) -> dict

**Recommended one-stop entry point.** Automatically wraps the script, generates a function name, creates the function, invokes and waits, and cleans up.

```python
from fc import create_and_invoke

# Option 1: pass a script file path directly (recommended; auto-detects .sh / .py and base64-wraps it)
result = create_and_invoke(
    script_path="/path/to/script.sh",     # auto read, base64-wrap, infer type
    # function_name=None,                 # if omitted, auto-generated: acf-task-20260423-143052-a1b2c3
    memory_size=512,
    cpu=0.35,
    timeout=60,
    # auto_cleanup=True,                  # delete_function is called automatically after execution by default
)

# Option 2: pass raw script content (auto base64-wrapped)
result = create_and_invoke(
    script_content="echo hello world",    # raw shell script
    script_type="shell",                  # "shell" | "python"
    timeout=60,
)

# Option 3: pass already-wrapped handler code (wrapping is skipped when `def handler(` is detected)
result = create_and_invoke(
    script_content=handler_code,          # already contains def handler(event, context)
    timeout=600,                          # >60s -> auto async + poll-and-wait
    auto_cleanup=False,                   # keep the function instead of deleting it
)
```

| Parameter | Description |
|------|------|
| `function_name` | Function name. `None` triggers auto-generation `acf-task-{timestamp}-{uuid6}` |
| `script_path` | Local script file path; the file is read and wrapped according to its extension |
| `script_content` | Script content string; mutually exclusive with `script_path` |
| `script_type` | `"shell"` \| `"python"`; auto-inferred when `script_path` is provided |
| `memory_size` | Memory in MB. Default `512`. Must satisfy Memory(GB)/CPU ratio ∈ [1, 4] |
| `cpu` | vCPU cores. Default `0.35`. Must satisfy Memory(GB)/CPU ratio ∈ [1, 4] |
| `auto_cleanup` | Defaults to `True`; cleans up after execution (deletes the async config first, then the function) |

> **Tip**: For simple scripts, use the defaults (`memory_size=512, cpu=0.35`) — they are safe and cheap. Only increase memory/CPU for compute-heavy or memory-heavy tasks.

### wrap_shell_script(script, timeout) -> str

Embeds a shell script safely into the FC handler via **base64 encoding**, eliminating escaping issues with triple quotes, backticks, `$`, and other special characters. The subprocess timeout is derived from the FC timeout automatically (timeout - 20s safety margin). The result includes `elapsed_s` and `env` diagnostics.

```python
from fc import wrap_shell_script

handler_code = wrap_shell_script("echo 'hello $USER'", timeout=60)
# -> in handler: base64.b64decode("...").decode() then execute
# -> returns: {stdout, stderr, returncode, elapsed_s, env}
```

### wrap_python_script(script, timeout) -> str

Embeds a Python script into the FC handler via base64 encoding. stdout/stderr are captured through StringIO.

```python
from fc import wrap_python_script

handler_code = wrap_python_script("print('hello')\nresult = 1+1", timeout=60)
```

### create_script_executor(...) -> str

**Reusable executor.** Creates a generic function that can run different scripts repeatedly through the payload, without recreating the function each time.

```python
from fc import create_script_executor, invoke_function_and_wait, delete_function

# 1. Create the executor (once)
executor_name = create_script_executor(
    memory_size=1024, cpu=1, timeout=600,
)

# 2. Run different scripts via payload (reused many times)
r1 = invoke_function_and_wait(executor_name, payload={"script": "echo hello", "type": "shell"}, timeout=30)
r2 = invoke_function_and_wait(executor_name, payload={"script": "print(1+1)", "type": "python"}, timeout=30)

# The payload also accepts base64 (handy for scripts with special characters):
import base64
b64 = base64.b64encode("echo 'special $chars'".encode()).decode()
r3 = invoke_function_and_wait(executor_name, payload={"script_b64": b64, "type": "shell"}, timeout=30)

# 3. Delete when finished
delete_function(executor_name)
```

### put_async_invoke_config(function_name, ...) -> dict

Enables or configures async task tracking on a function. **This is a precondition for `GetAsyncTask`**; without it async tasks have no recorded state.

> `invoke_function_and_wait` and `create_and_invoke` already call this automatically, so manual use is rarely needed.

```python
from fc import put_async_invoke_config

put_async_invoke_config(
    function_name="acf-task",
    async_task=True,                      # enable async task tracking
    max_async_retry_attempts=0,           # number of retry attempts on failure (0 = no retry)
)
```

### delete_async_invoke_config(function_name, ...) -> dict

Deletes the async invocation config. `create_and_invoke` calls this automatically when `auto_cleanup=True`.

### Other APIs

- `get_function(function_name, region, account_id)` -> dict - retrieve function details
- `list_functions(prefix, limit, region, account_id)` -> dict - list functions
- `delete_function(function_name, region, account_id)` -> dict - delete a function
- `list_regions()` -> list - return the list of commonly used available regions

## Wrapping a Script as a handler

When `create_and_invoke` is called with `script_path` or raw `script_content`, the wrapping is **performed automatically**, so no manual handler construction is required.

If manual wrapping is needed, use `wrap_shell_script` / `wrap_python_script`:

```python
from fc import wrap_shell_script, wrap_python_script

# Shell script - base64 safe encoding, no escaping issues
handler_code = wrap_shell_script('echo "hello $USER" && ls -la', timeout=60)

# Python script - captures stdout/stderr
handler_code = wrap_python_script('import math\nprint(math.pi)', timeout=60)
```

The wrapped handler returns a unified shape:
```json
{
    "stdout": "...",
    "stderr": "...",
    "returncode": 0,
    "elapsed_s": 1.234,
    "env": {"python": "3.10.x", "cwd": "/"}
}
```

> **Why base64**: embedding the script directly inside `"""script"""` breaks when the script contains triple quotes, `$`, or backticks.
> After base64 encoding all characters are safe; the handler restores the original via `base64.b64decode()` before executing it.

> **Timeout safety margin**: in `wrap_shell_script` / `wrap_python_script` the subprocess timeout is `FC_timeout - 20s`.
> For example, with FC timeout=120s the script effectively has 100s; the remaining 20s are reserved for handler initialization and result serialization.
> Mention this limit to the user during cost estimation.

## Cost Confirmation (Mandatory)

**Before creating a function you must estimate costs and obtain user confirmation; do not call create_function without it.**

FC is billed by invocation count plus execution duration (no describe_price API; estimate using public unit prices):

| Billing item | Unit price |
|--------|------|
| Invocations | CNY 0.0133 per 10,000 calls (first 1,000,000 per month free) |
| vCPU time | CNY 0.00003167 per vCPU-second |
| Memory time | CNY 0.00000397 per GB-second |
| GPU time | Charged separately based on GPU memory |

Estimation formula:
```
estimated_cost = vCPU * exec_seconds * CNY 0.00003167 + memory_GB * exec_seconds * CNY 0.00000397 + invocation_cost
```

Cost display template:
```
[FC cost estimate]
  Spec: 0.35 vCPU, 512MB memory
  Estimated execution time: ~10 seconds
  vCPU cost: 0.35 x 10 x CNY 0.00003167 = CNY 0.00011
  Memory cost: 0.5 x 10 x CNY 0.00000397 = CNY 0.00002
  Invocation cost: CNY 0 (within free quota)
  Estimated total: CNY 0.00013
  Billing mode: pay-as-you-go, stops automatically once execution finishes

Proceed with creation?
```

> FC short-task costs are extremely low, but estimation and disclosure to the user are still required.

## Sync vs Async Invocation

| Aspect | Sync | Async |
|--------|-----------|------------|
| Maximum timeout | 600 seconds (10 minutes) | 86400 seconds (24 hours) |
| Auto-switch threshold | timeout <= 60s | timeout > 60s |
| Return mode | Returns the result directly | Returns a taskId; poll via GetAsyncTask |
| Required configuration | None | Requires `PutAsyncInvokeConfig(asyncTask=true)` (handled automatically) |
| SDK body_type | `json` (parses JSON response) | `none` (async returns an empty body, not parsed) |
| Logs | Retrieved via x-fc-log-type: Tail | Retrieved via the logTail field within GetAsyncTask events |
| Use cases | Quick computations, simple scripts | Long-running training, batch jobs, data processing |

> `invoke_function_and_wait` handles all of the differences above (asyncTask config, body_type, polling fault tolerance) and is the recommended way to invoke functions.

### Internal Async Invocation Flow

When managing async invocations manually, keep in mind (`invoke_function_and_wait` / `create_and_invoke` already wrap all of this):

```
1. put_async_invoke_config(fn, asyncTask=True)   <- required, otherwise GetAsyncTask returns 404
2. invoke_function(fn, invocation_type="Async")   <- returns taskId
3. time.sleep(3)                                  <- task registration has some latency
4. poll get_async_task(fn, taskId)                <- tolerate AsyncTaskNotFound and retry
5. stop on status Succeeded/Failed/Stopped/Expired
6. cleanup: delete_async_invoke_config -> delete_function
```

## Resource Limits

| Dimension | Limit |
|------|------|
| vCPU | 16 max |
| Memory | 32 GB max |
| Sync invocation timeout | 600 seconds max (10 minutes) |
| Async invocation timeout | 86400 seconds max (24 hours) |
| Code package size | <= 50 MB compressed |
| Disk | 512 MB or 10 GB |

## Documentation Search

When you encounter unfamiliar parameters, unknown error codes, or need the latest API documentation, use `scripts/doc_search.py` to search the official Alibaba Cloud docs:

```python
from doc_search import search_and_format

# Search FC-related docs
print(search_and_format("CreateFunction runtime supported list", product="fc"))
print(search_and_format("InvokeFunction async invocation", product="fc"))
print(search_and_format("Function Compute GPU instance", product="fc"))

# Search by error code
print(search_and_format("ResourceExhausted concurrency limit", product="fc"))
```

The search returns titles, summaries, and links; use the web_fetch tool to view the full documentation content.

## Choosing a Usage Pattern

| Scenario | Recommended API | Notes |
|------|----------|------|
| Run a one-off script | `create_and_invoke(script_path=...)` | Auto wrap, create, configure asyncTask, execute, and clean up |
| Run multiple different scripts on the same function | `create_script_executor` + `invoke_function_and_wait` | Create once, reuse via payload, asyncTask managed automatically |
| Fine-grained control over function lifecycle | `create_function` + `put_async_invoke_config` + `invoke_function` + `get_async_task` + `delete_function` | Manual management; for async you **must** enable asyncTask first |

## Common Error Handling

| Error | Cause | Resolution |
|------|------|------|
| FunctionNotFound | The function does not exist | Verify that function_name is correct |
| FunctionAlreadyExists | Function name conflict | Use auto-naming (`function_name=None`) or delete the existing function first |
| AsyncTaskNotFound | GetAsyncTask returns 404 | asyncTask config is not enabled or the task is not registered yet. `invoke_function_and_wait` already handles this (enables config + retry tolerance) |
| Empty body JSON parse failure | Async invocation/DELETE returns an empty body | SDK body_type should be `"none"` rather than `"json"`. Already handled inside `invoke_function` and `delete_function` |
| ResourceExhausted | Concurrency limit exceeded | Wait and retry, or request a quota increase |
| RequestTimeout | Execution timed out | Increase timeout (>60s switches to async automatically) |
| CodeSizeExceed | Code package too large | Trim the code or deploy via OSS |


path: references/pai.md
content: # PAI (DLC Deep Learning Container) Reference

Script: `scripts/pai.py` | API style: ROA | Version: pai-dlc/2020-12-03

PAI-DLC is purpose-built for AI training/inference and supports distributed training frameworks such as PyTorch, TensorFlow, and MPI.

## End-to-End Workflow

```
1. Analyze task requirements (does it need a GPU?)
2. list_ecs_specs to query available specs -> pick the most suitable CPU/GPU spec
3. Cost confirmation
4. create_training_job (auto-resolves the image)
5. wait_job_complete (built-in ImagePullBackOff detection + auto log dump when stuck)
6. Inspect results
7. cleanup_jobs to clean up failed jobs when needed
```

**Key principle**: analyze first, query specs next, then choose on demand. Do not default to GPU. Simple scripts (shell, data processing, non-ML) only need a CPU spec.

PAI differs from ECS/FC/ACK: the script is passed directly into CreateJob via the `user_command` parameter, with no separate "run script" step.

## Step 1: Analyze the Task & Query Specs

### Decide Whether a GPU Is Needed

| Needs GPU | Does Not Need GPU |
|-----------|-------------------|
| PyTorch/TensorFlow model training | Shell scripts, data processing |
| CUDA compute workloads | CPU-bound compute |
| Large-model inference / fine-tuning | Simple test jobs |

### Query Available Specs

```python
from pai import list_ecs_specs

# Query CPU specs
cpu_specs = list_ecs_specs(accelerator_type="CPU")
for s in cpu_specs[:5]:
    print(f"{s['InstanceType']}: {s['Cpu']}C {s['Memory']}GB")

# Query GPU specs
gpu_specs = list_ecs_specs(accelerator_type="GPU")
for s in gpu_specs[:5]:
    print(f"{s['InstanceType']}: {s['Cpu']}C {s['Memory']}GB "
          f"{s['Gpu']}x{s.get('GpuType','')} {s.get('GpuMemory','')}GB")
```

Returned fields: `InstanceType, AcceleratorType, Cpu, Gpu, GpuType, Memory, GpuMemory, IsAvailable, SpotStockStatus, PaymentTypes`

### Spec Selection Guide

- **Simple scripts / tests**: CPU spec, e.g. `ecs.c6.large` (2C 4GB)
- **Data processing**: CPU spec, e.g. `ecs.c6.xlarge` (4C 8GB)
- **Small-model inference / fine-tuning**: `ecs.gn6i-c4g1.xlarge` (1x T4 16GB)
- **Mid-size model training**: `ecs.gn6v-c8g1.2xlarge` (1x V100 16GB)
- **Large-model fine-tuning**: `ecs.gn7i-c32g1.8xlarge` (4x A10 24GB)
- **Large-model pre-training**: `ecs.ebmgn7i.32xlarge` (8x A100 80GB)

## Step 2: Cost Confirmation (Mandatory)

**Before creating a training job you MUST estimate cost and confirm with the user. Do not call create_job / create_training_job without confirmation.**

PAI-DLC public resource groups are billed using ECS specs; estimate via the ECS `describe_price` API:

```python
from ecs import describe_price

result = describe_price(
    instance_type="ecs.c6.large",  # use the ecs_spec returned from list_ecs_specs
    price_unit="Hour",
    instance_charge_type="PostPaid",
    system_disk_category="cloud_essd",
    system_disk_size=120,
)
hourly_per_worker = result["PriceInfo"]["Price"]["TradePrice"]
total_hourly = hourly_per_worker * worker_count
```

Cost display template:
```
PAI-DLC cost estimate:
  Job type: PyTorchJob
  Spec: ecs.c6.large (2vCPU 4GB, CPU) x 1 Worker
  Per-worker price: CNY 0.17/hour
  Estimated runtime: ~2 minutes
  Estimated total cost: < CNY 0.01
  Billing: pay-as-you-go, billing stops automatically when the job finishes

Proceed with creation?
```

> Lingjun / resource-quota mode pricing depends on the specific quota plan; refer to the PAI console pricing page. For public resource groups, use the ECS price inquiry API.

## API Quick Reference

### list_ecs_specs(...) -> list[dict]

Query the ECS specs supported by PAI-DLC. **Always call this before creating a job to confirm available specs.**

```python
from pai import list_ecs_specs

# CPU specs only
specs = list_ecs_specs(accelerator_type="CPU")

# GPU specs only, sorted by GPU count
specs = list_ecs_specs(accelerator_type="GPU", sort_by="GPU", order="asc")

# All specs
specs = list_ecs_specs()
```

Parameters:
- `accelerator_type`: `"CPU"` | `"GPU"` | `None` (all)
- `sort_by`: `"CPU"` | `"GPU"` | `"Memory"` | `"GmtCreateTime"`
- `order`: `"asc"` | `"desc"`

### resolve_image(...) -> str

Dynamically resolve the best image URI. **You do not need to specify an image manually; it is queried automatically based on `job_type` and `chip_type`.**

```python
from pai import resolve_image

# CPU image
cpu_img = resolve_image("PyTorchJob", chip_type="CPU")

# GPU image
gpu_img = resolve_image("PyTorchJob", chip_type="GPU")
```

Parameters:
- `job_type`: `PyTorchJob` | `TFJob` | `MPIJob` | `XGBoostJob` | `RayJob`
- `chip_type`: `"GPU"` | `"CPU"`

Image selection strategy:
1. Query the aiworkspace ListImages API (`system.supported.dlc=true,system.chipType={chip_type}`)
2. Filter by framework name prefix (e.g. `pytorch:`)
3. Prefer standard images (exclude accl/ngc/deepEp variants)
4. For CPU prefer images whose tag contains "cpu"; for GPU prefer images whose tag contains "gpu"
5. Results are cached to avoid repeated queries

### Workspace Auto-Management

The `workspace_id` parameter is optional in every job-creation function:
- If omitted, `ListWorkspaces` is called automatically to detect existing workspaces
- If the account has no workspace at all, **a workspace named `alibabacloud_compute_default` is created automatically** (simple mode, production environment only)
- Results are cached so subsequent calls do not repeat the request

### create_training_job(...) -> dict

**Recommended entry point.** Creates a training job, automatically building the JobSpec and inferring the image from the spec. Returns `{JobId, RequestId}`.

```python
from pai import create_training_job

# CPU job (simple script)
result = create_training_job(
    display_name="test-script",
    user_command="bash /root/a.sh",
    job_type="PyTorchJob",
    ecs_spec="ecs.c6.large",        # CPU spec -> CPU image is selected automatically
    worker_count=1,
)

# GPU job (model training)
result = create_training_job(
    display_name="qwen-7b-finetune",
    user_command="python train.py",
    job_type="PyTorchJob",
    ecs_spec="ecs.gn6i-c4g1.xlarge", # GPU spec -> GPU image is selected automatically
    worker_count=1,
)
# result["JobId"] -> "dlc-xxx"
```

Image inference logic: detects whether `ecs_spec` contains `.gn` or `.ebmgn` to decide CPU vs GPU,
then calls `resolve_image()` to fetch a matching image. You can also override via the `image` parameter.

Parameters:
- `display_name`: job name (required)
- `user_command`: launch command (required)
- `job_type`: `PyTorchJob` | `TFJob` | `MPIJob` | `XGBoostJob` | `RayJob`
- `image`: Docker image (None means auto-resolve)
- `worker_count`: number of worker replicas
- `ecs_spec`: ECS spec (for public resource groups)
- `gpu/gpu_type/cpu/memory`: for resource-quota mode; pick either these or `ecs_spec`
- `workspace_id`: workspace ID (auto-detected)
- `resource_id`: resource group ID (empty = public resource pool)
- `envs`: dict of environment variables
- `thirdparty_libs`: list of pip packages

**Resource specification modes**:
- Public resource group: use `ecs_spec` to pick the ECS instance type
- Lingjun / resource quota: use a combination of `cpu`, `memory`, `gpu`, `gpu_type`

### create_pytorch_job(...) -> dict

PyTorch-specific convenience entry point. Same parameters as `create_training_job`, but `job_type` is fixed to `PyTorchJob`.

### create_job(...) -> dict

Low-level API. You must build `job_specs` manually. Prefer `create_training_job` instead.

```python
from pai import create_job, build_job_spec

specs = [build_job_spec(
    spec_type="Worker",
    image="...",   # obtain via resolve_image()
    pod_count=4,
    ecs_spec="ecs.gn6i-c4g1.xlarge",
)]

result = create_job(
    display_name="custom-job",
    job_type="PyTorchJob",
    user_command="torchrun --nproc_per_node=1 train.py",
    job_specs=specs,
)
```

### build_job_spec(...) -> dict

Builds a single JobSpec dict. `image` is required.

### get_job(job_id, region) -> dict

Fetch job details. Returned fields include: `JobId, Status, SubStatus, DisplayName, Duration, Pods, ReasonMessage, ...`

Status values: Creating, Queuing, Running, Succeeded, Failed, Stopped

### wait_job_complete(job_id, ...) -> dict

Wait for the job to complete. **Built-in safeguards:**

1. **Active ImagePullBackOff detection**: once an image-pull error is detected, the job is auto-terminated with an error after `image_pull_timeout` (default 300s)
2. **Stall auto-diagnosis**: if the job status does not change for more than `interval * 4` seconds, pod events and logs are fetched automatically to print diagnostics
3. **Timeout protection**: auto-terminate beyond `timeout` (default 7200s)
4. **Failure auto-diagnosis**: when the job is Failed/Stopped, pod logs and events are dumped automatically

```python
from pai import wait_job_complete

detail = wait_job_complete(
    "dlc-xxx",
    timeout=7200,
    interval=30,
    image_pull_timeout=300,
)
print(detail["Status"])  # Succeeded
```

### Logs and Events

```python
from pai import get_job_logs, get_pod_logs, get_pod_events

# Recommended: iterates over all pods automatically; no need to know the pod ID
all_logs = get_job_logs(job_id="dlc-xxx")
# Returns dict: {"dlc-xxx-master-0": ["line1", "line2", ...], ...}

# Fetch a single pod's logs (obtain the pod ID via get_job() first)
logs = get_pod_logs(job_id="dlc-xxx", pod_id="dlc-xxx-master-0")

# Fetch pod system events
events = get_pod_events(job_id="dlc-xxx", pod_id="dlc-xxx-master-0")
```

### Other APIs

- `list_jobs(workspace_id, display_name, status, page_number, page_size, region)` -> dict
- `stop_job(job_id, region)` -> dict (HTTP POST)
- `delete_job(job_id, region)` -> dict
- `list_images(labels, name, page_size, region)` -> list[dict]

### cleanup_jobs(...) -> list[str]

Clean up failed/abnormal jobs.

```python
from pai import cleanup_jobs

# Clean up all Failed jobs (only stops those still running)
affected = cleanup_jobs(status="Failed")

# Clean up and delete
affected = cleanup_jobs(status="Failed", delete=True)

# Clean up Stopped jobs
affected = cleanup_jobs(status="Stopped", delete=True)
```

## Error Handling

### Built-in Retries

The `_call` function automatically retries the following transient errors (up to 3 times, exponential backoff):
- Throttling (rate limiting)
- ServiceUnavailable
- InternalError
- OperationTimeout
- ETIMEDOUT / ConnectionReset

### Common Errors

| Error | Cause | Resolution |
|-------|-------|-----------|
| InvalidAction.NotFound (404) | Action name or HTTP method mismatch | Check the API docs |
| OperationForbidden (403) | Missing WorkspaceId | Code auto-detects; a workspace is created automatically when none exists |
| ImagePullBackOff | Image does not exist or is not accessible | Use resolve_image dynamic lookup to avoid this |
| Throttling | API call rate too high | Built-in automatic retry |

## Documentation Search

When uncertain about a parameter, encountering an unknown error code, or needing the latest API docs, use `scripts/doc_search.py` to search the Alibaba Cloud official documentation:

```python
from doc_search import search_and_format

print(search_and_format("CreateJob PyTorchJob parameters", product="pai"))
print(search_and_format("ListEcsSpecs instance specs", product="pai"))
print(search_and_format("DLC distributed training configuration", product="pai"))
```


path: references/ram-policies.md
content: # RAM 权限清单

## 本 Skill 所需权限（required_permissions）

以下按产品分组列出本 Skill 执行所需的全部 RAM 权限。每条权限标注类型（读 / 写）和用途。

如果开放更多权限，Skill 可能通过 search_and_format 找到合适的 API 进行调用，用以完成任务。

### ECS（Elastic Compute Service）

| 权限 | 类型 | 用途 |
|------|------|------|
| `ecs:DescribeRegions` | 读 | 查询可用地域列表 |
| `ecs:DescribeZones` | 读 | 查询地域下可用区列表 |
| `ecs:DescribeInstanceTypes` | 读 | 查询实例规格列表（vCPU、内存、GPU 等） |
| `ecs:DescribeInstanceTypeFamilies` | 读 | 查询实例规格族信息 |
| `ecs:DescribeRecommendInstanceType` | 读 | 获取推荐实例规格 |
| `ecs:DescribeAvailableResource` | 读 | 查询可用区库存与可售规格 |
| `ecs:DescribeImages` | 读 | 查询可用镜像列表 |
| `ecs:DescribePrice` | 读 | 查询实例价格（成本估算） |
| `ecs:DescribeInstances` | 读 | 查询已创建的实例信息 |
| `ecs:DescribeInvocations` | 读 | 查询 Cloud Assistant 命令执行状态 |
| `ecs:DescribeInvocationResults` | 读 | 查询 Cloud Assistant 命令执行结果 |
| `ecs:RunInstances` | 写 | 创建 ECS 实例 |
| `ecs:DeleteInstances` | 写 | 释放 ECS 实例 |
| `ecs:RunCommand` | 写 | 通过 Cloud Assistant 在实例上执行脚本 |
| `ecs:CreateSecurityGroup` | 写 | 创建安全组 |
| `ecs:AuthorizeSecurityGroup` | 写 | 配置安全组入方向规则 |
| `ecs:AuthorizeSecurityGroupEgress` | 写 | 配置安全组出方向规则 |
| `ecs:DeleteSecurityGroup` | 写 | 删除安全组（资源清理） |

### VPC（Virtual Private Cloud）

| 权限 | 类型 | 用途 |
|------|------|------|
| `vpc:DescribeVpcs` | 读 | 查询已有 VPC 列表 |
| `vpc:DescribeVSwitches` | 读 | 查询已有交换机列表 |
| `vpc:CreateVpc` | 写 | 创建 VPC |
| `vpc:DeleteVpc` | 写 | 删除 VPC（资源清理） |
| `vpc:CreateVSwitch` | 写 | 创建交换机 |
| `vpc:DeleteVSwitch` | 写 | 删除交换机（资源清理） |

### FC（Function Compute）

| 权限                             | 类型 | 用途 |
|--------------------------------|------|------|
| `fc:GetFunction`               | 读 | 查询函数详情 |
| `fc:ListFunctions`           | 读 | 列出已有函数 |
| `fc:CreateFunction`          | 写 | 创建函数 |
| `fc:DeleteFunction`          | 写 | 删除函数（资源清理） |
| `fc:InvokeFunction`          | 写 | 调用函数执行任务 |
| `fc:PutAsyncInvokeConfig`    | 写 | 配置异步调用参数 |
| `fc:DeleteAsyncInvokeConfig` | 写 | 删除异步调用配置（资源清理） |
| `fc:GetAsyncTask`            | 读 | 查询异步任务执行状态与结果 |

### STS（Security Token Service）

| 权限 | 类型 | 用途 |
|------|------|------|
| `sts:GetCallerIdentity` | 读 | 获取当前调用者身份（用于构建 FC 端点地址） |

### ACK（Container Service for Kubernetes）

| 权限 | 类型 | 用途 |
|------|------|------|
| `cs:DescribeRegions` | 读 | 查询 ACK 可用地域 |
| `cs:DescribeClustersV1` | 读 | 查询集群列表 |
| `cs:DescribeClusterDetail` | 读 | 查询集群详情与状态 |
| `cs:DescribeClusterNodePools` | 读 | 查询集群节点池列表 |
| `cs:DescribeClusterUserKubeconfig` | 读 | 获取集群 kubeconfig（用于 kubectl 操作） |
| `cs:CreateCluster` | 写 | 创建 ACK 集群 |
| `cs:DeleteCluster` | 写 | 删除 ACK 集群（资源清理） |
| `cs:CreateClusterNodePool` | 写 | 创建集群节点池 |

### PAI-DLC（Platform for AI — Deep Learning Containers）

| 权限 | 类型 | 用途 |
|------|------|------|
| `paidlc:ListEcsSpecs` | 读 | 查询 DLC 可用实例规格列表 |
| `paidlc:ListJobs` | 读 | 查询训练任务列表 |
| `paidlc:GetJob` | 读 | 查询训练任务详情与状态 |
| `paidlc:GetPodLogs` | 读 | 获取训练 Pod 日志 |
| `paidlc:GetPodEvents` | 读 | 获取训练 Pod 事件（诊断用） |
| `paidlc:CreateJob` | 写 | 创建训练任务 |
| `paidlc:DeleteJob` | 写 | 删除训练任务 |
| `paidlc:StopJob` | 写 | 停止运行中的训练任务 |

### PAI-Workspace

| 权限 | 类型 | 用途 |
|------|------|------|
| `paiworkspace:ListWorkspaces` | 读 | 查询已有工作空间列表 |
| `paiworkspace:ListImages` | 读 | 查询可用镜像列表（DLC 镜像） |
| `paiworkspace:CreateWorkspace` | 写 | 创建工作空间 |


path: references/select-resource.md
content: # Alibaba Cloud Compute Resource Selection Guide

## Product Overview

| Product | Full Name | Positioning | Billing Model | Startup Speed | Task Type |
|------|------|------|----------|----------|----------|
| ECS | Elastic Compute Service | General-purpose VM | Pay-as-you-go / Subscription | 1-3 minutes | One-shot + long-running |
| FC | Function Compute | Serverless function execution | Per invocation + execution duration | Sub-second cold start | **One-shot tasks only** |
| ACK | Container Service for Kubernetes | Container orchestration platform | Cluster management fee + node ECS fee | 5-15 minutes | One-shot + long-running |
| PAI | Platform for AI (DLC) | AI training / inference | Pay-as-you-go (public resource group) / Lingjun cluster | 1-5 minutes | **One-shot tasks only** |

> **Important**: FC and PAI-DLC are one-shot task engines and are not suitable for long-running scenarios such as website deployment or background services.
> Long-running services (e.g. nginx, databases, API services) can only run on ECS or ACK.

## Detailed Comparison

### ECS - Elastic Compute Service

**Pros:**
- The most general-purpose compute resource, suitable for all scenarios
- Rich instance families (general-purpose g, compute c, memory r, GPU, etc.)
- Full control over the operating system; can install any software
- Supports Cloud Assistant RunCommand to execute scripts remotely
- Mature pricing and inventory query APIs

**Cons:**
- Requires manual management of instance lifecycle
- Higher long-term cost than Serverless
- Requires manual configuration of network, security groups, and other infrastructure

**Use cases:**
- General compute tasks (compilation, data processing, web services)
- Tasks that need a full Linux environment
- Long-running background tasks
- Workloads that need a specific software environment (custom images)
- Default choice when the user does not specify a product

### FC - Function Compute

**Pros:**
- Serverless; no server management required
- Sub-second elastic scaling; billed only for actual usage
- Extremely cost-effective for short tasks (per invocation + vCPU-second + memory-second)
- Supports multiple runtimes (Python, Node.js, Go, Java, Custom Runtime)
- Supports GPU instances (fc.gpu.tesla.1, etc.)

**Cons:**
- Per-invocation execution time limits (synchronous up to 10 minutes, asynchronous up to 24 hours)
- Per-instance resource caps (vCPU <= 16, memory <= 32GB)
- Ephemeral filesystem; data is lost after restart
- Not suitable for workloads requiring persistent state
- ROA-style API; invocation pattern differs from ECS / VPC

**Use cases:**
- CPU-intensive short tasks (data transformation, image processing)
- Event-driven compute
- Budget-sensitive intermittent tasks
- Compute that does not require persistent storage
- Tasks with vCPU demand <= 16

**Not suitable for:**
- Website deployment (nginx, Apache, etc.)
- Long-resident background services (databases, message queues, etc.)
- Tasks that require a persistent filesystem

### ACK - Container Service for Kubernetes

**Pros:**
- Native Kubernetes orchestration capabilities
- Suitable for microservices and containerized workloads
- Elastic node pools support auto-scaling
- Rich ecosystem (Helm, Operator, Service Mesh)
- Supports GPU scheduling and heterogeneous compute

**Cons:**
- Cluster creation takes a long time (5-15 minutes)
- Cluster management fee applies (managed edition is free; pro edition is paid)
- Steep learning curve; requires Kubernetes knowledge
- Not suitable for one-shot short tasks
- Script execution requires kubectl exec or submitting a Job

**Use cases:**
- Containerized application deployment and orchestration
- Services that require auto-scaling
- Microservice architectures
- CI/CD pipelines
- Container workloads that need GPUs but are not AI training

### PAI (DLC) - Platform for AI

**Pros:**
- Designed specifically for AI training / inference
- Supports distributed training (TFJob, PyTorchJob, MPIJob, etc.)
- Lingjun clusters provide high-performance GPUs (A100, H100, etc.)
- Built-in data source mounting and code source management
- Automatically handles multi-node multi-GPU communication

**Cons:**
- Suitable only for AI / ML workloads
- Lingjun clusters require reserved resources, so cost is higher
- Public resource group GPUs may queue
- Does not support general-purpose compute tasks

**Use cases:**
- Large model training (LLM, CV, NLP)
- Distributed multi-node multi-GPU training
- GPU-intensive AI tasks
- Model fine-tuning
- Scenarios requiring high-end GPUs such as A100 / H100

**Not suitable for:**
- Website deployment, web services
- Non-AI/ML general-purpose compute tasks
- Long-running service processes

## Selection Decision Tree

```
User intent
│
├─ Did the user explicitly specify a product (e.g. "use ECS", "use FC")?
│  └─ Yes -> Use the specified product directly; skip the selection comparison
│
├─ Is the task a long-running service (website, database, API service, etc.)?
│  ├─ Yes -> Only ECS or ACK (FC / PAI-DLC are not supported)
│  │       ├─ Need Kubernetes / container orchestration? -> ACK
│  │       └─ No -> ECS
│  └─ No (one-shot task)
│
├─ Does it involve AI/ML training or inference?
│  ├─ Yes -> Need multi-node multi-GPU / high-end GPU (A100+)?
│  │       ├─ Yes -> PAI (DLC)
│  │       └─ No -> Is a single GPU enough?
│  │               ├─ Yes, and the task is short -> FC (GPU instance)
│  │               └─ No -> PAI (DLC) or ECS (GPU family)
│  └─ No
│
├─ Is the task containerized / does it require Kubernetes?
│  ├─ Yes -> ACK
│  └─ No
│
├─ No clear rule restricts the choice; multiple options are viable?
│  └─ Yes -> Compare candidate options in parallel (see "Multi-option Comparison" below)
│
└─ Default -> ECS
```

## Multi-option Comparison — MANDATORY PARALLEL SUB-AGENT DISPATCH

When no clear rule narrows the choice to a single product (e.g. "run a simple shell script" could fit either ECS or FC),
you **MUST launch parallel sub-agents** (one per candidate product) to evaluate them. **Do NOT compare options in the main agent thread using documentation knowledge or heuristic judgment — each sub-agent must call real APIs (DescribeInstanceTypes, DescribePrice, etc.) to collect data.** Compare on the dimensions below:

| Dimension | Description |
|----------|------|
| Estimated end-to-end duration | End-to-end time from resource creation to script completion (including startup, configuration, network preparation, etc.) |
| Estimated cost | Estimated total cost under pay-as-you-go billing |
| Complexity | Number of resources to create and the number of configuration steps |
| Resource cleanup | Whether manual release is required and whether residual charges may occur |

**Implementation (mandatory)**: you MUST use sub-agents to evaluate each candidate option in parallel; each sub-agent independently calls the corresponding product's APIs (instance lookup, pricing, inventory, etc.),
then results are aggregated and compared. Performing this comparison in the main agent thread without sub-agents is strictly forbidden. Benefits:
1. **Faster** - multi-product research runs in parallel
2. **Context isolation** - per-product API calls and intermediate results do not pollute the main agent

Sub-agent task template:
```
Evaluate the feasibility of using {product} to run the following task:
Task description: {user intent}
Script content: {script}

Please complete:
1. Look up a suitable instance type (recommendation or list)
2. Confirm inventory availability
3. Estimate cost (via pricing API or formula)
4. Estimate end-to-end duration (creation + execution + cleanup)
5. List the resources that need to be created

Return format:
- Recommended instance type: ...
- Estimated cost: CNY ...
- End-to-end duration: ~... minutes
- Resources to create: ...
- Feasible: yes / no (explain the reason if not feasible)
```

Comparison result presentation template:
```
Option comparison:

| Dimension | ECS | FC |
|------|-----|-----|
| Instance type | ecs.t6-c1m2.large (2c4g) | 0.35vCPU, 512MB |
| Estimated cost | CNY 0.002 | CNY 0.00013 |
| End-to-end duration | ~3 minutes | ~15 seconds |
| Resources to create | VPC + VSwitch + security group + instance | Function |
| Resource cleanup | Must release 4 resources in reverse order | Cleaned up automatically |

Recommendation: FC (lower cost, faster, no infrastructure to manage)

Proceed with the recommended option?
```

## Budget Estimation Reference

| Product | Billing Unit | Approximate Price (cn-hangzhou) |
|------|----------|------------------------|
| ECS (ecs.c7.xlarge, 4c8g) | Pay-as-you-go / hour | ~CNY 0.5 / hour |
| ECS (ecs.g7.2xlarge, 8c32g) | Pay-as-you-go / hour | ~CNY 1.5 / hour |
| ECS (ecs.gn6i-c4g1.xlarge, T4 GPU) | Pay-as-you-go / hour | ~CNY 8 / hour |
| FC (1vCPU, 2GB) | vCPU-second + memory-second | ~CNY 0.0001 / second |
| FC (16vCPU, 32GB) | vCPU-second + memory-second | ~CNY 0.002 / second |
| ACK (managed cluster) | Cluster management fee | Free |
| ACK (pro cluster) | Cluster management fee | ~CNY 0.64 / hour |
| PAI DLC (public resource group, V100) | Pay-as-you-go | ~CNY 20 / hour |
| PAI DLC (Lingjun, A100x8) | Pay-as-you-go | ~CNY 150 / hour |

> Note: prices are reference values; actual prices depend on DescribePrice / the official website. When using this skill, fetch real-time prices via the pricing API.


path: references/vpc.md
content: # VPC Reference

Script: `scripts/vpc.py` | API style: RPC | Version: vpc/2016-04-28

Almost all products (ECS, ACK, etc.) depend on VPC and VSwitch. Make sure the network is ready before creating compute resources.

## API Quick Reference

### ensure_vpc_and_vswitch(zone_id, ...) → (vpc_id, vswitch_id, vpc_is_new, vswitch_is_new)

**Recommended entry point.** Automatically creates or reuses an existing VPC + VSwitch. **Auto-detects an available CIDR range, no manual conflict avoidance needed.**

Returns four values: the VPC/VSwitch IDs and two booleans indicating whether each was freshly created (`True`) or reused from an existing resource (`False`). Use the boolean flags to decide cleanup scope — only delete newly created resources.

```python
from vpc import ensure_vpc_and_vswitch

vpc_id, vsw_id, vpc_is_new, vsw_is_new = ensure_vpc_and_vswitch(
    zone_id="cn-hangzhou-h",           # required, target zone
    vpc_cidr="192.168.0.0/16",         # VPC CIDR, default 192.168.0.0/16
    vsw_cidr=None,                     # VSwitch CIDR, default None = auto-detect available /24 subnet
    vpc_name="acf-vpc",                # VPC name, used for reuse detection
    vsw_name="acf-vsw",                # VSwitch name
    region=None,                       # region, defaults to env
)
# vpc_is_new=False means the VPC was reused — do NOT delete it during cleanup
# vsw_is_new=True  means the VSwitch was freshly created — safe to delete
```

Logic:
1. Look up an existing VPC by vpc_name → reuse if found → otherwise create
2. Look up an existing VSwitch in the target zone → reuse if found
3. No usable VSwitch → **scan CIDRs of existing VSwitches in the VPC** and pick an unused /24 subnet (e.g. if 192.168.0.0/24 is taken, try 192.168.1.0/24, 192.168.2.0/24, ...)
4. You may also pass `vsw_cidr="192.168.10.0/24"` explicitly to skip auto-detection

### create_vpc(cidr_block, vpc_name, region, description) → dict

Create a VPC and wait for Available. Returns VPC details including `VpcId`.

### describe_vpcs(vpc_id, vpc_name, region, page_size) → dict

Query the VPC list. Returns `Vpcs.Vpc[]`.

### create_vswitch(vpc_id, zone_id, cidr_block, vswitch_name, region, description) → dict

Create a VSwitch and wait for Available. Returns VSwitch details including `VSwitchId`.

### describe_vswitches(vpc_id, vswitch_id, zone_id, region, page_size) → dict

Query the VSwitch list. Returns `VSwitches.VSwitch[]`.

### delete_vswitch(vswitch_id, region) → dict

Delete a VSwitch. The VSwitch must have no associated resources (release ECS instances, security groups, etc. first).

### delete_vpc(vpc_id, region) → dict

Delete a VPC. All VSwitches inside the VPC must be deleted first.

### Release Order

Delete network resources in reverse dependency order: `ECS instances → security groups → VSwitch → VPC`.

```python
from vpc import delete_vswitch, delete_vpc

delete_vswitch(vswitch_id="vsw-xxx")
delete_vpc(vpc_id="vpc-xxx")
```

> Use the `vpc_is_new` / `vsw_is_new` flags from `ensure_vpc_and_vswitch()` to decide: only delete resources where the flag is `True`. Reused resources (`False`) belong to other workloads and must NOT be deleted.

## Documentation Search

When you are unsure about a parameter, hit an unknown error code, or need the latest API docs, use `scripts/doc_search.py` to search the Alibaba Cloud official documentation:

```python
from doc_search import search_and_format

# Search VPC-related docs
print(search_and_format("CreateVpc parameter description", product="vpc"))
print(search_and_format("VSwitch zone constraints", product="vpc"))

# Search by error code
print(search_and_format("InvalidCidrBlock", product="vpc"))
```

Search results include title, summary, and link. Use the web_fetch tool to retrieve full document content.

## Notes

- A VSwitch must specify zone_id, and zone_id must match the zone of any compute resources created later
- Get zone_id via `ecs.describe_zones()` or `ecs.describe_available_resource()`
- VPC CIDR planning: use 10.0.0.0/16 for production; 192.168.0.0/16 or /24 for testing/personal use


path: scripts/ack.py
content: #!/usr/bin/env python3
"""Alibaba Cloud ACK (Container Service Kubernetes) management.

Capabilities:
  - describe_regions / describe_clusters  (query)
  - create_cluster / ensure_cluster       (create or reuse a cluster)
  - wait_cluster_running / wait_cluster_deleted  (wait for state)
  - run_script_as_job                     (submit a Job via the Kubernetes SDK)
  - delete_cluster / cleanup_resources    (reverse-order cleanup)
  - create_and_run                        (convenience entry, auto cleanup)

ACK uses ROA-style API (version=2015-12-15).
Endpoint: cs.{region}.aliyuncs.com

Job execution uses the ``kubernetes`` Python SDK (no kubectl dependency).
"""

from __future__ import annotations

import base64
import json
import sys
import os
import time
from typing import Any

sys.path.insert(0, os.path.dirname(__file__))

from common import call_roa_api, get_default_region, wait_until, pp

VERSION = "2015-12-15"


def _get_endpoint(region: str | None = None) -> str:
    r = region or get_default_region()
    return f"cs.{r}.aliyuncs.com"


def _call(
    pathname: str,
    method: str = "GET",
    query: dict | None = None,
    body: dict | None = None,
    region: str | None = None,
    action: str = "anonymous",
) -> dict | str:
    ep = _get_endpoint(region)
    return call_roa_api(
        version=VERSION,
        pathname=pathname,
        method=method,
        query=query,
        body=body,
        region=region,
        endpoint=ep,
        action=action,
    )


# ---------------------------------------------------------------------------
# Region query
# ---------------------------------------------------------------------------

def describe_regions(region: str | None = None) -> dict:
    """List ACK available regions."""
    return _call("/regions", region=region, action="DescribeRegions")


# ---------------------------------------------------------------------------
# Cluster query
# ---------------------------------------------------------------------------

def describe_clusters(
    name: str | None = None,
    cluster_type: str | None = None,
    region: str | None = None,
    page_size: int = 50,
    page_number: int = 1,
) -> list[dict]:
    """List ACK clusters with optional filters.

    Args:
        name: Filter by cluster name (exact match).
        cluster_type: Filter by type (ManagedKubernetes, etc.).

    Returns:
        List of cluster info dicts.
    """
    query: dict[str, Any] = {
        "page_size": str(page_size),
        "page_number": str(page_number),
    }
    if name:
        query["name"] = name
    if cluster_type:
        query["cluster_type"] = cluster_type

    result = _call(
        "/api/v1/clusters",
        query=query,
        region=region,
        action="DescribeClustersV1",
    )
    return result.get("clusters", [])


_K8S_PROBE_TIMEOUT = 3
_PROBE_CONCURRENCY = 10
_PROBE_MAX_CANDIDATES = 100


def _check_cluster_rbac(cluster_id: str, region: str | None = None) -> bool:
    """Verify that the current account has batch/jobs RBAC permission on the cluster."""
    try:
        batch_v1, _, _ = _build_k8s_clients(cluster_id, region)
        batch_v1.list_namespaced_job("default", _request_timeout=_K8S_PROBE_TIMEOUT)
        return True
    except Exception as e:
        err_msg = str(e)
        if "forbidden" in err_msg.lower() or "RBAC" in err_msg:
            print(f"  RBAC check failed for {cluster_id}: {e}")
        else:
            print(f"  Cannot reach cluster {cluster_id}: {e}")
        return False


def _probe_cluster(cluster_id: str, name: str, region: str | None) -> str | None:
    """Check if a single cluster is reusable (nodes ready + RBAC). Returns cluster_id or None."""
    try:
        batch_v1, core_v1, _ = _build_k8s_clients(cluster_id, region)
        nodes = core_v1.list_node(_request_timeout=_K8S_PROBE_TIMEOUT)
        ready = sum(
            1 for n in nodes.items
            for cond in (n.status.conditions or [])
            if cond.type == "Ready" and cond.status == "True"
        )
        if ready <= 0:
            return None
        batch_v1.list_namespaced_job("default", _request_timeout=_K8S_PROBE_TIMEOUT)
        print(f"Found reusable cluster: {cluster_id} ({name}), {ready} ready node(s)")
        return cluster_id
    except Exception as e:
        print(f"  Cluster {cluster_id} ({name}) not reusable: {e}")
        return None


def _find_reusable_cluster(
    cluster_type: str = "ManagedKubernetes",
    region: str | None = None,
    name_prefix: str = "acf-",
) -> str | None:
    """Find any running cluster with ready nodes, checked concurrently.

    Only considers clusters whose name starts with *name_prefix*.
    At most ``_PROBE_MAX_CANDIDATES`` clusters are checked, with
    ``_PROBE_CONCURRENCY`` probes running in parallel.

    Returns cluster_id if a suitable cluster is found, None otherwise.
    """
    from concurrent.futures import ThreadPoolExecutor, as_completed

    all_clusters = describe_clusters(cluster_type=cluster_type, region=region)
    candidates = [
        c for c in all_clusters
        if c.get("state") == "running"
        and int(c.get("size", 0)) > 0
        and c.get("name", "").startswith(name_prefix)
    ][:_PROBE_MAX_CANDIDATES]
    if not candidates:
        return None

    with ThreadPoolExecutor(max_workers=_PROBE_CONCURRENCY) as pool:
        futures = {
            pool.submit(_probe_cluster, c["cluster_id"], c.get("name", "?"), region): c
            for c in candidates
        }
        for future in as_completed(futures):
            result = future.result()
            if result:
                for f in futures:
                    f.cancel()
                return result
    return None


def ensure_cluster(
    cluster_name: str = "acf-cluster",
    cluster_type: str = "ManagedKubernetes",
    vpcid: str | None = None,
    vswitch_ids: list[str] | None = None,
    worker_instance_types: list[str] | None = None,
    num_of_nodes: int = 1,
    worker_system_disk_category: str = "cloud_essd",
    worker_system_disk_size: int = 120,
    region: str | None = None,
    reuse_any: bool = False,
    **extra,
) -> str:
    """Find an existing running cluster or create a new one.

    Search order:
    1. Exact name match (running or initializing)
    2. Any ``acf-`` prefixed running cluster with ready nodes (if reuse_any=True)
    3. Create a new cluster

    Returns the cluster_id of a running cluster with ready nodes.
    """
    # 1. Exact name match
    clusters = describe_clusters(name=cluster_name, cluster_type=cluster_type, region=region)
    for c in clusters:
        if c.get("state") == "running" and c.get("name") == cluster_name:
            cluster_id = c["cluster_id"]
            if _check_cluster_rbac(cluster_id, region):
                print(f"Reusing existing cluster (name match): {cluster_id} ({cluster_name})")
                return cluster_id
            print(f"  Cluster {cluster_id} ({cluster_name}) matched by name but lacks RBAC permissions, skipping")
        if c.get("state") == "initial" and c.get("name") == cluster_name:
            cluster_id = c["cluster_id"]
            print(f"Found initializing cluster: {cluster_id}, waiting...")
            wait_cluster_running(cluster_id, region)
            wait_nodes_ready(cluster_id, min_nodes=1, region=region)
            return cluster_id

    # 2. Any acf- prefixed running cluster with ready nodes
    if reuse_any:
        reused = _find_reusable_cluster(cluster_type=cluster_type, region=region)
        if reused:
            return reused

    # 3. Create new cluster
    result = create_cluster(
        cluster_name=cluster_name,
        cluster_type=cluster_type,
        vpcid=vpcid,
        vswitch_ids=vswitch_ids,
        worker_instance_types=worker_instance_types or ["ecs.g7.xlarge"],
        num_of_nodes=num_of_nodes,
        worker_system_disk_category=worker_system_disk_category,
        worker_system_disk_size=worker_system_disk_size,
        region=region,
        **extra,
    )
    cluster_id = result["cluster_id"]
    wait_cluster_running(cluster_id, region)
    wait_nodes_ready(cluster_id, min_nodes=max(1, num_of_nodes), region=region)
    return cluster_id


# ---------------------------------------------------------------------------
# Cluster management
# ---------------------------------------------------------------------------

def create_cluster(
    cluster_name: str,
    cluster_type: str = "ManagedKubernetes",
    kubernetes_version: str | None = None,
    vpcid: str | None = None,
    vswitch_ids: list[str] | None = None,
    container_cidr: str = "172.20.0.0/16",
    service_cidr: str = "172.21.0.0/20",
    worker_instance_types: list[str] | None = None,
    num_of_nodes: int = 2,
    worker_system_disk_category: str = "cloud_essd",
    worker_system_disk_size: int = 120,
    security_group_id: str | None = None,
    login_password: str | None = None,
    key_pair: str | None = None,
    endpoint_public_access: bool = False,
    region: str | None = None,
    **extra,
) -> dict:
    """Create an ACK managed cluster.

    If neither security_group_id, login_password, nor key_pair is provided,
    they are auto-generated (enterprise SG, random password).

    Args:
        endpoint_public_access: Whether the API server has a public endpoint.
            Defaults to False (private only). Set to True if the orchestrator
            runs outside the cluster VPC and needs to reach the API server.

    Returns:
        Dict with cluster_id, task_id, etc.
    """
    import secrets
    import string

    r = region or get_default_region()

    if not login_password and not key_pair:
        alphabet = string.ascii_letters + string.digits + "!@#$%"
        login_password = (
            secrets.choice(string.ascii_uppercase)
            + secrets.choice(string.ascii_lowercase)
            + secrets.choice(string.digits)
            + secrets.choice("!@#$%")
            + "".join(secrets.choice(alphabet) for _ in range(12))
        )

    body: dict[str, Any] = {
        "name": cluster_name,
        "region_id": r,
        "cluster_type": cluster_type,
        "container_cidr": container_cidr,
        "service_cidr": service_cidr,
        "num_of_nodes": num_of_nodes,
        "worker_system_disk_category": worker_system_disk_category,
        "worker_system_disk_size": worker_system_disk_size,
        "snat_entry": True,
        "endpoint_public_access": endpoint_public_access,
    }

    if login_password:
        body["login_password"] = login_password
    if key_pair:
        body["key_pair"] = key_pair

    if security_group_id:
        body["security_group_id"] = security_group_id
    else:
        body["is_enterprise_security_group"] = True

    if kubernetes_version:
        body["kubernetes_version"] = kubernetes_version
    if vpcid:
        body["vpcid"] = vpcid
    if vswitch_ids:
        body["vswitch_ids"] = vswitch_ids
        body["worker_vswitch_ids"] = vswitch_ids
    if worker_instance_types:
        body["worker_instance_types"] = worker_instance_types

    body.update(extra)

    result = _call("/clusters", method="POST", body=body, region=region, action="CreateCluster")
    cluster_id = result.get("cluster_id", "")
    print(f"Cluster creation started: {cluster_id}")
    return result


def describe_cluster_detail(cluster_id: str, region: str | None = None) -> dict:
    """Get cluster detail. state: initial, running, failed, deleted, deleting."""
    return _call(f"/clusters/{cluster_id}", region=region, action="DescribeClusterDetail")


def wait_cluster_running(cluster_id: str, region: str | None = None, timeout: int = 900) -> dict:
    """Wait until cluster reaches 'running' state (default 15 min)."""
    def _check():
        detail = describe_cluster_detail(cluster_id, region)
        return {"Status": detail.get("state", "unknown"), **detail}

    result = wait_until(
        _check,
        target_statuses={"running"},
        fail_statuses={"failed", "deleted", "delete_failed"},
        interval=30,
        timeout=timeout,
        status_key="Status",
    )
    print(f"Cluster {cluster_id} is running")
    return result


def wait_nodes_ready(
    cluster_id: str,
    min_nodes: int = 1,
    region: str | None = None,
    timeout: int = 300,
    interval: int = 15,
) -> int:
    """Wait until at least *min_nodes* K8s nodes are Ready.

    Uses the Kubernetes Python SDK (no kubectl needed).
    Returns the number of ready nodes.
    """
    batch_v1, core_v1, api_client = _build_k8s_clients(cluster_id, region)
    deadline = time.time() + timeout

    while time.time() < deadline:
        try:
            nodes = core_v1.list_node()
        except Exception as e:
            remaining = int(deadline - time.time())
            print(f"  [{remaining}s left] K8s API not reachable yet: {e}")
            time.sleep(interval)
            continue

        ready_count = 0
        for node in nodes.items:
            for cond in (node.status.conditions or []):
                if cond.type == "Ready" and cond.status == "True":
                    ready_count += 1
                    break

        remaining = int(deadline - time.time())
        if ready_count >= min_nodes:
            print(f"Nodes ready: {ready_count}/{len(nodes.items)}")
            return ready_count

        print(f"  [{remaining}s left] ready={ready_count}/{len(nodes.items)}, need {min_nodes}")
        time.sleep(interval)

    raise TimeoutError(
        f"Only {ready_count} of {min_nodes} required nodes ready after {timeout}s"
    )


def delete_cluster(cluster_id: str, region: str | None = None) -> dict:
    """Submit cluster deletion (async). Use wait_cluster_deleted to block."""
    return _call(f"/clusters/{cluster_id}", method="DELETE", region=region, action="DeleteCluster")


def wait_cluster_deleted(cluster_id: str, region: str | None = None, timeout: int = 600) -> None:
    """Wait until cluster is fully deleted."""
    deadline = time.time() + timeout
    while time.time() < deadline:
        try:
            detail = describe_cluster_detail(cluster_id, region)
            state = detail.get("state", "unknown")
        except Exception as e:
            err = str(e)
            if "NotFound" in err or "404" in err or "ErrorClusterNotFound" in err:
                print(f"Cluster {cluster_id} deleted")
                return
            raise
        if state in ("deleted",):
            print(f"Cluster {cluster_id} deleted")
            return
        if state in ("delete_failed",):
            raise RuntimeError(f"Cluster {cluster_id} deletion failed")
        print(f"  Cluster state: {state}, waiting 15s...")
        time.sleep(15)
    raise TimeoutError(f"Cluster {cluster_id} deletion timed out after {timeout}s")


def cleanup_resources(
    cluster_id: str,
    security_group_id: str | None = None,
    region: str | None = None,
    delete_timeout: int = 600,
) -> None:
    """Reverse-order resource cleanup: cluster → wait → optional SG.

    If security_group_id is None (cluster auto-created SG), the SG is
    automatically released with the cluster — no manual cleanup needed.
    """
    print(f"Deleting cluster {cluster_id}...")
    delete_cluster(cluster_id, region)
    wait_cluster_deleted(cluster_id, region, timeout=delete_timeout)

    if security_group_id:
        try:
            from ecs import delete_security_group
            print(f"Deleting security group {security_group_id}...")
            delete_security_group(security_group_id, region)
            print(f"Security group {security_group_id} deleted")
        except Exception as e:
            print(f"Warning: SG cleanup failed (may already be deleted): {e}")


# ---------------------------------------------------------------------------
# Node pool management
# ---------------------------------------------------------------------------

def create_cluster_node_pool(
    cluster_id: str,
    nodepool_name: str = "acf-nodepool",
    instance_types: list[str] | None = None,
    vswitch_ids: list[str] | None = None,
    desired_size: int = 2,
    system_disk_category: str = "cloud_essd",
    system_disk_size: int = 120,
    region: str | None = None,
    **extra,
) -> dict:
    """Create a node pool in an existing cluster."""
    body: dict[str, Any] = {
        "nodepool_info": {"name": nodepool_name},
        "scaling_group": {
            "instance_types": instance_types or ["ecs.g7.xlarge"],
            "vswitch_ids": vswitch_ids or [],
            "system_disk_category": system_disk_category,
            "system_disk_size": system_disk_size,
            "desired_size": desired_size,
        },
    }
    body.update(extra)

    result = _call(
        f"/clusters/{cluster_id}/nodepools",
        method="POST",
        body=body,
        region=region,
        action="CreateClusterNodePool",
    )
    nodepool_id = result.get("nodepool_id", "")
    print(f"NodePool created: {nodepool_id}")
    return result


def describe_cluster_node_pools(cluster_id: str, region: str | None = None) -> dict:
    """List node pools in a cluster."""
    return _call(f"/clusters/{cluster_id}/nodepools", region=region, action="DescribeClusterNodePools")


# ---------------------------------------------------------------------------
# Kubeconfig & Kubernetes SDK helpers
# ---------------------------------------------------------------------------

def get_cluster_kubeconfig(cluster_id: str, region: str | None = None, private: bool = False) -> str:
    """Get kubeconfig YAML string for a cluster.

    Args:
        private: If True, return private (VPC-internal) endpoint kubeconfig.
                 If False (default), return public endpoint kubeconfig.
    """
    result = _call(
        f"/k8s/{cluster_id}/user_config",
        query={"PrivateIpAddress": "true" if private else "false"},
        region=region,
        action="DescribeClusterUserKubeconfig",
    )
    return result.get("config", "")


def _build_k8s_clients(cluster_id: str, region: str | None = None):
    """Build Kubernetes API clients from cluster kubeconfig.

    Returns (BatchV1Api, CoreV1Api, api_client).
    """
    import yaml
    from kubernetes import client as k8s_client, config as k8s_config

    kubeconfig_str = get_cluster_kubeconfig(cluster_id, region)
    kubeconfig_dict = yaml.safe_load(kubeconfig_str)

    api_client = k8s_config.new_client_from_config_dict(kubeconfig_dict)
    batch_v1 = k8s_client.BatchV1Api(api_client)
    core_v1 = k8s_client.CoreV1Api(api_client)
    return batch_v1, core_v1, api_client


def _resolve_image(
    core_v1,
    script_type: str = "shell",
    explicit_image: str | None = None,
) -> str:
    """Dynamically resolve a container image from the cluster's own registry.

    Queries kube-system pods to extract the registry prefix that is
    guaranteed to be accessible from within the cluster, then maps
    script_type to a suitable base image.

    Falls back to Docker Hub images if registry detection fails
    (e.g. for overseas clusters that can access Docker Hub directly).
    """
    if explicit_image:
        return explicit_image

    image_map = {"shell": "acs/busybox:v1.29.2", "python": "acs/python:3.10-slim"}
    fallback_map = {"shell": "busybox:1.36", "python": "python:3.10-slim"}

    try:
        pods = core_v1.list_namespaced_pod("kube-system", limit=10)
        for pod in pods.items:
            for container in pod.spec.containers:
                img = container.image or ""
                if "/acs/" in img:
                    registry_prefix = img.split("/acs/")[0]
                    resolved = f"{registry_prefix}/{image_map.get(script_type, image_map['shell'])}"
                    print(f"Resolved image from cluster registry: {resolved}")
                    return resolved
    except Exception as e:
        print(f"Warning: failed to detect cluster registry, using fallback: {e}")

    return fallback_map.get(script_type, fallback_map["shell"])


# ---------------------------------------------------------------------------
# Script execution (via Kubernetes SDK)
# ---------------------------------------------------------------------------

_FATAL_POD_REASONS = {
    "ImagePullBackOff", "ErrImagePull", "InvalidImageName",
    "CreateContainerConfigError", "CrashLoopBackOff",
    "RunContainerError", "ErrImageNeverPull",
}


def _check_pod_fatal_errors(core_v1, job_name: str, namespace: str) -> str | None:
    """Check if any pod of the Job has a fatal container error.

    Returns an error description string, or None if no fatal errors.
    """
    pods = core_v1.list_namespaced_pod(
        namespace, label_selector=f"job-name={job_name}",
    )
    for pod in pods.items:
        for cs in (pod.status.container_statuses or []):
            if cs.state and cs.state.waiting:
                reason = cs.state.waiting.reason or ""
                if reason in _FATAL_POD_REASONS:
                    msg = cs.state.waiting.message or ""
                    return f"Pod {pod.metadata.name}: {reason} — {msg[:300]}"
    return None


def run_script_as_job(
    cluster_id: str,
    script_content: str,
    job_name: str = "acf-job",
    image: str | None = None,
    script_type: str = "shell",
    namespace: str = "default",
    region: str | None = None,
    poll_interval: int = 10,
    timeout: int = 600,
) -> str:
    """Execute a script as a K8s Job using the Kubernetes Python SDK.

    Dynamically resolves container image from cluster registry.
    Polls pod status; raises immediately on fatal errors
    (ImagePullBackOff, CrashLoopBackOff, etc.).

    Args:
        cluster_id: ACK cluster ID.
        script_content: Script content to execute.
        job_name: Kubernetes Job name.
        image: Container image override. Auto-resolved if None.
        script_type: "shell" or "python".
        namespace: Kubernetes namespace.
        region: Alibaba Cloud region.
        poll_interval: Seconds between status checks.
        timeout: Max seconds to wait.

    Returns:
        Job logs output string.
    """
    from kubernetes import client as k8s_client
    from kubernetes.client.rest import ApiException

    batch_v1, core_v1, api_client = _build_k8s_clients(cluster_id, region)

    resolved_image = _resolve_image(core_v1, script_type, image)
    interpreter = "/bin/sh" if script_type == "shell" else "python3"
    encoded_script = base64.b64encode(script_content.encode()).decode()

    job = k8s_client.V1Job(
        api_version="batch/v1",
        kind="Job",
        metadata=k8s_client.V1ObjectMeta(name=job_name, namespace=namespace),
        spec=k8s_client.V1JobSpec(
            backoff_limit=3,
            ttl_seconds_after_finished=300,
            template=k8s_client.V1PodTemplateSpec(
                spec=k8s_client.V1PodSpec(
                    restart_policy="Never",
                    containers=[k8s_client.V1Container(
                        name="runner",
                        image=resolved_image,
                        command=["/bin/sh", "-c",
                                 f"echo '{encoded_script}' | base64 -d | {interpreter}"],
                    )],
                ),
            ),
        ),
    )

    try:
        batch_v1.delete_namespaced_job(
            job_name, namespace, propagation_policy="Background",
        )
        time.sleep(3)
        print(f"Cleaned up previous Job '{job_name}'")
    except ApiException as e:
        if e.status != 404:
            raise

    print(f"Creating Job '{job_name}' (image: {resolved_image})...")
    batch_v1.create_namespaced_job(namespace, job)

    print(f"Polling Job status (interval={poll_interval}s, timeout={timeout}s)...")
    deadline = time.time() + timeout
    while time.time() < deadline:
        job_status = batch_v1.read_namespaced_job_status(job_name, namespace)
        conditions = job_status.status.conditions or []

        for cond in conditions:
            if cond.type == "Complete" and cond.status == "True":
                print(f"Job '{job_name}' completed successfully.")
                return _collect_job_logs(core_v1, job_name, namespace)
            if cond.type == "Failed" and cond.status == "True":
                logs = _collect_job_logs(core_v1, job_name, namespace)
                raise RuntimeError(f"Job '{job_name}' failed.\nLogs:\n{logs}")

        fatal = _check_pod_fatal_errors(core_v1, job_name, namespace)
        if fatal:
            raise RuntimeError(f"Job '{job_name}' fatal error: {fatal}")

        active = job_status.status.active or 0
        succeeded = job_status.status.succeeded or 0
        failed = job_status.status.failed or 0
        remaining = int(deadline - time.time())
        print(f"  [{remaining}s left] active={active} succeeded={succeeded} failed={failed}")
        time.sleep(poll_interval)

    raise TimeoutError(f"Job '{job_name}' timed out after {timeout}s")


def _collect_job_logs(core_v1, job_name: str, namespace: str) -> str:
    """Collect logs from all pods belonging to a Job."""
    pods = core_v1.list_namespaced_pod(
        namespace, label_selector=f"job-name={job_name}",
    )
    logs_parts = []
    for pod in pods.items:
        try:
            log = core_v1.read_namespaced_pod_log(
                pod.metadata.name, namespace, tail_lines=200,
            )
            logs_parts.append(log)
        except Exception:
            pass

    output = "\n".join(logs_parts)
    print(f"Job '{job_name}' output:\n{output}")
    return output


# ---------------------------------------------------------------------------
# Convenience entry
# ---------------------------------------------------------------------------

def create_and_run(
    script_content: str,
    cluster_name: str = "acf-cluster",
    vpcid: str | None = None,
    vswitch_ids: list[str] | None = None,
    zone_id: str | None = None,
    worker_instance_types: list[str] | None = None,
    num_of_nodes: int = 1,
    image: str | None = None,
    script_type: str = "shell",
    auto_cleanup: bool = True,
    endpoint_public_access: bool = False,
    region: str | None = None,
) -> dict:
    """Convenience: ensure cluster → run script → optional cleanup.

    If vpcid/vswitch_ids are not provided but zone_id is, automatically
    creates network resources via vpc.ensure_vpc_and_vswitch.

    Security group is auto-created by ACK (enterprise SG) and released
    with the cluster — no manual SG management needed.

    Args:
        script_content: Script to execute.
        cluster_name: Cluster name (used for reuse detection).
        vpcid: VPC ID. Auto-created if None and zone_id is given.
        vswitch_ids: VSwitch IDs.
        zone_id: Used to auto-create VPC/VSwitch if vpcid is None.
        worker_instance_types: ECS instance types for worker nodes.
        num_of_nodes: Number of worker nodes.
        image: Container image override.
        script_type: "shell" or "python".
        auto_cleanup: Delete cluster after execution.
        endpoint_public_access: Whether the cluster API server has a public
            endpoint. Defaults to False; pass True when the orchestrator runs
            outside the cluster VPC.
        region: Alibaba Cloud region.

    Returns:
        Dict with cluster_id and job_output.
    """
    if not vpcid and zone_id:
        from vpc import ensure_vpc_and_vswitch
        vpcid, vsw_id, _, _ = ensure_vpc_and_vswitch(zone_id=zone_id, region=region)
        vswitch_ids = [vsw_id]

    cluster_id = ensure_cluster(
        cluster_name=cluster_name,
        vpcid=vpcid,
        vswitch_ids=vswitch_ids,
        worker_instance_types=worker_instance_types,
        num_of_nodes=num_of_nodes,
        region=region,
        endpoint_public_access=endpoint_public_access,
    )

    try:
        job_output = run_script_as_job(
            cluster_id, script_content,
            image=image, script_type=script_type, region=region,
        )
        return {"cluster_id": cluster_id, "job_output": job_output}
    finally:
        if auto_cleanup:
            try:
                cleanup_resources(cluster_id, region=region)
            except Exception as e:
                print(f"Warning: cleanup failed: {e}")


path: scripts/bootstrap.py
content: #!/usr/bin/env python3
"""Environment bootstrap — zero third-party dependencies.

This module MUST only use the Python standard library so it can run
before any pip packages are installed.
"""

from __future__ import annotations

import subprocess
import sys

_MIN_PYTHON = (3, 8)

_REQUIRED_PACKAGES = [
    ("alibabacloud_credentials", "alibabacloud_credentials==1.0.8"),
    ("alibabacloud_tea_openapi", "alibabacloud_tea_openapi==0.4.4"),
    ("darabonba", "darabonba-core==1.0.5"),
]


def ensure_dependencies() -> None:
    """Check Python version and install missing pip packages.

    Call this once at the start of a session before any API work.
    Raises SystemExit if the Python version is too old.
    """
    if sys.version_info < _MIN_PYTHON:
        sys.exit(
            f"Python >= {_MIN_PYTHON[0]}.{_MIN_PYTHON[1]} is required, "
            f"but current version is {sys.version.split()[0]}."
        )

    missing: list[str] = []
    for import_name, pip_spec in _REQUIRED_PACKAGES:
        try:
            __import__(import_name)
        except ImportError:
            missing.append(pip_spec)

    if missing:
        print(f"Installing missing packages: {', '.join(missing)}")
        # only install packages in _REQUIRED_PACKAGES, these packages are Alibaba Cloud's official SDK.
        subprocess.check_call(
            [sys.executable, "-m", "pip", "install", *missing],
            stdout=subprocess.DEVNULL,
        )


path: scripts/common.py
content: #!/usr/bin/env python3
"""Shared utilities for Alibaba Cloud API calls."""

from __future__ import annotations

import json
import os
import time
import warnings
from typing import Any

warnings.filterwarnings("ignore", message="urllib3 v2 only supports OpenSSL")
warnings.filterwarnings("ignore", category=DeprecationWarning, module="urllib3")

from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_tea_openapi.client import Client as OpenApiClient
from darabonba.runtime import RuntimeOptions

USER_AGENT = "AlibabaCloud-Agent-Skills/alibabacloud-compute-provision"


def get_credential() -> CredentialClient:
    """Return a credential client using the default provider chain.

    The default chain automatically resolves credentials from environment
    variables, ~/.alibabacloud/credentials, ~/.aliyun/config.json, and
    instance RAM roles, in priority order.
    """
    return CredentialClient(None)


def get_default_region() -> str:
    """Return the default region (env override, otherwise cn-hangzhou)."""
    return os.environ.get("ALIBABA_CLOUD_REGION", "cn-hangzhou")


def build_client(endpoint: str, region: str | None = None) -> OpenApiClient:
    """Build an OpenAPI client for the given endpoint."""
    config = open_api_models.Config(
        credential=get_credential(),
        endpoint=endpoint,
        region_id=region or get_default_region(),
        user_agent=USER_AGENT,
    )
    return OpenApiClient(config)


def call_rpc_api(
    product: str,
    version: str,
    action: str,
    params: dict[str, Any] | None = None,
    region: str | None = None,
    endpoint: str | None = None,
) -> dict:
    """Call an Alibaba Cloud RPC-style API.

    Args:
        product: Product code (e.g. 'ecs', 'vpc').
        version: API version (e.g. '2014-05-26').
        action: API action name.
        params: Query parameters.
        region: Region ID. Falls back to env default.
        endpoint: Override endpoint. Auto-resolved if not given.
    """
    r = region or get_default_region()
    ep = endpoint or f"{product}.{r}.aliyuncs.com"
    client = build_client(ep, r)

    query = dict(params or {})
    query.setdefault("RegionId", r)
    serialized = _serialize(query)

    api_params = open_api_models.Params(
        action=action,
        version=version,
        protocol="HTTPS",
        pathname="/",
        method="POST",
        auth_type="AK",
        style="RPC",
        body_type="json",
        req_body_type="json",
    )
    request = open_api_models.OpenApiRequest(query=serialized)
    result = client.call_api(api_params, request, RuntimeOptions())
    return result.get("body", result)


def call_roa_api(
    version: str,
    pathname: str,
    method: str = "GET",
    query: dict[str, Any] | None = None,
    body: dict[str, Any] | None = None,
    region: str | None = None,
    endpoint: str | None = None,
    body_type: str = "json",
    action: str = "anonymous",
) -> dict | str:
    """Call an Alibaba Cloud ROA-style API.

    Args:
        version: API version.
        pathname: URL path (e.g. '/2023-03-30/functions').
        method: HTTP method.
        query: Query string parameters.
        body: Request body (for POST/PUT).
        region: Region ID.
        endpoint: API endpoint.
        body_type: Response body type — "json" (parse as JSON) or "none" (no parsing).
        action: API action name for gateway routing (some products like
            PAI-DLC require the real action name, not "anonymous").
    """
    if not endpoint:
        raise ValueError("endpoint is required for ROA API calls")

    r = region or get_default_region()
    client = build_client(endpoint, r)

    serialized_query = _serialize(query) if query else None

    api_params = open_api_models.Params(
        action=action,
        version=version,
        protocol="HTTPS",
        pathname=pathname,
        method=method,
        auth_type="AK",
        style="ROA",
        body_type=body_type,
        req_body_type="json",
    )
    request = open_api_models.OpenApiRequest(
        query=serialized_query,
        body=body,
    )
    result = client.call_api(api_params, request, RuntimeOptions())
    return result.get("body", result)


def wait_until(
    check_fn,
    target_statuses: set[str],
    fail_statuses: set[str] | None = None,
    interval: int = 10,
    timeout: int = 600,
    status_key: str = "Status",
) -> dict:
    """Poll check_fn until status reaches target or fails.

    Args:
        check_fn: Callable returning a dict with status info.
        target_statuses: Success statuses.
        fail_statuses: Failure statuses (raises on match).
        interval: Seconds between polls.
        timeout: Max wait seconds.
        status_key: Key to read status from check_fn result.

    Returns:
        The final check_fn result.
    """
    fail_statuses = fail_statuses or set()
    deadline = time.time() + timeout
    while time.time() < deadline:
        result = check_fn()
        status = result.get(status_key, "")
        if status in target_statuses:
            return result
        if status in fail_statuses:
            raise RuntimeError(f"Resource entered failure state: {status}. Detail: {json.dumps(result, ensure_ascii=False)}")
        print(f"  Current status: {status}, waiting {interval}s...")
        time.sleep(interval)
    raise TimeoutError(f"Timed out after {timeout}s waiting for status in {target_statuses}")


def pp(data: Any) -> str:
    """Pretty-print a dict/list as JSON string and print it."""
    s = json.dumps(data, ensure_ascii=False, indent=2)
    print(s)
    return s


def _serialize(params: dict[str, Any]) -> dict[str, str]:
    result: dict[str, str] = {}
    for k, v in params.items():
        if v is None:
            continue
        if isinstance(v, str):
            result[k] = v
        elif isinstance(v, bool):
            result[k] = "true" if v else "false"
        elif isinstance(v, (dict, list)):
            result[k] = json.dumps(v, ensure_ascii=False)
        else:
            result[k] = str(v)
    return result


path: scripts/doc_search.py
content: #!/usr/bin/env python3
"""Alibaba Cloud documentation search.

Search help.aliyun.com for product documentation.
Returns titles, summaries, and links.
"""

from __future__ import annotations

import json
import os
import sys
import urllib.request
import urllib.parse
from typing import Any

sys.path.insert(0, os.path.dirname(__file__))

from common import USER_AGENT

SEARCH_URL = "https://help.aliyun.com/help/json/search.json"
PAGE_SIZE = 10
TIMEOUT = 15

PRODUCT_CATEGORY_IDS = {
    "ecs": 25365,
    "vpc": 27706,
    "fc": 2508973,
    "ack": 85222,
    "pai": 30347,
    "ros": 28850,
    "oss": 31815,
    "rds": 26090,
    "slb": 27537,
}


def search(
    keywords: str,
    product: str | None = None,
    category_id: int | None = None,
    page_size: int = PAGE_SIZE,
    page_num: int = 1,
) -> list[dict[str, str]]:
    """Search Alibaba Cloud documentation.

    Args:
        keywords: Search keywords.
        product: Product shortname (ecs, vpc, fc, ack, pai, etc.) for scoped search.
        category_id: Explicit category ID. Overrides product if both given.
        page_size: Results per page.
        page_num: Page number.

    Returns:
        List of dicts with 'title', 'content', 'url' keys.
    """
    if not keywords.strip():
        raise ValueError("keywords cannot be empty")

    cid = category_id
    if cid is None and product:
        cid = PRODUCT_CATEGORY_IDS.get(product.lower())

    params: dict[str, Any] = {
        "keywords": keywords,
        "topics": "DOCUMENT,PRODUCT",
        "language": "zh",
        "website": "cn",
        "pageSize": page_size,
        "pageNum": page_num,
    }
    if cid is not None:
        params["categoryId"] = cid

    query_string = urllib.parse.urlencode(params)
    url = f"{SEARCH_URL}?{query_string}"

    req = urllib.request.Request(url, method="GET")
    req.add_header("User-Agent", USER_AGENT)

    try:
        with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
            raw = resp.read().decode("utf-8")
    except Exception as e:
        print(f"Search request failed: {e}")
        return []

    try:
        data = json.loads(raw)
    except json.JSONDecodeError:
        print("Failed to parse search response")
        return []

    if not data.get("success"):
        print("Search API returned failure")
        return []

    documents = data.get("data", {}).get("documents", {})
    items = documents.get("data", [])
    total = documents.get("totalCount", 0)

    results = []
    for item in items:
        results.append({
            "title": item.get("title", ""),
            "content": item.get("content", ""),
            "url": item.get("url", ""),
        })

    print(f"Found {len(results)} documents (total {total}) for: {keywords}")
    return results


def search_and_format(
    keywords: str,
    product: str | None = None,
    category_id: int | None = None,
) -> str:
    """Search and return a formatted string for display."""
    results = search(keywords, product, category_id)
    if not results:
        return f"No documents found for: {keywords}"

    lines = []
    for i, item in enumerate(results, 1):
        lines.append(f"{i}. {item['title']}")
        if item["content"]:
            lines.append(f"   {item['content']}")
        if item["url"]:
            lines.append(f"   Link: {item['url']}")
        lines.append("")

    lines.append(f"Use web_fetch tool to read full document content if needed.")
    return "\n".join(lines)


if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: doc_search.py <keywords> [product]")
        print("  product: ecs, vpc, fc, ack, pai, ros, oss, rds, slb")
        print()
        print("Examples:")
        print("  doc_search.py 'RunInstances' ecs")
        print("  doc_search.py 'CreateFunction' fc")
        sys.exit(1)

    kw = sys.argv[1]
    prod = sys.argv[2] if len(sys.argv) > 2 else None
    output = search_and_format(kw, prod)
    print(output)


path: scripts/ecs.py
content: #!/usr/bin/env python3
"""Alibaba Cloud ECS management.

Capabilities:
  - describe_instance_types / describe_instance_type_families  (instance type query)
  - describe_regions / describe_zones / describe_available_resource  (inventory query)
  - describe_price  (pricing)
  - run_instances / describe_instances  (instance creation)
  - create_security_group / authorize_security_group / authorize_security_group_egress
  - describe_images
  - run_command / describe_invocations / describe_invocation_results  (script execution)

All functions use RPC-style API (product=ecs, version=2014-05-26).
"""

from __future__ import annotations

import base64
import sys
import os
import time
from typing import Any

sys.path.insert(0, os.path.dirname(__file__))

from common import call_rpc_api, wait_until, pp

PRODUCT = "ecs"
VERSION = "2014-05-26"


def _call(action: str, params: dict | None = None, region: str | None = None) -> dict:
    return call_rpc_api(PRODUCT, VERSION, action, params=params, region=region)


# ---------------------------------------------------------------------------
# Instance type query
# ---------------------------------------------------------------------------

def describe_instance_types(
    instance_type_family: str | None = None,
    instance_types: list[str] | None = None,
    min_cpu: int | None = None,
    max_cpu: int | None = None,
    min_memory: float | None = None,
    max_memory: float | None = None,
    gpu_amount: int | None = None,
    region: str | None = None,
) -> dict:
    """Query ECS instance type specifications.

    Use filters to narrow results (cpu, memory, gpu, family).
    Returns InstanceTypes.InstanceType[].
    """
    params: dict[str, Any] = {}
    if instance_type_family:
        params["InstanceTypeFamily"] = instance_type_family
    if instance_types:
        for i, t in enumerate(instance_types):
            params[f"InstanceTypes.{i+1}"] = t
    if min_cpu is not None:
        params["MinimumCpuCoreCount"] = min_cpu
    if max_cpu is not None:
        params["MaximumCpuCoreCount"] = max_cpu
    if min_memory is not None:
        params["MinimumMemorySize"] = min_memory
    if max_memory is not None:
        params["MaximumMemorySize"] = max_memory
    if gpu_amount is not None:
        params["GPUAmount"] = gpu_amount
    return _call("DescribeInstanceTypes", params, region)


def describe_instance_type_families(
    generation: str | None = None,
    region: str | None = None,
) -> dict:
    """List instance type families (e.g. ecs.g7, ecs.c7)."""
    params: dict[str, Any] = {}
    if generation:
        params["Generation"] = generation
    return _call("DescribeInstanceTypeFamilies", params, region)


def describe_recommend_instance_type(
    cores: int | None = None,
    memory: float | None = None,
    instance_type: str | None = None,
    instance_family_level: str | None = None,
    instance_charge_type: str = "PostPaid",
    spot_strategy: str = "NoSpot",
    priority_strategy: str = "InventoryFirst",
    zone_id: str | None = None,
    system_disk_category: str | None = None,
    max_price: float | None = None,
    scene: str = "CREATE",
    instance_type_family: list[str] | None = None,
    region: str | None = None,
) -> dict:
    """Recommend instance types based on requirements.

    Provide EITHER (cores + memory) OR instance_type, not both.

    Args:
        cores: vCPU count.
        memory: Memory in GiB.
        instance_type: Base instance type to find alternatives for.
        instance_family_level: EntryLevel | EnterpriseLevel | CreditEntryLevel.
        priority_strategy: InventoryFirst | PriceFirst | NewProductFirst.
        max_price: Max hourly price (only for SpotWithPriceLimit).
        scene: CREATE | UPGRADE.
        instance_type_family: Limit to these families, e.g. ["ecs.g7", "ecs.c7"].

    Returns:
        Data.RecommendInstanceType[] with InstanceType, ZoneId, Priority etc.
    """
    params: dict[str, Any] = {
        "NetworkType": "vpc",
        "InstanceChargeType": instance_charge_type,
        "SpotStrategy": spot_strategy,
        "PriorityStrategy": priority_strategy,
        "Scene": scene,
    }
    if cores is not None:
        params["Cores"] = cores
    if memory is not None:
        params["Memory"] = memory
    if instance_type:
        params["InstanceType"] = instance_type
    if instance_family_level:
        params["InstanceFamilyLevel"] = instance_family_level
    if zone_id:
        params["ZoneId"] = zone_id
    if system_disk_category:
        params["SystemDiskCategory"] = system_disk_category
    if max_price is not None:
        params["MaxPrice"] = max_price
    if instance_type_family:
        for i, f in enumerate(instance_type_family):
            params[f"InstanceTypeFamily.{i+1}"] = f
    result = _call("DescribeRecommendInstanceType", params, region)

    for rec in result.get("Data", {}).get("RecommendInstanceType", []):
        inst = rec.get("InstanceType", {})
        mem_mb = inst.get("Memory")
        if isinstance(mem_mb, (int, float)) and mem_mb >= 1024:
            inst["MemoryGB"] = round(mem_mb / 1024, 1)

    return result


# ---------------------------------------------------------------------------
# Inventory query
# ---------------------------------------------------------------------------

def describe_regions(region: str | None = None) -> dict:
    """List all available regions."""
    return _call("DescribeRegions", region=region)


def describe_zones(region: str | None = None) -> dict:
    """List zones in a region."""
    return _call("DescribeZones", region=region)


def describe_available_resource(
    destination_resource: str = "InstanceType",
    zone_id: str | None = None,
    instance_type: str | None = None,
    instance_charge_type: str = "PostPaid",
    io_optimized: str = "optimized",
    region: str | None = None,
) -> dict:
    """Query available resources (instance types, disk types, etc.) by zone.

    destination_resource: InstanceType | SystemDisk | DataDisk | Network
    """
    params: dict[str, Any] = {
        "DestinationResource": destination_resource,
        "InstanceChargeType": instance_charge_type,
        "IoOptimized": io_optimized,
    }
    if zone_id:
        params["ZoneId"] = zone_id
    if instance_type:
        params["InstanceType"] = instance_type
    return _call("DescribeAvailableResource", params, region)


# ---------------------------------------------------------------------------
# Pricing
# ---------------------------------------------------------------------------

def describe_price(
    instance_type: str,
    period: int = 1,
    price_unit: str = "Hour",
    instance_charge_type: str = "PostPaid",
    system_disk_category: str = "cloud_essd",
    system_disk_size: int = 40,
    internet_max_bandwidth_out: int = 0,
    amount: int = 1,
    region: str | None = None,
    zone_id: str | None = None,
) -> dict:
    """Query price for an ECS instance configuration.

    Returns price info including TradePrice, OriginalPrice, DiscountPrice.
    """
    params: dict[str, Any] = {
        "ResourceType": "instance",
        "InstanceType": instance_type,
        "Period": period,
        "PriceUnit": price_unit,
        "InstanceChargeType": instance_charge_type,
        "SystemDisk.Category": system_disk_category,
        "SystemDisk.Size": system_disk_size,
        "InternetMaxBandwidthOut": internet_max_bandwidth_out,
        "Amount": amount,
    }
    if zone_id:
        params["ZoneId"] = zone_id
    return _call("DescribePrice", params, region)


# ---------------------------------------------------------------------------
# Image query
# ---------------------------------------------------------------------------

def describe_images(
    image_name: str | None = None,
    os_type: str | None = None,
    image_family: str | None = None,
    instance_type: str | None = None,
    region: str | None = None,
    page_size: int = 20,
) -> dict:
    """Query available images.

    Common image_family values:
      acs:alibaba_cloud_linux_3_2104_lts_x64, acs:ubuntu_24_04_x64,
      acs:ubuntu_22_04_x64, acs:centos_7_9_x64
    """
    params: dict[str, Any] = {
        "PageSize": page_size,
        "ImageOwnerAlias": "system",
        "Status": "Available",
    }
    if image_name:
        params["ImageName"] = image_name
    if os_type:
        params["OSType"] = os_type
    if image_family:
        params["ImageFamily"] = image_family
    if instance_type:
        params["InstanceType"] = instance_type
    return _call("DescribeImages", params, region)


# ---------------------------------------------------------------------------
# Security groups
# ---------------------------------------------------------------------------

def create_security_group(
    vpc_id: str,
    sg_name: str = "acf-sg",
    description: str = "Created by Alibaba Cloud Compute Provision skill",
    region: str | None = None,
) -> str:
    """Create a security group. Returns SecurityGroupId."""
    params: dict[str, Any] = {
        "VpcId": vpc_id,
        "SecurityGroupName": sg_name,
        "Description": description,
    }
    result = _call("CreateSecurityGroup", params, region)
    sg_id = result.get("SecurityGroupId", "")
    print(f"SecurityGroup created: {sg_id}")
    return sg_id


def authorize_security_group(
    security_group_id: str,
    ip_protocol: str = "tcp",
    port_range: str = "22/22",
    source_cidr_ip: str = "0.0.0.0/0",
    region: str | None = None,
) -> dict:
    """Add an ingress rule to a security group."""
    params: dict[str, Any] = {
        "SecurityGroupId": security_group_id,
        "IpProtocol": ip_protocol,
        "PortRange": port_range,
        "SourceCidrIp": source_cidr_ip,
        "Policy": "Accept",
    }
    return _call("AuthorizeSecurityGroup", params, region)


def authorize_security_group_egress(
    security_group_id: str,
    ip_protocol: str = "all",
    port_range: str = "-1/-1",
    dest_cidr_ip: str = "0.0.0.0/0",
    region: str | None = None,
) -> dict:
    """Add an egress rule to a security group."""
    params: dict[str, Any] = {
        "SecurityGroupId": security_group_id,
        "IpProtocol": ip_protocol,
        "PortRange": port_range,
        "DestCidrIp": dest_cidr_ip,
        "Policy": "Accept",
    }
    return _call("AuthorizeSecurityGroupEgress", params, region)


def setup_security_group(
    vpc_id: str,
    sg_name: str = "acf-sg",
    ingress_rules: list[dict] | None = None,
    region: str | None = None,
) -> str:
    """Create a security group with caller-specified ingress rules + full egress.

    Ingress rules MUST be derived from what the deployed application actually
    needs. Open only the ports the workload requires, and scope the source CIDR
    as tightly as possible. Pass ``None`` or ``[]`` for an internal-only SG
    (no ingress, full egress).

    Args:
        ingress_rules: List of dicts. Each dict accepts:
            - ``port_range``  (str, required): e.g. ``"80/80"``, ``"443/443"``,
              ``"8000/9000"``.
            - ``source_cidr_ip``  (str, default ``"0.0.0.0/0"``): tighten this
              for management ports (SSH/RDP/DB) — e.g. the operator's public
              ``/32`` or an internal CIDR. Only use ``0.0.0.0/0`` for ports
              that genuinely serve the public internet (HTTP/HTTPS).
            - ``ip_protocol``  (str, default ``"tcp"``): ``tcp``/``udp``/``icmp``/``all``.

    Examples:
        # Public web server
        setup_security_group(vpc_id, ingress_rules=[
            {"port_range": "80/80"},
            {"port_range": "443/443"},
        ])

        # Web server + SSH from operator only
        setup_security_group(vpc_id, ingress_rules=[
            {"port_range": "80/80"},
            {"port_range": "443/443"},
            {"port_range": "22/22", "source_cidr_ip": "203.0.113.5/32"},
        ])

        # Internal worker, no ingress
        setup_security_group(vpc_id, ingress_rules=None)

    Returns:
        security_group_id.
    """
    sg_id = create_security_group(vpc_id=vpc_id, sg_name=sg_name, region=region)

    rules = ingress_rules or []
    for rule in rules:
        port_range = rule["port_range"]
        protocol = rule.get("ip_protocol", "tcp")
        source = rule.get("source_cidr_ip", "0.0.0.0/0")
        authorize_security_group(sg_id, protocol, port_range, source, region)

    authorize_security_group_egress(sg_id, "all", "-1/-1", "0.0.0.0/0", region)

    if rules:
        summary = ", ".join(
            f"{r.get('ip_protocol', 'tcp')}/{r['port_range']} from {r.get('source_cidr_ip', '0.0.0.0/0')}"
            for r in rules
        )
        print(f"SecurityGroup {sg_id} configured: ingress=[{summary}] + full egress")
    else:
        print(f"SecurityGroup {sg_id} configured with full egress (no ingress)")
    return sg_id


def delete_security_group(
    security_group_id: str,
    region: str | None = None,
) -> dict:
    """Delete a security group. The SG must have no associated instances."""
    params: dict[str, Any] = {"SecurityGroupId": security_group_id}
    result = _call("DeleteSecurityGroup", params, region)
    print(f"SecurityGroup deleted: {security_group_id}")
    return result


# ---------------------------------------------------------------------------
# Instance creation
# ---------------------------------------------------------------------------

def run_instances(
    instance_type: str,
    vswitch_id: str,
    security_group_id: str,
    image_id: str,
    instance_name: str = "acf-instance",
    system_disk_category: str = "cloud_essd",
    system_disk_size: int = 40,
    internet_max_bandwidth_out: int = 0,
    instance_charge_type: str = "PostPaid",
    amount: int = 1,
    region: str | None = None,
    **extra_params,
) -> dict:
    """Create ECS instance(s) and wait until Running.

    Args:
        internet_max_bandwidth_out: Public network bandwidth in Mbps (0~100).
            0 = no public IP assigned (default).
            >0 = auto-assign public IP with this bandwidth.
            Typical values: 5 (light use), 10-50 (moderate), 100 (heavy).

    Returns RunInstances response with InstanceIdSets.

    Raises:
        ValueError: If UserData is passed. Use run_command() or
            run_command_and_wait() after instance creation instead.
    """
    if "UserData" in extra_params:
        raise ValueError(
            "UserData is not allowed in run_instances(). "
            "Create the instance first, then use run_command() or "
            "run_command_and_wait() to execute scripts on it."
        )

    params: dict[str, Any] = {
        "InstanceType": instance_type,
        "VSwitchId": vswitch_id,
        "SecurityGroupId": security_group_id,
        "ImageId": image_id,
        "InstanceName": instance_name,
        "SystemDisk.Category": system_disk_category,
        "SystemDisk.Size": system_disk_size,
        "InternetMaxBandwidthOut": internet_max_bandwidth_out,
        "InstanceChargeType": instance_charge_type,
        "Amount": amount,
        "InternetChargeType": "PayByTraffic",
    }
    params.update(extra_params)

    result = _call("RunInstances", params, region)
    instance_ids = result.get("InstanceIdSets", {}).get("InstanceIdSet", [])
    print(f"Instances created: {instance_ids}")

    for iid in instance_ids:
        def _check(instance_id=iid):
            info = describe_instances(instance_ids=[instance_id], region=region)
            items = info.get("Instances", {}).get("Instance", [])
            return items[0] if items else {"Status": "Pending"}

        wait_until(_check, {"Running"}, {"Stopped", "Error"}, interval=10, timeout=300)
        print(f"Instance {iid} is Running")

    return result


_FALLBACK_REGIONS = ["cn-hangzhou", "cn-shanghai", "cn-beijing", "cn-shenzhen", "cn-zhangjiakou"]


def find_available_instance_type(
    cores: int = 2,
    memory: float = 4,
    instance_type: str | None = None,
    instance_charge_type: str = "PostPaid",
    system_disk_category: str = "cloud_essd",
    regions: list[str] | None = None,
) -> dict:
    """Find an available instance type with pricing across zones and regions.

    Searches across multiple regions/zones until it finds an instance type
    with stock. Returns the first available option with its pricing info
    so the agent can present cost to the user BEFORE creating anything.

    Args:
        cores: Desired vCPU count (used if instance_type is None).
        memory: Desired memory in GiB (used if instance_type is None).
        instance_type: Explicit instance type to check availability for.
        regions: Regions to try in order. Defaults to _FALLBACK_REGIONS.

    Returns:
        dict with keys: instance_type, region, zone_id, cores, memory,
        price_per_hour, currency. Raise RuntimeError if nothing found.
    """
    candidate_regions = regions or _FALLBACK_REGIONS
    errors: list[str] = []

    for region in candidate_regions:
        print(f"Checking region: {region} ...")

        try:
            recs = describe_recommend_instance_type(
                cores=cores if not instance_type else None,
                memory=memory if not instance_type else None,
                instance_type=instance_type,
                instance_charge_type=instance_charge_type,
                priority_strategy="InventoryFirst",
                system_disk_category=system_disk_category,
                region=region,
            )
        except Exception as e:
            errors.append(f"{region}: {e}")
            continue

        rec_list = recs.get("Data", {}).get("RecommendInstanceType", [])
        if not rec_list:
            errors.append(f"{region}: no instance types available")
            continue

        for rec in rec_list:
            type_info = rec.get("InstanceType", {})
            rec_type = type_info.get("InstanceType", "")
            rec_cores = type_info.get("Cores", cores)
            rec_mem_mb = type_info.get("Memory")
            rec_mem = round(rec_mem_mb / 1024, 1) if isinstance(rec_mem_mb, (int, float)) and rec_mem_mb >= 1024 else rec_mem_mb
            zones = rec.get("Zones", {}).get("zone", [])
            if not zones or not rec_type:
                continue

            zone_id = zones[0].get("ZoneNo", "")
            if not zone_id:
                continue

            try:
                price_info = describe_price(
                    instance_type=rec_type,
                    price_unit="Hour",
                    region=region,
                )
                price_data = price_info.get("PriceInfo", {}).get("Price", {})
                trade_price = price_data.get("TradePrice", 0)
                currency = price_data.get("Currency", "CNY")
            except Exception:
                trade_price = None
                currency = "CNY"

            print(f"  ✅ Found: {rec_type} ({rec_cores}C/{rec_mem}G) in {zone_id}")
            return {
                "instance_type": rec_type,
                "region": region,
                "zone_id": zone_id,
                "cores": rec_cores,
                "memory": rec_mem,
                "price_per_hour": trade_price,
                "currency": currency,
            }

    raise RuntimeError(
        f"No available ECS instance type found after trying {len(candidate_regions)} regions.\n"
        + "\n".join(f"  - {e}" for e in errors)
    )


def create_instance_with_infra(
    instance_type: str,
    zone_id: str,
    image_family: str = "acs:alibaba_cloud_linux_3_2104_lts_x64",
    instance_name: str = "acf-instance",
    system_disk_category: str = "cloud_essd",
    system_disk_size: int = 40,
    internet_max_bandwidth_out: int = 0,
    instance_charge_type: str = "PostPaid",
    region: str | None = None,
    **extra_params,
) -> dict:
    """Create an ECS instance with automatic VPC/VSwitch/SecurityGroup setup.

    Call this AFTER cost confirmation. This function handles all
    infrastructure (VPC, VSwitch, SecurityGroup, Image) automatically.

    Args:
        instance_type: The instance type from find_available_instance_type().
        zone_id: The zone from find_available_instance_type().
        region: The region from find_available_instance_type().

    Returns:
        dict with keys: instance_id, region, zone_id, instance_type,
        vpc_id, vswitch_id, security_group_id, image_id.
    """
    sys.path.insert(0, os.path.dirname(__file__))
    from vpc import ensure_vpc_and_vswitch

    if "UserData" in extra_params:
        raise ValueError(
            "UserData is not allowed. Use run_command() or "
            "run_command_and_wait() after instance creation instead."
        )
    if "image_id" in extra_params or "ImageId" in extra_params:
        raise ValueError(
            "image_id/ImageId should not be passed — it is resolved "
            "internally via image_family. Use the image_family parameter instead."
        )

    vpc_id, vsw_id, vpc_is_new, vsw_is_new = ensure_vpc_and_vswitch(
        zone_id=zone_id, region=region,
    )

    sg_id = create_security_group(vpc_id=vpc_id, region=region)
    authorize_security_group_egress(security_group_id=sg_id, region=region)
    if internet_max_bandwidth_out > 0:
        authorize_security_group(
            security_group_id=sg_id, port_range="22/22", region=region
        )

    imgs = describe_images(
        image_family=image_family, instance_type=instance_type, region=region
    )
    img_list = imgs.get("Images", {}).get("Image", [])
    if not img_list:
        _rollback_infra(sg_id, vsw_id if vsw_is_new else None,
                        vpc_id if vpc_is_new else None, region)
        raise RuntimeError(f"No image found for family={image_family}, type={instance_type}, region={region}")
    img_id = img_list[0]["ImageId"]

    try:
        result = run_instances(
            instance_type=instance_type,
            vswitch_id=vsw_id,
            security_group_id=sg_id,
            image_id=img_id,
            instance_name=instance_name,
            system_disk_category=system_disk_category,
            system_disk_size=system_disk_size,
            internet_max_bandwidth_out=internet_max_bandwidth_out,
            instance_charge_type=instance_charge_type,
            region=region,
            **extra_params,
        )
    except Exception:
        _rollback_infra(sg_id, vsw_id if vsw_is_new else None,
                        vpc_id if vpc_is_new else None, region)
        raise

    iid = result.get("InstanceIdSets", {}).get("InstanceIdSet", [""])[0]
    print(f"✅ Instance created: {iid} ({instance_type} in {zone_id})")
    return {
        "instance_id": iid,
        "region": region,
        "zone_id": zone_id,
        "instance_type": instance_type,
        "vpc_id": vpc_id,
        "vswitch_id": vsw_id,
        "security_group_id": sg_id,
        "image_id": img_id,
        "vpc_is_new": vpc_is_new,
        "vswitch_is_new": vsw_is_new,
    }


def _rollback_infra(
    sg_id: str | None, vsw_id: str | None, vpc_id: str | None, region: str | None
) -> None:
    """Best-effort cleanup of infra created before instance creation failed."""
    print("⚠️ Instance creation failed, rolling back infrastructure ...")
    if sg_id:
        try:
            delete_security_group(sg_id, region=region)
        except Exception as e:
            print(f"  ⚠️ Rollback SG {sg_id}: {e}")
    if vsw_id:
        try:
            from vpc import delete_vswitch
            delete_vswitch(vsw_id, region=region)
        except Exception as e:
            print(f"  ⚠️ Rollback VSwitch {vsw_id}: {e}")
    if vpc_id:
        try:
            from vpc import delete_vpc
            delete_vpc(vpc_id, region=region)
        except Exception as e:
            print(f"  ⚠️ Rollback VPC {vpc_id}: {e}")


def describe_instances(
    instance_ids: list[str] | None = None,
    instance_name: str | None = None,
    status: str | None = None,
    region: str | None = None,
    page_size: int = 50,
) -> dict:
    """Query ECS instances."""
    params: dict[str, Any] = {"PageSize": page_size}
    if instance_ids:
        import json as _json
        params["InstanceIds"] = _json.dumps(instance_ids)
    if instance_name:
        params["InstanceName"] = instance_name
    if status:
        params["Status"] = status
    return _call("DescribeInstances", params, region)


def delete_instances(
    instance_ids: list[str],
    force: bool = True,
    region: str | None = None,
    max_retries: int = 5,
    retry_interval: int = 15,
) -> dict:
    """Release (delete) one or more ECS instances.

    Retries automatically on IncorrectInstanceStatus (e.g. instance still
    initializing after RunInstances). Waits retry_interval seconds between
    attempts, up to max_retries times.
    """
    params: dict[str, Any] = {
        "Force": force,
    }
    for i, iid in enumerate(instance_ids):
        params[f"InstanceId.{i+1}"] = iid

    for attempt in range(1, max_retries + 1):
        try:
            result = _call("DeleteInstances", params, region)
            print(f"Instances released: {instance_ids}")
            return result
        except Exception as e:
            err_str = str(e)
            if "IncorrectInstanceStatus" in err_str and attempt < max_retries:
                print(f"  Instance not ready for release (attempt {attempt}/{max_retries}), "
                      f"retrying in {retry_interval}s...")
                time.sleep(retry_interval)
            else:
                raise


# ---------------------------------------------------------------------------
# Script execution (Cloud Assistant)
# ---------------------------------------------------------------------------

def run_command(
    instance_id: str,
    command_content: str | None = None,
    command_type: str = "RunShellScript",
    timeout: int = 600,
    region: str | None = None,
    script_path: str | None = None,
) -> str:
    """Execute a script on an ECS instance via Cloud Assistant.

    Args:
        instance_id: Target ECS instance.
        command_content: Script content (shell or python). Ignored if script_path is set.
        command_type: RunShellScript | RunPythonScript | RunBatScript | RunPowerShellScript.
        timeout: Command timeout in seconds.
        script_path: Local file path to read script content from.

    Provide either command_content or script_path. script_path takes priority.

    Returns:
        InvokeId for tracking.
    """
    if script_path:
        with open(os.path.expanduser(script_path), "r", encoding="utf-8") as f:
            command_content = f.read()
        print(f"Script loaded from: {script_path} ({len(command_content)} bytes)")
    if not command_content:
        raise ValueError("Either command_content or script_path must be provided")

    encoded = base64.b64encode(command_content.encode("utf-8")).decode("utf-8")
    params: dict[str, Any] = {
        "Type": command_type,
        "CommandContent": encoded,
        "InstanceId.1": instance_id,
        "Timeout": timeout,
        "ContentEncoding": "Base64",
    }
    result = _call("RunCommand", params, region)
    invoke_id = result.get("InvokeId", "")
    print(f"Command submitted, InvokeId: {invoke_id}")
    return invoke_id


def describe_invocations(
    invoke_id: str,
    region: str | None = None,
) -> dict:
    """Query invocation status."""
    params: dict[str, Any] = {"InvokeId": invoke_id}
    return _call("DescribeInvocations", params, region)


def describe_invocation_results(
    invoke_id: str,
    instance_id: str | None = None,
    region: str | None = None,
) -> dict:
    """Get invocation results (stdout/stderr)."""
    params: dict[str, Any] = {"InvokeId": invoke_id}
    if instance_id:
        params["InstanceId"] = instance_id
    return _call("DescribeInvocationResults", params, region)


def run_command_and_wait(
    instance_id: str,
    command_content: str | None = None,
    command_type: str = "RunShellScript",
    timeout: int = 600,
    region: str | None = None,
    script_path: str | None = None,
) -> dict:
    """Execute a script and wait for completion. Returns invocation result.

    Provide either command_content or script_path. script_path takes priority.
    Result includes Output (Base64 encoded), ExitCode, ErrorInfo, etc.
    """
    invoke_id = run_command(instance_id, command_content, command_type, timeout, region, script_path)

    def _check():
        inv = describe_invocations(invoke_id, region)
        items = inv.get("Invocations", {}).get("Invocation", [])
        if not items:
            return {"InvokeStatus": "Pending"}
        status = items[0].get("InvocationStatus", "Pending")
        return {"Status": status}

    wait_until(
        _check,
        target_statuses={"Success", "Finished", "Failed", "PartialFailed", "Stopped"},
        interval=5,
        timeout=timeout + 60,
        status_key="Status",
    )

    result = describe_invocation_results(invoke_id, instance_id, region)
    items = result.get("Invocation", {}).get("InvocationResults", {}).get("InvocationResult", [])
    if items:
        item = items[0]
        output_b64 = item.get("Output", "")
        if output_b64:
            try:
                item["DecodedOutput"] = base64.b64decode(output_b64).decode("utf-8", errors="replace")
            except Exception:
                pass
        print(f"Command exit code: {item.get('ExitCode')}")
        if item.get("DecodedOutput"):
            print(f"Output:\n{item['DecodedOutput']}")
        return item

    return result


def run_command_and_cleanup(
    instance_id: str,
    command_content: str | None = None,
    command_type: str = "RunShellScript",
    timeout: int = 600,
    region: str | None = None,
    script_path: str | None = None,
    security_group_id: str | None = None,
    vpc_id: str | None = None,
    vswitch_id: str | None = None,
    infra: dict | None = None,
) -> dict:
    """Execute a script, wait for completion, then release the instance and infra.

    For one-shot tasks: runs the script, collects output, then cleans up
    the instance + security group (always) and VSwitch + VPC (only if newly
    created, not shared/reused).

    Preferred usage — pass the return value of create_instance_with_infra()
    as ``infra`` and omit the individual ID args:

        infra = create_instance_with_infra(...)
        run_command_and_cleanup(
            instance_id=infra["instance_id"],
            command_content="echo hi",
            region=infra["region"],
            infra=infra,
        )

    When ``infra`` is provided, vpc_id/vswitch_id/security_group_id are
    read from it, and VPC/VSwitch are only deleted if they were freshly
    created (``vpc_is_new`` / ``vswitch_is_new`` flags).

    Returns:
        The command invocation result (same as run_command_and_wait).
    """
    if infra is not None:
        security_group_id = security_group_id or infra.get("security_group_id")
        region = region or infra.get("region")
        if infra.get("vswitch_is_new"):
            vswitch_id = vswitch_id or infra.get("vswitch_id")
        else:
            vswitch_id = None
        if infra.get("vpc_is_new"):
            vpc_id = vpc_id or infra.get("vpc_id")
        else:
            vpc_id = None

    try:
        result = run_command_and_wait(
            instance_id=instance_id,
            command_content=command_content,
            command_type=command_type,
            timeout=timeout,
            region=region,
            script_path=script_path,
        )
    except Exception as e:
        print(f"⚠️ Command failed: {e}")
        result = {"Error": str(e), "ExitCode": -1}

    print(f"\n🧹 Cleaning up resources ...")

    try:
        delete_instances([instance_id], force=True, region=region)
        _wait_instance_deleted(instance_id, region=region)
    except Exception as e:
        print(f"  ⚠️ Failed to delete instance {instance_id}: {e}")

    if security_group_id:
        _retry_cleanup(
            lambda: delete_security_group(security_group_id, region=region),
            f"SecurityGroup {security_group_id}",
        )

    if vswitch_id:
        sys.path.insert(0, os.path.dirname(__file__))
        from vpc import delete_vswitch
        _retry_cleanup(
            lambda: delete_vswitch(vswitch_id, region=region),
            f"VSwitch {vswitch_id}",
        )

    if vpc_id:
        from vpc import delete_vpc
        _retry_cleanup(
            lambda: delete_vpc(vpc_id, region=region),
            f"VPC {vpc_id}",
        )

    print("🧹 Cleanup complete.")
    return result


def _retry_cleanup(
    fn, resource_label: str, max_retries: int = 5, interval: int = 15
) -> None:
    """Retry a cleanup operation, tolerating DependencyViolation delays."""
    for attempt in range(1, max_retries + 1):
        try:
            fn()
            print(f"  {resource_label} deleted.")
            return
        except Exception as e:
            err_str = str(e)
            if "DependencyViolation" in err_str and attempt < max_retries:
                print(f"  {resource_label}: dependency not yet released (attempt {attempt}/{max_retries}), retrying in {interval}s...")
                time.sleep(interval)
            else:
                print(f"  ⚠️ Failed to delete {resource_label}: {e}")
                return


def _wait_instance_deleted(
    instance_id: str, region: str | None = None, timeout: int = 120, interval: int = 5
) -> None:
    """Poll until the instance disappears from DescribeInstances."""
    deadline = time.time() + timeout
    while time.time() < deadline:
        info = describe_instances(instance_ids=[instance_id], region=region)
        items = info.get("Instances", {}).get("Instance", [])
        if not items:
            print(f"  Instance {instance_id} released.")
            return
        status = items[0].get("Status", "")
        print(f"  Instance {instance_id} status: {status}, waiting ...")
        time.sleep(interval)
    print(f"  ⚠️ Timed out waiting for instance {instance_id} to be released.")


path: scripts/vpc.py
content: #!/usr/bin/env python3
"""Alibaba Cloud VPC & VSwitch management.

Capabilities:
  - create_vpc / describe_vpcs
  - create_vswitch / describe_vswitches

All functions use RPC-style API (product=vpc, version=2016-04-28).
"""

from __future__ import annotations

import sys
import os

sys.path.insert(0, os.path.dirname(__file__))

from common import call_rpc_api, wait_until, pp

PRODUCT = "vpc"
VERSION = "2016-04-28"


def _call(action: str, params: dict | None = None, region: str | None = None) -> dict:
    return call_rpc_api(PRODUCT, VERSION, action, params=params, region=region)


# ---------------------------------------------------------------------------
# VPC
# ---------------------------------------------------------------------------

def create_vpc(
    cidr_block: str = "192.168.0.0/16",
    vpc_name: str = "acf-vpc",
    region: str | None = None,
    description: str = "",
) -> dict:
    """Create a VPC and wait until Available.

    Returns the full VPC description dict.
    """
    params: dict = {
        "CidrBlock": cidr_block,
        "VpcName": vpc_name,
    }
    if description:
        params["Description"] = description

    result = _call("CreateVpc", params, region)
    vpc_id = result.get("VpcId", "")
    print(f"VPC created: {vpc_id}")

    def _check():
        vpcs = describe_vpcs(vpc_id=vpc_id, region=region)
        items = vpcs.get("Vpcs", {}).get("Vpc", [])
        return items[0] if items else {"Status": "Pending"}

    info = wait_until(_check, {"Available"}, {"Error"}, interval=5, timeout=120)
    print(f"VPC {vpc_id} is Available")
    return info


def describe_vpcs(
    vpc_id: str | None = None,
    vpc_name: str | None = None,
    region: str | None = None,
    page_size: int = 50,
) -> dict:
    """List VPCs with optional filters."""
    params: dict = {"PageSize": page_size}
    if vpc_id:
        params["VpcId"] = vpc_id
    if vpc_name:
        params["VpcName"] = vpc_name
    return _call("DescribeVpcs", params, region)


# ---------------------------------------------------------------------------
# VSwitch
# ---------------------------------------------------------------------------

def create_vswitch(
    vpc_id: str,
    zone_id: str,
    cidr_block: str = "192.168.0.0/24",
    vswitch_name: str = "acf-vsw",
    region: str | None = None,
    description: str = "",
) -> dict:
    """Create a VSwitch and wait until Available.

    Returns the full VSwitch description dict.
    """
    params: dict = {
        "VpcId": vpc_id,
        "ZoneId": zone_id,
        "CidrBlock": cidr_block,
        "VSwitchName": vswitch_name,
    }
    if description:
        params["Description"] = description

    result = _call("CreateVSwitch", params, region)
    vsw_id = result.get("VSwitchId", "")
    print(f"VSwitch created: {vsw_id}")

    def _check():
        vsws = describe_vswitches(vswitch_id=vsw_id, region=region)
        items = vsws.get("VSwitches", {}).get("VSwitch", [])
        return items[0] if items else {"Status": "Pending"}

    info = wait_until(_check, {"Available"}, {"Error"}, interval=3, timeout=60)
    print(f"VSwitch {vsw_id} is Available")
    return info


def describe_vswitches(
    vpc_id: str | None = None,
    vswitch_id: str | None = None,
    zone_id: str | None = None,
    region: str | None = None,
    page_size: int = 50,
) -> dict:
    """List VSwitches with optional filters."""
    params: dict = {"PageSize": page_size}
    if vpc_id:
        params["VpcId"] = vpc_id
    if vswitch_id:
        params["VSwitchId"] = vswitch_id
    if zone_id:
        params["ZoneId"] = zone_id
    return _call("DescribeVSwitches", params, region)


def delete_vswitch(
    vswitch_id: str,
    region: str | None = None,
) -> dict:
    """Delete a VSwitch. It must have no associated resources."""
    result = _call("DeleteVSwitch", {"VSwitchId": vswitch_id}, region)
    print(f"VSwitch deleted: {vswitch_id}")
    return result


def delete_vpc(
    vpc_id: str,
    region: str | None = None,
) -> dict:
    """Delete a VPC. All VSwitches in it must be deleted first."""
    result = _call("DeleteVpc", {"VpcId": vpc_id}, region)
    print(f"VPC deleted: {vpc_id}")
    return result


# ---------------------------------------------------------------------------
# Convenience: ensure VPC + VSwitch exist
# ---------------------------------------------------------------------------

def _find_available_vsw_cidr(vpc_id: str, vpc_cidr_prefix: str = "192.168", region: str | None = None) -> str:
    """Find an available /24 CIDR within the VPC by checking existing VSwitches.

    Scans 192.168.0.0/24 through 192.168.255.0/24 and returns the first
    subnet not used by any existing VSwitch.
    """
    vsws = describe_vswitches(vpc_id=vpc_id, region=region)
    vsw_items = vsws.get("VSwitches", {}).get("VSwitch", [])
    used_cidrs = {v.get("CidrBlock", "") for v in vsw_items}

    for third_octet in range(256):
        candidate = f"{vpc_cidr_prefix}.{third_octet}.0/24"
        if candidate not in used_cidrs:
            return candidate

    raise RuntimeError(f"No available /24 CIDR in {vpc_cidr_prefix}.0.0/16, all 256 subnets occupied")


def ensure_vpc_and_vswitch(
    zone_id: str,
    vpc_cidr: str = "192.168.0.0/16",
    vsw_cidr: str | None = None,
    vpc_name: str = "acf-vpc",
    vsw_name: str = "acf-vsw",
    region: str | None = None,
) -> tuple[str, str, bool, bool]:
    """Create or reuse a VPC and VSwitch.

    Returns (vpc_id, vswitch_id, vpc_is_new, vswitch_is_new).
    The boolean flags indicate whether the resource was freshly created
    (True) or reused from an existing one (False).

    If vsw_cidr is None (default), auto-detects an available /24 subnet
    within the VPC to avoid CIDR conflicts with existing VSwitches.
    """
    vpc_is_new = False
    vsw_is_new = False

    vpcs = describe_vpcs(vpc_name=vpc_name, region=region)
    items = vpcs.get("Vpcs", {}).get("Vpc", [])
    available = [v for v in items if v.get("Status") == "Available"]

    if available:
        vpc_id = available[0]["VpcId"]
        print(f"Reusing existing VPC: {vpc_id}")
    else:
        vpc_info = create_vpc(cidr_block=vpc_cidr, vpc_name=vpc_name, region=region)
        vpc_id = vpc_info["VpcId"]
        vpc_is_new = True

    vsws = describe_vswitches(vpc_id=vpc_id, zone_id=zone_id, region=region)
    vsw_items = vsws.get("VSwitches", {}).get("VSwitch", [])
    vsw_available = [v for v in vsw_items if v.get("Status") == "Available"]

    if vsw_available:
        vsw_id = vsw_available[0]["VSwitchId"]
        print(f"Reusing existing VSwitch: {vsw_id}")
    else:
        if vsw_cidr is None:
            vpc_prefix = vpc_cidr.rsplit(".", 2)[0]  # "192.168.0.0/16" → "192.168"
            vsw_cidr = _find_available_vsw_cidr(vpc_id, vpc_prefix, region)
            print(f"Auto-detected available CIDR: {vsw_cidr}")

        vsw_info = create_vswitch(
            vpc_id=vpc_id, zone_id=zone_id,
            cidr_block=vsw_cidr, vswitch_name=vsw_name, region=region,
        )
        vsw_id = vsw_info["VSwitchId"]
        vsw_is_new = True

    return vpc_id, vsw_id, vpc_is_new, vsw_is_new


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)