Skill: wbavon/flagrelease-entrance-flagos (version 1.0.0)

--- name: flagrelease-entrance-flagos description: | Full FlagRelease pipeline orchestrator. Runs the complete LLM deployment, verification, and benchmarking pipeline for multi-chip GPU backends. Executes: install-stack → env-verify → model-verify → perf-test in sequence, passing state between steps and producing a final structured report. Assumes gpu-container-setup (Step 1) is already done — a running container with PyTorch + GPU access must exist. user-invokable: true allowed-tools: "Bash(*) Read Edit Write Glob Grep WebSearch WebFetch AskUserQuestion Agent" --- # FlagRelease Pipeline Orchestrator End-to-end LLM deployment + testing pipeline for multi-chip GPU backends. Orchestrates 4 sub-skills in sequence and produces a final report. ## Skill Components ``` flagrelease/ ├── SKILL.md # This file — orchestration flow └── references/ └── pipeline-state.md # Pipeline state schema, gate logic, data flow ``` **Sub-skills (each independently invokable):** ``` ../install-stack/ # Step 2: Install 5 packages │ ├── SKILL.md │ ├── scripts/ │ │ ├── detect_network.py # Probe GitHub/PyPI, return mirror config │ │ ├── collect_env_info.py # Python/glibc/arch/vendor/disk info │ │ ├── select_flagtree_wheel.py # Match vendor+python+glibc → wheel │ │ └── validate_packages.py # Import-test all 5 packages │ └── references/ │ ├── vendor-mappings.md # FlagCX make flags, adaptor names │ └── network-mirrors.md # Mirror config rules ../env-verify/ # Step 3: Qwen3-0.6B smoke test │ ├── SKILL.md │ ├── scripts/ │ │ ├── run_offline_inference.py # Phase A: offline inference test │ │ └── test_serve_mode.py # Phase B: serve + health + chat test │ └── references/ │ └── error-classification.md # Layer-based error classification ../model-verify/ # Step 4: Target model ± multi-chip │ ├── SKILL.md │ ├── scripts/ │ │ └── diff_analysis.py # Compare Run A vs Run B results │ └── references/ │ └── multichip-errors.md # Multi-chip error patterns ../perf-test/ # Steps 5+6: Accuracy + Performance │ ├── SKILL.md │ ├── scripts/ │ │ ├── run_benchmark.py # Run single benchmark profile │ │ └── run_all_benchmarks.py # Run all profiles + summarize │ └── references/ │ └── benchmark-profiles.md # Profile definitions and metrics ``` ## Pipeline Overview ``` [Prerequisite: /gpu-container-setup already done by another team] │ ▼ install-stack → Install 5 packages (vLLM, FlagTree, FlagGems, FlagCX, plugin) │ scripts: detect_network, collect_env_info, select_flagtree_wheel │ │ GATE: vLLM + plugin must succeed ▼ env-verify → Smoke test with Qwen3-0.6B (FlagGems/CX OFF) │ scripts: run_offline_inference, test_serve_mode │ │ Verify Layers 0-3 ▼ model-verify → Target model test (OFF then ON), diff analysis │ scripts: run_offline_inference, test_serve_mode, diff_analysis │ │ Determine which stack works (full vs base) ▼ perf-test → Accuracy (placeholder) + Performance benchmarks │ scripts: run_benchmark, run_all_benchmarks ▼ Final Report ``` ## Prerequisites A running Docker container with: - PyTorch installed and GPU-accessible - Container name known (e.g. `flagrelease-worker`) This container is produced by `/gpu-container-setup` (maintained by another team). ## Execution Flow Read `references/pipeline-state.md` for the full state schema and gate logic. ### Step 0: Gather Initial Context Ask user for container name (or detect running containers): ```bash docker ps --format '{{.Names}}' | head -10 ``` Verify the container is running: ```bash docker inspect --format='{{.State.Status}}' <CONTAINER> | grep -q running ``` Initialize pipeline state (see `references/pipeline-state.md`). ### Step 1: Install Software Stack Read and follow `../install-stack/SKILL.md`. The install-stack skill will: 1. Copy `scripts/collect_env_info.py` into container → get vendor, Python, glibc 2. Copy `scripts/detect_network.py` into container → get mirror config 3. Install 5 packages in order, using `scripts/select_flagtree_wheel.py` for FlagTree 4. Run `scripts/validate_packages.py` inside container → get final status **Gate check:** If `gate_passed` is false (vLLM or plugin failed) → **STOP pipeline**. Report FAIL with install errors. Store result in pipeline state. ### Step 2: Environment Verification Read and follow `../env-verify/SKILL.md`. The env-verify skill will: 1. Download Qwen3-0.6B (if not cached) 2. Copy `scripts/run_offline_inference.py` into container → Phase A 3. Copy `scripts/test_serve_mode.py` into container → Phase B 4. Classify errors using `references/error-classification.md` **Decision:** Fatal error → STOP. Non-fatal → record and continue. Store result in pipeline state. ### Step 3: Model Verification Read and follow `../model-verify/SKILL.md`. **This step is interactive** — will ask user for model path. The model-verify skill will: 1. Get model info from user (AskUserQuestion) 2. Reuse `run_offline_inference.py` and `test_serve_mode.py` for Run A and Run B 3. Run `scripts/diff_analysis.py` to compare results 4. Determine `recommended_stack` (full/base/none) **Decision:** If `recommended_stack` is `none` (Run A failed) → STOP. Store result in pipeline state (including model_path, tp_size, recommended_stack). ### Step 4: Performance Test Read and follow `../perf-test/SKILL.md`. The perf-test skill will: 1. Start vllm serve with recommended stack 2. Copy `scripts/run_all_benchmarks.py` into container → run 5 profiles 3. Collect metrics and produce summary table Store result in pipeline state. ### Step 5: Final Report Compile all results from pipeline state into a final report: ```json { "status": "PASS | PARTIAL | FAIL", "pipeline": "flagrelease", "container": "<name>", "vendor": "<vendor>", "model": "<path>", "tensor_parallel_size": 8, "steps": { "install_stack": { "status": "...", "packages": {...} }, "env_verify": { "status": "...", "phase_a": "...", "phase_b": "..." }, "model_verify": { "status": "...", "run_a": "...", "run_b": "...", "recommended_stack": "..." }, "perf_test": { "status": "...", "profiles_passed": "5/5", "summary_table": "..." } }, "errors": [...], "conclusion": "Pipeline completed. ..." } ``` Present to user with clear summary: 1. Which packages installed / failed 2. Whether base stack works 3. Whether multi-chip stack works (and which component failed if not) 4. Performance numbers (summary table) 5. All errors with layer classification **Overall status:** - `PASS` — all steps pass, full multi-chip stack works - `PARTIAL` — model works with degraded stack, or some perf profiles failed - `FAIL` — model cannot serve (gate or Run A failure) ## Design Rules - **Every operation has a timeout** — no hangs allowed - **Every error is caught** with precise location (step, phase, layer, cause) - **Pipeline always completes** with success or structured error report - **One sub-step failure does NOT skip unrelated steps** (unless gate failure) - **Network uses mirrors** when direct access fails - **Scripts produce JSON** — structured, parseable, comparable across runs


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal:  | SkillSpector: clean (LOW)