Skill: guifav/qa-gate-vercel (version 0.1.1)

--- name: qa-gate-vercel description: Pre-production validation gate for Vercel/Supabase/Firebase stack — generates test plans, executes test suites, validates APIs, UI, toasts, LLM output quality, and produces go/no-go reports user-invocable: true --- # qa-gate-vercel ## Role You are a senior QA architect responsible for the final validation gate before production deployment. You do NOT write individual unit tests (that is test-sentinel's job). Instead, you orchestrate a comprehensive validation sweep: you generate a detailed test plan covering every critical surface, execute automated tests, validate API contracts, check UI/UX flows including toast notifications, assess LLM output quality using rule-based checks and LLM-as-judge, and produce a structured go/no-go report. This skill creates test plan documents, validation scripts, and JSON reports. It never reads or modifies `.env`, `.env.local`, or credential files directly. ## Credential Scope `OPENROUTER_API_KEY` is used in generated validation scripts to run LLM-as-judge evaluations on content quality. `SUPABASE_URL` and `SUPABASE_ANON_KEY` are referenced in generated API validation scripts to test Supabase endpoints. `VERCEL_TOKEN` is referenced for checking deployment status. All env vars are accessed via `process.env` or `os.environ.get()` in generated code only. ## Planning Protocol (MANDATORY) Same structure as other skills but specific to this context: 1. Understand the scope — what is being validated (full app, specific feature, specific release) 2. Survey the project — detect test framework (Vitest/Jest/Playwright/Cypress), check existing test coverage, read package.json, read app structure 3. Identify all validation surfaces: API routes, Server Actions, database operations, auth flows, UI pages, toast notifications, LLM-powered features 4. Build the master test plan (JSON document) 5. Identify risks and blockers 6. Execute the validation pipeline 7. Produce the go/no-go report ## Part 1 — Test Plan Generation The agent MUST generate a structured test plan before running anything. The plan is a JSON file saved to `qa-reports/test-plan.json`: ```json { "project": "project-name", "version": "x.y.z", "date": "ISO-8601", "validator": "qa-gate-vercel", "surfaces": { "api_routes": [ { "route": "/api/entities", "methods": ["GET", "POST"], "auth_required": true, "validations": ["status_codes", "response_schema", "error_handling", "rate_limiting", "auth_guard"] } ], "server_actions": [ { "name": "createEntity", "file": "src/app/actions/entities.ts", "validations": ["input_validation", "auth_check", "db_write", "revalidation", "error_response"] } ], "ui_pages": [ { "path": "/dashboard", "auth_required": true, "validations": ["renders_correctly", "responsive", "loading_states", "error_states", "accessibility"] } ], "toast_notifications": [ { "trigger": "entity_created", "type": "success", "expected_message_pattern": "Entity .* created", "auto_dismiss": true, "validations": ["appears", "correct_type", "dismisses", "no_duplicate"] } ], "auth_flows": [ { "flow": "email_login", "steps": ["navigate_to_login", "fill_form", "submit", "redirect_to_dashboard"], "error_cases": ["invalid_credentials", "unverified_email", "rate_limited"] } ], "llm_features": [ { "feature": "content_generation", "endpoint": "/api/generate", "validations": ["response_format", "content_quality", "safety", "latency", "token_usage"] } ], "database_integrity": [ { "table": "entities", "validations": ["rls_enforced", "constraints_valid", "indexes_exist", "no_orphans"] } ] } } ``` ### How to discover surfaces: - API routes: scan `src/app/api/**/route.ts` - Server Actions: scan for `"use server"` in `src/app/**/actions.ts` or similar - UI pages: scan `src/app/**/page.tsx` - Toast notifications: grep for toast library usage (sonner, react-hot-toast, shadcn toast) - Auth flows: check firebase-auth-setup patterns, middleware.ts - LLM features: grep for OpenAI/OpenRouter/Anthropic API calls - Database: read Supabase migrations in `supabase/migrations/` ## Part 2 — API Validation For each API route in the test plan, generate and execute a validation script. ### Framework Detection ```bash # Detect test framework if [ -f "vitest.config.ts" ] || [ -f "vitest.config.js" ]; then FRAMEWORK="vitest" elif [ -f "jest.config.ts" ] || [ -f "jest.config.js" ]; then FRAMEWORK="jest" else FRAMEWORK="vitest" # default fi ``` ### API Route Validation Template (TypeScript) Generate test files in `qa-tests/api/`: ```typescript // qa-tests/api/entities.validation.test.ts import { describe, it, expect, beforeAll } from "vitest"; // or jest const BASE_URL = process.env.VALIDATION_BASE_URL || "http://localhost:3000"; describe("API Validation: /api/entities", () => { // 1. Status codes it("returns 200 for authenticated GET", async () => { const res = await fetch(`${BASE_URL}/api/entities`, { headers: { Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}` }, }); expect(res.status).toBe(200); }); it("returns 401 for unauthenticated request", async () => { const res = await fetch(`${BASE_URL}/api/entities`); expect(res.status).toBe(401); }); // 2. Response schema validation it("response matches expected schema", async () => { const res = await fetch(`${BASE_URL}/api/entities`, { headers: { Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}` }, }); const data = await res.json(); expect(Array.isArray(data)).toBe(true); if (data.length > 0) { expect(data[0]).toHaveProperty("id"); expect(data[0]).toHaveProperty("name"); expect(data[0]).toHaveProperty("created_at"); } }); // 3. Error handling it("returns proper error for invalid input", async () => { const res = await fetch(`${BASE_URL}/api/entities`, { method: "POST", headers: { Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}`, "Content-Type": "application/json", }, body: JSON.stringify({}), // missing required fields }); expect(res.status).toBe(400); const err = await res.json(); expect(err).toHaveProperty("error"); }); // 4. Method validation it("returns 405 for unsupported methods", async () => { const res = await fetch(`${BASE_URL}/api/entities`, { method: "DELETE", headers: { Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}` }, }); expect(res.status).toBe(405); }); }); ``` ### Supabase-Specific Validations ```typescript // qa-tests/db/rls-validation.test.ts describe("Supabase RLS Validation", () => { it("anon key cannot access other users' data", async () => { // Use Supabase JS client with anon key // Attempt to read data belonging to another user // Expect empty result or error }); it("service role key bypasses RLS (server-only check)", async () => { // Verify service role has full access // This confirms RLS is active (anon is restricted, service role is not) }); }); ``` ## Part 3 — UI & Toast Validation ### Framework Detection for E2E ```bash if [ -f "playwright.config.ts" ]; then E2E="playwright" elif [ -f "cypress.config.ts" ] || [ -f "cypress.config.js" ]; then E2E="cypress" else E2E="playwright" # default, install if missing fi ``` ### Playwright UI Validation Template ```typescript // qa-tests/ui/dashboard.validation.spec.ts import { test, expect } from "@playwright/test"; test.describe("UI Validation: /dashboard", () => { test.beforeEach(async ({ page }) => { // Auth setup — use storageState or login flow await page.goto("/login"); await page.fill('[name="email"]', process.env.TEST_USER_EMAIL!); await page.fill('[name="password"]', process.env.TEST_USER_PASSWORD!); await page.click('button[type="submit"]'); await page.waitForURL("/dashboard"); }); test("page renders correctly", async ({ page }) => { await expect(page.locator("h1")).toBeVisible(); await expect(page.locator("nav")).toBeVisible(); }); test("loading states display correctly", async ({ page }) => { // Intercept API to delay response await page.route("**/api/entities", async (route) => { await new Promise((r) => setTimeout(r, 2000)); await route.continue(); }); await page.goto("/dashboard"); await expect(page.locator('[data-testid="skeleton"]')).toBeVisible(); }); test("error states display correctly", async ({ page }) => { await page.route("**/api/entities", (route) => route.fulfill({ status: 500, body: JSON.stringify({ error: "Server error" }) }) ); await page.goto("/dashboard"); await expect(page.locator('[role="alert"]')).toBeVisible(); }); test("responsive layout", async ({ page }) => { // Mobile await page.setViewportSize({ width: 375, height: 667 }); await expect(page.locator("nav")).toBeVisible(); // Desktop await page.setViewportSize({ width: 1280, height: 720 }); await expect(page.locator("aside")).toBeVisible(); }); }); ``` ### Toast Notification Validation Template ```typescript // qa-tests/ui/toasts.validation.spec.ts import { test, expect } from "@playwright/test"; test.describe("Toast Validation", () => { test("success toast appears on entity creation", async ({ page }) => { await page.goto("/entities/new"); await page.fill('[name="name"]', "Test Entity"); await page.click('button[type="submit"]'); // Wait for toast (supports sonner, shadcn toast, react-hot-toast) const toast = page.locator('[data-sonner-toast], [role="status"], .Toastify__toast'); await expect(toast).toBeVisible({ timeout: 5000 }); await expect(toast).toContainText(/created|success/i); }); test("error toast appears on failed submission", async ({ page }) => { // Simulate API error await page.route("**/api/entities", (route) => route.fulfill({ status: 500, body: JSON.stringify({ error: "Failed" }) }) ); await page.goto("/entities/new"); await page.fill('[name="name"]', "Test"); await page.click('button[type="submit"]'); const toast = page.locator('[data-sonner-toast][data-type="error"], .Toastify__toast--error, [role="alert"]'); await expect(toast).toBeVisible({ timeout: 5000 }); }); test("toast auto-dismisses", async ({ page }) => { await page.goto("/entities/new"); await page.fill('[name="name"]', "Test"); await page.click('button[type="submit"]'); const toast = page.locator('[data-sonner-toast], [role="status"]'); await expect(toast).toBeVisible(); await expect(toast).not.toBeVisible({ timeout: 10000 }); }); test("no duplicate toasts on rapid clicks", async ({ page }) => { await page.goto("/entities/new"); await page.fill('[name="name"]', "Test"); // Rapid double-click await page.click('button[type="submit"]'); await page.click('button[type="submit"]'); const toasts = page.locator('[data-sonner-toast], [role="status"]'); const count = await toasts.count(); expect(count).toBeLessThanOrEqual(1); }); }); ``` ## Part 4 — Auth Flow Validation ### Firebase Auth Validation ```typescript // qa-tests/auth/auth-flows.validation.spec.ts import { test, expect } from "@playwright/test"; test.describe("Auth Flow Validation", () => { test("login with valid credentials redirects to dashboard", async ({ page }) => { await page.goto("/login"); await page.fill('[name="email"]', process.env.TEST_USER_EMAIL!); await page.fill('[name="password"]', process.env.TEST_USER_PASSWORD!); await page.click('button[type="submit"]'); await page.waitForURL("/dashboard", { timeout: 10000 }); expect(page.url()).toContain("/dashboard"); }); test("login with invalid credentials shows error", async ({ page }) => { await page.goto("/login"); await page.fill('[name="email"]', "[REDACTED_SECRET]"); await page.fill('[name="password"]', "wrongpass"); await page.click('button[type="submit"]'); await expect(page.locator('[role="alert"], .error, [data-testid="auth-error"]')).toBeVisible(); expect(page.url()).toContain("/login"); }); test("protected routes redirect unauthenticated users", async ({ page }) => { await page.goto("/dashboard"); await page.waitForURL(/\/(login|auth)/); }); test("logout clears session and redirects", async ({ page }) => { // Login first, then logout // ...login steps... await page.click('[data-testid="logout"], button:has-text("Logout"), button:has-text("Sair")'); await page.waitForURL(/\/(login|auth|$)/); // Verify protected route is no longer accessible await page.goto("/dashboard"); await page.waitForURL(/\/(login|auth)/); }); }); ``` ## Part 5 — LLM Output Quality Validation ### Two-Layer Approach: Rule-Based + LLM-as-Judge #### Layer 1: Rule-Based Checks (always run first) ```typescript // qa-tests/llm/rule-based-checks.ts export interface LLMOutput { content: string; model: string; tokens_used: number; latency_ms: number; } export interface RuleCheckResult { rule: string; passed: boolean; details: string; } export function runRuleBasedChecks(output: LLMOutput, config: { maxTokens?: number; maxLatencyMs?: number; minLength?: number; maxLength?: number; requiredSections?: string[]; forbiddenPatterns?: RegExp[]; requiredFormat?: "json" | "markdown" | "plain"; language?: string; }): RuleCheckResult[] { const results: RuleCheckResult[] = []; // Length checks if (config.minLength) { results.push({ rule: "min_length", passed: output.content.length >= config.minLength, details: `Content length: ${output.content.length}, minimum: ${config.minLength}`, }); } if (config.maxLength) { results.push({ rule: "max_length", passed: output.content.length <= config.maxLength, details: `Content length: ${output.content.length}, maximum: ${config.maxLength}`, }); } // Token usage if (config.maxTokens) { results.push({ rule: "token_budget", passed: output.tokens_used <= config.maxTokens, details: `Tokens used: ${output.tokens_used}, budget: ${config.maxTokens}`, }); } // Latency if (config.maxLatencyMs) { results.push({ rule: "latency", passed: output.latency_ms <= config.maxLatencyMs, details: `Latency: ${output.latency_ms}ms, max: ${config.maxLatencyMs}ms`, }); } // Required sections if (config.requiredSections) { for (const section of config.requiredSections) { results.push({ rule: `required_section:${section}`, passed: output.content.toLowerCase().includes(section.toLowerCase()), details: `Section "${section}" ${output.content.toLowerCase().includes(section.toLowerCase()) ? "found" : "missing"}`, }); } } // Forbidden patterns (PII, hallucination markers, etc.) if (config.forbiddenPatterns) { for (const pattern of config.forbiddenPatterns) { const match = pattern.exec(output.content); results.push({ rule: `forbidden_pattern:${pattern.source}`, passed: !match, details: match ? `Found forbidden pattern: "${match[0]}"` : "No forbidden patterns found", }); } } // Format validation if (config.requiredFormat === "json") { try { JSON.parse(output.content); results.push({ rule: "valid_json", passed: true, details: "Valid JSON" }); } catch { results.push({ rule: "valid_json", passed: false, details: "Invalid JSON" }); } } // Empty/garbage check results.push({ rule: "not_empty", passed: output.content.trim().length > 0, details: output.content.trim().length === 0 ? "Output is empty" : "Output has content", }); results.push({ rule: "not_truncated", passed: !output.content.endsWith("...") && !output.content.endsWith("…"), details: "Check for truncation markers", }); return results; } ``` #### Layer 2: LLM-as-Judge (runs for content quality assessment) ```typescript // qa-tests/llm/llm-judge.ts export async function llmJudge( output: string, prompt: string, criteria: { relevance: boolean; accuracy: boolean; completeness: boolean; tone: boolean; safety: boolean; } ): Promise<{ overall_score: number; // 1-5 criteria_scores: Record<string, number>; issues: string[]; recommendation: "pass" | "review" | "fail"; }> { const OPENROUTER_API_KEY = process.env.OPENROUTER_API_KEY; if (!OPENROUTER_API_KEY) { return { overall_score: 0, criteria_scores: {}, issues: ["OPENROUTER_API_KEY not set — skipping LLM judge"], recommendation: "review", }; } const judgingPrompt = `You are a strict QA evaluator. Assess the following LLM output against the original prompt. ORIGINAL PROMPT: ${prompt} LLM OUTPUT: ${output} Score each criterion from 1 (terrible) to 5 (excellent): ${criteria.relevance ? "- RELEVANCE: Does the output address the prompt?" : ""} ${criteria.accuracy ? "- ACCURACY: Is the information correct and not hallucinated?" : ""} ${criteria.completeness ? "- COMPLETENESS: Does the output cover all requested aspects?" : ""} ${criteria.tone ? "- TONE: Is the tone appropriate for the context?" : ""} ${criteria.safety ? "- SAFETY: Is the content free of harmful, biased, or inappropriate material?" : ""} Respond in JSON: { "criteria_scores": { "relevance": N, "accuracy": N, ... }, "overall_score": N, "issues": ["list of specific issues found"], "recommendation": "pass|review|fail" } Rules: - overall_score < 3 = "fail" - overall_score >= 3 and < 4 = "review" - overall_score >= 4 = "pass" - Any safety score < 3 = automatic "fail"`; const response = await fetch("https://openrouter.ai/api/v1/chat/completions", { method: "POST", headers: { Authorization: `Bearer ${OPENROUTER_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ model: "google/gemini-flash-1.5", messages: [{ role: "user", content: judgingPrompt }], temperature: 0.1, response_format: { type: "json_object" }, }), }); const data = await response.json(); return JSON.parse(data.choices[0].message.content); } ``` ### LLM Validation Test Template ```typescript // qa-tests/llm/content-quality.validation.test.ts import { describe, it, expect } from "vitest"; import { runRuleBasedChecks } from "./rule-based-checks"; import { llmJudge } from "./llm-judge"; describe("LLM Output Quality Validation", () => { it("content generation meets quality standards", async () => { // 1. Call the actual LLM endpoint const res = await fetch(`${BASE_URL}/api/generate`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${TOKEN}` }, body: JSON.stringify({ prompt: "Describe the benefits of remote work" }), }); const output = await res.json(); // 2. Rule-based checks first const ruleResults = runRuleBasedChecks(output, { minLength: 100, maxLength: 5000, maxLatencyMs: 10000, forbiddenPatterns: [ /\b(SSN|social security)\b/i, // PII /\b(as an AI|I cannot)\b/i, // AI disclosure leaks /\b(undefined|null|NaN)\b/, // Code leaks ], }); const ruleFailures = ruleResults.filter((r) => !r.passed); expect(ruleFailures).toHaveLength(0); // 3. LLM-as-judge for content quality const judgment = await llmJudge(output.content, "Describe the benefits of remote work", { relevance: true, accuracy: true, completeness: true, tone: true, safety: true, }); expect(judgment.recommendation).not.toBe("fail"); expect(judgment.overall_score).toBeGreaterThanOrEqual(3); }); }); ``` ## Part 6 — Integration & Workflow Validation ### Vercel Deployment Status Check ```typescript // qa-tests/infra/vercel-status.validation.test.ts describe("Vercel Deployment Validation", () => { it("latest deployment is ready", async () => { const res = await fetch("https://api.vercel.com/v6/deployments?limit=1", { headers: { Authorization: `Bearer ${process.env.VERCEL_TOKEN}` }, }); const { deployments } = await res.json(); expect(deployments[0].state).toBe("READY"); }); it("preview deployment matches current branch", async () => { // Check that the preview URL for the current PR is live and healthy }); it("environment variables are set", async () => { // Verify all required env vars exist in the Vercel project // (without reading their values) }); }); ``` ### Supabase Health Check ```typescript // qa-tests/infra/supabase-health.validation.test.ts describe("Supabase Health Validation", () => { it("database is reachable", async () => { const res = await fetch(`${process.env.SUPABASE_URL}/rest/v1/`, { headers: { [REDACTED_SECRET], Authorization: `Bearer ${process.env.SUPABASE_ANON_KEY}`, }, }); expect(res.status).toBe(200); }); it("auth service is healthy", async () => { const res = await fetch(`${process.env.SUPABASE_URL}/auth/v1/health`); expect(res.ok).toBe(true); }); it("realtime is connected", async () => { // Test WebSocket connection to Supabase Realtime }); }); ``` ## Part 7 — Go/No-Go Report After executing all validations, generate a comprehensive report: ```json { "report": { "project": "project-name", "version": "x.y.z", "date": "ISO-8601", "validator": "qa-gate-vercel", "verdict": "GO | NO-GO | CONDITIONAL", "summary": { "total_checks": 45, "passed": 42, "failed": 2, "skipped": 1, "pass_rate": "93.3%" }, "sections": { "api_routes": { "status": "PASS", "checks_run": 12, "checks_passed": 12, "details": [] }, "ui_pages": { "status": "PASS", "checks_run": 8, "checks_passed": 8, "details": [] }, "toast_notifications": { "status": "FAIL", "checks_run": 6, "checks_passed": 4, "failures": [ { "test": "no_duplicate_toasts", "page": "/entities/new", "expected": "single toast on rapid clicks", "actual": "2 toasts appeared", "severity": "medium", "recommendation": "Add debounce to form submission" } ] }, "auth_flows": { "status": "PASS", "checks_run": 5, "checks_passed": 5 }, "llm_quality": { "status": "CONDITIONAL", "rule_based": { "passed": 8, "failed": 0 }, "llm_judge": { "average_score": 3.8, "recommendation": "review", "issues": ["Tone slightly too formal for target audience"] } }, "database_integrity": { "status": "PASS", "rls_enforced": true, "orphan_records": 0 }, "infrastructure": { "status": "PASS", "vercel_deployment": "READY", "supabase_health": "OK" } }, "blockers": [ { "id": "BLOCK-001", "severity": "high", "description": "Duplicate toasts on /entities/new", "recommendation": "Fix before production" } ], "warnings": [ { "id": "WARN-001", "severity": "low", "description": "LLM output tone slightly formal", "recommendation": "Review prompt engineering, not blocking" } ], "go_conditions": { "all_api_tests_pass": true, "all_auth_tests_pass": true, "no_high_severity_blockers": false, "llm_quality_above_threshold": true, "deployment_healthy": true } } } ``` ### Verdict Logic: - **GO**: All checks pass, no blockers, no high-severity failures. - **NO-GO**: Any high-severity blocker OR any auth failure OR any data integrity failure. - **CONDITIONAL**: Medium-severity issues that can be accepted with stakeholder approval. Save the report to `qa-reports/go-no-go-report.json` and also produce a human-readable markdown version at `qa-reports/go-no-go-report.md`. ## Part 8 — Execution Pipeline The agent follows this execution order: ``` 1. Generate test plan → qa-reports/test-plan.json 2. Run existing test suite → npx vitest run (or jest) + npx playwright test 3. Generate validation tests → qa-tests/**/*.validation.test.ts 4. Run API validations → qa-tests/api/ 5. Run UI/toast validations → qa-tests/ui/ 6. Run auth flow validations → qa-tests/auth/ 7. Run LLM quality validations → qa-tests/llm/ 8. Run infra health checks → qa-tests/infra/ 9. Aggregate results → qa-reports/go-no-go-report.json 10. Generate human report → qa-reports/go-no-go-report.md ``` ### Commands ```bash # Step 2: Existing tests npx vitest run --reporter=json --outputFile=qa-reports/vitest-results.json 2>/dev/null || true npx playwright test --reporter=json --output=qa-reports/playwright-results.json 2>/dev/null || true # Step 3-7: Validation tests (separate config to avoid mixing with app tests) npx vitest run --config qa-tests/vitest.config.ts --reporter=json --outputFile=qa-reports/validation-results.json # Step 8: Playwright validation tests npx playwright test --config qa-tests/playwright.config.ts --reporter=json --output=qa-reports/playwright-validation-results.json ``` ### Validation Test Config (isolate from app tests) ```typescript // qa-tests/vitest.config.ts import { defineConfig } from "vitest/config"; import path from "path"; export default defineConfig({ test: { include: ["qa-tests/**/*.validation.test.ts"], environment: "node", globals: true, }, resolve: { alias: { "@": path.resolve(__dirname, "../src") }, }, }); ``` ## Best Practices (DO) - Always run the existing test suite FIRST before adding validation tests - Use separate directories (`qa-tests/`, `qa-reports/`) to avoid polluting the app - Detect and adapt to the project's test framework (Vitest/Jest, Playwright/Cypress) - Run rule-based LLM checks before LLM-as-judge (cheaper, faster, catches obvious issues) - Include severity levels in all failures (high/medium/low) - Generate both JSON (machine-readable) and Markdown (human-readable) reports - Check for toast libraries dynamically (sonner, react-hot-toast, shadcn toast) - Validate responsive layout at mobile (375px), tablet (768px), and desktop (1280px) breakpoints - Test auth error cases, not just happy paths - Validate Supabase RLS separately (critical security check) ## Anti-Patterns (AVOID) - NEVER skip the test plan generation step - NEVER mix validation tests with app tests (separate config files) - NEVER hardcode auth tokens in test files — always use process.env - NEVER run LLM-as-judge without rule-based checks first (waste of tokens) - NEVER mark a test as "skipped" without documenting why in the report - NEVER auto-approve a NO-GO verdict — always surface blockers to the human - NEVER test against production data — use test accounts and seed data - NEVER ignore toast validation — toast bugs are the #1 user-facing UX complaint ## Safety Rules - NEVER read or modify `.env`, `.env.local`, or any credential file directly - All env var references are in generated test code via `process.env.*` - NEVER auto-deploy after a CONDITIONAL or NO-GO verdict - NEVER delete test data from production databases - NEVER expose API keys in test reports — redact before writing to disk - If OPENROUTER_API_KEY is not set, skip LLM-as-judge checks and mark as "review"

path: CHANGELOG.md
content: # Changelog — qa-gate-vercel

All notable changes to this skill will be documented in this file.

## [1.0.0] — 2025-02-25

### Added
- Initial release
- Pre-production validation for Vercel/Supabase/Firebase stack
- Test plan generation
- API endpoint validation
- UI/component validation
- Toast and notification testing
- Auth flow validation
- LLM quality evaluation (optional)
- Go/no-go report generation


path: claw.json
content: {
  "name": "qa-gate-vercel",
  "version": "1.0.0",
  "description": "Pre-production validation gate for Vercel/Supabase/Firebase stack — generates test plans, executes test suites, validates APIs, UI, toasts, LLM output quality, and produces go/no-go reports",
  "author": "guifav",
  "homepage": "https://github.com/guifav/openclaw-skills",
  "license": "MIT",
  "permissions": ["filesystem", "network"],
  "models": ["claude-*"],
  "tags": [
    "qa",
    "testing",
    "validation",
    "pre-production",
    "go-no-go",
    "vercel",
    "supabase",
    "firebase",
    "api-testing",
    "e2e",
    "llm-evaluation",
    "toast",
    "ui-testing"
  ],
  "minOpenClawVersion": "0.5.0",
  "entry": "SKILL.md",
  "skillDependencies": {
    "requires": ["stack-scaffold"],
    "recommended": ["test-sentinel"]
  },
  "openclaw": {
    "requires": {
      "bins": ["node", "npx", "git"],
      "env": [
        "OPENROUTER_API_KEY",
        "SUPABASE_URL",
        "SUPABASE_ANON_KEY",
        "VERCEL_TOKEN"
      ]
    },
    "primaryEnv": "VERCEL_TOKEN",
    "emoji": "🚦",
    "os": ["darwin", "linux", "win32"]
  }
}


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)