Skill: tahseen137/qa-patrol (version 1.0.3)

--- name: qa-patrol description: > Automated QA testing for web apps using local browser automation. Runs entirely on your machine — no data leaves, no cloud services, no external servers. Level 1 (smoke tests) needs only a URL. Level 2 (auth/payment testing) uses optional env vars for test credentials. Level 3 (static analysis, DB checks) optionally reads local files and connects to a user-provided database. Supports Supabase/Firebase auth, Stripe payments, React Native Web, Next.js, and SPAs. --- # QA Patrol Automated QA testing skill for web applications. Catches bugs that unit tests miss: cross-platform issues, auth state problems, data integrity failures, and integration breakages. ## Security & Privacy **All tests run locally on your machine. Nothing is sent to external servers. The browser automation uses OpenClaw's built-in browser control — no cloud services involved.** ### Permissions by Level | Level | What it does | Permissions needed | Env vars needed | |-------|-------------|-------------------|-----------------| | **1 — Smoke** | Loads pages, checks for errors | `browser` only | `APP_URL` (or pass `--url`) | | **2 — Auth/Payments** | Tests sign-in, checkout flows | `browser` only | Test account credentials (see below) | | **3 — Static Analysis** | Scans local source code for bug patterns | `browser` + `read` | None (uses local `repo_path`) | | **3 — DB Integrity** | Compares DB values to UI display | `browser` | `DATABASE_URL` | **The `read` permission is ONLY needed for Level 3 static analysis.** Level 1 and Level 2 tests use browser automation exclusively. If you only run Level 1/2 tests, the skill never accesses local files. ### Environment Variables (all optional) | Variable | Required | Used by | Purpose | |----------|----------|---------|---------| | `APP_URL` | No | Level 1+ | Target app URL (can also use `--url` flag) | | `ADMIN_EMAIL` | No | Level 2 | Admin test account email | | `ADMIN_PASSWORD` | No | Level 2 | Admin test account password | | `FREE_EMAIL` | No | Level 2 | Free-tier test account email | | `FREE_PASSWORD` | No | Level 2 | Free-tier test account password | | `PRO_EMAIL` | No | Level 2 | Pro test account email | | `PRO_PASSWORD` | No | Level 2 | Pro test account password | | `DATABASE_URL` | No | Level 3 | DB connection for data integrity checks | **⚠️ Use test credentials only — never supply production passwords or production DATABASE_URL.** ### Secrets Handling - **NEVER hardcode secrets** in test plans — always use environment variable interpolation: `${env.ADMIN_PASSWORD}` - Credentials are read from your local environment at runtime - Test plans in this skill's examples use only `${env.VAR}` placeholders - The skill does not persist, log, or transmit credentials ### Security Pattern Detection (Not Exploitation) The `references/bug-patterns.md` file contains regex patterns for **detecting exposed secrets** in codebases (e.g., `sk_live_`, `api_key=`). These are **detection patterns** used to help developers find and fix security issues — they are NOT exploitation tools. This is standard practice in security linters like ESLint, Semgrep, and GitHub's secret scanning. ### No Install Scripts, No Code Files This is an **instruction-only skill** — it contains no executable code, no install scripts, and no third-party dependencies. The entire security surface is the SKILL.md instructions and OpenClaw's built-in browser/read capabilities. ## Quick Start ### Level 1: Zero-Config Smoke Test ```bash # Just provide a URL qa-patrol https://example.com ``` ### Level 2: With Auth/Payments ```bash # Use a test plan template qa-patrol --plan auth-supabase.yaml --url https://example.com ``` ### Level 3: Full Config ```bash # Custom test plan with data integrity checks qa-patrol --plan my-app.yaml ``` ## Workflow ### 1. Load or Generate Test Plan If a YAML test plan is provided, load it. Otherwise, generate a basic plan: ```yaml app: url: <provided URL> name: <extracted from page title> tests: smoke: - name: Homepage loads navigate: / assert: - element_exists: main - no_console_errors: true ``` See `assets/templates/` for test plan templates: - `basic.yaml` - Zero-config smoke test - `auth-supabase.yaml` - Supabase auth flows - `payments-stripe.yaml` - Stripe checkout testing - `full-saas.yaml` - Complete SaaS test plan ### 2. Execute Tests Run tests in order: smoke → auth → payments → data_integrity → static_analysis. For each test: 1. Navigate to the target URL 2. Execute steps (click, type, wait) 3. Capture snapshot and console logs 4. Evaluate assertions 5. Record PASS/FAIL/SKIP with evidence #### Browser Automation Patterns ```python # Navigate and snapshot browser(action="navigate", targetUrl="https://example.com/page") browser(action="snapshot") # Form interaction browser(action="act", request={"kind": "click", "ref": "email_input"}) browser(action="act", request={"kind": "type", "ref": "email_input", "text": "[REDACTED_SECRET]"}) browser(action="act", request={"kind": "click", "ref": "submit_button"}) # Check console for errors browser(action="console", level="error") ``` See `references/test-patterns.md` for complete automation patterns. ### 3. Check for Known Bug Patterns Scan codebase (if accessible) for anti-patterns: | Pattern | What to grep | Severity | |---------|-------------|----------| | Alert.alert on web | `Alert.alert` without `Platform.OS` guard | High | | Linking in Modal | `Linking.openURL` inside Modal component | High | | Missing RLS | Supabase queries without proper auth context | High | | Hardcoded secrets | API keys in client code | Critical | See `references/bug-patterns.md` for the full catalog. ### 4. Data Integrity Checks (Level 3) When `data_integrity` tests are defined: 1. Execute the DB query (requires DB access) 2. Navigate to the UI path 3. Extract the displayed value 4. Compare against query result 5. Flag mismatches with severity based on % difference ### 5. Generate Report Output a structured report: ```markdown # QA Report: [App Name] **Date**: YYYY-MM-DD HH:MM **URL**: https://example.com **Confidence**: 87% ## Summary | Category | Pass | Fail | Skip | |----------|------|------|------| | Smoke | 5 | 0 | 0 | | Auth | 3 | 1 | 0 | | Payments | 0 | 0 | 2 | ## Failures ### [FAIL] Auth: Session persistence after refresh **Steps**: Sign in → Refresh page → Check auth state **Expected**: User remains signed in **Actual**: Redirected to login page **Evidence**: [screenshot] **Severity**: High ## Recommendations 1. Fix session persistence (likely cookie/localStorage issue) 2. Add Platform.OS guards to Alert.alert calls ``` See `references/report-format.md` for the complete template. ## Test Plan Reference ### App Configuration ```yaml app: url: https://example.com # Required: base URL name: My App # Optional: display name stack: expo-web # expo-web | nextjs | spa | static ``` ### Auth Configuration ```yaml auth: provider: supabase # supabase | firebase | auth0 | custom login_path: /auth # Path to login page accounts: admin: email: [REDACTED_SECRET] [REDACTED_SECRET] # Use env vars for secrets free: email: [REDACTED_SECRET] [REDACTED_SECRET] guest: true # Test anonymous/guest mode ``` ### Test Types #### Smoke Tests ```yaml tests: smoke: - name: Homepage loads navigate: / assert: - element_exists: main - no_console_errors: true - no_network_errors: true - name: Navigation works navigate: / steps: - click: { ref: nav_link } - assert: { url_contains: "/target" } ``` #### Auth Tests ```yaml tests: auth: - name: Sign in flow steps: - navigate: /auth - type: { ref: email_input, text: "${auth.accounts.free.email}" } - type: { ref: password_input, text: "${auth.accounts.free.password}" } - click: { ref: sign_in_button } - wait: { url_contains: "/home", timeout: 5000 } - assert: { element_exists: "user_avatar" } - name: Sign out flow requires: signed_in steps: - click: { ref: user_menu } - click: { ref: sign_out_button } - assert: { url_contains: "/auth" } - name: Session persistence requires: signed_in steps: - navigate: /home - refresh: true - assert: { element_exists: "user_avatar" } ``` #### Payment Tests ```yaml tests: payments: provider: stripe tests: - name: Checkout creation steps: - navigate: /pricing - click: { ref: pro_plan_button } - wait: { url_contains: "checkout.stripe.com", timeout: 10000 } - assert: { element_exists: "cardNumber" } ``` #### Data Integrity Tests ```yaml tests: data_integrity: - name: Card count matches query: "SELECT count(*) FROM cards WHERE country='CA'" ui_path: /settings ui_selector: "[data-testid='card-count']" tolerance: 0 # Exact match required - name: Points calculation query: "SELECT points_rate FROM tiers WHERE name='Gold'" ui_path: /calculator ui_selector: ".points-display" tolerance: 0.01 # 1% tolerance ``` #### Static Analysis ```yaml tests: static_analysis: scan_path: ./src patterns: - name: Alert.alert without Platform guard grep: "Alert\\.alert" exclude_grep: "Platform\\.OS" severity: high fix_hint: "Wrap in Platform.OS check or use cross-platform alert" - name: Hardcoded API keys grep: "(sk_live_|pk_live_|api_key.*=.*['\"][a-zA-Z0-9]{20,})" severity: critical ``` ### Assertions Reference | Assertion | Description | |-----------|-------------| | `element_exists: "ref"` | Element with ref is in DOM | | `element_visible: "ref"` | Element is visible | | `text_contains: "string"` | Page contains text | | `url_contains: "/path"` | URL includes path | | `no_console_errors: true` | No console.error calls | | `no_network_errors: true` | No failed network requests | | `value_equals: { ref, value }` | Input value matches | | `count_equals: { ref, count }` | Number of matching elements | ### Variable Interpolation Use `${...}` for dynamic values: - `${auth.accounts.free.email}` - From test plan - `${env.API_KEY}` - From environment - `${captured.user_id}` - From previous step capture ## Confidence Scoring Calculate confidence based on test coverage and results: ``` base_confidence = 50 per_smoke_pass = +5 (max 20) per_auth_pass = +8 (max 24) per_payment_pass = +10 (max 20) per_data_check_pass = +6 (max 18) static_analysis_clean = +8 no_critical_failures = +10 final_confidence = min(base + bonuses - penalties, 100) ``` Penalties: - Critical failure: -20 - High severity failure: -10 - Medium severity failure: -5 - Skipped critical test: -5 ## Files ### References - `references/test-patterns.md` - Browser automation patterns and examples - `references/bug-patterns.md` - Known bug patterns to detect - `references/report-format.md` - QA report template ### Templates - `assets/templates/basic.yaml` - Zero-config smoke test - `assets/templates/auth-supabase.yaml` - Supabase auth testing - `assets/templates/payments-stripe.yaml` - Stripe payment testing - `assets/templates/full-saas.yaml` - Complete SaaS test plan ### Examples - `assets/examples/rewardly.yaml` - Real-world React Native Web app test plan ## Tips 1. **Start with smoke tests** - Verify basic functionality before auth/payments 2. **Use guest mode first** - Test without auth to establish baseline 3. **Check console early** - Console errors often reveal root causes 4. **Screenshot failures** - Always capture evidence for debugging 5. **Test cache states** - Sign out and clear cache to expose hidden issues 6. **Verify cross-platform** - If React Native Web, test alert/linking patterns


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (MEDIUM)