npm package: @qate/cli
Description: Qate CLI for CI/CD pipeline integration - run test sets and test sequences
Author: Qate Team
Latest version: 1.1.5

# @qate/cli

Qate CLI for CI/CD pipeline integration. Run test sets and test sequences from your CI pipelines.

## Installation

```bash
npm install -g @qate/cli
```

## Authentication

Generate an API key from Settings > CI/CD API Keys in the Qate dashboard.

Set the API key as an environment variable:
```bash
export QATE_API_KEY=qate_xxxxxxxxxxxx
```

Or pass it with each command:
```bash
qate run -n "My Test Set" --api-key qate_xxxxxxxxxxxx
```

## Commands

### Test Sets (Parallel Execution)

Test sets run all tests in parallel for maximum speed.

```bash
# List all test sets
qate list:testsets
qate list  # alias

# Run a test set
qate run:testset -n "Regression Suite"
qate run -n "Regression Suite"  # alias

# Run and wait for completion
qate run -n "Regression Suite" --wait

# Run with timeout (default: 600s)
qate run -n "Regression Suite" --wait --timeout 300

# Check execution status
qate status:testset -e ci_xxx_xxx

# Export test set definition
qate export:testset -n "Regression Suite" -o tests.json
```

### Test Sequences (Sequential Execution)

Test sequences run tests in order, optionally sharing browser state between tests.

```bash
# List all test sequences
qate list:sequences

# Run a test sequence
qate run:sequence -n "User Flow"

# Run and wait for completion
qate run:sequence -n "User Flow" --wait

# Check execution status
qate status:sequence -e seq_xxx_xxx

# Export test sequence definition
qate export:sequence -n "User Flow" -o sequence.json
```

### Generic Status Command

Auto-detects whether the execution is a test set or sequence based on the ID prefix:

```bash
qate status -e ci_xxx_xxx   # Test set
qate status -e seq_xxx_xxx  # Test sequence
```

### Generate Playwright Tests

Generate standalone Playwright test files from your Qate tests. These can be run locally or in your CI pipeline.

```bash
# Generate from a test set
qate generate -n "Regression Suite" -o ./e2e-tests
qate generate:testset -n "Regression Suite" -o ./e2e-tests  # explicit

# Generate from a test sequence
qate generate:sequence -n "User Flow" -o ./e2e-tests

# Specify provider (for cloud testing)
qate generate -n "Regression Suite" -o ./e2e-tests --provider browserstack
qate generate -n "Regression Suite" -o ./e2e-tests --provider saucelabs
qate generate -n "Regression Suite" -o ./e2e-tests --provider lambdatest

# Specify browsers
qate generate -n "Regression Suite" -o ./e2e-tests --browsers "chromium,firefox,webkit"
```

After generating, run the tests:

```bash
cd ./e2e-tests
npm install
npx playwright install
npx playwright test
```

## CI/CD Examples

### GitHub Actions (Local Playwright Execution)

Run tests locally in your CI using generated Playwright tests:

```yaml
name: E2E Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Qate CLI
        run: npm install -g @qate/cli

      - name: Generate Playwright Tests
        env:
          QATE_API_KEY: ${{ secrets.QATE_API_KEY }}
        run: qate generate -n "Regression Suite" -o ./e2e

      - name: Install Dependencies
        working-directory: ./e2e
        run: npm ci

      - name: Install Playwright Browsers
        working-directory: ./e2e
        run: npx playwright install --with-deps chromium

      - name: Run Tests
        working-directory: ./e2e
        run: npx playwright test

      - name: Upload Test Results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: e2e/playwright-report/
```

### GitHub Actions (BrowserStack)

Run tests on BrowserStack's cloud infrastructure:

```yaml
name: E2E Tests (BrowserStack)
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Qate CLI
        run: npm install -g @qate/cli

      - name: Generate Playwright Tests
        env:
          QATE_API_KEY: ${{ secrets.QATE_API_KEY }}
        run: qate generate -n "Regression Suite" -o ./e2e --provider browserstack

      - name: Install Dependencies
        working-directory: ./e2e
        run: npm ci

      - name: Run Tests on BrowserStack
        working-directory: ./e2e
        env:
          BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
          BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
        run: npx playwright test
```

### GitHub Actions (LambdaTest)

Run tests on LambdaTest's cloud infrastructure:

```yaml
name: E2E Tests (LambdaTest)
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Qate CLI
        run: npm install -g @qate/cli

      - name: Generate Playwright Tests
        env:
          QATE_API_KEY: ${{ secrets.QATE_API_KEY }}
        run: qate generate -n "Regression Suite" -o ./e2e --provider lambdatest

      - name: Install Dependencies
        working-directory: ./e2e
        run: npm ci

      - name: Run Tests on LambdaTest
        working-directory: ./e2e
        env:
          LT_USERNAME: ${{ secrets.LT_USERNAME }}
          LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }}
        run: npx playwright test
```

### GitHub Actions (Sauce Labs)

Run tests on Sauce Labs using saucectl:

```yaml
name: E2E Tests (Sauce Labs)
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Qate CLI
        run: npm install -g @qate/cli

      - name: Generate Playwright Tests
        env:
          QATE_API_KEY: ${{ secrets.QATE_API_KEY }}
        run: qate generate -n "Regression Suite" -o ./e2e --provider saucelabs

      - name: Install Dependencies
        working-directory: ./e2e
        run: npm ci

      - name: Install saucectl
        run: npm install -g saucectl

      - name: Run Tests on Sauce Labs
        working-directory: ./e2e
        env:
          SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
          SAUCE_ACCESS_KEY: ${{ secrets.SAUCE_ACCESS_KEY }}
        run: saucectl run
```

> **Note:** Sauce Labs uses `saucectl` CLI instead of direct Playwright execution. The generated `.sauce/config.yml` file contains the test configuration.

### GitLab CI

```yaml
e2e-tests:
  stage: test
  image: mcr.microsoft.com/playwright:v1.40.0-jammy
  script:
    - npm install -g @qate/cli
    - qate generate -n "Regression Suite" -o ./e2e
    - cd e2e && npm ci
    - npx playwright test
  variables:
    QATE_API_KEY: $QATE_API_KEY
  artifacts:
    when: always
    paths:
      - e2e/playwright-report/
    expire_in: 1 week
```

### Azure DevOps

```yaml
- task: NodeTool@0
  inputs:
    versionSpec: '20.x'

- script: npm install -g @qate/cli
  displayName: 'Install Qate CLI'

- script: qate generate -n "Regression Suite" -o ./e2e
  displayName: 'Generate Playwright Tests'
  env:
    QATE_API_KEY: $(QATE_API_KEY)

- script: |
    cd e2e
    npm ci
    npx playwright install --with-deps chromium
    npx playwright test
  displayName: 'Run E2E Tests'

- task: PublishTestResults@2
  inputs:
    testResultsFiles: 'e2e/test-results.json'
  condition: always()
```

### Jenkins

```groovy
pipeline {
    agent any
    environment {
        QATE_API_KEY = credentials('qate-api-key')
    }
    stages {
        stage('Generate Tests') {
            steps {
                sh 'npm install -g @qate/cli'
                sh 'qate generate -n "Regression Suite" -o ./e2e'
            }
        }
        stage('Run E2E Tests') {
            steps {
                dir('e2e') {
                    sh 'npm ci'
                    sh 'npx playwright install --with-deps chromium'
                    sh 'npx playwright test'
                }
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: 'e2e/playwright-report/**', allowEmptyArchive: true
        }
    }
}
```

## Options

### Global Options

| Option | Description |
|--------|-------------|
| `-a, --api-key <key>` | API key (or use QATE_API_KEY env var) |
| `-u, --url <url>` | Qate API URL (default: https://api.qate.ai) |
| `--json` | Output results as JSON |

### Run Options

| Option | Description |
|--------|-------------|
| `-n, --name <name>` | Name of the test set/sequence to run |
| `--app <id>` | Application ID (if name is not unique) |
| `-w, --wait` | Wait for execution to complete |
| `--timeout <seconds>` | Timeout for waiting (default: 600) |

### Export Options

| Option | Description |
|--------|-------------|
| `-n, --name <name>` | Name of the test set/sequence to export |
| `-o, --output <file>` | Output file path (default: stdout) |
| `--app <id>` | Application ID (if name is not unique) |

### Generate Options

| Option | Description |
|--------|-------------|
| `-n, --name <name>` | Name of the test set/sequence |
| `-o, --output <dir>` | Output directory for generated files |
| `--app <id>` | Application ID (if name is not unique) |
| `--provider <provider>` | Test provider: local, browserstack, saucelabs, lambdatest (default: local) |
| `--browsers <browsers>` | Comma-separated list of browsers (default: chromium) |

## Exit Codes

| Code | Meaning |
|------|---------|
| 0 | Success (tests passed or still running) |
| 1 | Failure (tests failed or error occurred) |

## Test Sets vs Test Sequences

| Feature | Test Set | Test Sequence |
|---------|----------|---------------|
| Execution | Parallel | Sequential |
| Speed | Faster | Slower |
| State sharing | Independent | Optional |
| Use case | Regression testing | User flows, E2E journeys |
| Stop on failure | No | Configurable |

## Windows Desktop Applications

For Windows desktop applications, tests are executed on a connected desktop agent. The CLI triggers server-side execution and polls for results.

```bash
# Run tests on a connected desktop agent
qate run -n "Desktop App Tests"

# Run a sequence on a connected desktop agent
qate run:sequence -n "Desktop Flow"

# Install a build on the agent machine before running tests
qate install --app <applicationId> --path "C:\builds\MyApp-Setup.exe" --args "/silent"

# Full CI pipeline: install build, then run tests
qate install --app 64a... --path "C:\builds\setup.exe" --args "/S"
qate run -n "Smoke Tests" --app 64a...
```

### Install Command Options

| Option | Description |
|--------|-------------|
| `--app <id>` | Application ID (required) |
| `--path <path>` | Path to installer on the agent machine (required) |
| `--args <arguments>` | Installer arguments (e.g., `/silent`, `/S`) |
| `--timeout <seconds>` | Timeout in seconds (default: 300) |

## License

MIT