npm package: @lytics/playwright-reporter
Description: Adapter-based Playwright reporter with pluggable storage backends
Latest version: 0.1.1

# @lytics/playwright-reporter

> Adapter-based Playwright reporter with pluggable storage backends

[![npm version](https://img.shields.io/npm/v/@lytics/playwright-reporter)](https://www.npmjs.com/package/@lytics/playwright-reporter)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Overview

`CoreReporter` is a flexible Playwright reporter that uses the adapter pattern to write test results to multiple storage backends simultaneously. It validates required annotations, tracks unique tests, handles retries, and generates comprehensive test run statistics.

## Features

- ✅ **Adapter Pattern** - Write to multiple backends (filesystem, database, cloud storage, etc.)
- ✅ **Annotation Validation** - Enforces required `testCaseId` and `journeyId` annotations
- ✅ **Unique Test Tracking** - Handles retries and tracks latest test state
- ✅ **Flaky Test Detection** - Identifies tests that pass after retries
- ✅ **Structured Errors** - Parses and structures error information
- ✅ **Environment Enrichment** - Add custom environment data to test runs
- ✅ **Parallel Adapter Execution** - All adapters run concurrently

## Installation

```bash
npm install @lytics/playwright-reporter @lytics/playwright-annotations
# or
pnpm add @lytics/playwright-reporter @lytics/playwright-annotations
# or
yarn add @lytics/playwright-reporter @lytics/playwright-annotations
```

## Quick Start

```typescript
// playwright.config.ts
import { defineConfig } from '@playwright/test';
import { CoreReporter } from '@lytics/playwright-reporter';
import { FilesystemAdapter } from '@lytics/playwright-adapters/filesystem';

export default defineConfig({
  reporter: [
    ['list'], // Built-in reporter
    [
      new CoreReporter({
        adapters: [
          new FilesystemAdapter({ outputDir: './test-results' })
        ]
      })
    ]
  ],
  // ... rest of config
});
```

## Usage

### Basic Configuration

```typescript
import { CoreReporter } from '@lytics/playwright-reporter';
import { FilesystemAdapter } from '@lytics/playwright-adapters/filesystem';

const reporter = new CoreReporter({
  adapters: [
    new FilesystemAdapter({ outputDir: './test-results' })
  ]
});

export default reporter;
```

### Multiple Adapters

```typescript
import { CoreReporter } from '@lytics/playwright-reporter';
import { FilesystemAdapter } from '@lytics/playwright-adapters/filesystem';
import { SlackAdapter } from '@lytics/playwright-adapters/slack';

const reporter = new CoreReporter({
  adapters: [
    new FilesystemAdapter({ outputDir: './test-results' }),
    new SlackAdapter({ 
      webhookUrl: process.env.SLACK_WEBHOOK_URL,
      notifyOnFailure: true 
    })
  ]
});

export default reporter;
```

### Environment Enrichment

Add custom environment data to test runs:

```typescript
const reporter = new CoreReporter({
  adapters: [/* ... */],
  environmentEnricher: () => ({
    branch: process.env.GITHUB_REF,
    commit: process.env.GITHUB_SHA,
    author: process.env.GITHUB_ACTOR,
    buildNumber: process.env.GITHUB_RUN_NUMBER,
    nodeVersion: process.version,
  })
});
```

## Core Types

### CoreTestResult

Individual test result with full metadata:

```typescript
interface CoreTestResult {
  testCaseId: string;
  journeyId: string;
  title: string;
  annotations: TestAnnotations;
  status: "passed" | "failed" | "timedOut" | "skipped" | "cancelled" | "interrupted" | "unknown";
  projectName: string;
  durationMs: number;
  error?: TestError;
  timestamp: Date;
  buildId: string;
  reportLink?: string;
}
```

### CoreTestRun

Test run summary with aggregate statistics:

```typescript
interface CoreTestRun {
  runId: string;
  timestamp: Date;
  overallStatus: "passed" | "failed" | "timedOut" | "skipped" | "cancelled" | "interrupted" | "unknown";
  totalTests: number;           // Unique tests (deduped)
  totalExecutions: number;      // Total test executions (including retries)
  passed: number;
  failed: number;
  skipped: number;
  durationMs: number;
  passRate: number;             // 0-1
  averageTestDuration: number;
  slowestTestDuration: number;
  flakyTests: number;           // Tests that passed after retries
  environment: Record<string, unknown>;
}
```

### TestError

Structured error information:

```typescript
interface TestError {
  matcher: string;              // e.g., "toHaveText", "toBeVisible"
  expected: string;
  actual: string;
  locator: string;
  location: {
    file: string;
    line: number;
    column: number;
  };
  message: string;
  snippet: string[];            // Code snippet around error
}
```

## Creating Custom Adapters

Implement the `ResultAdapter` interface:

```typescript
import { ResultAdapter, CoreTestResult, CoreTestRun } from '@lytics/playwright-reporter';

class MyCustomAdapter implements ResultAdapter {
  async initialize(): Promise<void> {
    // Connect to your storage backend
    console.log('Initializing adapter...');
  }

  async writeTestResult(result: CoreTestResult): Promise<void> {
    // Write individual test result
    console.log(`Test ${result.testCaseId}: ${result.status}`);
  }

  async writeTestRun(run: CoreTestRun): Promise<void> {
    // Write test run summary
    console.log(`Run ${run.runId}: ${run.passed}/${run.totalTests} passed`);
  }

  async close(): Promise<void> {
    // Cleanup connections
    console.log('Closing adapter...');
  }
}

// Use your custom adapter
const reporter = new CoreReporter({
  adapters: [new MyCustomAdapter()]
});
```

## Annotation Requirements

The reporter requires tests to have proper annotations. Use `@lytics/playwright-annotations`:

```typescript
import { pushSuiteAnnotation, pushTestAnnotations } from '@lytics/playwright-annotations';

test.describe('My Feature', () => {
  test.beforeEach(async ({}, testInfo) => {
    pushSuiteAnnotation(testInfo, 'MY_FEATURE');
  });

  test('validates user can perform action', async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: 'MY_FEATURE_ACTION',
      testCaseId: 'MY_FEATURE_ACTION_VALID',
    });

    // Test implementation...
  });
});
```

**Tests missing required annotations will be skipped with a warning.**

## API Reference

### CoreReporter

**Constructor:**
```typescript
new CoreReporter(config: CoreReporterConfig)
```

**Config:**
```typescript
interface CoreReporterConfig {
  adapters: ResultAdapter[];
  environmentEnricher?: () => Record<string, unknown>;
}
```

**Methods:**
- `onBegin()` - Called once before test suite starts
- `onTestBegin(test)` - Called before each test
- `onTestEnd(test, result)` - Called after each test
- `onEnd(result)` - Called once after test suite completes

### parseError

Parse raw Playwright errors into structured format:

```typescript
import { parseError } from '@lytics/playwright-reporter';

const error = parseError(result.errors[0]);
if (error) {
  console.log(`Expected: ${error.expected}`);
  console.log(`Actual: ${error.actual}`);
  console.log(`Locator: ${error.locator}`);
}
```

## How It Works

### Test Execution Flow

```
1. onBegin()
   └─> Initialize all adapters in parallel

2. For each test:
   onTestBegin(test)
   └─> Track test start
   
   onTestEnd(test, result)
   └─> Validate annotations
   └─> Track unique test state (handle retries)
   └─> Map to CoreTestResult
   └─> Write to all adapters in parallel

3. onEnd(result)
   └─> Calculate aggregate statistics
   └─> Map to CoreTestRun
   └─> Write to all adapters in parallel
   └─> Close all adapters
```

### Retry Handling

The reporter intelligently handles test retries:

- Tracks **unique tests** by `testCaseId`
- Only the **latest execution** of each test is counted in final stats
- Detects **flaky tests** (passed after retry)
- Reports both `totalTests` (unique) and `totalExecutions` (including retries)

Example:
```
Test A: fail → retry → pass  ✅ (flaky)
Test B: fail → retry → fail  ❌
Test C: pass                  ✅

totalTests: 3
totalExecutions: 5
passed: 2
failed: 1
flakyTests: 1
```

## Environment Variables

The reporter uses these environment variables (can be customized via `environmentEnricher`):

- `GITHUB_RUN_ID` - Used for `runId` and `buildId`
- `ARTIFACT_BASE_URL` - Used for `reportLink`

## Adapters

See these packages for ready-to-use adapters:

- [`@lytics/playwright-adapters`](../adapters) - Filesystem, Slack, Firestore

## Troubleshooting

### Tests are skipped with "missing annotation" warning

**Solution:** Ensure all tests have required annotations:

```typescript
pushTestAnnotations(testInfo, {
  journeyId: 'YOUR_JOURNEY_ID',
  testCaseId: 'YOUR_TEST_CASE_ID',
});
```

### Adapter initialization fails

**Solution:** Check adapter configuration and credentials. Adapters log errors to console.

### Stats don't match expected values

**Cause:** Retries are handled automatically. The reporter tracks:
- `totalTests` = unique tests (deduped by `testCaseId`)
- `totalExecutions` = all test runs (including retries)

## Contributing

See [CONTRIBUTING.md](../../CONTRIBUTING.md) in the repository root.

## License

MIT

## Related Packages

- [`@lytics/playwright-annotations`](../annotations) - Annotation framework
- [`@lytics/playwright-adapters`](../adapters) - Storage adapters (Filesystem, Slack, Firestore)