npm package: @m-software-engineering/heat-collector
Description: Collector server for heat-tracker sessions and events
Latest version: 0.3.4

# @m-software-engineering/heat-collector

Express collector service for heat-tracker. It provides ingestion and query APIs, supports SQL/MongoDB backends, and can authenticate by project key, JWT, or both.

## Install

```bash
npm install @m-software-engineering/heat-collector
# or
pnpm add @m-software-engineering/heat-collector
# or
yarn add @m-software-engineering/heat-collector
```

## Database drivers (peer dependencies)

- Postgres: `pg`
- MySQL: `mysql2`
- SQLite: `better-sqlite3`
- MongoDB: `mongodb`

## Quick start (Express)

```ts
import express from "express";
import { createCollector } from "@m-software-engineering/heat-collector";

const collector = await createCollector({
  db: { dialect: "sqlite", file: "./heat-tracker.db" },
  auth: { mode: "projectKey" },
  autoMigrate: true
});

const app = express();
app.use(collector.router);

app.listen(3000, () => {
  console.log("Collector listening on http://localhost:3000");
});
```

## Quick start (NestJS)

Register `collector.router` in `main.ts` after creating the Nest application:

```ts
import { NestFactory } from "@nestjs/core";
import { AppModule } from "./app.module";
import type { NestExpressApplication } from "@nestjs/platform-express";
import { createCollector } from "@m-software-engineering/heat-collector";

async function bootstrap() {
  const app = await NestFactory.create<NestExpressApplication>(AppModule);

  app.enableCors();

  const collector = await createCollector({
    db: { dialect: "sqlite", file: "./heat.db" },
    auth: { mode: "projectKey" },
    autoMigrate: true
  });

  app.use(collector.router);

  await app.listen(3000, "0.0.0.0");
}

bootstrap();
```

## How the collector works

1. `POST /ingest` receives event batches from the SDK.
2. The request is authenticated via `x-project-key`, JWT, or both.
3. The collector upserts project/user/session data and stores events.
4. Query APIs return aggregated points and session timelines for dashboards.

## Endpoints

### Ingestion

`POST /ingest`

Accepted headers:

- `x-project-key`: required for `projectKey` and `both` modes.
- `authorization: Bearer <jwt>`: required for `jwt`, optional for `both`.

### Query APIs

- `GET /api/projects/:projectId/heatmap`
- `GET /api/projects/:projectId/events`
- `GET /api/projects/:projectId/sessions`
- `GET /api/sessions/:sessionId`
- `GET /api/metrics`

### Response headers and error payloads

All endpoints now include:

- `X-Request-Id`: request correlation id for logs/troubleshooting.
- `X-Heat-Collector`: static collector identifier.

Query endpoints also send:

- `Cache-Control: no-store`
- `X-Total-Count` or endpoint-specific count headers.

Error payload format is consistent across routes:

```json
{
  "error": "invalid query",
  "code": "invalid_query",
  "message": "Invalid heatmap query parameters.",
  "requestId": "..."
}
```

Validation failures include a `details` object.

### Heatmap query behavior notes

- `type` now accepts: `all`, `click`, `move`, `scroll`, `pageview`, `custom`, `input`, `keyboard`.
- If `type` is omitted, the collector first tries `click` (backward compatible) and automatically falls back to `all` when click has no matches.
- Heatmap points are only plotted from `click`, `move`, and `scroll`; other event types still appear in heatmap metadata (`total`, `typeBreakdown`, `warning`) for better debugging.

## Configuration

```ts
type CollectorConfig = {
  db:
    | { dialect: "pg" | "mysql" | "sqlite"; connectionString?: string; file?: string }
    | { dialect: "mongodb"; connectionString?: string; database?: string };
  auth:
    | { mode: "projectKey" }
    | { mode: "jwt"; jwksUrl: string; issuer: string; audience: string }
    | { mode: "both"; jwksUrl: string; issuer: string; audience: string };
  autoMigrate?: boolean;
  ingestion?: {
    maxBodyBytes?: number;
    rateLimit?: { windowMs: number; max: number };
  };
  hooks?: {
    onBeforeInsert?: (payload: any) => any | Promise<any>;
  };
  logging?: { level: "debug" | "info" | "warn" | "error" };
};
```

## MongoDB permissions note

When `autoMigrate` is enabled, the collector attempts to create indexes. If the MongoDB user does not have index-management permissions, the collector now continues without failing startup. For controlled production environments, you can still set `autoMigrate: false` and manage indexes externally.

### Troubleshooting `not authorized on <db> to execute command { find: "projects" }`

This error usually means the collector is querying a different database than the one your MongoDB user can access.

Use one of these options:

1. Put the database directly in the connection string path:

```ts
db: {
  dialect: "mongodb",
  connectionString: "mongodb://user:password@host:27017/heat_tracker?authSource=admin"
}
```

2. Or explicitly set `database` in the collector config:

```ts
db: {
  dialect: "mongodb",
  connectionString: "mongodb://user:password@host:27017/?authSource=admin",
  database: "heat_tracker"
}
```

The collector now prefers `db.database` when provided, otherwise it uses the database from the MongoDB connection string.

## Production migration command

```bash
HEAT_DIALECT=pg DATABASE_URL=postgres://... heat-collector-migrate
```