npm package: @principal-ai/otel-collector-server
Description: OpenTelemetry collector service for Electron desktop apps
Author: Principal AI
Latest version: 0.3.15

# @principal-ai/otel-collector-server

Node.js wrapper around the official OpenTelemetry Collector with dual-mode support for development and Electron integration.

## 🎯 Contract-Based Observability (Coming Soon)

This package is being enhanced to support **contract-based observability** - validate traces against expected workflows, reduce storage costs by 90%+, and surface unknown-unknowns automatically.

**Key Features:**
- ✅ Validate traces against Principal-View canvas contracts
- ✅ Emit metrics-only for expected workflows (95% of traces)
- ✅ Store full traces only for anomalies (5% of traces)
- ✅ Real-time contract violations in Electron UI
- ✅ Production mode for EC2 deployment

See [CONTRACT-BASED-PLATFORM.md](./docs/architecture/CONTRACT-BASED-PLATFORM.md) for full design.

## Features

- 🚀 **Standalone Mode** - Console or file output for development and testing
- ⚡ **Electron Mode** - MessagePort routing to renderer windows
- 🔧 **Auto-managed Binaries** - Handles OpenTelemetry Collector binaries per platform
- 📊 **Pretty Printing** - Human-readable trace output
- 🎯 **Source Routing** - Routes traces based on origin URL
- 🔄 **Auto-restart** - Monitors and restarts collector on crashes

## Quick Start

### Installation

```bash
npm install @principal-ai/otel-collector-server
```

### Standalone Mode (CLI)

```bash
# Install globally
npm install -g @principal-ai/otel-collector-server

# Start with console output
otel-collector-server

# Start with file output
otel-collector-server --output file --file ./traces.jsonl

# Custom configuration
otel-collector-server --port 4318 --log-level debug --pretty

# Using environment variables
OTLP_PORT=4318 OUTPUT=file OUTPUT_FILE=./traces.jsonl otel-collector-server
```

### Production Deployment

**AWS App Runner (Recommended)** - Serverless, managed deployment:
- ✅ **Currently deployed**: https://5hmsn3wzue.us-east-1.awsapprunner.com
- 📋 **Architecture & Decisions**: [production-deployment.md](./.principal-views/production-deployment.md)
- 🔧 **Operational Guide**: [PRODUCTION.md](./docs/deployment/PRODUCTION.md)
- 🚀 **Setup Guide**: [APPRUNNER-SETUP.md](./docs/deployment/APPRUNNER-SETUP.md)
- Automatic deployments via GitHub Actions (push to `main` branch)
- Auto-scaling and zero-downtime updates
- ~$51-60/month for 24/7 operation

**AWS EC2** - Self-hosted for more control:
- See **[DEPLOYMENT.md](./docs/deployment/DEPLOYMENT.md)** for complete guide
- Docker or PM2 deployment options
- Manual deployment and management
- ~$15/month for t3.small instance

### Programmatic Usage

```typescript
import { OTELCollectorServer } from '@principal-ai/otel-collector-server';

const server = new OTELCollectorServer({
  mode: 'standalone',
  output: 'console',
  prettyPrint: true,
  otlpPort: 4318,
  logLevel: 'info',
});

await server.start();

// Graceful shutdown
process.on('SIGINT', async () => {
  await server.stop();
  process.exit(0);
});
```

## Configuration

### Standalone Mode Options

The server can be configured via:
1. **Environment variables** (recommended for production)
2. **Command-line arguments** (overrides environment variables)
3. **Programmatic config** (for library usage)

**Environment Variables:**
```bash
OTLP_PORT=4318              # OTLP endpoint port
WRAPPER_PORT=4319           # Wrapper HTTP port
OUTPUT=console              # Output type: console | file
OUTPUT_FILE=./traces.jsonl  # File path (when OUTPUT=file)
PRETTY_PRINT=true           # Pretty print console output
LOG_LEVEL=info              # Log level: debug | info | warn | error
COLLECTOR_VERSION=v0.144.0  # OTEL collector version
BINARY_PATH=/path/to/binary # Custom binary path
```

**TypeScript Interface:**
```typescript
interface OTELCollectorServerConfig {
  mode: 'standalone' | 'electron';
  otlpPort?: number;           // Default: 4318
  wrapperPort?: number;        // Default: 4319
  collectorVersion?: string;   // Default: latest
  logLevel?: 'debug' | 'info' | 'warn' | 'error';
  autoStart?: boolean;
  restartOnCrash?: boolean;

  // Standalone mode
  output?: 'console' | 'file';
  outputFile?: string;
  prettyPrint?: boolean;

  // Optional custom binary path
  binaryPath?: string;
}
```

## Architecture

```
External App (instrumented with OTEL SDK)
         ↓ HTTP POST /v1/traces (OTLP)
OpenTelemetry Collector (Go binary)
         ↓ forwards to
Node.js Wrapper (this package)
         ↓
Console Output | File Output | MessagePort (Electron)
```

## CLI Options

```
Usage: otel-collector-server [options]

Options:
  -p, --port <port>            OTLP endpoint port (default: 4318)
  --wrapper-port <port>        Wrapper HTTP port (default: 4319)
  -o, --output <type>          Output type: console | file (default: console)
  -f, --file <path>            Output file path (sets output to file)
  --pretty                     Pretty print console output (default: true)
  --no-pretty                  Disable pretty printing
  -l, --log-level <level>      Log level: debug | info | warn | error
  --version <version>          Collector version (default: latest)
  -h, --help                   Show help message
```

## Sending Test Traces

### Production Endpoint (App Runner)

```bash
# Send a test trace to production
curl -X POST https://5hmsn3wzue.us-east-1.awsapprunner.com/v1/traces \
  -H "Content-Type: application/json" \
  -d '{
    "resourceSpans": [{
      "resource": {
        "attributes": [
          { "key": "service.name", "value": { "stringValue": "test-service" } },
          { "key": "dev.server.url", "value": { "stringValue": "http://localhost:3000" } }
        ]
      },
      "scopeSpans": [{
        "spans": [{
          "traceId": "abc123",
          "spanId": "def456",
          "name": "test-span",
          "kind": 2,
          "startTimeUnixNano": "1737835200000000000",
          "endTimeUnixNano": "1737835200150000000"
        }]
      }]
    }]
  }'
```

### Local Development

```bash
# Send a test trace to local instance
curl -X POST http://localhost:4318/v1/traces \
  -H "Content-Type: application/json" \
  -d '{
    "resourceSpans": [{
      "resource": {
        "attributes": [
          { "key": "service.name", "value": { "stringValue": "test-service" } },
          { "key": "dev.server.url", "value": { "stringValue": "http://localhost:3000" } }
        ]
      },
      "scopeSpans": [{
        "spans": [{
          "traceId": "abc123",
          "spanId": "def456",
          "name": "test-span",
          "kind": 2,
          "startTimeUnixNano": "1737835200000000000",
          "endTimeUnixNano": "1737835200150000000"
        }]
      }]
    }]
  }'
```

## Example Output

### Pretty Print (Default)

```
======================================================================
TRACE FROM http://localhost:3000
Received at: 2025-01-25T12:00:00.000Z
======================================================================

Service: web-ade

Scope: my-tracer
  - HTTP GET /api/users (SERVER) 45.23ms [OK]
     http.method=GET, http.status_code=200
    └─ DB Query: SELECT * FROM users (CLIENT) 30.15ms [OK]
       db.system=postgresql, db.statement=SELECT * FROM users
    └─ Cache lookup (INTERNAL) 2.08ms [OK]

======================================================================
```

### File Output (JSONL)

```json
{"timestamp":"2025-01-25T12:00:00.000Z","source":"http://localhost:3000","payload":{...}}
```

## Requirements

- Node.js >= 18.19.0
- OpenTelemetry Collector binary (auto-downloaded or bundled)

## Supported Platforms

- macOS (Intel & Apple Silicon)
- Linux (x64 & ARM64)
- Windows (x64)

## Development

### Setup

```bash
# Install dependencies
npm install

# Download OpenTelemetry Collector binaries (required for first run)
npm run download-binaries

# Build
npm run build

# Run in dev mode
npm run dev:standalone

# Run tests
npm test
```

### Download Binaries

The package bundles OpenTelemetry Collector binaries for all platforms. To download them:

```bash
npm run download-binaries
```

This downloads ~250-350MB of binaries for:
- macOS (Intel & Apple Silicon)
- Linux (x64 & ARM64)
- Windows (x64)

Binaries are automatically downloaded when publishing via `prepublishOnly` hook.

## Project Status

### ✅ Completed (Phase 1 & 2)
- [x] Standalone mode with console output
- [x] Standalone mode with file output
- [x] Auto-managed collector binaries (platform detection)
- [x] Binary bundling for all platforms (v0.144.0)
- [x] Pretty-print trace output
- [x] Source URL routing
- [x] CLI interface
- [x] Comprehensive test coverage (57+ test cases)
- [x] TypeScript types and API exports
- [x] Full ESLint configuration and compliance
- [x] Tested and verified with live OTEL Collector

### ✅ Completed (Phase 3)
- [x] Integration tests with actual Go collector (14 comprehensive tests)
- [x] FileOutput unit tests (16 comprehensive tests)

### 📋 Roadmap (Future Phases)
- [ ] Electron mode with MessagePort routing (Phase 4)
- [ ] Desktop app integration (Phase 5)
- [ ] Metrics support (/v1/metrics)
- [ ] Logs support (/v1/logs)
- [ ] Custom processors configuration
- [ ] Multiple exporters support

## License

MIT

## Contributing

Issues and PRs welcome!

## See Also

### Documentation
- [docs/architecture/DESIGN.md](./docs/architecture/DESIGN.md) - Detailed system architecture
- [.principal-views/production-deployment.md](./.principal-views/production-deployment.md) - Production architecture decisions
- [docs/deployment/PRODUCTION.md](./docs/deployment/PRODUCTION.md) - Production operational guide

### External Resources
- [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/)
- [OTLP Specification](https://opentelemetry.io/docs/specs/otlp/)