npm package: @memoreco/memoreco-js
Description: Memoreco Web SDK - JavaScript/TypeScript interface for recording, uploading, and managing media assets with AI processing
Author: Memoreco
Latest version: 0.5.3

# @memoreco/memoreco-js

[![npm version](https://img.shields.io/npm/v/%40memoreco%2Fmemoreco-js.svg)](https://www.npmjs.com/package/@memoreco/memoreco-js)
[![npm downloads](https://img.shields.io/npm/dm/%40memoreco%2Fmemoreco-js.svg)](https://www.npmjs.com/package/@memoreco/memoreco-js)
[![MIT license](https://img.shields.io/badge/license-MIT-black.svg)](https://github.com/memoreco-dev/memoreco/blob/main/LICENSE)

The official Memoreco Web SDK for JavaScript and TypeScript. Record, upload, and manage media assets with optional AI processing.

## Features

- 🎥 **Screen & Camera Recording** - Built on top of `@memoreco/recorder`
- 📤 **Smart Upload Strategy** - Automatic selection between Memoreco-managed and custom storage
- 🤖 **AI Processing** - Integrate AI services for transcription, analysis, and enhancement
- 🎬 **Media Player** - Embed and play recordings with customizable themes
- 📊 **Real-time Events** - Listen to recording, upload, and AI processing events
- 🔧 **TypeScript Support** - Full type definitions included
- ⚛️ **React Integration** - Hooks and components for React apps
- 🎯 **Smart UI Behavior** - Device overlay auto-hides during capture/recording and restores when finished
- 🧠 **AI Overlay** - Optional guidance overlay during recording (not captured in screen recordings)
- 🎛️ **Recording Controls** - Simplified API for managing recording UI state
- 🖼️ **Recording Gallery** - Automatic display of captured videos and photos below preview

## Recent Updates
- 2026-01-19: **v0.4.3** - Bundle `@memoreco/shared` at build time so external devs don't need to install it separately.
- 2026-01-19: **v0.4.2** - Relax `@memoreco/recorder` peer dependency to include `^0.3.0` (pairs with recorder v0.3.0).
- 2026-01-19: **v0.4.1** - Fix npm installation by replacing the internal `workspace:*` dependency on `@memoreco/shared` with a published semver range.
- 2026-01-19: **v0.4.0** - AI Assist improvements (start decision logic, countdown handling), MemoryModule integration, AI guidance use cases with gaze tracking, and streaming transcription AI assist config.
- 2026-01-16: **AI Assist Persona IDs** - SDK AI Assist now requires `personaId` and no longer sends `systemPrompt` or `role` in recording configs.
- 2025-12-06: **UI Consistency & Polish (v0.3.19)** - Major visual refinement pass to match Figma designs:
  - **Countdown Timer**: Fixed positioning (36px/28px from edges), font (Neue Haas Medium 600), size (16px red text), dot (16x16px), and pill dimensions (89x40px).
  - **AI Overlay**: Eliminated duplicate borders (CSS + monoco library conflict), fixed border sync during fade-in with requestAnimationFrame, prevented premature dismissal during typewriter animation.
  - **Streaming Transcript**: Enforced Inter font with !important, centered pills that expand left when reaching max width, fixed padding (container 16px, pill 16px/32px), eliminated background opacity changes.
  - **Review & Pause Buttons**: Standardized all buttons to pill style (32px height, 50px border-radius, Neue Haas Medium).
  - All UI components now load correct fonts from CDN (Neue Haas Display Roman/Medium, Inter).
- 2025-12-02: **Language-Aware Captions & Provider Sync** - Real-time captions now follow Netflix-style guidelines (max two lines, ~42 characters each) so long sentences wrap into stacked subtitle lines instead of stretching across the frame. We also keep streaming provider preferences in lockstep with the recorder config, ensuring Speechmatics-only builds stay on Speechmatics all the way down to the transcription service.
- 2025-12-02: **AI Overlay Countdown & SSE Resilience** - The AI Assist overlay now shows the full countdown duration immediately, even while the typewriter animation is running, so interviewees see exactly how much time remains before the prompt retires. On the transport side, the SSE client now reuses the centralized API constants (no more hard-coded `/v1`) and automatically retries with capped backoff whenever a browser drops the connection, preventing the “three interventions then silence” behaviour during longer takes.
- 2025-11-30: **AI Assist Preview Integration** - When `ai_assist` is included in `aiServices`, the SDK now auto-connects to the AI Assist SSE stream, renders the translucent overlay inside the camera container, and tears everything down as soon as the recording stops or is cancelled. No extra wiring is needed in apps—the helper mirrors the persona configured on the recording and keeps overlays hidden from the captured media.
- 2025-11-29: **QR Handoff Complete** - Release 0.3.0 marks the first stable QR handoff feature. Desktop users can open a QR overlay, scan with their phone, and the captured photo/video streams back to the desktop in real-time via SSE. Includes fetch-based SSE (bypasses ngrok interstitial), mirror-aware transforms, credit-check whitelisting, and proper submission sequencing so the desktop overlay always receives the `submission_complete` event.
- 2025-11-29: **QR Handoff Mobile Fixes** - Release 0.2.39 fixes logo and sounds not loading on mobile `/r/` recording pages by whitelisting `/brand/` and `/sounds/` paths in waitlist middleware. Fixes recording creation failing with 500 error by using the request UUID (not slug) in ephemeral token metadata. Restores mirror transform after QR overlay closes so selfie preview stays flipped.
- 2025-11-29: **QR Handoff Production Fixes** - Release 0.2.38 fixes 403 errors on public recording request pages by whitelisting the endpoint in visibility middleware. Fixes 401 errors when cancelling QR handoff sessions by using the dedicated cancel-handoff endpoint with proper ephemeral token auth. Also suppresses PTZ `OverconstrainedError` warnings for devices that don't support zoom/pan/tilt, and fixes token provider for voice demo in profile photo explainer.
- 2025-11-28: **QR Handoff SSE Improvements** - Release 0.2.37 replaces native `EventSource` with fetch-based SSE implementation for QR handoff, adding support for the `ngrok-skip-browser-warning` header to bypass ngrok free tier interstitial pages during development. Also adds `Cache-Control` to allowed CORS headers.
- 2025-11-25: **Staged Photo Validation Loading** - New `preloadOnInit: 'staged'` option loads face detection on SDK init (~2-3s) and auto-loads NSFW detection when camera starts. This improves page load performance while keeping capture responsive. Also adds `preloadFaceDetection()`, `preloadNsfwDetection()`, `isFaceDetectionReady()`, and `isNsfwDetectionReady()` methods for manual control. See [Photo Validation docs](./docs/photo-validation.md).
- 2025-11-24: Release 0.2.19 adds **URL Thumbnail Capture** for Voice Mode - automatically detects URLs in user input, captures full-page screenshots, displays loading states, and shows clickable previews. Includes new `useThumbnailCapture` and `useDebouncedUrlParser` hooks, plus `ThumbnailCard`, `ThumbnailPreview`, and `ThumbnailStrip` UI components in `@memoreco/ui`. Thumbnails continue loading in chat after messages are sent.
- 2025-11-20: Release 0.2.15 adds `voiceMode.mountVoiceButton(container, options)` helper for instant-loading Voice Mode with eager resource preloading (AudioWorklet, intro sound) before initialization. External devs no longer need to manually wire up promises, handlers, and DOM mounting—just call the helper and it handles initialization, styling, and callbacks automatically.
- 2025-11-20: Release 0.2.14 adds `clearStreamingTranscript()` API for chat interfaces to reset the buffer mid-recording, hover stop icon (white square appears on hover during recording), and intro sound cooldown (prevents duplicate playback). Also fixes loader rendering to preserve button structure.
- 2025-11-19: Release 0.2.13 fixes audio level indicator (pulsating mask inside microphone) scaling by converting AnimatedMicrophone SVGs to use `viewBox` and percentage dimensions, ensuring proper resizing with the button.
- 2025-11-19: Release 0.2.12 fixes preview overlay visibility (now fully hidden with `display: none` when `showPreviewOverlay: false`) and progress ring scaling (SVG now uses `viewBox` and percentage dimensions for proper button resizing).
- 2025-11-19: Release 0.2.11 adds `showStatusBubble` and `showPreviewOverlay` toggles to Voice Mode so inline chat integrations can hide SDK-managed UI elements while streaming transcripts into their own inputs. Also bumps the exposed `SDK_VERSION` constant so provider logs report 0.2.11.
- 2025-11-19: Release 0.2.10 fixes warm-mic double intro sound—Voice Mode reactivations now play a single chime by gating the preview layer's audio when the mic is already warm. Adds `playIntroSound` option to `CameraPreviewOptions` for fine-grained control. Includes SDK_VERSION constant update.
- 2025-11-19: Release 0.2.9 (deprecated - use 0.2.10) - initial fix for warm-mic double intro sound.
- 2025-11-19: Release 0.2.8 streamlines Voice Mode for chat integrations: the Siri-style button now supports click-to-talk activation, the intro sound only plays once per cold start, the streaming transcript overlay gets balanced padding plus text-replacement hooks, and prompt enhancement updates both the preview and chat input as soon as recordings stop.
- 2025-11-12: Release 0.2.7 fixes **two critical Firefox issues**: (1) voice status bubble positioning now uses `position: fixed` matching the button's strategy for correct resize behavior, and (2) resolves "Connecting AudioNodes with different sample-rate" error by creating fresh AudioContexts at the mic's native rate (e.g., 48kHz) with worklet-based downsampling. The SDK now reports its own version plus the recorder and UI package versions via `sdk.version` (object) as well as convenience fields `sdk.sdkVersion`, `sdk.recorderVersion`, and `sdk.uiVersion`.
- 2025-11-12: Release 0.2.6 keeps the voice status bubble pinned to the button during viewport changes, disables verbose logging in production by default, and pairs with recorder 0.2.5 for Firefox-safe audio fallbacks.
- 2025-11-11: Release 0.2.5 regenerates the internal API client so webhook testing helpers stay available during development and syncs recorder dependency requirements.
- 2025-11-11: Release 0.2.4 hardens push-to-talk race handling—recordings always stop on keyup even if the previous command was still “Understanding…”, preventing phantom sessions during rapid retries.
- 2025-11-10: Release 0.2.3 refreshes the Siri for Sites MCP quick start, updates voice-task samples, and syncs docs ahead of the npm publish.
- 2025-11-11: Voice Tasks debug logging now respects production mode so live deployments stay clean while errors remain visible.
- 2025-11-10: Warm mic pre-roll capture, streaming transcript stabilization, and token lifecycle fixes ensure instant Speechmatics sessions with accurate UI updates.
- 2025-10-30: Recording flows fall back to the permission helper when preview streams are missing, ensuring users see the guided modal before recording fails. New SDK config flags (`usePermissionHelper`, `permissionHelperOptions`) let you customize this per recording.
- 2025-10-29: Voice Tasks now triggers the permission helper before the first recording so users see the browser prompt instead of a silent failure. Note: If permissions are already blocked in browser settings, the helper is skipped and a standard error is shown.
- 2025-10-29: Added idle overlay diagnostics in `VoiceTasksModule` and `DeviceOverlay` to trace audio idle screen reappearance.

## Installation

```bash
npm install @memoreco/memoreco-js
```

## Quick Start

```typescript
import { createMemoreco } from "@memoreco/memoreco-js";

const memoreco = await createMemoreco({
  apiKey: "your-api-key",
  debug: true,
});

const logger = memoreco.getLogger("quick-start");

// Check browser capabilities
const capabilities = memoreco.getCapabilities();
logger.info("Screen recording supported", { supported: capabilities.screenRecording });

// Listen to events with structured logging
memoreco.on("recording.start", (event) => {
  logger.info("Recording started", { recordingId: event.recordingId });
});

memoreco.on("upload.complete", (event) => {
  logger.info("Upload complete", { assetUrl: event.url });
});

memoreco.on("upload.cancelled", (event) => {
  logger.warn("Upload cancelled", { reason: event.reason });
});
```

Cancel uploads
```javascript
// Abort the active upload by recordingId
memoreco.recordings.cancelUpload(recordingId);
```

## Configuration

```typescript
interface MemorecoConfig {
  /** API key for authentication */
  apiKey: string;
  /** API base URL (optional, defaults to production) */
  apiUrl?: string;
  /** 
   * Debug mode for additional logging
   * When enabled, logs API requests, responses, events, and internal operations to console
   * Includes timing information, error details, and state changes
   * Should be disabled in production for performance and security
   */
  debug?: boolean;
  /** 
   * Custom user agent identifier for API requests
   * Helps identify your application in logs and analytics
   * Format: "YourApp/1.0.0 (contact@yourapp.com)"
   * If not provided, uses default SDK user agent
   */
  userAgent?: string;
  /** UI behavior configuration */
  ui?: {
    /** AI overlay configuration */
    aiOverlay?: {
      /** Enable/disable the AI overlay (default: false) */
      enabled?: boolean;
      /** Custom text to display in the overlay */
      defaultMessage?: string;
      /** Position of the overlay */
      position?: 'top-left' | 'top-right' | 'bottom-left' | 'bottom-right' | 'center';
      /** Custom CSS styles for the overlay container */
      style?: Partial<CSSStyleDeclaration>;
      /** Whether to show a spinner in the overlay (default: true) */
      showSpinner?: boolean;
      /** Whether to show a timer in the overlay (default: true) */
      showTimer?: boolean;
    };
    /** 
     * Enable review mode for captures (default: false)
     * When enabled, users can preview and accept/reject media before it's uploaded and persisted.
     * Applies to all capture types: photos, videos, audio, and screen recordings.
     */
    reviewMode?: boolean;
  };
}
```

### UI Behavior

The SDK automatically manages UI elements during capture and recording:

- **Device Overlay**: Automatically slides out during photo capture, multi-shot sequences, and video recording
- **AI Overlay**: Optional guidance overlay during recording (configurable via `ui.aiOverlay`)
- **Gallery Display**: Captured videos and photos automatically appear below the camera preview
- **Overlay Restoration**: Device overlay slides back in when capture/recording finishes and no blocking overlays (QR, pause, restart) are active

### Permission Helper (First-Time Recording)

When you enable the built-in permission helper, the SDK displays Memoreco's guided modal _before_ calling `getUserMedia`. This ensures the browser prompt appears on the first click instead of failing silently when access is missing.

```javascript
await memoreco.camera.showPreview("voice-tasks-preview", {
  selectedModeId: "audio",
  usePermissionHelper: true,
  permissionHelperOptions: {
    explanation: "Allow microphone access so Memoreco can capture your voice command.",
    darkMode: true
  }
});
```

> Voice Tasks integrations now pass `usePermissionHelper` automatically, so the helper runs whenever a user hasn't granted microphone or camera permissions yet.

### Review Mode

[Full gallery documentation](./docs/galleries.md)

Enable review mode to give users the ability to preview and accept/reject captures before they're uploaded:

```typescript
const sdk = new Memoreco({
  apiKey: 'your-api-key',
  ui: {
    reviewMode: true  // Enable review mode for all captures
  }
});

// Listen for review events
const reviewLogger = sdk.getLogger('review-mode');
sdk.on('snapshot.taken', (event) => {
  reviewLogger.info('Capture ready for review', { snapshotId: event.snapshotId });
});

sdk.on('snapshot.accepted', (event) => {
  reviewLogger.info('User accepted capture', { assetId: event.assetId, assetUrl: event.url });
});

sdk.on('snapshot.rejected', (event) => {
  reviewLogger.warn('User rejected capture', { snapshotId: event.snapshotId });
});

// Capture with review
const result = await sdk.capture({ media: 'photo', source: 'camera' });
// Review overlay will appear automatically
// User can click "Accept" to upload or "Retake" to discard
```

**Review Mode Behavior:**
- Works for all capture types: photos, videos, audio, and screen recordings
- Shows an overlay with the captured media and Accept/Retake buttons
- Upload is deferred until user accepts the capture
- No server-side storage or asset creation until acceptance
- Rejected captures are immediately discarded from memory
- Events: `snapshot.taken` (pending), `snapshot.accepted` (uploaded), `snapshot.rejected` (discarded)

### Streaming Transcript Overlay

Need to see what the AI Assist service receives in real time? Enable the optional streaming transcript overlay so the live transcript floats over the preview:

```typescript
const sdk = new Memoreco({
  apiKey: 'your-api-key',
  ui: {
    streamingTranscriptOverlay: {
      enabled: true // Default: false
    }
  }
});
```

When enabled, the SDK mounts the `StreamingTranscript` component inside the camera container so every transcript event is visible. Keep it disabled in production if you prefer a cleaner recording UI.

### Recording Controls

[Full documentation](./docs/recording-controls.md)

The Recording Controls API simplifies recording UI management by providing a single state subscription instead of manual event wiring:

```typescript
// Get the recording controls manager
const controls = sdk.getRecordingControls();
const controlsLogger = sdk.getLogger('recording-controls');

// Subscribe to state changes
controls.onStateChange((state) => {
  // state is: 'idle' | 'recording' | 'paused' | 'error'
  controlsLogger.info('Recording state updated', { state });
  
  // Update your UI based on state
  updateRecordingButtons(state);
});

// Helper methods
if (controls.isRecording()) {
  controlsLogger.info('Currently recording');
}
```

**Benefits:**
- ✅ No need to manually wire up multiple event listeners
- ✅ SDK automatically manages AI overlay mounting/unmounting
- ✅ Single source of truth for recording state
- ✅ Type-safe state transitions
- ✅ Simplified integration for UI developers

### Recording Hotkeys

The SDK supports configurable keyboard shortcuts for start/stop and restart (discard + re-record). Hotkeys are ignored when the user is typing in an input, textarea, or contenteditable element.

**Default shortcuts:**
- **Start / Stop toggle:** `Shift+Alt+R`
- **Restart (discard current + start new):** `Shift+Alt+N`

#### Using with RecordingManager

```typescript
const recorder = memoreco.recordings.createManager({
  onStateChange: (state) => console.log('State:', state),
  onComplete: (result) => console.log('Done!', result),
});

// Start a recording first so the manager knows which config to reuse
await recorder.start({
  capture: { media: 'video', sources: [{ kind: 'camera' }] },
  maxDuration: 300,
});

// Enable hotkeys (uses defaults)
const cleanup = recorder.enableHotkeys();

// Or customise shortcuts and cooldown
const cleanup = recorder.enableHotkeys({
  startStop: 'ctrl+shift+r',
  restart: 'ctrl+shift+n',
  cooldownMs: 400,
});

// Disable later
cleanup();
// or
recorder.disableHotkeys();
```

#### Using with the React `<MemorecoRecorder>` component

```tsx
import { MemorecoRecorder } from "@memoreco/recorder";

<MemorecoRecorder
  mode="auto"
  hotkeys={{ enabled: true, startStop: 'shift+s', restart: 'shift+n' }}
/>
```

#### Using the standalone React hook

```tsx
import { useRecorderHotkeys } from "@memoreco/recorder";

useRecorderHotkeys(
  {
    state: recorderState,      // 'idle' | 'recording' | 'paused' | ...
    onStart: handleStart,
    onStop: handleStop,
    onCancel: handleCancel,
  },
  { startStop: 'shift+s', restart: 'shift+n' }
);
```

## Browser Compatibility

The SDK automatically detects browser capabilities:

- **Screen Recording**: Chrome 72+, Firefox 66+, Safari 13+
- **Camera Recording**: Chrome 53+, Firefox 36+, Safari 11+
- **Audio Recording**: Chrome 47+, Firefox 29+, Safari 11+
- **File Upload**: All modern browsers



## Architecture

The SDK is built with a service-oriented architecture for maximum maintainability and performance:

```
@memoreco/memoreco-js
├── Core SDK (Memoreco class - 3,050 lines, reduced from 5,500)
├── Event System (EventEmitter)
├── Core Services:
│   ├── AuthManager (authentication & tokens)
│   ├── NetworkManager (connectivity management)
│   └── SystemLockHandler (system events)
├── Preview Services:
│   ├── PreviewController (camera lifecycle)
│   ├── InactivityManager (timeout handling)
│   ├── DeviceOverlay (device selection UI)
│   ├── Overlays (UI overlays & modals)
│   └── SnapshotEffects (visual/audio effects)
├── Capture & Display:
│   ├── CaptureCoordinator (capture orchestration)
│   └── MediaDisplay (media rendering)
├── Utilities:
│   ├── Logger (centralized logging)
│   ├── DOM utilities (element manipulation)
│   └── UI constants (theming & styles)
└── Types & Tests (TypeScript definitions & 100+ unit tests)
```

**Key Improvements:**
- 🔧 **45% code reduction** (5,500 → 3,050 lines) with 100% API compatibility
- 🧪 **100+ unit tests** with comprehensive service coverage  
- 🎯 **14 focused services** with single responsibilities
- ⚡ **Better performance** through optimized service architecture
- 📚 **Enhanced maintainability** with clear separation of concerns

For detailed architecture documentation, see the repo guide: [SDK_FILE_STRUCTURE.md](https://github.com/memoreco-dev/memoreco/blob/main/packages/memoreco-js/docs/SDK_FILE_STRUCTURE.md).

## Dependencies

- `@memoreco/recorder` - Screen and camera recording
- `@memoreco/api-sdk-public` - API client
- `@memoreco/types` - Shared types
- `@memoreco/utils` - Utility functions

## Release Workflow

Follow the internal release checklist: [`docs/PUBLISHING.md`](https://github.com/memoreco-dev/memoreco/blob/main/packages/memoreco-js/docs/PUBLISHING.md) for `pnpm run release:check`, the automated `scripts/publish.mjs` helper, and post-publish tagging guidance.


## Support

For questions and support:
- 📧 Email: support@memoreco.com
- 📚 Documentation: [docs.memoreco.com](https://docs.memoreco.com)