npm package: @mp4e/player
Description: MP4E React Player - Engine-powered interactive video player
Latest version: 1.0.20

# @mp4e/player

React player for MP4E interactive videos. The WASM engine is bundled inline for universal compatibility.

## Installation

```bash
npm install @mp4e/player
```

## Usage

```tsx
import { MP4EPlayer } from '@mp4e/player';
import '@mp4e/player/styles.css';

function App() {
  return (
    <MP4EPlayer
      src="/video.mp4"
      onMetadataLoaded={(metadata) => {
        console.log('Loaded:', metadata.schemaVersion);
      }}
    />
  );
}
```

The player automatically extracts metadata embedded in the MP4 file - you typically only need the `src` prop.

## Features

- **Automatic metadata extraction** - Reads MP4E metadata from the video file
- **WASM engine bundled** - Works offline, no external fetches
- **Universal bundler support** - Vite, webpack, Next.js
- **TypeScript types included**
- **WebGL-accelerated rendering** - GPU-powered object highlighting
- **Worker mode** - Offload engine to Web Worker for better performance

## Props

### Core Props

| Prop | Type | Default | Description |
|------|------|---------|-------------|
| `src` | `string` | required | Video URL |
| `metadata` | `MP4EMetadata` | - | Override embedded metadata |
| `controls` | `boolean` | `true` | Show playback controls |
| `autoplay` | `boolean` | `false` | Auto-start playback |
| `muted` | `boolean` | `false` | Start muted |
| `loop` | `boolean` | `false` | Loop playback |
| `interactive` | `boolean` | `true` | Enable object interactions |
| `showBboxes` | `boolean` | `false` | Show object bounding boxes |
| `showCanvasLabels` | `boolean` | `false` | Show labels on hover |
| `showObjectDisplays` | `boolean` | `true` | Show hover/click displays |
| `useWorker` | `boolean` | `false` | Run engine in Web Worker |
| `debug` | `boolean` | `false` | Show debug UI |

### Callback Props

| Prop | Type | Description |
|------|------|-------------|
| `onEngineReady` | `() => void` | Engine initialized |
| `onEngineError` | `() => void` | Engine failed to load |
| `onMetadataLoaded` | `(metadata: MP4EMetadata) => void` | Metadata loaded |
| `onTimeUpdate` | `(time: number, frame: number) => void` | Time changed (throttled ~60fps) |
| `onFrameChange` | `(frame: number) => void` | Frame changed (every frame) |
| `onVideoDuration` | `(duration: number) => void` | Duration available |
| `onObjectClick` | `(objectId: string, data: ObjectData) => void` | Object clicked |
| `onObjectHover` | `(objectId: string \| null, data: ObjectData \| null) => void` | Object hovered |
| `onPlay` | `() => void` | Video started |
| `onPause` | `() => void` | Video paused |
| `onEnded` | `() => void` | Video ended |
| `onSceneChange` | `(scene: Scene) => void` | Scene changed |

### Advanced Props

| Prop | Type | Description |
|------|------|-------------|
| `initialTime` | `number` | Start time in seconds |
| `objectGroups` | `ObjectGroup[]` | Object grouping configuration |
| `defaultDisplaySettings` | `ObjectDisplaySettings` | Default display behavior |
| `runtimeFlags` | `MP4ERuntimeFlags` | Feature flags |
| `disableInteraction` | `boolean` | Disable all hover/click interactions |

## Ref API

Access player methods via ref:

```tsx
import { useRef } from 'react';
import { MP4EPlayer, MP4EPlayerRef } from '@mp4e/player';

function App() {
  const playerRef = useRef<MP4EPlayerRef>(null);

  const handleSeek = () => {
    playerRef.current?.seek(10); // Seek to 10 seconds
  };

  return (
    <>
      <MP4EPlayer ref={playerRef} src="/video.mp4" />
      <button onClick={handleSeek}>Jump to 10s</button>
    </>
  );
}
```

### Playback Methods

| Method | Returns | Description |
|--------|---------|-------------|
| `play()` | `Promise<void>` | Start playback |
| `pause()` | `void` | Pause playback |
| `seek(time)` | `void` | Seek to time (seconds) |
| `seekToFrame(frame)` | `void` | Seek to frame number |
| `seekSilently(time)` | `void` | Seek without triggering rules |

### State Methods

| Method | Returns | Description |
|--------|---------|-------------|
| `getCurrentTime()` | `number` | Current time in seconds |
| `getCurrentFrame()` | `number` | Current frame number |
| `getDuration()` | `number` | Video duration in seconds |
| `isPaused()` | `boolean` | Is video paused |
| `isMuted()` | `boolean` | Is video muted |
| `getVolume()` | `number` | Volume (0-1) |
| `setVolume(v)` | `void` | Set volume (0-1) |
| `mute()` | `void` | Mute audio |
| `unmute()` | `void` | Unmute audio |
| `getPlaybackRate()` | `number` | Playback speed |
| `setPlaybackRate(r)` | `void` | Set playback speed |

### Fullscreen Methods

| Method | Returns | Description |
|--------|---------|-------------|
| `isFullscreen()` | `boolean` | Is player fullscreen |
| `enterFullscreen()` | `Promise<void>` | Enter fullscreen |
| `exitFullscreen()` | `Promise<void>` | Exit fullscreen |
| `toggleFullscreen()` | `Promise<void>` | Toggle fullscreen |

### Element Access

| Method | Returns | Description |
|--------|---------|-------------|
| `getVideoElement()` | `HTMLVideoElement \| null` | Video element |
| `getCanvasElement()` | `HTMLCanvasElement \| null` | Canvas element |
| `getOverlayContainerRef()` | `HTMLDivElement \| null` | Overlay container |

### Coordinate Conversion

| Method | Returns | Description |
|--------|---------|-------------|
| `getVideoDisplayTransform()` | `VideoDisplayTransform \| null` | Get scale/offset for coordinate conversion |
| `getOverlayPositions()` | `Map<string, RenderedOverlayPosition>` | Get current overlay positions |
| `getEffectiveMetadata()` | `MP4EMetadata \| null` | Get resolved metadata |

#### VideoDisplayTransform

```typescript
interface VideoDisplayTransform {
  scale: number       // Scale factor from video-native to display
  offsetX: number     // Horizontal offset (letterboxing)
  offsetY: number     // Vertical offset (letterboxing)
  displayWidth: number
  displayHeight: number
  videoWidth: number
  videoHeight: number
}
```

Use this to convert between video-native coordinates and screen coordinates:

```typescript
// Convert video-native position to screen position
const transform = playerRef.current?.getVideoDisplayTransform();
if (transform) {
  const screenX = videoNativeX * transform.scale + transform.offsetX;
  const screenY = videoNativeY * transform.scale + transform.offsetY;
}
```

### Imperative Overlay Updates

For smooth 60fps position updates (e.g., during drag operations):

```typescript
// Update overlay position imperatively (no React re-render)
playerRef.current?.updateOverlayStyle('overlay-id', {
  x: 100,  // video-native coordinates
  y: 50,
  width: 200,
  height: 100,
});
```

This directly updates the DOM element for smooth visual feedback, bypassing React's rendering cycle. Useful for:
- Drag operations
- Real-time position updates
- Animation

## Types

### MP4EMetadata

```typescript
interface MP4EMetadata {
  schemaVersion: string;
  videoDetails: VideoDetails;
  objects: {
    registry: ObjectRegistry;
    timeline: FrameData[];
  };
  layers: Layer[];
  variables?: Variable[];
  rules?: Rule[];
  scenes?: Scene[];
}
```

### ObjectData

```typescript
interface ObjectData {
  id: string;
  label: string;
  userLabel?: string;
  confidence: number;
  trackingType: ObjectTrackingType;
  groupIds?: string[];
  hidden?: boolean;
  data?: Record<string, any>;
}
```

## Development

```bash
# Run demo app
npm run dev

# Build library
npm run build:lib

# Build types
npm run build:types

# Publish to npm
npm publish
```

## Architecture

```
Player (React) → Engine (WASM) → Pure State Machine
```

- **Player** is the "body" - handles UI, video, rendering
- **Engine** is the "brain" - handles events, state, rules
- Zero coupling - player listens to engine events

## Performance

The player includes several performance optimizations:

1. **Phase 1**: Visibility-based re-rendering - only re-render when object visibility changes
2. **Phase 2**: DOM pooling - reuse overlay elements instead of creating/destroying
3. **Phase 3**: WebGL rendering - GPU-accelerated bounding box rendering
4. **Worker mode**: Offload engine computation to Web Worker

Enable worker mode for complex videos:

```tsx
<MP4EPlayer src="/video.mp4" useWorker />
```

## Changelog

### v0.2.79

- Added `disableInteraction` prop to disable all hover/click interactions
- Added `onFrameChange` callback (fires on every frame change)
- Added `onVideoDuration` callback (fires when duration is available)
- Added `getVideoDisplayTransform()` ref method for coordinate conversion
- Added `getOverlayPositions()` ref method to get current overlay positions
- Added `updateOverlayStyle()` ref method for imperative 60fps position updates
- Added `getEffectiveMetadata()` ref method
- Added `getOverlayContainerRef()` ref method

### v0.2.78

- WebGL-accelerated bounding box rendering (Phase 3 optimization)
- Improved performance for videos with many objects

### v0.2.77

- DOM pooling for group-bound overlays (Phase 2 optimization)
- Visibility-based re-rendering (Phase 1 optimization)