npm package: @sage-rsc/narrator-avatar
Description: React component for 3D talking avatars with lip-sync, Deepgram/Google TTS, content-aware gestures, and pause/resume
Latest version: 1.0.2

# narrator-avatar

[![npm version](https://img.shields.io/npm/v/@sage-rsc/narrator-avatar.svg)](https://www.npmjs.com/package/@sage-rsc/narrator-avatar)
[![license](https://img.shields.io/npm/l/@sage-rsc/narrator-avatar.svg)](https://www.npmjs.com/package/@sage-rsc/narrator-avatar)

React component for 3D talking avatars with lip-sync, Deepgram or Google TTS, content-aware hand gestures, and pause/resume. Built on [@met4citizen/talkinghead](https://www.npmjs.com/package/@met4citizen/talkinghead).

## Features

- **3D avatars** – Ready Player Me–compatible GLB models (full-body)
- **Lip-sync** – Word-level sync with Deepgram or Google TTS
- **TTS** – Deepgram (streaming) or Google Cloud Text-to-Speech
- **Gestures** – Content-aware hand gestures using all 8 built-in: handup, index, ok, thumbup, thumbdown, side, shrug, namaste
- **Playback** – Speak, Pause, Resume, Stop (phrase-level in accurate mode)
- **Accessibility** – Subtitle callback for closed captions

## Table of contents

- [Install](#install)
- [Usage](#usage)
- [Using with Vite](#using-with-vite)
- [Props](#props)
- [Ref API](#ref-api)
- [Environment variables](#environment-variables)
- [TypeScript](#typescript)
- [License](#license)

## Install

```bash
npm install @sage-rsc/narrator-avatar
```

Peer dependencies (React 18+) and `@met4citizen/talkinghead` are installed automatically.

**Using Vite?** Add one line to your Vite config or the avatar will fail to load. See [Using with Vite](#using-with-vite).

## Usage

```jsx
import { useRef } from 'react';
import NarratorAvatar from '@sage-rsc/narrator-avatar';

function MyPage() {
  const avatarRef = useRef(null);

  return (
    <div style={{ width: '400px', height: '500px' }}>
      <NarratorAvatar
        ref={avatarRef}
        avatarUrl="/avatars/brunette.glb"
        avatarBody="F"
        ttsService="deepgram"
        ttsVoice="aura-2-aurora-en"
        ttsApiKey={import.meta.env.VITE_DEEPGRAM_API_KEY}
        accurateLipSync={true}
        speechRate={0.9}
        onReady={() => console.log('Ready')}
        onSpeechStart={(text) => console.log('Started:', text)}
        onSpeechEnd={() => console.log('Ended')}
        onSubtitle={(text) => console.log('Subtitle:', text)}
      />
      <button onClick={() => avatarRef.current?.speakText('Hello! How are you?')}>
        Speak
      </button>
      <button onClick={() => avatarRef.current?.pauseSpeaking()}>Pause</button>
      <button onClick={() => avatarRef.current?.resumeSpeaking()}>Resume</button>
      <button onClick={() => avatarRef.current?.stopSpeaking()}>Stop</button>
    </div>
  );
}
```

## Using with Vite

**Required for Vite apps.** The underlying `@met4citizen/talkinghead` package loads lip-sync language modules via dynamic `import()`. If Vite pre-bundles it, those imports break and you get a 404 on `lipsync-en.mjs` and `Cannot read properties of undefined (reading 'preProcessText')`.

Exclude it from optimization:

```js
// vite.config.js or vite.config.ts
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';

export default defineConfig({
  plugins: [react()],
  optimizeDeps: { exclude: ['@met4citizen/talkinghead'] },
});
```

Restart the dev server after changing the config.

## Props

| Prop | Description |
|------|-------------|
| `avatarUrl` | URL to GLB model (e.g. `/avatars/brunette.glb`) |
| `avatarBody` | `'M'` or `'F'` for posture |
| `cameraView` | Camera framing (default `'mid'`) |
| `cameraRotateEnable` | Allow mouse drag to rotate view (default `false`). Set `true` to enable. |
| `cameraZoomEnable` | Allow mouse wheel to zoom (default `false`). Set `true` to enable. |
| `cameraPanEnable` | Allow mouse to pan (default `false`) |
| `ttsService` | `'google'` or `'deepgram'` |
| `ttsVoice` | Deepgram: e.g. `aura-2-mars-en`, `aura-2-aurora-en`. Google: e.g. `en-GB-Standard-A` |
| `ttsApiKey` | API key (or set `VITE_DEEPGRAM_API_KEY` / `VITE_GOOGLE_TTS_API_KEY`) |
| `lipsyncModules` | Array of language codes (default `['en']`) |
| `lipsyncLang` | Lip-sync language (default `'en'`) |
| `accurateLipSync` | `true` = REST per phrase, best lip-sync + pause/resume |
| `speechRate` | e.g. `0.9` for 10% slower (pitch-preserving) |
| `speechGestures` | Content-aware hand gestures (default `true`) |
| `onReady`, `onError`, `onSpeechStart`, `onSpeechEnd`, `onSubtitle` | Callbacks |

## Ref API

| Method / property | Description |
|-------------------|-------------|
| `speakText(text, options?)` | Speak text via TTS |
| `pauseSpeaking()` | Pause (phrase-level when `accurateLipSync` is true) |
| `resumeSpeaking()` | Resume from next phrase |
| `stopSpeaking()` | Stop and clear |
| `isReady` | Whether the avatar has finished loading |
| `isSpeaking` | Whether the avatar is currently speaking |

## Environment variables

| Variable | Use |
|----------|-----|
| `VITE_DEEPGRAM_API_KEY` | Deepgram TTS (or pass `ttsApiKey` prop) |
| `VITE_GOOGLE_TTS_API_KEY` | Google TTS when `ttsService="google"` (or pass `ttsApiKey`) |

## TypeScript

The package is JavaScript. For TypeScript, add a declaration file (e.g. `src/narrator-avatar.d.ts`) that declares the component props and ref type, or use the component with `// @ts-expect-error` if you prefer.

## License

MIT