npm package: @keyframelabs/elements
Description: Framework-agnostic primitives for Keyframe Labs.
Latest version: 0.4.0

# @keyframelabs/elements

Framework-agnostic primitives for Keyframe Labs.

**Which package should I use?**
- **@keyframelabs/sdk** (high control)
  - You implement the UI, state management, and agent/llm binding yourself
- **@keyframelabs/elements** (custom UI)
  - You implement the UI; we handle the state and agent/llm binding (framework-agnostic)
- **@keyframelabs/react**: (drop-in)
  - We handle the UI, state, and agent/llm binding

## Install

```bash
pnpm add @keyframelabs/elements
```

## Usage

This package provides three integration options depending on your strategy:
1. **`PersonaEmbed`**: Fully managed. Uses a publishable key. Best for rapid frontend integration.
2. **`PersonaView`**: Bring your own backend. Uses session tokens generated by your server.
3. **`<kfl-embed>`**: Drop-in web component. Wraps `PersonaEmbed` with a complete widget UI (states, controls, positioning). Zero framework dependencies.

### Option A: PersonaEmbed (managed)

Use this if you have configured an embed in the Keyframe platform dashboard.

```ts
import { PersonaEmbed } from '@keyframelabs/elements';

const embed = new PersonaEmbed({
  container: document.getElementById('persona-container'),
  publishableKey: 'kfl_pk_live_...',
  videoFit: 'cover',
  onStateChange: (status) => console.log('Status:', status),
  onAgentStateChange: (state) => console.log('Agent:', state),
  onDisconnect: () => console.log('Disconnected'),
  onError: (err) => console.error(err),
});

await embed.connect();

// Later...
embed.disconnect();
```

### Option B: PersonaView (bring your own backend)

Use this when you want to manage authentication through your own backend.

```ts
import { PersonaView } from '@keyframelabs/elements';

const view = new PersonaView({
  container: document.getElementById('persona-container')!,
  sessionDetails,
  voiceAgentDetails,
  videoFit: 'contain',
  onStateChange: (status) => console.log('Status:', status),
  onAgentStateChange: (state) => console.log('Agent:', state),
  onDisconnect: () => console.log('Disconnected'),
  onError: (err) => console.error(err),
});

await view.connect();

// Later...
view.disconnect();
```

### Option C: `<kfl-embed>` Web Component

A self-registering custom element that wraps `PersonaEmbed` with a widget UI shell: minimized/active/hidden states, "Join call" button, minimize/expand toggle, corner positioning, and button color styling. Drop it into any page with zero framework dependencies.

```html
<kfl-embed
  publishable-key="kfl_pk_live_..."
  initial-state="minimized"
  corner="bottom-right"
  minimized-width="144"
  minimized-height="216"
  active-width="252"
  active-height="377"
  button-color="#919191"
  button-color-opacity="0.3"
  video-fit="cover"
  preview-image="https://example.com/avatar.png"></kfl-embed>
<script type="module" src="https://unpkg.com/@keyframelabs/elements/dist/kfl-embed.js"></script>
```

#### Attributes

| Attribute                        | Type                                                       | Default          | Description                                                        |
| -------------------------------- | ---------------------------------------------------------- | ---------------- | ------------------------------------------------------------------ |
| `publishable-key`                | `string`                                                   | Required         | Your publishable embed key.                                        |
| `api-base-url`                   | `string`                                                   | Keyframe default | Base URL for the Keyframe API.                                     |
| `initial-state`                  | `'minimized' \| 'active' \| 'hidden'`                      | `'minimized'`    | Widget state on first load.                                        |
| `controlled-widget-state`        | `'minimized' \| 'active' \| 'hidden'`                      | —                | Externally control the widget state (overrides internal state).     |
| `corner`                         | `'bottom-right' \| 'bottom-left' \| 'top-right' \| 'top-left'` | `'bottom-right'` | Which corner to anchor the widget (fixed mode only).               |
| `inline`                         | boolean attribute                                          | —                | Use `position: relative` instead of `fixed`.                       |
| `minimized-width`                | `number`                                                   | `144`            | Width in px when minimized.                                        |
| `minimized-height`               | `number`                                                   | `216`            | Height in px when minimized.                                       |
| `active-width`                   | `number`                                                   | `252`            | Width in px when active (in-call).                                 |
| `active-height`                  | `number`                                                   | `377`            | Height in px when active (in-call).                                |
| `hide-ui`                        | boolean attribute                                          | —                | Hides all overlay controls. Useful for building your own UI shell. |
| `show-minimize-button`           | `'true' \| 'false'`                                        | `'true'`         | Show the X/minimize button on hover.                               |
| `controlled-show-minimize-button`| `'true' \| 'false'`                                        | —                | Externally control the minimize button visibility.                 |
| `button-color`                   | hex color string                                           | `'#919191'`      | Color of the "Join call" button.                                   |
| `button-color-opacity`           | `number` (0–1)                                             | `0.3`            | Opacity of the "Join call" button background.                      |
| `video-fit`                      | `'cover' \| 'contain'`                                     | `'cover'`        | Video scaling mode.                                                |
| `preview-image`                  | URL string                                                 | —                | Image shown in the widget before a call starts.                    |

#### Events

| Event              | `detail`                                | Description                            |
| ------------------ | --------------------------------------- | -------------------------------------- |
| `statechange`      | `{ status: EmbedStatus }`               | Connection status changed.             |
| `agentstatechange` | `{ state: AgentState }`                 | Avatar playback state changed.         |
| `widgetstatechange`| `{ state: 'minimized' \| 'active' \| 'hidden' }` | Widget UI state changed.      |
| `disconnected`     | —                                       | Session disconnected.                  |
| `error`            | `{ error: Error }`                      | A fatal error occurred.                |

#### JavaScript API

```ts
const el = document.querySelector('kfl-embed');

await el.micOn();     // Start session if needed, then unmute mic
await el.micOff();    // Mute mic
el.isMicOn();         // boolean

await el.mute();      // Mute speaker audio
await el.unmute();    // Unmute speaker audio
el.isMuted();         // boolean
```

#### Build

The web component is built as a self-contained ES module via `vite.config.embed.ts`:

```bash
pnpm build:embed   # → dist/kfl-embed.js
```

Import the auto-registering entry point (registers `<kfl-embed>` on `customElements`):

```ts
import '@keyframelabs/elements/kfl-embed';
```

Or import the class directly for manual registration:

```ts
import { KflEmbedElement } from '@keyframelabs/elements';
customElements.define('my-embed', KflEmbedElement);
```

## Supported agents and real-time LLMs

Supports ElevenLabs and OpenAI Realtime.

For `PersonaEmbed`, this is determined by the values you set in the Keyframe platform dashboard.

For `PersonaView`, this is determined by `voiceAgentDetails`.

## Emotion Controls

The avatar can display emotional expressions (`neutral`, `angry`, `sad`, `happy`) that affect its facial expression and demeanor. All supported voice agents can drive emotions automatically via tool/function calling.

### Agent Events

The `emotion` event is emitted when any agent triggers a `set_emotion` tool call:

```typescript
agent.on('emotion', (emotion) => {
  console.log('Emotion changed:', emotion); // 'neutral' | 'angry' | 'sad' | 'happy'
});
```

When using `PersonaEmbed` or `PersonaView`, emotion events are automatically wired to the avatar session -- no extra code is needed.

### ElevenLabs

Emotions are driven by a **client tool call** named `set_emotion`. The agent parses incoming `client_tool_call` WebSocket messages and sends a `client_tool_result` back.

**Setup:** Create a `set_emotion` [client tool](https://elevenlabs.io/docs/conversational-ai/customization/tools/client-tools) in the ElevenLabs dashboard for your agent with a single `emotion` parameter (enum: `neutral`, `angry`, `sad`, `happy`). Then instruct your agent (via its system prompt) to call `set_emotion` on each turn.

### OpenAI Realtime

The `set_emotion` function is **automatically declared** in the OpenAI Realtime session setup. The model calls it via Realtime [function calling](https://developers.openai.com/api/docs/guides/realtime-conversations#function-calling) (`response.done` with `function_call` output items), and the client responds by creating a `function_call_output` conversation item before asking the model to continue.

**Setup:** No additional dashboard configuration is needed. Instruct the model via its system prompt to call `set_emotion` on each turn to reflect the tone of its response.

### Manual Emotion Control

For custom emotion logic outside of tool calling, you can access the underlying session directly:

```typescript
import { createClient } from '@keyframelabs/sdk';

const session = createClient({ ... });
await session.setEmotion('happy');
```

## API

### `PersonaEmbed`

#### Options

| Option               | Type                            | Default                          | Description                                    |
| -------------------- | ------------------------------- | -------------------------------- | ---------------------------------------------- |
| `container`          | `HTMLElement`                   | Required                         | Target container for video and audio elements. |
| `publishableKey`     | `string`                        | Required                         | Your publishable embed key.                    |
| `apiBaseUrl`         | `string`                        | `'https://api.keyframelabs.com'` | Base URL for the Keyframe API.                 |
| `videoFit`           | `'cover' \| 'contain'`          | `'cover'`                        | Video scaling mode (`object-fit`).             |
| `onStateChange`      | `(status: EmbedStatus) => void` | —                                | Fired when connection status changes.          |
| `onAgentStateChange` | `(state: AgentState) => void`   | —                                | Fired when avatar playback state changes. Signaled by the GPU node via RPC, not the voice agent. |
| `onDisconnect`       | `() => void`                    | —                                | Fired when the session disconnects.            |
| `onError`            | `(err: Error) => void`          | —                                | Fired on fatal errors.                         |

#### Methods

| Method       | Signature             | Description                                        |
| ------------ | --------------------- | -------------------------------------------------- |
| `connect`    | `() => Promise<void>` | Starts the session and establishes the connection. |
| `disconnect` | `() => void`          | Disconnects the session and cleans up resources.   |
| `toggleMute` | `() => void`          | Toggles the user's microphone mute state.          |

#### Properties

| Property       | Type               | Description                                                                            |
| -------------- | ------------------ | -------------------------------------------------------------------------------------- |
| `status`       | `EmbedStatus`      | Current connection status: `'connecting' \| 'connected' \| 'disconnected' \| 'error'`. |
| `agentState`   | `AgentState`       | Avatar playback state: `'listening' \| 'speaking'`. Set by the GPU node, not the voice agent. |
| `isMuted`      | `boolean`          | Whether the microphone is currently muted.                                             |
| `videoElement` | `HTMLVideoElement` | The underlying video element used for rendering.                                       |
| `audioElement` | `HTMLAudioElement` | The underlying audio element used for playback.                                        |

### `PersonaView`

#### Options

| Option               | Type                            | Default   | Description                                    |
| -------------------- | ------------------------------- | --------- | ---------------------------------------------- |
| `container`          | `HTMLElement`                   | Required  | Target container for video and audio elements. |
| `sessionDetails`     | `SessionDetails`                | Required  | Session configuration from your backend.       |
| `voiceAgentDetails`  | `VoiceAgentDetails`             | Required  | Voice agent configuration from your backend.   |
| `videoFit`           | `'cover' \| 'contain'`          | `'cover'` | Video scaling mode (`object-fit`).             |
| `onStateChange`      | `(status: EmbedStatus) => void` | —         | Fired when connection status changes.          |
| `onAgentStateChange` | `(state: AgentState) => void`   | —         | Fired when agent state changes.                |
| `onDisconnect`       | `() => void`                    | —         | Fired when the session disconnects.            |
| `onError`            | `(err: Error) => void`          | —         | Fired on fatal errors.                         |

#### Methods

Same as `PersonaEmbed`.

#### Properties

Same as `PersonaEmbed`.

#### Types

```ts
type SessionDetails = {
  server_url: string;
  participant_token: string;
  agent_identity: string;
};

type VoiceAgentDetails = {
  type: 'elevenlabs' | 'openai';
  token?: string;        // For openai (ephemeral client secret)
  agent_id?: string;     // For elevenlabs
  signed_url?: string;   // For elevenlabs
  system_prompt?: string; // For openai
  voice?: string;         // For openai
};

type Emotion = 'neutral' | 'angry' | 'sad' | 'happy';
```

## License

MIT