npm package: @jona27/livetalking-streaming-sdk
Description: Streaming Avatar SDK for LiveTalking with Wav2Lip - JavaScript Version
Author: jona27
Latest version: 1.0.0

# LiveTalking Streaming SDK

A Node.js SDK for streaming avatars using LiveTalking with Wav2Lip technology, fully compatible with your enhanced streaming API backend.

## Installation

```bash
npm install @jona27/livetalking-streaming-sdk
```

## Quick Start

```javascript
const { StreamingAvatar } = require('@jona27/livetalking-streaming-sdk');

async function main() {
  // Initialize the streaming avatar
  const avatar = new StreamingAvatar({
    serverUrl: 'http://localhost:8010', // Your LiveTalking server URL
    debug: true
  });

  try {
    // Create authentication token
    await avatar.createToken({ user_id: 'your_user_id' });
    
    // Create and start avatar session
    const sessionData = await avatar.createStartAvatar({
      avatarId: 'wav2lip256_avatar1', // Your avatar ID
      quality: 'medium'
    });
    
    console.log('Session created:', sessionData.sessionId);

    // Set up event listeners
    avatar.on('avatar_start_talking', (text) => {
      console.log('Avatar started talking:', text);
    });

    avatar.on('stream_ready', (data) => {
      console.log('Stream ready:', data.sessionId);
    });

    // Make avatar speak
    await avatar.speak({
      text: 'Hello! This is your streaming avatar.',
      interrupt: false
    });

    // Interrupt current speech
    await avatar.interrupt();

    // Close session when done
    await avatar.closeAvatar();
    
  } catch (error) {
    console.error('Error:', error.message);
  }
}

main();
```

## API Reference

### StreamingAvatar

#### Constructor

```javascript
new StreamingAvatar(config)
```

**Parameters:**
- `config.serverUrl` (string): Your LiveTalking server URL
- `config.token` (string, optional): Pre-existing authentication token
- `config.debug` (boolean, optional): Enable debug logging

#### Methods

##### `createToken(request?)`

Creates an authentication token for API access.

**Parameters:**
- `request.user_id` (string, optional): User identifier

**Returns:** Promise<CreateTokenResponse>

##### `createStartAvatar(request)`

Creates and starts an avatar session with WebSocket connection.

**Parameters:**
- `request.avatarId` (string): Avatar ID (e.g., 'wav2lip256_avatar1')
- `request.quality` ('low' | 'medium' | 'high', optional): Video quality
- `request.voice` (object, optional): Voice configuration
- `request.knowledgeBase` (string, optional): Knowledge base for LLM

**Returns:** Promise<AvatarSessionData>

##### `startSession()`

Starts an existing session (if not auto-started).

##### `speak(request)`

Makes the avatar speak text via WebSocket or HTTP.

**Parameters:**
- `request.text` (string): Text to speak
- `request.interrupt` (boolean, optional): Whether to interrupt current speech

##### `interrupt()`

Interrupts current avatar speech.

##### `closeAvatar()`

Closes the avatar session and cleans up resources.

##### `getSessionId()`

Gets the current session ID.

**Returns:** string | null

##### `isAvatarConnected()`

Checks if avatar is connected via WebSocket.

**Returns:** boolean

##### `sendWebRTCOffer(sdp)`

Sends WebRTC offer for direct video streaming.

**Parameters:**
- `sdp` (string): WebRTC SDP offer

##### `ping()`

Sends ping to keep WebSocket connection alive.

#### Events

- `avatar_start_talking`: Fired when avatar starts speaking with text
- `avatar_stop_talking`: Fired when avatar stops speaking
- `stream_ready`: Fired when stream is ready with session data
- `stream_disconnected`: Fired when stream disconnects
- `error`: Fired when an error occurs
- `connection_established`: Fired when WebRTC connection is established
- `session_created`: Fired when session is created with session ID
- `websocket_connected`: Fired when WebSocket connects
- `websocket_disconnected`: Fired when WebSocket disconnects

## Enhanced API Integration

This SDK integrates with your enhanced streaming API backend:

### Authentication Flow
1. **Token Creation**: `POST /api/v1/streaming/create_token`
2. **Session Creation**: `POST /api/v1/streaming/new`
3. **WebSocket Connection**: `GET /ws/sdk/{session_id}`

### Real-time Communication
- **WebSocket**: Real-time bidirectional communication
- **HTTP Fallback**: API endpoints for speak/interrupt
- **Event System**: Real-time status updates

### Available Endpoints
- `POST /api/v1/streaming/create_token` - Create auth token
- `POST /api/v1/streaming/new` - Create session
- `POST /api/v1/streaming/start` - Start session
- `DELETE /api/v1/streaming/end_session` - End session
- `POST /api/v1/streaming/speak` - Make avatar speak
- `POST /api/v1/streaming/interrupt` - Interrupt speech
- `GET /api/v1/streaming/avatars` - List available avatars
- `GET /api/v1/streaming/health` - Health check

## Backend Requirements

Make sure your LiveTalking server is running with the enhanced API:

```bash
python app.py --model wav2lip --transport webrtc --listenport 8010
```

The enhanced API will be available at: `http://localhost:8010/api/v1/streaming/`

## Features

- ✅ JWT-based authentication
- ✅ WebSocket real-time communication
- ✅ HTTP API fallback
- ✅ Session management
- ✅ Text-to-speech with Wav2Lip
- ✅ Speech interruption
- ✅ Event-driven architecture
- ✅ TypeScript support
- ✅ WebRTC integration ready
- ✅ Keep-alive ping/pong
- ✅ Error handling and recovery

## License

Apache-2.0