npm package: @hunar.ai/voice-chat-client
Description: Voice chat client library for interacting with voice AI WebSocket servers
Author: Hunar
Latest version: 0.1.0

<h1 align='center'>Voice Chat Client (JS)</h1>

<p align="center">
  <a title="package-version" href="https://www.npmjs.com/package/@hunar.ai/voice-chat-client">
    <img alt="NPM Version" src="https://img.shields.io/npm/v/@hunar.ai/voice-chat-client?color=blue">
  </a>
  <a title="build status" href="#">
    <img src="https://img.shields.io/static/v1?label=build&message=passing&color=GREEN" />
  </a>
  <a href="https://github.com/Hunar-ai/voice-chat-client-js/blob/main/LICENSE.md">
        <img src="https://img.shields.io/static/v1?label=license&message=MIT&color=GREEN" alt="License">
    </a>
  <a href="https://yarnpkg.com/" alt="yarn">
     <img src="https://img.shields.io/static/v1?label=maintained%20with&message=yarn&color=2188b6"/>
  </a>
</p>

JS SDK for clients to connect to voice-chat ws server.

## Features

- 🎙️ **Real-time Audio Streaming**: Capture and stream microphone input with configurable sample rates
- 🔊 **Audio Playback**: Queue and play audio chunks with checkpoint synchronization
- ⚛️ **React Integration**: Ready-to-use hooks for React applications
- 🌐 **Framework Agnostic**: Core classes work in any JavaScript environment
- 📦 **Type Safe**: Full TypeScript support with complete type definitions
- 🎯 **WebSocket Protocol**: Standardized protocol for voice AI server communication

## Installation

```bash
yarn add @hunar.ai/voice-chat-client
```

## Quick Start

> Note: This library is primarily supports React applications. Vanilla JavaScript/HTML support will be added in the future.

```tsx
import React from "react";
import {
  RealtimeProvider,
  useRealtimeContext,
  useMicrophone,
  useMicrophoneSelector,
  useMicrophonePermission,
} from "@hunar.ai/voice-chat-client";

function VoiceChat() {
  const { client, clientState, connect, disconnect } = useRealtimeContext();
  const { currentDevice, audioDevices, changeAudioDevice } =
    useMicrophoneSelector();
  const { hasMicrophonePermission, requestMicrophonePermission } =
    useMicrophonePermission();

  const { isRecording, startRecording, stopRecording } = useMicrophone({
    currentDevice,
    onChunk: (audio, contentType, sampleRate) => {
      client.sendAudio(audio, contentType, sampleRate);
    },
  });

  React.useEffect(() => {
    connect({
      onConnect: () => console.log("Connected"),
      onError: (error) => console.error("Error:", error),
      onDisconnect: () => console.log("Disconnected"),
    });

    return () => disconnect();
  }, [connect, disconnect]);

  return (
    <div>
      <div>Status: {clientState}</div>

      {!hasMicrophonePermission && (
        <button onClick={requestMicrophonePermission}>Enable Microphone</button>
      )}

      <select onChange={(e) => changeAudioDevice(e.target.value)}>
        {Object.values(audioDevices).map((device) => (
          <option key={device.deviceId} value={device.deviceId}>
            {device.label}
          </option>
        ))}
      </select>

      <button onClick={isRecording ? stopRecording : startRecording}>
        {isRecording ? "Stop" : "Start"}
      </button>
    </div>
  );
}

function App() {
  return (
    <RealtimeProvider wsUrl="wss://your-server.com">
      <VoiceChat />
    </RealtimeProvider>
  );
}
```

## Audio Processing

The library uses an Audio Worklet for efficient real-time audio processing and PCM conversion. The worklet is automatically bundled with the package - no additional setup required.

The audio processing pipeline:

- Captures microphone input at configurable sample rates (default: 24kHz)
- Converts audio to 16-bit PCM format
- Streams audio chunks to your WebSocket server

**Note**: Audio worklets require HTTPS in production environments.

## API Reference

**Export Path Options:**

- `@hunar.ai/voice-chat-client` - All exports (core classes, React hooks, and providers)

### Core Classes

#### `RealtimeClient<TAudioPlayer>`

WebSocket client for voice AI communication.

```typescript
const client = new RealtimeClient<TAudioPlayer>(
  url: string, createAudioPlayer: () => TAudioPlayer
);

// Connect to server
client.connect({
  onConnect: () => void,
  onError: (error: Error) => void,
  onDisconnect: () => void,
});

// Send audio data
client.sendAudio(
  audio: ArrayBuffer | Int16Array,
  contentType?: string,    // default: 'audio/pcm'
  sampleRate?: number      // default: 24000
);

// Send context updates
client.sendContextUpdate(context: Record<string, unknown>);

// Send markers
client.sendMark(marker: string);

// Disconnect
client.disconnect();
```

#### `RealtimeClientState`

Connection state enumeration for the RealtimeClient.

```typescript
enum RealtimeClientState {
  CONNECTING = "connecting",
  CONNECTED = "connected",
  DISCONNECTED = "disconnected",
}
```

#### `AudioManager<TCheckpoint, TAudioPlayer>`

Manages audio playback queue with checkpoint synchronization.

```typescript
// TCheckpoint is the type of the checkpoint data. For example it could be a string or an object.
// TAudioPlayer extends BaseAudioPlayer for custom audio player implementations.
const audioManager = new AudioManager<TCheckpoint, TAudioPlayer>({
  onCheckpointPlayed: (checkpoint: TCheckpoint) => void,
  createAudioPlayer: () => TAudioPlayer
});

// Add audio chunk to queue
audioManager.addChunk(audioData: ArrayBuffer);

// Add checkpoint marker
audioManager.addCheckpoint(checkpointData: TCheckpoint);

// Control playback
audioManager.stop();
```

#### `BaseAudioPlayer`

Abstract base class for audio playback implementations.

```typescript
abstract class BaseAudioPlayer {
  abstract isPlaying: boolean;
  abstract get canPlay(): boolean;
  abstract play(props: AudioPlayerPlayProps): void;
  abstract stop(): void;
}
```

#### `BrowserAudioPlayer`

Web Audio API implementation of BaseAudioPlayer for browser environments.

```typescript
const audioPlayer = new BrowserAudioPlayer();

// Play audio buffer
audioPlayer.play({
  buffer: audioData,
  onended: () => console.log("Playback finished"),
});

// Stop playback
audioPlayer.stop();

// Check playback state
console.log(audioPlayer.isPlaying);
```

### React Hooks

#### `useRealtimeContext`

Provides access to the RealtimeClient instance and connection state within a RealtimeProvider.

```typescript
const {
  client, // RealtimeClient instance
  clientState, // RealtimeClientState (CONNECTED | DISCONNECTED | CONNECTING)
  connect, // (props: {onConnect, onError, onDisconnect}) => void
  disconnect, // () => void
  isPlaying, // boolean: current playback state
} = useRealtimeContext();
```

#### `RealtimeProvider`

Context provider that manages a RealtimeClient instance and tracks connection state.

```typescript
<RealtimeProvider wsUrl="wss://your-server.com">
  <YourApp />
</RealtimeProvider>
```

The provider automatically manages the WebSocket connection lifecycle and provides access to the current connection state via `useRealtimeContext`.

#### `useMicrophone`

Manages microphone recording with audio streaming.

```typescript
const {
  isRecording,              // boolean: current recording state
  startRecording,           // () => Promise<void>: start recording
  stopRecording,            // () => void: stop recording and cleanup
  toggleMicrophoneCapture,  // () => void: pause/resume recording without cleanup
} = useMicrophone({
  currentDevice: string | null,
  onChunk: (audio, contentType?, sampleRate?) => void,
  sampleRate?: number,      // default: 24000
});
```

#### `useMicrophoneSelector`

Lists and manages audio input devices.

```typescript
const {
  audioDevices, // AudioDeviceMap
  currentDevice, // string | null
  changeAudioDevice, // (deviceId: string | null) => Promise<void>
  handleDeviceChangeEvent, // () => void
} = useMicrophoneSelector({
  workletPath: string,
});
```

#### `useMicrophonePermission`

Handles microphone permission requests.

```typescript
const {
  hasMicrophonePermission, // boolean
  permissionError, // string
  checkMicrophonePermission, // () => Promise<void>
  requestMicrophonePermission, // () => Promise<boolean>
} = useMicrophonePermission();
```

#### `useMedia`

Low-level hook for media device access and audio recording.

```typescript
const {
  getAudioDevices, // () => Promise<AudioDeviceMap>
  getAudioDevice, // (deviceId: string) => Promise<MediaStream>
  createAudioRecorder, // (onChunk, options?) => MediaRecorder
} = useMedia({
  workletPath: string,
});
```

## WebSocket Protocol

The client follows this protocol for communication:

### Client → Server Messages

**Start Stream**

```json
{
  "event": "start",
  "start": { "streamId": "uuid" }
}
```

**Send Audio**

```json
{
  "event": "media",
  "streamId": "uuid",
  "media": {
    "contentType": "audio/pcm",
    "sampleRate": 24000,
    "payload": "base64-encoded-audio"
  }
}
```

**Context Update**

```json
{
  "event": "contextUpdate",
  "streamId": "uuid",
  "context": { "key": "value" }
}
```

**Mark Played**

```json
{
  "event": "playedStream",
  "streamId": "uuid",
  "name": "marker-name"
}
```

### Server → Client Messages

**Play Audio**

```json
{
  "event": "playAudio",
  "media": {
    "contentType": "audio/pcm",
    "sampleRate": 24000,
    "payload": "base64-encoded-audio"
  }
}
```

**Checkpoint**

```json
{
  "event": "checkpoint",
  "streamId": "uuid",
  "name": "checkpoint-name"
}
```

**Clear Audio**

```json
{
  "event": "clearAudio",
  "streamId": "uuid"
}
```

## TypeScript Support

The library is written in TypeScript and includes complete type definitions. All exports are fully typed.

```typescript
import type {
  // Audio Manager types
  QueuedItemType,
  QueuedAudioItem,
  QueuedCheckpointItem,
  QueueItem,

  // Audio Player types
  AudioPlayerPlayProps,

  // Realtime Client types
  RealtimeClientEvent,
  RealtimeServerEvent,
  RealtimeClientState,
  RealtimeClientMessageProps,
  RealtimeServerMessageProps,
  RealtimeStartMessageProps,
  RealtimeContextUpdateMessageProps,
  RealtimeMediaMessageProps,
  RealtimeMarkMessageProps,
  RealtimeStopMessageProps,
  RealtimePlayAudioMessageProps,
  RealtimeClearAudioMessageProps,
  RealtimeClearedAudioMessageProps,
  RealtimeCheckpointMessageProps,

  // React Hook types
  AudioDevice,
  AudioDeviceMap,
  UseMicrophoneOptions,
} from "@hunar.ai/voice-chat-client";
```

## Browser Compatibility

- Chrome/Edge: ✅ Full support
- Firefox: ✅ Full support
- Safari: ✅ Full support (14.1+)
- Opera: ✅ Full support

Requires:

- WebSocket API
- Web Audio API
- AudioWorklet API
- MediaDevices API

## Examples

TBD

## License

[MIT](LICENSE.md)

## Attributions

This project includes code derived from external sources:

### Audio Processor Worklet

The `src/core/audio-processor-worker.js` file is derived from the Pipecat Client Web Transports library:

- **Source**: <https://github.com/pipecat-ai/pipecat-client-web-transports>
- **Original file**: lib/wavtools/lib/worklets/audio_processor.js
- **Raw URL**: <https://raw.githubusercontent.com/pipecat-ai/pipecat-client-web-transports/ad419d77b1221528ca1ea8ccaa973c0a8176f81a/lib/wavtools/lib/worklets/audio_processor.js>
- **Date accessed**: October 24, 2025
- **License**: BSD 2-Clause License (Copyright © 2024, Daily)

## Contributing

This is a source available project. Direct PR for contributions are not accepted. Please open issues for any bugs or feature requests.