npm package: @sathsarabandaraj/audio-handler
Description: A comprehensive audio handling library for STT (Speech-to-Text) and TTS (Text-to-Speech) functionality using Deepgram
Author: Sath
Latest version: 0.1.2

# @sathsarabandaraj/audio-handler

> A unified TypeScript library for browser-based audio management, featuring Speech-to-Text (STT) and Text-to-Speech (TTS) powered by Deepgram AI.

[![npm version](https://img.shields.io/npm/v/@sathsarabandaraj/audio-handler.svg)](https://www.npmjs.com/package/@sathsarabandaraj/audio-handler)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## 🌟 Features

- **🎤 Speech-to-Text**: Real-time speech transcription using Deepgram's AI
- **🔊 Text-to-Speech**: High-quality voice synthesis with multiple voice options
- **🔌 Provider System**: Pluggable architecture for different AI providers
- **⚡️ Event-Driven**: React to audio events in real-time
- **🌐 Browser Ready**: Works seamlessly in web applications
- **🛡️ TypeScript**: Full type safety and IntelliSense support
- **🔄 Smart Fallbacks**: Automatic fallback to Web Speech API when needed
- **🎯 Easy Integration**: Simple API, minimal setup

## 📦 Installation

```bash
npm install @sathsarabandaraj/audio-handler
```

Or with pnpm:

```bash
pnpm add @sathsarabandaraj/audio-handler
```

## 🚀 Quick Start

### 1. Initialize AudioManager

```javascript
import AudioManager from '@sathsarabandaraj/audio-handler';

// Create instance
const audioManager = new AudioManager({
  apiKey: 'your-deepgram-api-key',
  language: 'en-US'  // Optional
});

// Initialize
await audioManager.initialize();
```

### 2. Speech-to-Text

```javascript
// Start listening
await audioManager.startListening((result) => {
  console.log('Transcript:', result.text);
  console.log('Is Final:', result.isFinal);
  console.log('Confidence:', result.confidence);
});

// Stop listening
await audioManager.stopListening();
```

### 3. Text-to-Speech

```javascript
// Generate speech
const result = await audioManager.speak('Hello, world!', {
  model: 'aura-asteria-en'
});

// Play audio
const blob = new Blob([result.audioBuffer], { type: 'audio/mpeg' });
const url = URL.createObjectURL(blob);
const audio = new Audio(url);
audio.play();
```

## 📚 Complete Examples

We provide **three complete, working examples** that demonstrate different implementation approaches:

### 1. HTML Example (Vanilla JavaScript)

Single-file HTML application with no build step required.

```bash
cd examples/html-example
python3 -m http.server 8080
# Open http://localhost:8080
```

**Features:**
- ✅ Single HTML file
- ✅ No framework needed
- ✅ ES6 modules
- ✅ Smart TTS fallback
- ✅ Deferred transcription

[📖 HTML Example Documentation](examples/html-example/README.md)

### 2. React Example (Component-Based)

Modern React application with Vite build system.

```bash
cd examples/react-example
pnpm install
pnpm run dev
# Open http://localhost:5173
```

**Features:**
- ✅ React 18 + Hooks
- ✅ Component architecture
- ✅ Hot Module Replacement
- ✅ TypeScript-ready
- ✅ Production-ready structure

[📖 React Example Documentation](examples/react-example/README.md)

### 3. Backend Proxy (Node.js/Express)

Secure proxy server to solve CORS issues and protect API keys.

```bash
cd examples/backend
npm install
cp .env.example .env
# Edit .env and add your API key
npm start
# Server runs on http://localhost:3001
```

**Features:**
- ✅ Express.js REST API
- ✅ CORS-enabled
- ✅ API key security
- ✅ Audio streaming
- ✅ Health checks

[📖 Backend Documentation](examples/backend/README.md)

## 🎯 Complete Documentation

For a comprehensive understanding of how everything works together, read our detailed guide:

- **[Examples Documentation](examples/README.md)** - Complete architecture overview, data flow, and system design

## 📖 API Reference

### AudioManager

#### Constructor

```typescript
new AudioManager(config: AudioManagerConfig)
```

**Config Options:**

```typescript
interface AudioManagerConfig {
  apiKey: string;         // Required: Deepgram API key
  language?: string;      // Optional: Language code (default: 'en-US')
}
```

#### Methods

##### `initialize()`

Initialize the audio manager. Must be called before using STT/TTS.

```javascript
await audioManager.initialize();
```

##### `startListening(callback)`

Start speech-to-text transcription.

```javascript
await audioManager.startListening((result) => {
  console.log(result.text);        // Transcribed text
  console.log(result.isFinal);     // Is this a final result?
  console.log(result.confidence);  // Confidence score (0-1)
});
```

**Result Interface:**

```typescript
interface TranscriptionResult {
  text: string;          // Transcribed text
  isFinal: boolean;      // Final result or interim?
  confidence: number;    // Confidence score (0-1)
}
```

##### `stopListening()`

Stop speech-to-text transcription.

```javascript
await audioManager.stopListening();
```

##### `speak(text, options)`

Generate speech from text.

```javascript
const result = await audioManager.speak('Hello, world!', {
  model: 'aura-asteria-en'  // Optional: Voice model
});

console.log(result.audioBuffer);  // ArrayBuffer
console.log(result.duration);     // Duration in seconds
```

**Options:**

```typescript
interface TTSOptions {
  model?: string;  // Voice model (default: 'aura-asteria-en')
}
```

**Result Interface:**

```typescript
interface TTSResult {
  audioBuffer: ArrayBuffer;  // Audio data
  duration: number;          // Duration in seconds
}
```

### Available Voice Models

| Model | Gender | Language | Description |
|-------|--------|----------|-------------|
| `aura-asteria-en` | Female | English | Warm and friendly |
| `aura-luna-en` | Female | English | Clear and professional |
| `aura-stella-en` | Female | English | Expressive |
| `aura-athena-en` | Female | English | Authoritative |
| `aura-hera-en` | Female | English | Confident |
| `aura-orion-en` | Male | English | Deep and commanding |
| `aura-arcas-en` | Male | English | Neutral and clear |
| `aura-perseus-en` | Male | English | Energetic |
| `aura-angus-en` | Male | English | Smooth |
| `aura-orpheus-en` | Male | English | Rich and resonant |
| `aura-helios-en` | Male | English | Bright |
| `aura-zeus-en` | Male | English | Powerful |

## 🏗️ Architecture

```text
AudioManager (Core)
│
├── DeepgramSTTProvider
│   ├── Microphone Permission
│   ├── AudioContext
│   ├── WebSocket (Live Transcription)
│   └── Event Emitter
│
└── DeepgramTTSProvider
    ├── HTTP Client (REST API)
    ├── Audio Buffer Handler
    └── Duration Calculator
```

## 🔒 Security Considerations

### API Key Protection

**❌ Don't expose API keys in frontend:**

```javascript
// Bad - API key visible in browser
const audioManager = new AudioManager({
  apiKey: 'abc123...'  // Visible in source code!
});
```

**✅ Use backend proxy:**

1. Set up the [backend proxy server](examples/backend/)
2. Frontend calls your proxy
3. Proxy adds API key server-side
4. Proxy calls Deepgram API

### CORS Issues

- **STT (WebSocket)**: Works directly from browser
- **TTS (REST)**: Requires backend proxy due to CORS

See our [backend example](examples/backend/) for a complete proxy implementation.

## 🛠️ Development

### Build from Source

```bash
# Clone repository
git clone https://github.com/sathsarabandaraj/audio-handler.git
cd audio-handler

# Install dependencies
pnpm install

# Build package
pnpm run build

# Output: dist/index.js
```

### Project Structure

```text
audio-handler/
├── src/
│   ├── index.ts              # Main exports
│   ├── core/
│   │   ├── audio-manager.ts  # Core AudioManager class
│   │   ├── types.ts          # TypeScript interfaces
│   │   └── mic-permission.ts # Microphone permission handler
│   └── providers/
│       └── deepgram/
│           ├── index.ts      # Provider exports
│           ├── stt.ts        # STT implementation
│           ├── tts.ts        # TTS implementation
│           └── types.ts      # Provider types
├── examples/
│   ├── html-example/         # Vanilla JS example
│   ├── react-example/        # React example
│   └── backend/              # Express proxy server
├── dist/                     # Build output
├── package.json
├── tsconfig.json
└── rollup.config.js
```

## 🧪 Testing

### Manual Testing

Use the provided examples to test functionality:

```bash
# Test HTML example
cd examples/html-example
python3 -m http.server 8080

# Test React example
cd examples/react-example
pnpm run dev

# Test backend proxy
cd examples/backend
npm start
```

## 🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Commit your changes: `git commit -m 'Add amazing feature'`
4. Push to the branch: `git push origin feature/amazing-feature`
5. Open a Pull Request

## 📝 License

MIT License

Copyright (c) 2025 Sathsara Bandaraj

## 🙏 Acknowledgments

- **[Deepgram](https://deepgram.com/)** for their excellent AI-powered speech APIs
- **[Rollup](https://rollupjs.org/)** for module bundling
- **[TypeScript](https://www.typescriptlang.org/)** for type safety

## 🔗 Links

- [Deepgram API Documentation](https://developers.deepgram.com/)
- [Get Deepgram API Key](https://console.deepgram.com/signup)
- [Web Speech API Documentation](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API)

## 💬 Support

- 📧 Email: support@example.com
- 🐛 Issues: [GitHub Issues](https://github.com/sathsarabandaraj/audio-handler/issues)
- 💬 Discussions: [GitHub Discussions](https://github.com/sathsarabandaraj/audio-handler/discussions)

## 📊 Browser Compatibility

| Feature | Chrome | Firefox | Safari | Edge |
|---------|--------|---------|--------|------|
| STT (getUserMedia) | ✅ 53+ | ✅ 36+ | ✅ 11+ | ✅ 79+ |
| AudioContext | ✅ 35+ | ✅ 25+ | ✅ 14.1+ | ✅ 79+ |
| WebSocket | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
| Web Speech API | ✅ 33+ | ✅ 49+ | ✅ 14.1+ | ✅ 79+ |

**Recommendation:** Use the latest versions of Chrome, Firefox, Edge, or Safari for the best experience.

## 📈 Roadmap

- [ ] Additional provider support (Azure, AWS, Google)
- [ ] Language detection
- [ ] Custom vocabulary support
- [ ] Audio recording and export
- [ ] Real-time waveform visualization
- [ ] Voice activity detection
- [ ] Noise cancellation
- [ ] Multiple speaker detection

## 🎉 Getting Started

Ready to add voice capabilities to your app? Start with our examples:

1. **Quick Test**: Try the [HTML example](examples/html-example/) - no build required
2. **Production Ready**: Use the [React example](examples/react-example/) as a template
3. **Security**: Set up the [backend proxy](examples/backend/) for production

Have questions? Check out our [comprehensive documentation](examples/README.md)!

---

**Built with ❤️ by [Sathsara Bandaraj](https://github.com/sathsarabandaraj)**

**Star ⭐ this repo if you find it useful!**