npm package: @lautmaler/jovo-platform-aiflow
Description: Sipgate AI Flow integration for Jovo Framework
Author: lautmaler
Latest version: 0.2.8

# Jovo Platform Plugin for Sipgate AI Flow

This plugin allows you to build voice applications for the [Sipgate AI Flow](https://www.sipgate.de/) platform with the [Jovo Framework](https://www.jovo.tech/).

## Installation

```bash
npm install @lautmaler/jovo-platform-aiflow
```

## Quick Start

### Basic Setup

```typescript
import { App } from '@jovotech/framework';
import { SipgateAiflowPlatform } from '@lautmaler/jovo-platform-aiflow';

const app = new App({
  plugins: [
    new SipgateAiflowPlatform({
      apiKey: process.env.AIFLOW_API_KEY, // Optional: Sipgate API key
      debug: true,
    }),
  ],
});
```

The platform integrates with Jovo's standard server adapters to handle HTTP POST requests from Sipgate AI Flow.

## Event Types

The platform handles all Sipgate AI Flow event types and maps them to Jovo's input system:

| Event Type | Description | Jovo InputType |
|------------|-------------|----------------|
| `session_start` | Call session begins | `InputType.SESSION_START` |
| `user_speak` | User speech detected | `InputType.USER_SPEAK` |
| `assistant_speak` | Assistant finished speaking | `InputType.ASSISTANT_SPEAK` |
| `user_barge_in` | User interrupted assistant | `InputType.USER_BARGE_IN` |
| `user_speak` (with `barged_in: true`) | User spoke over assistant | `InputType.USER_BARGE_IN` |
| `session_end` | Call session ends | `InputType.SESSION_END` |

**Note:** Barge-in events can arrive either as a dedicated `user_barge_in` event or as a regular `user_speak` event with `barged_in: true`. Both are mapped to `InputType.USER_BARGE_IN` automatically, so your handler covers both cases.

### Handling Events

```typescript
import { BaseComponent, Component, Handle } from '@jovotech/framework';
import { InputType } from '@lautmaler/jovo-platform-aiflow';

@Component()
export class GlobalComponent extends BaseComponent {

  @Handle({
    types: [InputType.SESSION_START],
  })
  sessionStart() {
    console.log('New call session started');
    // Initialize session data
  }

  @Handle({
    types: [InputType.SESSION_END],
  })
  sessionEnd() {
    console.log('Call session ended');
    // Cleanup
    this.$session.state = [];
    return this.$send('{}');
  }

  @Handle({
    types: [InputType.USER_BARGE_IN],
  })
  userBargeIn() {
    console.log('User interrupted during playback');
    // Handle interruption — see "Barge-In Handling" section below
  }
}
```

## Output Types

All output types require a `sessionId` parameter. You can access the session ID from `this.$session.sessionInfo.id`:

### SpeakOutput

Send text-to-speech output to the caller.

```typescript
import { SpeakOutput } from '@lautmaler/jovo-platform-aiflow';

@Intents('HelpIntent')
helpHandler() {
  const sessionId = (this.$session as any).sessionInfo?.id;

  return this.$send(SpeakOutput, {
    message: 'How can I help you today?',
    sessionId,
    // Optional parameters:
    ssml: '<speak>How can I <emphasis>help</emphasis> you?</speak>',
    provider: {
      provider: 'azure',
      language: 'en-US',
      voice: 'en-US-JennyNeural',
    },
    bargeIn: {
      strategy: 'minimum_characters',
      minimum_characters: 3,
    },
  });
}
```

**Options:**
- `message` (string | string[]): Text to speak
- `sessionId` (string, required): Session identifier
- `ssml` (string, optional): SSML markup (overrides message)
- `provider` (TtsProviderConfig, optional): TTS provider configuration
- `bargeIn` (BargeInConfig, optional): Barge-in behavior

### HangupOutput

End the call.

```typescript
import { HangupOutput } from '@lautmaler/jovo-platform-aiflow';

@Intents('EndConversationIntent')
async endConversation() {
  const sessionId = (this.$session as any).sessionInfo?.id;

  await this.$send(SpeakOutput, {
    message: 'Goodbye!',
    sessionId,
  });

  return this.$send(HangupOutput, { sessionId });
}
```

**Options:**
- `sessionId` (string, required): Session identifier

### TransferOutput

Transfer the call to another number.

```typescript
import { TransferOutput } from '@lautmaler/jovo-platform-aiflow';

@Intents('HumanOperatorIntent')
async transferToOperator() {
  const sessionId = (this.$session as any).sessionInfo?.id;

  await this.$send(SpeakOutput, {
    message: 'Transferring you to an operator.',
    sessionId,
  });

  return this.$send(TransferOutput, {
    sessionId,
    targetPhoneNumber: '+491234567890',
    callerIdName: 'Support',
    callerIdNumber: '+490987654321',
  });
}
```

**Options:**
- `sessionId` (string, required): Session identifier
- `targetPhoneNumber` (string, required): Phone number to transfer to
- `callerIdName` (string, required): Caller ID name
- `callerIdNumber` (string, required): Caller ID number

### AudioOutput

Play a base64-encoded WAV audio file.

```typescript
import { AudioOutput } from '@lautmaler/jovo-platform-aiflow';

@Intents('PlayMusicIntent')
playMusic() {
  const sessionId = (this.$session as any).sessionInfo?.id;
  const audioBase64 = '...'; // Base64 encoded WAV

  return this.$send(AudioOutput, {
    audio: audioBase64,
    sessionId,
    bargeIn: {
      strategy: 'minimum_characters',
      minimum_characters: 5,
    },
  });
}
```

**Options:**
- `audio` (string, required): Base64-encoded WAV audio
- `sessionId` (string, required): Session identifier
- `bargeIn` (BargeInConfig, optional): Barge-in behavior

### DtmfOutput

Send DTMF tones (0-9).

```typescript
import { DtmfOutput } from '@lautmaler/jovo-platform-aiflow';

sendDtmf() {
  const sessionId = (this.$session as any).sessionInfo?.id;

  return this.$send(DtmfOutput, {
    sessionId,
    dtmf: 9, // Single digit 0-9
  });
}
```

**Options:**
- `sessionId` (string, required): Session identifier
- `dtmf` (number, required): DTMF digit (0-9)

### BargeInOutput

Acknowledge a barge-in event.

```typescript
import { BargeInOutput } from '@lautmaler/jovo-platform-aiflow';

@Handle({
  types: [InputType.USER_BARGE_IN],
})
handleBargeIn() {
  const sessionId = (this.$session as any).sessionInfo?.id;

  return this.$send(BargeInOutput, { sessionId });
}
```

**Options:**
- `sessionId` (string, required): Session identifier

## Barge-In Handling

Barge-in occurs when a caller interrupts the assistant while it's speaking. The platform provides tools for both configuring when barge-in is allowed and handling the interruption intelligently.

### Barge-In Strategies

Configure how sensitive barge-in detection should be via `BargeInConfig`:

| Strategy | Latency | Reliability | Use Case |
|----------|---------|-------------|----------|
| `immediate` | 20-100ms | May trigger on noise | Natural conversations |
| `minimum_characters` | 50-200ms | Very reliable | Balanced approach (recommended) |
| `manual` | N/A | Perfect | Custom logic |
| `none` | N/A | Perfect | Critical info only (legal disclaimers, confirmation codes) |

```typescript
import { BargeInStrategy } from '@lautmaler/jovo-platform-aiflow';

// Most natural — triggers on voice detection
{ strategy: BargeInStrategy.IMMEDIATE, allow_after_ms: 500 }

// Balanced — waits for a few characters
{ strategy: BargeInStrategy.MINIMUM_CHARACTERS, minimum_characters: 3 }

// For critical information — higher threshold
{ strategy: BargeInStrategy.MINIMUM_CHARACTERS, minimum_characters: 10, allow_after_ms: 3000 }

// No interruption allowed
{ strategy: BargeInStrategy.NONE }
```

The `allow_after_ms` option prevents accidental triggers during the first milliseconds of playback.

### Classifying Barge-In Intent

Use `BargeInHelper` to classify what the user meant by their interruption:

```typescript
import { BargeInHelper, BargeInIntent } from '@lautmaler/jovo-platform-aiflow';

@Handle({ types: [InputType.USER_BARGE_IN] })
handleBargeIn() {
  const text = this.$request.getInputText();
  const intent = BargeInHelper.classify(text);

  switch (intent) {
    case BargeInIntent.NOISE:
    case BargeInIntent.STOP:
      // Short noise or "stop"/"quiet" — return empty (204)
      return this.$send(EmptyOutput);

    case BargeInIntent.ACKNOWLEDGMENT:
      // "Got it", "okay", "yes" — brief follow-up
      return this.$send(SpeakOutput, { message: 'Great! What else can I help with?' });

    case BargeInIntent.REDIRECT:
      // "Actually", "wait", "hold on" — let user continue
      return this.$send(SpeakOutput, { message: 'Sure, go ahead.' });

    case BargeInIntent.CORRECTION:
    case BargeInIntent.NEW_QUESTION:
      // Substantial input (>25 chars or contains "?") — process as new input
      // Re-route to your normal speech handling logic
      return this.handleUserInput(text);
  }
}
```

Convenience methods:
- `BargeInHelper.shouldSilentAcknowledge(text)` — returns `true` for noise/stop (return 204)
- `BargeInHelper.isSubstantialInput(text)` — returns `true` for new questions/corrections (process normally)

### Tracking Impatient Users

The session automatically tracks how many times a user has barged in via `bargeInCount`. Use this to adapt response brevity:

```typescript
@Handle({ types: [InputType.USER_BARGE_IN] })
handleBargeIn() {
  const session = this.$session as SipgateAiflowSession;
  const text = this.$request.getInputText();

  // Frequent interrupters get shorter responses
  if (session.bargeInCount > 3) {
    if (BargeInHelper.shouldSilentAcknowledge(text)) {
      return this.$send(EmptyOutput);
    }
    return this.$send(SpeakOutput, { message: 'Yes?' });
  }

  // Normal barge-in handling for patient users
  // ...
}
```

### Checking barged_in on Input

The `barged_in` flag is available on `AiflowInput` for any handler that needs to know if the current input was an interruption:

```typescript
const input = this.$input as AiflowInput;
if (input.barged_in) {
  // This input interrupted the assistant
}
```

### Cancelling Async Operations

When using async mode (`SipgateAiflowAsync`), cancel pending operations on barge-in so the user isn't waiting for stale responses:

```typescript
@Handle({ types: [InputType.USER_BARGE_IN] })
handleBargeIn() {
  if (this.jovo instanceof SipgateAiflowAsync) {
    this.jovo.cancelPendingOperations();
  }
  // Handle the interruption...
}
```

Use `this.jovo.abortSignal` in your own async operations to respect cancellation:

```typescript
const response = await fetch(url, { signal: this.jovo.abortSignal });
```

## Session Information

Session information from Sipgate AI Flow is available via `this.$session.sessionInfo`:

```typescript
interface SessionInfo {
  id: string;           // UUID of the session
  account_id: string;   // Account identifier
  phone_number: string; // Caller's phone number
}
```

Example usage:

```typescript
LAUNCH() {
  const sessionInfo = (this.$session as any).sessionInfo;

  if (sessionInfo) {
    console.log(`Call from: ${sessionInfo.phone_number}`);
    console.log(`Session ID: ${sessionInfo.id}`);
  }

  return this.$redirect(MainComponent);
}
```

## Platform Configuration

```typescript
interface SipgateAiflowConfig {
  apiKey?: string;              // Sipgate API key (optional)
  debug?: boolean;              // Enable debug logging
  tts?: TtsConfig;              // Default TTS provider settings
  bargeIn?: BargeInConfig;      // Default barge-in configuration
  userInputTimeoutSec: number;  // Timeout for user input (default: 4)
  asyncOutput?: boolean;        // Enable/disable async output (auto-detects by default)
  plugins?: Plugin[];           // Additional platform-specific plugins
}
```

### Default Configuration

The platform provides sensible defaults:

```typescript
{
  provider: {
    provider: 'azure',
    language: 'en-US',
    voice: 'en-US-JennyNeural',
  },
  bargeIn: {
    strategy: 'minimum_characters',
    minimum_characters: 3,
  },
  debug: false,
}
```

**Note:** These default configurations are defined but not currently applied automatically to outputs. You must explicitly pass `provider` and `bargeIn` options to each output that supports them.

### Custom Configuration

```typescript
new SipgateAiflowPlatform({
  apiKey: process.env.AIFLOW_API_KEY,
  debug: true,
  provider: {
    provider: 'azure',
    language: 'de-DE',
    voice: 'de-DE-KatjaNeural',
  },
  bargeIn: {
    strategy: 'minimum_characters',
    minimum_characters: 3
  },
})
```

## Async Output (WebSocket)

When using a WebSocket connection, the platform supports **async output mode** where each `$send()` call delivers the response immediately over the WebSocket, instead of batching all outputs into a single HTTP response.

### How It Works

- In **sync mode** (default for HTTP): All `$send()` calls are collected and returned as a single response at the end of the handler.
- In **async mode** (default for WebSocket): Each `$send()` call immediately sends its output over the WebSocket connection. The final HTTP response is suppressed to avoid duplicate delivery.

This is useful for scenarios like sending a speak action followed by a hangup, where each action should be delivered to Sipgate AI Flow as soon as it's ready.

### Configuration

Async output is controlled via the `asyncOutput` config option:

```typescript
new SipgateAiflowPlatform({
  asyncOutput: true,  // Force enable async output
  // asyncOutput: false, // Force disable async output
  // asyncOutput: undefined, // Auto-detect (default)
})
```

| Value | Behavior |
|-------|----------|
| `undefined` (default) | Auto-detect: enabled when using `WebSocketServer`, disabled for HTTP servers |
| `true` | Force enable async output regardless of server type |
| `false` | Force disable async output, even when using WebSocket |

### Example: Multiple Outputs with WebSocket

```typescript
@Intents('EndConversationIntent')
async endConversation() {
  const sessionId = (this.$session as any).sessionInfo?.id;

  // With async mode, this is sent immediately over WebSocket
  await this.$send(SpeakOutput, {
    message: 'Goodbye!',
    sessionId,
  });

  // This is also sent immediately, right after the speak
  return this.$send(HangupOutput, { sessionId });
}
```

### WebSocket Server Setup

To use async output with the built-in `WebSocketServer`:

```typescript
import { WebSocketServer } from '@lautmaler/jovo-platform-aiflow';
import WebSocket from 'ws';

const wss = new WebSocket.Server({ port: 3000 });

wss.on('connection', (ws) => {
  ws.on('message', async (data) => {
    await app.handle(new WebSocketServer(ws, data));
  });
});
```

When `WebSocketServer` is used, async output is enabled automatically — no additional configuration needed.

## Server Integration

The platform works with Jovo's standard Express.js server adapter:

### Express.js with Jovo Webhook

```typescript
import { Webhook } from '@jovotech/server-express';
import { app } from './app'; // Your Jovo app

const port = process.env.JOVO_PORT || 3000;

(async () => {
  await app.initialize();

  Webhook.listen(port, () => {
    console.log(`Server listening on port ${port}`);
  });

  Webhook.post('/webhook', async (req, res) => {
    await app.handle(new ExpressJs(req, res));
  });
})();
```

### Custom Express Server

If you need a custom Express server adapter, extend Jovo's `Server` class:

```typescript
import { AnyObject, Server } from '@jovotech/framework';
import type { Request, Response } from 'express';

export class ExpressJsExt extends Server {
  constructor(public req: Request, public res: Response) {
    super();
  }

  fail(error: Error): void {
    if (!this.res.headersSent) {
      this.res.status(500).json({
        code: 500,
        msg: error.message,
      });
    }
  }

  getRequestObject(): AnyObject {
    return this.req.body;
  }

  setResponse(response: unknown): Promise<void> {
    return new Promise<void>((resolve) => {
      if (!this.res.headersSent) {
        this.res.json(response);
      }
      resolve();
    });
  }
}
```

## Complete Example

```typescript
import { App } from '@jovotech/framework';
import {
  SipgateAiflowPlatform,
  SpeakOutput,
  HangupOutput,
  InputType,
} from '@lautmaler/jovo-platform-aiflow';
import { BaseComponent, Component, Intents, Handle } from '@jovotech/framework';

// Define your component
@Component()
class MyComponent extends BaseComponent {

  LAUNCH() {
    const sessionInfo = (this.$session as any).sessionInfo;

    return this.$send(SpeakOutput, {
      message: 'Welcome to my voice app!',
      sessionId: sessionInfo?.id,
    });
  }

  @Intents('HelpIntent')
  helpHandler() {
    const sessionId = (this.$session as any).sessionInfo?.id;

    return this.$send(SpeakOutput, {
      message: 'I can help you with various tasks.',
      sessionId,
    });
  }

  @Intents('EndConversationIntent')
  async endConversation() {
    const sessionId = (this.$session as any).sessionInfo?.id;

    await this.$send(SpeakOutput, {
      message: 'Goodbye!',
      sessionId,
    });

    return this.$send(HangupOutput, { sessionId });
  }

  @Handle({
    types: [InputType.SESSION_END],
  })
  sessionEnd() {
    this.$session.state = [];
    return this.$send('{}');
  }
}

// Create your app
const app = new App({
  components: [MyComponent],
  plugins: [
    new SipgateAiflowPlatform({
      apiKey: process.env.AIFLOW_API_KEY,
      debug: true,
    }),
  ],
});

export { app };
```

## Testing

The platform supports Jovo's TestSuite for unit testing:

```typescript
import { TestSuite } from '@jovotech/framework';
import { InputType } from '@lautmaler/jovo-platform-aiflow';

const testSuite = new TestSuite({
  stage: 'dev',
  locale: 'en-US',
});

beforeEach(() => {
  testSuite.$session.sessionInfo = {
    id: 'test-session-id',
    account_id: 'test-account',
    phone_number: '+1234567890',
  };
});

test('should handle help intent', async () => {
  await testSuite.run({
    type: InputType.Launch,
  });

  const { output } = await testSuite.run({
    intent: 'HelpIntent',
  });

  expect(output.length).toBe(1);
  expect(output[0]?.platforms?.['sipgate-aiflow']?.text).toBeDefined();
});
```

## Sample Requests

Example event payloads can be found in the `sample-requests/` directory:

- `SessionStart.json` - Call session begins
- `UserSpeak.json` - User speech detected
- `AssistantSpeak.json` - Assistant finished speaking
- `UserBargeIn.json` - User interrupted assistant (dedicated event)
- `UserSpeakBargedIn.json` - User speech with `barged_in` flag
- `SessionEnd.json` - Call session ends

## Links

- [Jovo Framework Documentation](https://www.jovo.tech/docs)
- [Sipgate AI Flow SDK](https://www.npmjs.com/package/@sipgate/ai-flow-sdk)
- [Sipgate AI Flow Documentation](https://www.sipgate.de/)

## License

See LICENSE file