npm package: @lautmaler/platform-audiocodes
Description: Voice.AI gateway integration
Author: lautmaler
Latest version: 1.4.7

## Jovo plugin library for AudioCodes VoiceAI Connect

This library allows you to build voice apps for the [AudioCodes VoiceAI Connect](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Combined/voiceai_connect.htm) platform with the [Jovo Framework](https://www.jovo.tech/).
The Audiocodes API documentation is available [here](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_API/API_1.htm)

### Usage

For context, a typical conversation flow is documented [here](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_API/API_1.htm).<br/>
As a matter of implementation, Audiocodes-side requests are handled by AudiocodesRequest.ts which maps them to [JovoInput](https://www.jovo.tech/docs/input) objects. [Handlers](https://www.jovo.tech/docs/handlers) are responsible for processing the input and generating [Output](https://www.jovo.tech/docs/output) objects. This library provides implementations for several Output types in `src/output`.</br>
Finally, `AudiocodesPlatformOutputTemplateConverterStrategy` maps the output to an `AudiocodesResponse` and sent back to the Audiocodes platform as JSON.

Your **Jovo App needs to use a DBPlugin** to store Jovos dialogue state and session data in between requests. The Audiocodes protocol does not allow to send this data within the requests and responses as Jovo would do normally.

#### Supported Output types

`src/output` contains all supported Audiocodes output implementations:

- `MessageActivityOutput`: a `message` type output, used to send a voice prompt to the user.
- `SessionConfigOutput`: configures bot delay options [doc](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/handle-bot-delay.htm?TocPath=Bot%2520integration%257CBasic%2520behavior%257C_____5)
- `EndConversationOutput`: end call [doc](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Combined/hangup.htm)
- `CallTransferOutput`: transfer call to other number [doc](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Integration/call-transfer.htm)
- `AbortPromptOutput`: abort currently playing prompts [doc](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/aborting-prompts.htm?TocPath=Bot%2520integration%257CControlling%2520the%2520call%257C_____8)
- `PlayUrlActivityOutput`: play audio file to caller [doc](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Combined/play-url.htm?Highlight=wav)

#### Using Output types

An intent handler that return a bot response would look like this:

```typescript
import { BaseComponent } from '@jovotech/framework';

class Component extends BaseComponent {
  @Intents('HelpIntent')
  helpHandler() {
    return this.$send(MessageActivityOutput, {
      message: '<speak> I can demo you some features.</speak>',
    });
  }
}
```

The`message` attribute of `MessageActivityOutput` config is a (SSML enabled) string which is sent to the user as a voice prompt.

Below is a more complex example where the call is transferred to a human operator. There are two activities sent to the user: a voice prompt using `MessageActivityOutput` and a call transfer request using `CallTransferOutput`. The latter has extra `activityParams`

```typescript
class Component extends BaseComponent {
  @Intents(['HumanOperatorIntent'])
  humanOperator() {
    this.$send(MessageActivityOutput, {
      message: 'Ok. I will transfer you to a human.',
    });
    this.$send(CallTransferOutput, {
      activityParams: {
        transferTarget: 'sip:some-sip-target-number',
        handoverReason: 'User requests to talk to a human',
      },
    });
    return this.$redirect(CallTransferComponent);
  }
}
```

#### Session Parameter Plugin

The `AudiocodesPlatform` automatically use the `AudiocodesSessionPlugin`.  
The `AudiocodesSessionPlugin` ensures that a sensible set of session parameters gets set upon starting a call (notions _session parameters_ and _call settings_ are used interchangeably here).
It also provides the `$updateCallSettings` method on the jovo object that lets you update session parameters explicitly.

At the start of the call the plugin will send a set of session parameters to the Audicodes backend.  
The Platform config expects you to hand in a subset of mandatory session parameters under `sessionConfig` (they have no plausible defaults, so you have to decide). You can also can set non mandatory session parameters under the `sessionConfig` property.

```typescript
const app = new App({
  plugins: [
    new AudiocodesPlatform({
      fallbackLocale: 'de-DE',
      sessionConfig: {
        language: 'de-DE',
        voiceName: 'de-DE-elkeNeural',
        alternativeLanguages: [
          { language: 'en-US', voiceName: 'en-US-GuyNeural' },
          { language: 'de-DE', voiceName: 'de-DE-KatjaNeural' },
        ],
      },
    }),
  ],
});
```

The given `sessionConfig` overwrites values of the default config, or adds them if not present.  
The default config defines the following parameters

```json
{
  "botNoInputSpeech": "Einen Moment Bitte.",
  "userNoInputTimeoutMS": 8000,
  "userNoInputRetries": 3,
  "userNoInputGiveUpTimeoutMS": 60000,
  "enableSpeechInput": true,
  "bargeIn": true,
  "languageDetectionActivate": true,
  "botNoInputRetries": 2,
  "sendEventsToBot": [
    "VoiceAiNotificationEvent.DIALOUT_INITIATED",
    "VoiceAiNotificationEvent.NO_USER_INPUT",
    "VoiceAiNotificationEvent.CONVERSATION_END"
  ],
  "notifyParamChange": ["language"],
  "sendDTMF": true,
  "sendDTMFAsText": false,
  "bargeInOnDTMF": true,
  "dtmfCollect": true,
  "dtmfCollectSubmitDigit": "#",
  "dtmfCollectInterDigitTimeoutMS": 2000,
  "dtmfCollectMaxDigits": 0
}
```

The available parameters will be described in the following.

Please only use the `this.updateCallSettings(callSettings)` method to update session parameters. The underlying mechanism merges the current session parameters with the update and

1. syncs it with the data in `this.$session` at the end of the jovo request cycle
2. adds a session config output as the first element in `jovo.$output` array. This way the new parameters are already in action when the rest of the bots outputs reach the audiocodes backend.

And an example of updating `sessionParams` in a handler:

```typescript
class Component extends BaseComponent {
  @Handle()
  welcome() {
    return this.$updateCallSettings({
      language: 'de-DE',
      voiceName: 'new-voice',
    });
  }
}
```

#### Configurable session and activity parameters

Certain session and activity parameters are configurable by controlling attribute values of Output classes. (see Output Types section above for a list of supported output types).
A configuration change can be done for the entire call duration via the `this.updateCallSettings(callSettings)` method (within one of your handlers) that manipulates `this.$session.sessionParams` and takes care of updating the Audiocodes backend.  
You can also change the configuration just for the next turn of the dialogue by setting `activityParams` of the output classes your are using in the previous turn for the bot response.

**activityParams and sessionParams**<br/>
`CallSettingsParams` is the data holder for such common attributes and it can be used to set `activityParams` and/or `sessionParams`. Here is the exhaustive list of these attributes<br/>

- `language` and `voiceName` (depending on the speech provider) can be set for each message individually. See [API doc](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/speech_customization.htm?TocPath=Bot%2520integration%257CBasic%2520behavior%257C_____4).
- `sttSpeechContexts` providing hints for improving the accuracy of speech recognitions. See [API doc](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/speech_customization.htm?Highlight=sttSpeechContexts).<br/>
- `sendEventsToBot` if specified, sends notification events to the bot. See [API doc](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/controlling_events_to_bot.htm?TocPath=Bot%2520integration%257CReceiving%2520notifications%257C_____5). By default no events are sent to the bot
- `enableSpeechInput` Enables the activation of the speech-to-text service after the bot message playback is finished. Defaults to true
- `bargeIn` allow user to interrupt when the bot plays a message to the user. default is true
- `languageDetectionActivate`: Activates language detection. Defaults to true.
- `alternativeLanguages`: Defines a list of up to three alternative languages (in addition to the current language) that will be used to detect the language spoken. By default English and German are configured.

DTMF attributes. If `sendDTMF` is not set, the defaults below are used. Several attributes have defaults in place so they can be omitted if that default value is desired.

- `sendDTMF`: enables collecting DTMF. default is true.
- `sendDTMFAsText` boolean default is false
- `bargeInOnDTMF` boolean default is true
- `dtmfCollect` if false, each DTMF digit is sent on a separate event to the bot. default is true
- `dtmfCollectInterDigitTimeoutMS`: how long to wait before sending all digits to the bot. Default is 2000
- `dtmfCollectMaxDigits` max number of digits collected. Default is 0 (no limit)
- `dtmfCollectSubmitDigit` Defines a special DTMF "submit" digit that when received from the user, VoiceAI sends all the collected digits to the bot. default is #.To disable set to empty string.

Bot delay attributes

- `botNoInputTimeoutMS`: timeout before a prompt is played to the user in case the bot takes more time to process the input . Default is not set.
- `botNoInputSpeech`: textual prompt to play to the user when no input is received from the bot.
- `botNoInputRetries`: maximum number of recurring timeouts for bot response. Default is not set.
- `botNoInputGiveUpTimeoutMS`: timeout response before the call is disconnected. Default is not set.
- `botNoInputUrl`: Audio file to play when the bot does not respond.
- `resumeRecognitionTimeoutMS`: When Barge-In is disabled, speech input is not expected before the bot's response has finished playback. If no reply from the bot arrives within this configured timeout (in milliseconds), VoiceAI Connect expects speech input from the user and speech-to-text recognition is re-activated.

User delay [attributes](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/speech_customization.htm#nouserinputconfig)
`userNoInputTimeoutMS`: Maximum time in ms to wait for user input. Defaults to 2000
`userNoInputGiveUpTimeoutMS`: Maximum time in ms to wait for user input after which the call is terminated. Defaults to 15000
`userNoInputRetries`: Maximum number of retries to wait for user input. Defaults to 1 or two attempts.

Play URL [attributes](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/play-url.htm?TocPath=Bot%2520integration%257CControlling%2520the%2520call%257C_____6)
Attributes configurable via `PlayUrlActivityOutput`:

- `playUrlUrl`: URL of where the audio file is located.
- `playUrlMediaFormat`: format of the pre-recorded audio file.
- `playUrlAltText`: text to display in the transcript page of the user interface while the pre-recorded audio is played.
- `playUrlFailureHandling`: behaviour in case of failing to play audio: `disconnect`, `notofy-bot` and `ignore` are supported. Use `InputType.PLAY_URL_FAILED` to handle the event.
- `playUrlCaching` boolean. If true, the audio file is cached on the VoiceAI Connect server. Defaults to true.

Example setting of `activityParams` when sending a message activity:

```typescript
class Component extends BaseComponent {
  @Intents(['HumanOperatorIntent'])
  humanOperator() {
    this.$send(MessageActivityOutput, {
      message: 'Frag wie das Wetter ist.',
      activityParams: {
        sttSpeechContexts: [
          {
            phrases: ['Wetter'],
            boost: 20,
          },
          {
            phrases: ['$ADDRESSNUM'],
          },
        ],
        sendEventsToBot: ['noUserInput'],
      },
    });
  }
}
```

Some activities allow for setting activityParams specific only to them. These are:

- `handoverReason` for call transfers. [API doc](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Integration/call-transfer.htm?Highlight=handoverReason):<br/>
- `hangupReason` when ending a conversation. [API doc](https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Combined/hangup.htm?Highlight=hangupReason):<br/>
  There are ActivityParam structures defined for each such activity. They all inherit from `CallSettingsParams` which allows (re)setting all other common attributes as well.<br/>

`handoverReason` example:

```typescript
this.$send(CallTransferOutput, {
  transferTarget: '<sip:target>',
  transferNotifications: true,
  handoverReason: 'bot call handover',
  transferSipHeaders: [],
});
```

`hangupReason` example:

```typescript
class Component extends BaseComponent {
  @Intents('EndConversationIntent')
  endConversation() {
    return this.$send(EndConversationOutput, {
      hangupReason: 'call successful',
      hangupSipHeaders: [],
    });
  }
}
```

##### Configuring session params

All sessionParams like [Bot delay](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/handle-bot-delay.htm?TocPath=Bot%2520integration%257CBasic%2520behavior%257C_____5) and [DTMF](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/receive-dtmf.htm?Highlight=dtmfCollectSubmitDigit) should be configured using the `SessionConfigOutput.sessionParams` interface.<br/>. The config can be set once and it applies for the entire session.<br/>
Example:

```typescript
class Component extends BaseComponent {
  @Handle()
  welcome() {
    this.updateCallSettings({
      botNoInputTimeoutMS: 1000, // timeout before a prompt is played to the user.
      botNoInputSpeech: 'Please wait for bot input.', // textual prompt to play to the user when no input is received from the bot.
      botNoInputGiveUpTimeoutMS: 10000, // timeout response before the call is disconnected.
      botNoInputRetries: 2, // maximum number of recurring timeouts for bot response.
      botNoInputUrl: 'https://audiofile-to-play-when-bot-does-not-respond.url',
      resumeRecognitionTimeoutMS: 1000,
    });

    return this.$send(MessageActivityOutput, {
      message: 'What do you want to try?',
    });
  }
}
```

- `sendEventsToBot` session-wide configuration for sending notification events to the bot. Similar to the eventParams attribute.<br/>

### Handling inputs

The Jovo Audiocodes platform maps the different incoming events and notifications from the Audiocodes backend to an `InputType`. This way you can contrain handlers to trigger on certain event by using `InputType`.
Events from the Audiocodes backend can have additional payloads with reside in the `parameters` or the `value` property. These properties are mapped into the `this.$input` object directly and because of that have the same structure as documented in the Audiocodes documentation.
This means you can infer from the `InputType` which property should be populated and what structure resides under `parameters` or `value`.

#### DTMF input

When configured, the bot can respond to user's DTMF input. The `sessionParams.sendDTMF` boolean flag (true by default) is responsible for enabling/disabling DTMF collection. The session is configured by default to collect all digits the user inputs and if an input pauses for more than 2sec, the collected digits are sent to the bot.

To handle user's DTMF input a Jovo component has to handle the `InputType.DTMF_EVENT` event:

```typescript
class Component extends BaseComponent {
  @Handle({ types: [InputType.DTMF_EVENT] })
  handleDTMF() {
    const userDtmf = this.jovo.$input.value; //dtmf input can be configured to deliver string or number
    if (userDtmf === '9') {
      this.$send(MessageActivityOutput, {
        message: '<speak> Sorry to see you leave.</speak>',
      });
      return this.endConversation();
    } else if (userDtmf === '11') {
      this.$send(MessageActivityOutput, {
        message: '<speak> You typed 11. Win!</speak>',
      });
    }
  }
}
```

#### Handling Audiocodes notifications

Audiocodes will send notifications when configured using the `sendEventsToBot` session or activity param.
These events are mapped by the `BotNotificationEvent` enum to Jovo InputTypes:

- `InputType.LAUNCH` maps `start` event
- `InputType.TEXT` maps `message` event
- `InputType.DIALOUT_INITIATED` maps `dialoutInitiated` event
- `InputType.NO_USER_INPUT` maps `noUserInput` event
- `InputType.SPEECH_HYPOTHESIS` maps `speechHypothesis` event
- `InputType.CONVERSATION_END` maps `conversationEnd` event
- `InputType.CALL_TRANSFER_FAILED` maps `transferStatus` event with status value `failed`
- `InputType.CALL_TRANSFER_ANSWERED` maps `transferStatus` event with status value `answered`
- `InputType.PLAY_URL_FAILED` maps `playUrlFailed` event

`notifyParamChange.notifyParamChange` configures whether to receive notifications when a call param changed. This attribute takes an array of parameters for which VoiceAI Connect notifies the bot whenever their values change. [Doc](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/notifying_bot_parameter_changes.htm?TocPath=Bot%2520integration%257CReceiving%2520notifications%257C_____6)

Usage example:

```typescript
class Component extends BaseComponent {
  @Handle({
    types: [InputType.NO_USER_INPUT],
  })
  handleBotNotificationEvent() {
    return this.$send(MessageActivityOutput, { message: 'Please state your request.' });
  }
}
```

Check [documentation](https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/controlling_events_to_bot.htm) for a detailed description for each notification.

#### Configuration

AudiocodesPlatform can be configured with the following attributes:

- `fallbackLanguage`: the language to use when the requested language is not supported by the Audiocodes platform. Defaults to empty string.
- `fallbackTransferPhoneNumber`: the fallback phone number to transfer the call if the intended transfer phone number fails. Needs to be preconfigured in the Audiocodes service. Defaults to empty string.
- `supportedLanguages`: a list of locale+voiceName pairs preconfigured in the Audiocodes account. Defaults to empty list.
- `sessionParams`: default session params to use for the entire session. The defaults are:

```
sessionParams: {
    botNoInputTimeoutMS: 6000,
    botNoInputGiveUpTimeoutMS: 30000,
    botNoInputRetries: 2,
    sendEventsToBot: [
        VoiceAiNotificationEvent.DIALOUT_INITIATED,
        VoiceAiNotificationEvent.NO_USER_INPUT,
        VoiceAiNotificationEvent.CONVERSATION_END,
        VoiceAiNotificationEvent.SPEECH_HYPOTHESIS,
    ],
    userNoInputGiveUpTimeoutMS: 60000,
    notifyParamChange: ['language'],
}
```

When instantiating the AudiocodesPlatform, the above defaults can be overridden by passing a `sessionParams` object to the constructor.

Example where the empty defaults are set and sessionParams.sendEventsToBot is overridden to an empty array (no events are sent to the bot):

```typescript
app.configure({
  plugins: [
    // Add Jovo plugins here
    new AudiocodesPlatform({
      fallbackLocale: 'en-US',
      transferPhoneNumber: 'sip:+12345@sip-provider',
      supportedLanguages: [
        { locale: 'en-US', voiceName: 'en-US-GuyNeural' },
        { locale: 'de-DE', voiceName: 'de-DE-KatjaNeural' },
      ],
      sessionParams: {
        sendEventsToBot: [],
      },
      plugins: [...somePlugins],
    }),
  ],
});
```

#### Language handling

The conversation language can be set explicitly by passing a locale in the `language` attribute of an activity. The locale format is `<language code>-<country code>`, eg `en-US` or `de-DE`.
If the desired language is not supported by the Audiocodes configuration, the AudiocodesPlatform config attribute `fallbackLanguage` is used.
In order to set a desired language for an entire user session, set the locale in the `AudiocodesResponse.sessionParams.language` attribute.

#### Experimental features

Audiocodes requests handled by the Jovo service are mapped by `AudiocodesEventActivityRequest`. Audiocodes sends context-specific attributes with the request which are mapped by `AudiocodesEventActivityRequest.parameters`. Extra attributes can be send as well, depending on the admin configuration of Audiocodes.<br/>

For demonstration purposes, an additional request attribute called `EventRequestParameters.sipUserToUser`is also implemented. This maps the `User-to-User` SIP header which is sent by the Audiocodes platform during call initiation.