Migrating Twilio Flex Webchat from Autopilot to AWS Lex
Tags: twilio, chatbot, lex, autopilot
Published: 2022-12-14 02:34:54

Twilio Autopilot is a Natural Language Understanding Service (NLU), which I suspect, given Twilio's existing relationship with IBM, is just a façade over the [Watson Digital Assistant](https://www.ibm.com/products/watson-assistant). As part of a customer support platform we've integrated the default Flex Webchat Studio Flow with Autopilot to provide a simple chatbot for deflecting customer requests to our knowledge base articles so they can self-service and providing pre-engagement (e.g. selecting a issue to follow-up) before handing off to a customer support representative. This has all worked very well, up until we were forced to find a replacement for Autopilot.

In September Twilio announced the [end of life](https://support.twilio.com/hc/en-us/articles/8279929323547-Notice-Twilio-Autopilot-End-of-Life-August-2023-) of their [Autopilot conversational AI Platform](https://www.twilio.com/docs/autopilot):

> Twilio's Autopilot product will officially shut down on August 25, 2023. After this date, Autopilot APIs will no longer function, and starting February 25, 2023, Customer Support will cease responding to new requests.

> While we don’t have a single in-house replacement product available at this time, we have a number of recommended solutions below for different use cases. For continuity of service, all Autopilot users must migrate their integrations prior to the August 25, 2023 end of service date.

While they haven't provided a replacement or migration strategy they have given a list of suggested alternatives, of which we decided to go with AWS's Lex NLU service. The only issue was that AWS Lex channel integrations for Twilio is currently limited to SMS.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g71bqeaimquwmj8po1g3.png)

So we started by doing an investigation into Twilio Webchat's current system architecture to understand what bits would need replacing to support integrating with Lex.

## Architecture
### Twilio Webchat with Autopilot (existing)

The following is an architectural diagram of how the Twilio Webchat (with Autopilot) currently works, based on documentation available at [How Messaging Works in Twilio Flex](https://www.twilio.com/docs/flex/developer/messaging#inbound-webchat). One point to note is that a Twilio Studio flow is at the heart of this design, and it orchestrates the initiate chat session with the Autopilot chatbot, up until the subsequent agent hand-off, after which it is removed from the chat channel and the Flex Agent Desktop joins.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2ft3s8clf40y5esbxm0p.png)

### Twilio Webchat with Lex Chatbot (new)

This new webchat design uses a new Webhook Flex Flow config to establish the customer chat session with a custom Lex integration webhook (implemented using AWS Lambda) which replaces the Studio Webchat Flow. It is this AWS lambda webhook which integrates the Twilio Programmable Chat channel with the Lex Chatbot and orchestrates the subsequent agent hand-off.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rk4nwcj0tohuyr6nzluw.png)

## Twilio Webhook Integration with AWS Lambda

Before getting into the code it is worth understanding how the custom webhook is configured into Twilio. As shown in the screenshot below, we modified the stock WebChat Flex Flow and changed the *Integration Type* from *Studio* to *Webhook*, then configured the URL to the AWS Lambda (sitting behind API Gateway) into the *Webhook URL* field: 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bva6dmytpz6el72ptz03.png)

[Webhook requests from Twilio](https://www.twilio.com/docs/messaging/guides/webhook-request) are URL encoded (i.e. `application/x-www-form-urlencoded`) so we use a [custom Velocity Templating Language (VTL) mapping](https://gist.github.com/krisdover/80a19b1e8acc609d40fce32bca474f3a) in API Gateway so that the Lambda receives JSON:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/beyoy8pgvwprk6xyzxn9.png)

### Lambda Webhook Code Explanation

You'll need the following packages installed to get the code working:

```sh
npm install @aws-sdk/client-lex-runtime-v2 twilio aws-lambda
npm install -D @types/aws-lambda @twilio-labs/serverless-runtime-types
```

It is also assumed that the configuration parameters for the Lambda will be passed as environment variables, but for productionised code you might want to use the AWS Parameter Store for any secrets.

A [full TypeScript code sample](https://gist.github.com/krisdover/dbbfe7cec3b0351d5160565240e99b59) is available in a Gist, but the following is an explanation of the key bits:

Firstly, our Lambda function is configured in API Gateway to be [invoked as an async function](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-integration-async.html) (i.e. it won't wait for a response), since Twilio expects the webhook to respond in less than 5 seconds, hence the `Promise<void>` return type:

```typescript
export async function handler(event: ChatMessageEvent): Promise<void> {
   ....
}
```

This is achieved using the following API Gateway Request Header config:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ca8sukkltuny0rav9b18.png)

As a security measure, we validate the signatures in the headers of all Twilio Webhook requests using our `TWILIO_AUTH_TOKEN`:

```typescript
  if (
    !twilio.validateRequest(
      process.env.TWILIO_AUTH_TOKEN!,
      request.header["X-Twilio-Signature"]!,
      `https://${request.domainName}${request.requestPath}`,
      chatEvent
    )
  ) {
    console.warn(
      "X-Twilio-Signature validation failed:",
      request.header["X-Twilio-Signature"]
    );
    return;
  }
```

Currently, Twilio only invokes our webhook with `onMessageSent` [events ](https://www.twilio.com/docs/chat/webhook-events) but it's best to future proof against this changing by existing early for unsupported `EventType`s.

```typescript
  if (chatEvent.EventType !== "onMessageSent") {
    return;
  }
```
This means that the Lex chatbot won't be invoked when chat is first initiated (i.e. no `onChannelAdded` event), but rather after the first messaging is sent by the User. In our customised WebChat UI we workaround this by auto-sending a hidden "Hi" message on the user's behalf when chat starts.

We retrieve the chat channel attributes for the request using Twilio's [Node Helper Library](https://www.npmjs.com/package/twilio) because it contains the chat pre-engagement details and chat user name:

```typescript
    const channelCtx = twilio(chatEvent.AccountSid, 
      process.env.TWILIO_AUTH_TOKEN!, {
      edge: "sydney", // change to match your AWS region
    })
      .chat.services(chatEvent.InstanceSid)
      .channels(chatEvent.ChannelSid);

    const channel = await channelCtx.fetch();

    const channelAttribs = JSON.parse(channel.attributes) as {
      // Flex WebChat UI Attributes
      from: string;
      pre_engagement_data: {
        friendlyName?: string;
        question?: string;
        location?: string;
      };
      // Custom attributes
      botBusy: boolean;
    };
```

Here we setup the [LexV2 RecognizeTextCommand](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-lex-runtime-v2/classes/recognizetextcommand.html) with the IDs of our chatbot - which for the purpose this article I assume you have already have or know how to create, as there is already good documentation on how to [get started with AWS Lex V2](https://docs.aws.amazon.com/lexv2/latest/dg/getting-started.html).

Things to note are:
* we use the unique chat `channelSid` as our Lex session ID,
* pass some of the `chatEvent` and `chatChannel` attributes through as Lex `requestAttributes` so that our [Lex Code Hook lambda](https://docs.aws.amazon.com/lexv2/latest/dg/lambda.html) can use them to retrieve/modify the chat channel, and
* pass the entire user chat message as the `text` of the Lex `RecognizeTextCommand`.

```typescript
    const requestAttributes: LexRequestAttributes = {
      channelType: "web",
      channelSid: chatEvent.ChannelSid,
      userIdentifier: chatEvent.ClientIdentity,
      name: channelAttribs?.from,
      subject: (channelAttribs?.pre_engagement_data?.question || "").replace(
        /\n+/g,
        ""
      ),
      instanceSid: chatEvent.InstanceSid,
      accountSid: chatEvent.AccountSid,
    };

    const command: RecognizeTextCommandInput = {
      botId: process.env.BOT_ID,
      localeId: process.env.BOT_LOCALE,
      botAliasId: process.env.BOT_ALIAS_ID,
      sessionId: chatEvent.ChannelSid,
      text: chatEvent.Body,
      requestAttributes,
    };
```

Now we send the Lex command and `await` the response:

```typescript
    try {
      console.log("sending: ", command);
      response = await lexClient.recognizeText(command);
      console.log(response);
    } finally {
      // apply any channel status returned by the chatbot and
      // clear bot busy indicator (don't await)
      const { status } = response?.sessionState?.sessionAttributes || {};
      clearBotBusyPromise = channel.update({
        attributes: JSON.stringify({
          ...channelAttribs,
          botBusy: false,
          status,
        }),
      });
    }
```

You'll also notice that we any `status` returned by the chatbot in the `sessionAttributes` on the chat channel. This allows the chatbot to at any time signal the end of the chat engagement to the Flex WebChat UI (which changes state to disabled further messaging), simply by setting `status: 'INACTIVE'`.

At this point, if everything went according to plan we should have a response from our Lex chatbot which we just need to format and send back to our chat user as a message:

```typescript
    if (response.messages) {
      const from = "Bot";
      for (const msg of response.messages) {
        const { contentType, content, imageResponseCard } = msg;
        if (contentType === "CustomPayload" && content) {
          // display the custom message
          const createOption: MessageListInstanceCreateOptions =
            JSON.parse(content);
          await channelCtx.messages.create({
            from,
            ...createOption,
          });
        } else if (contentType === "PlainText") {
          // display a text message
          await channelCtx.messages.create({
            from,
            body: content,
          });
        } else if (contentType === "ImageResponseCard") {
          // display quick-response buttons
          await channelCtx.messages.create({
            from,
            ...responseCardToTwilioMessage(imageResponseCard!),
          });
        }
      }
    }
```

This code handles multiple returned Lex messages with up to three (3) `contentType`s currently used in our chatbot:
* `PlainText` - this is the most basic and just consists of plain text which we can directly return as the body of a new Twilio chat message.
* `ImageResponseCard` - this consists of a textual title and one or more option button text/value pairs. We return the textual version of title concatenated with the options, but also add the options into the message attributes so they can be used by  our custom Flex WebChat UI code to render quick-response option buttons to the chat user (in place of the textual message body).
* `CustomPayload` - this consists of just JSON data which we parse and pass directly to `messages.create()`, which allows our chatbot Code Hook to have direct control over all attributes of the created message.

Finally, the last bit of code in our Lambda handles unregistering the webhook during the execution of an *agent handoff* Lex chatbot intent, since we don't want the bot responding after an agent is added to the chat channel:

```typescript
    const { intent } = response.sessionState || {};
    if (intent?.name === "agent-handoff" && intent?.state === "Fulfilled") {
      // remove this integration webhook for agent hand-off
      await channelCtx.webhooks(chatEvent.WebhookSid).remove();
    }
```
## Lex Chatbot Code Hooks

Twilio Autopilot had a concept of a [Actions](https://www.twilio.com/docs/autopilot/actions), which describe what should be done in response the NLU engine matching a user message to a given [Task](https://www.twilio.com/docs/autopilot/api/task). [Handoff](https://www.twilio.com/docs/autopilot/actions/hand-off) was one such pre-defined action, via which a chat could be handed-off from the Autopilot chatbot to an agent. [Redirect](https://www.twilio.com/docs/autopilot/actions/redirect#redirecting-to-urls) was another such action, which allowed custom Twilio Serverless Functions which can be invoked as part of completing an Autopilot Task.

Well, in AWS Lex we have the Code Hook as a generalised analogue of the Autopilot's Actions, just as an Intent is the analogue of the Task. We can configure a given Lambda function to run as a Code Hook under the Alias Language settings:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dcbkeyha6jirpib8qk8v.png)

### Lambda Code Hook Explanation

Some [sample code demonstrating how to create a Code Hook](https://gist.github.com/krisdover/5902070bfcd6ddfb0638d7680c777b0d) to do agent-handoff or end a chat session is also provided in a Gist, which again assumes you have created the associated intents in Lex (not covered here).

At the heart of the Code Hook is a switch on the invoked chatbot intent, which either creates a Twilio task-router chat task for `agent-handoff` or sets an inactive status for `end-chat`:

```typescript
export const handler: LexV2Handler = async (event): Promise<LexV2Result> => {
  console.log("event:", event);

  const {
    sessionState: { intent },
  } = event;

  if (intent.name === "agent-handoff") {
    const { accountSid, instanceSid, channelSid: chatChannelSid } =
      event.requestAttributes as LexRequestAttributes;

    // create agent chat task
    await twilio(accountSid, process.env.TWILIO_AUTH_TOKEN!, {
      edge: "sydney",
    })
      .taskrouter.workspaces(process.env.TASKROUTER_WORKSPACE_SID)
      .tasks.create({
        attributes: JSON.stringify(event.requestAttributes),
        taskChannel: "chat",
        process.env.TASKROUTER_WORKFLOW_SID,
      });

    return plainTextResponse(event, "I'm connecting you with an agent now.");
  }
  if (intent.name === "end-chat") {
    const sessionAttribs =
      intent.confirmationState === "Confirmed"
        ? { status: "INACTIVE" }
        : undefined;

    return codeHookResult(event, "Fulfilled", sessionAttribs);
  }

  throw new Error(`Unsupported intent: ${intent.name}`);
};
```

## Conclusion

Having fully transitioned from Twilio Autopilot to AWS Lex v2, the chat user experience has been great and the change in performance only slightly noticeable. This is mainly due to a reduction in chatbot responsiveness, which is caused by us using the default [Twilio Region](https://www.twilio.com/docs/global-infrastructure/understanding-twilio-regions) (i.e. where the data-processing happens) which is located on the US East Coast, whereas our AWS (Lex) infrastructure is running out of Sydney (i.e. Asia Pacific South East 2). Using [Twilio's Sydney edge location](https://www.twilio.com/docs/global-infrastructure/understanding-edge-locations) has also helped to improve this tremendously, and you would have probably noticed this configuration on end of the webhook URLs configured into Twilio and the Twilio API calls in the Lambdas.

So if you've been considering to make a similar migration, but have been struggling to find information on how, I hope you've found this guide useful. And please, feel free to leave any thoughts, improvements, or even corrections you might have in the comments section. Bye for now.