Conversations With Alexa
By: Lance Ulanoff
Tags: Amazon, Technology, Alexa, AI, Analysis

We were engaged in a spirited debate about whether to order dishes or combo meals from our local Chinese Food restaurant when Alexa chimed in: “Maybe you should get one platter and then two orders of rice to split.”

“Is that enough food for both of us?” I asked.

“According to Lotus Flower’s menu choices, which indicate that the combos are family style and the rice comes in quarts, it is,” said Alexa.

“But we’re really hungry,” my wife pleaded with Alexa.

‘Okay, okay,” said Alexa, “Then I suggest two kids dinners and three..”

“Wait.” I interrupted, “Aren’t kids dinners really small?”

“No,” Alexa answered without losing a step, “They’re just 3 ounces smaller. But you should also get three orders of rice,” said Alexa, sounding a bit exasperated.

“Hey, that all sounds pretty good. We want Kung Pao chicken and Chow Mein kids meals,” said my wife.

“Ordering,” said Alexa.

Full disclosure: This didn’t happen and is not possible with current Alexa, but Natural Turn Taking (not the most elegant name), which Amazon revealed this week as part of the near-future of its Alexa voice assistant (and a small part of a major fall product rollout), could make a three-way conversation like this more than possible.

Think about it this way: The fears you’ve had that, when talking in front of an Echo or other Alexa-enabled devices, that there’s always someone else in the room, possibly listening to you, is sort of coming true.

Or put another way, you’ll soon be giving Alexa the okay to do something it’s been doing for ages anyway, butting in.

The State of Alexa

If there is a race to make our voice assistants more human-like in their conversational abilities, Amazon is firmly in the middle. Alexa doesn’t possess the Turing Test challenging skill of Google’s Duplex, which, to be fair is highly-trained, narrowly focused add-on for Google Assistant that works in 48 of 50 states to help you make dinner and hair salon reservations. It can, however, answer follow-ups to initial questions without requiring the wake word.

It’s no better than Apple’s Siri or Google Assistant about not speaking unless spoken to. Alexa still responds to words similar to its watch word, from casual conversations in its vicinity, and commercials. Thank goodness I didn’t name either of my children “Alex.”

I grew up on a steady diet of Star Trek where the Enterprise AI “Computer” engaged in lengthy conversations with the Captain and Crew with one watch word and, after that, only the context provided by, usually, Captain Kirk. As a result, I know we’re still in the early days of real-world AI-based voice assistants.

A real conversation

Amazon’s Natural Turn Taking, though, is a pivotal step toward a more human-to-AI conversational future.

The ability to engage in a one way conversation is difficult enough for most AI voice assistants, a three way convo is a digital party in which the AI has to navigate who is speaking to whom (to Alexa or the other party), pauses as one speaker waits for a response, and the inevitable cross-talk, which is even more likely when you’re trying to make a decision.

In a blog post on the still-under-development technology, Amazon said Alexa uses a mixture of technologies to assess everything from acoustic to linguistic and even visual (I guess this will work on Echo Shows) cues to figure out when it’s time for Alexa to listen or interject.

As it’s currently being built, Alexa wouldn’t do any of that if someone doesn’t say, “Alexa, join my conversation,” which is about as natural as asking someone to pick up the fork to eat the food you just placed before them. Obviously, since Alexa is a disembodied observer in the conversation, it makes sense to formally invite her to engage.

While that invite might feel awkward, the rest of the conversation should not. Once Alexa is invited, there’s no need to use the watch word again until the end. Removing the need to say “Alexa” during these conversations greases the wheels of discussion nicely but doesn’t necessarily make it feel natural.

The concept of “barge-ins,” though, does help. Essentially, Amazon is now programming Alexa to identify when speakers break into an Alexa response to, perhaps, change the course of conversation or, maybe, identify which of a list of options they prefer. Amazon calls this “contextual barge-ins.” Instead of Alexa ignoring your interruption or, more likely, resetting to an earlier conversational state, this Alexa will know exactly when you interrupted and understand that you selected something or indicated heightened interest. This can help steer the three-way conversation to a more natural conclusion.

Brain training

In preparing for these future conversational abilities, the current Alexa is now teachable, learning from its mistakes and getting a firmer grasp on the difference between entities (a mode-based request like “set the volume to five”) and declarative statements (“the music is too loud”). Understanding how to handle the latter comment and that the response should be, “Okay, do you want me to lower the volume?” could also lead to future interactions where you saying, “The music is too loud,” automatically leads to Alexa lowering the volume to five.

Amazon is also working on improving Alexa’s tone of voice, letting her adapt her cadence and emphasis based on the context of the discussion. During the virtual launch event, Company reps played before and after versions of the same Alexa utterance but, to be honest, I think I still prefer the unadjusted one.

With some of this learning and adaptation in hand, the future Alexa could handle not just who is talking and when, but understand the context, nuance, and meaning of everything you say, and respond in kind during a spirited, likely task-based, discussion.

When the conversation is done, you’ll still have to tell Alexa to stop listening. Right now, Amazon has you saying, “Alexa, leave my conversation,” an especially awkward phrase that would probably hurt a human’s feelings. It’s a good reminder, though, that Alexa isn’t human, doesn’t have feelings, and is still a tool designed to help you get things done. It’s not a new party guest or best friend.

Now, pardon me while I go order some Chinese food.