Smart Speech Assistants – Part 10 – What Should a Skill Say?
Channel: Andreas Jakl
Date: 2022-10-13

Smart Speech Assistants 🗣️
Part 10: What Should a Skill Say?

In this video series, you will learn about the voice user interfaces in the world of health. Smart Speakers are one of the most accessible interfaces for technology. Therefore, they play a vital role in learning about health (patient education), educating professionals as well as staying connected during your rehabilitation process.

👂 This part covers:

- Why simply answering a question might not be enough: implicit conversations.
- How elaborate does the response of a voice assistant have to be?
- Can multimodality improve the conversations?
- How should the confidence if an artificial intelligence understood you correctly influence its answer.
- Why conversational markers are important for creating natural conversations.
- Distinguishing between beginners & professionals in your dialog design.


☑️ General overview:

Why is speech interaction at the center of creating health-related digital services? First, we tackle four common assumptions:

🤔 1) Who uses smart speakers anyway? – voice assistants in the world and how they can be used for health
🤔 2) Does artificial voice sound bad? – Speech understanding, dialects, and next-gen language models.
🤔 3) Voice assistants are only used for telling jokes? – Samples of real-life interactions and where speech assistants are part of everyday life.
🤔 4) Voice assistants only work at home, not for business? – professional settings from online meetings, internet of things, Alexa for business and offline voice recognition.

Afterwards, let's dive into the world of speech assistants. What is Echo, what is Amazon Alexa? How do they work behind the scenes? What can smart speakers *not* do today? How do the components of a speech service look like?

For your next hands-on steps, we will look at development options. Starting with easy-to-use development tools like Voiceflow (no prior coding experience required), up to native skill development using the Alexa Skills Kit (ASK) or AWS services.

But to create a good user experience, you need to design for voice. First, you need to understand unique properties of voice interaction. This includes speed and listening rate. Next, you need to be aware of user expectations on voice UIs.

Once you are aware of the context, you can design what a skill should say. This includes many concepts like implicit confirmation, multimodality, and conversational markers.

You can hardly ever script conversations. How can you make a skill react to changes from the golden path? How do you handle errors in the conversations and make sure the user knows what to say?

Privacy and ethics are one of the most important final topics. What access does your skill have on user data? What requirements do you have to fulfil to comply with certification guidelines and laws?

The final part gives insights into the open source "Voice Learning" Alexa skill. It helps students learn for courses. According to scientific studies, a voice-based quiz can be a better learning method, compared to studying the traditional way.

---
🌍 Developed for the DIRENE project (Competences for the new era of user-driven digital rehabilitation), co-funded by the Erasmus+ Program of the European Union