AI Learning Helper
By: Olga Generozova
Tags: Design, System Design, Artificial Intelligence, Learning To Read, Education

AI Learning Helper

AI Learning Helper is a product designed to help children learn at home. It is developed by NTT DATA Group innovation centres in Denmark, UK and Romania. My role on this project is bringing system design and product design experience to the innovation team.

In this article I will share my approach to designing for AI and using system design methods to develop business propositions. We will look at the methods the team used to develop the product vision, business model ideas and and plan the MVP. The team is developing the MVP and engaging with potential partners now.

Challenge

When the project started in 2019, the goal was to help children, who cannot attend school, learn at home. There are many different reasons why children cannot go school. It may be disability, temporary illness, psychological problems, parent’s or guardian’s choice. Homeschooling is expensive and time-consuming. Often, these children do not receive the same quality of education as children going to school.

In 2020, as the pandemic took hold of most of the world, parents had to become teachers overnight. Suddenly, homeschooling became relevant to everyone. The scope and the potential impact of the project expanded.

It became a society-wide educational challenge.

We united as system design, service design, data and innovation experts to find a solution to this challenge.

How can artificial intelligence help children learn, teachers teach and parents homeschool?

We wanted to develop a scalable solution, a personalised learning helper, that any child, parent or teacher with a connected device can use.

After considering many areas of early education, we chose to focus on learning to read. It is a fundamental skill, which lays foundation for most other types of learning.

Research

NTT DATA has a partnership with MIT Media Labs. It gives us access to a treasure trove of research insights into innovative applications of advanced technologies. We reviewed research in the field of using robots for early education and social companionship and AI augmented learning. The key concepts of the product are based on the following learnings.

Social learning

Children in early primary school learn socially. The relationships are very important for their engagement and motivation.

Children have better learning outcomes and knowledge retention when they have an emotional connection with their learning companion.

Research has shown that children respond very well to learning with a buddy character. Many existing solutions focus on testing children’s knowledge and finding the right content. Social and emotional components are often overlooked.

Pedagogical methods for learning to read

We conducted desk research and interviewed teachers to understand key pedagogical methods. It led us to ”The simple view of reading”, a widely accepted scientific theory, explaining how reading skills are developed. A student’s reading comprehension depends on pronunciation (decoding) and understanding (language comprehension):

Decoding (D) x Language Comprehension (LC)= Reading Comprehension (RC)

We chose to use this formula for evaluating learning progress and recommending next steps.

We reviewed typical reading lesson plans, discussion guides and activities teachers use to introduce a book. These materials provided guidance and inspiration for a range of teaching methods used by the AI Learning Helper.

Homeschooling challenges: interviewing teachers, parents and kids

To understand the challenges of remote learning, I interviewed children, teachers and parents. Below are the highlights of what we learnt.

The teachers find it hard to keep students engaged remotely. It is difficult to track learning progress and give individual attention and support to each student.

The children miss learning with friends, feel less motivated and more distracted at home. They liked the idea of learning with a buddy, like an older brother or sister and commented that “it would feel less lonely”. Having their favourite game, book or movie characters as their learning buddy avatar would make learning more fun.

Parents commented, that they struggle with playing a teacher role, and balancing it with other commitments. They would want to receive guidance on the best way to teach their child and have more visibility of how well they are doing.

Approach

We set out to create a personalised AI Learning Helper, that can develop an emotional connection with every child. It should evaluate their learning progress and adapt teaching strategy. We wanted to make learning to read fun, using edutainment and gamification. The solution needs to be scalable and accessible to anyone with a laptop or a tablet.

We designed the AI Learning Helper to have 3 layers:

Teaching engine — a platform that can import educational content and help children learn to read and understand it

— a platform that can import educational content and help children learn to read and understand it Emotion AI — a learning companion that detects emotions and responds in a human-like way

— a learning companion that detects emotions and responds in a human-like way Engaging AI Learning Helper avatars that can be part of an app, exist in a 3D learning environment or in a physical space

Teaching engine

AI provides a way to codify pedagogical methods and best practices for learning to read. We developed AI capabilities to support key skills important for learning to read. For each book AI Learning Helper provides real time pronunciation feedback. It can test how well children can summarise text, describe images, ask and answer questions about the text.

The AI Learning Helper is evaluating student’s learning progress to recommend the best next steps. We are planning to continue refining the evaluation criteria in collaboration with teachers.

To make the solution scaleable, we aim to minimise human effort when importing a new book into the engine. Each book will be “onboarded” once and made available to all users. AI capabilities supporting pronunciation and understanding will be added to enrich content. Eventually, the book onboarding process will be automated. For the MVP some of the AI capabilities will be done or reviewed by humans.

Emotion AI framework

Our NTT DATA Group colleagues at the itelligence Transformation Lab in Denmark have developed it.human platform over the last couple of years. They offer digital human employees companies can hire for providing customer service.

The it.human platform became an integral part of the AI Learning Helper. It enables the AI Learning Helper to have a natural conversation with users and build an emotional connection. The AI Learning Helper monitors facial expressions, vocal intonations and body language. It detects emotional reactions and responds in a human-like way.

Engaging helper avatars

To improve engagement and motivation, the AI Learning Helper can look like children’s favourite characters. We designed the solution that is flexible enough to work with many different types of avatars. The it.human platform is able to work with many types of 3D models. It can bring the avatars to life, giving the ability to speak, move, understand and express emotions.

Designing for AI

Bringing together AI, design and business

Our multi-disciplinary innovation team worked in close collaboration throughout the project. We combined our research insights from AI, business and human perspectives to co-create the product vision.

Behind our friendly learning buddy is a carefully orchestrated system of AI components. AI experts in 4 countries from Northern Europe to Australia are developing it. Defining an end-to-end user experience was essential to align the team around the product vision and coordinate work. I created Storyboards for the product vision, the MVP and the vision video.

Testing AI Learning Helper

We are testing the product in stages:

Making sure it works as expected, and is intuitive and easy to use

Get feedback on the learning activities, choice of avatars, voices etc.

Responding to children's behaviour and emotional reactions

Evaluating learning progress, adapting teaching methods and choosing the right content

Improving learning outcomes

Often in early tests many aspects of the experience are imitated by a person. Early feedback is valuable for AI teams to focus development efforts and consider real life scenarios that might be less obvious. Gradually they are replaced with AI algorithms.

Testing prototypes with children gave us many surprising insights. The way they interacted with the product and the questions they asked often were unexpected. It helped us understand what is important from a child’s perspective and provide input for UX and conversational design.

The focus of this article is on the design process. To read about user research for AI and collaboration between AI and design teams at NTT DATA Group in more detail see these articles:

AIDA: the challenge of user research for AI in the autism field

Using qualitative research to structure an unbiased AI dataset

Thoughts on the role of Designer of AI products and services

The role of an AI Designer is a Translator between human needs and advanced technology. Let’s look at some aspects of this role:

Understand the art of possible, the potential and limitations of AI and robotics now and in the near future

now and in the near future Invent genuinely useful applications of AI , based on solving real needs, humanising technology

, based on solving real needs, humanising technology Distill knowledge and codify experience of experts, collaborate with subject matter experts throughout the project

of experts, collaborate with subject matter experts throughout the project Test with humans, who are going to use the service , carefully planning what can be tested at each stage

, carefully planning what can be tested at each stage Design responsible AI , considering ethics of AI, handling sensitive data, as well as longer term consequences

, considering ethics of AI, handling sensitive data, as well as longer term consequences Define enablers and orchestrate backstage actions for each aspect of the experience

Designing how to get the product info in the hands of users is as important as creating a great product.

System design approach

Helping children learn to read is a great example of a problem that cannot be solved by a single company. Many people and organisations support children on their journey to becoming confident readers.

System design offers methods and tools to create solutions, delivered through partnerships. I will share the system design approach we used for AI Learning Helper.

The strength of our team is in design and ability to develop cutting edge AI solutions. We do not specialise in education or entertainment. Partnerships in these areas are important to create this product.

To teach children reading skills, we need educational books. We can work with publishers and authors to offer them an innovative distribution channel and provide books to users.

To deliver the best learning outcomes for children, AI Learning Helper needs to codify the best pedagogical methods. Input from leading educators and academia would benefit the product.

To make the learning helper avatars more engaging, we can make them look and behave like children’s favourite toys, game, book or movie characters. Collaborations with companies, which own characters children love would benefit the product.

Teachers and parents are the best people to introduce the AI Helper to children. Organisations working with schools and parents can deliver the product to their customers.

For the AI Learning Helper to succeed, we need to build an ecosystem. A carefully orchestrated partnership of organisations contributing to and delivering the service.

We started with understanding the product ecosystem.

Product Ecosystem

Product ecosystem is a network of organisations, co-creating, delivering or consuming a product. It may include suppliers, partners, distributors, customers, competitors or government agencies.

In service design, we map an existing Ecosystem, to understand the context a service operates in. In system design we design a new Ecosystem that can deliver a service through partnerships of organisations.

Benefits and outcomes

Identify potential partners and clients , understand all parties involved in creating, delivering and consuming a product or service

, understand all parties involved in creating, delivering and consuming a product or service Consider all types of value stakeholders can exchange , if they participate in the product ecosystem, what each of them can contribute and receive in return

, if they participate in the product ecosystem, what each of them can contribute and receive in return Provide context for creating business models, partnership agreements and positioning the product to potential clients and partners

I created an Ecosystem Map to visualise our collective knowledge of the stakeholders and the values they exchange. The Ecosystem Map helps to understand the big picture, the lay of the land, all potential partnerships the product team can have.