AI and the Ainu Language
On the 15th of July, we organised a webinar on AI and the Ainu Language. Ainu is recognised by UNESCO
as a critically endangered language and there is almost nobody who speaks it as their first language ormother tongue. Sachiko Kibata is one of the few surviving speakers of Ainu. First, we welcomed
Sachiko to our seminar where she answered some questions and presented a traditional folk tale(yukar). After this, Professor Tatsuya Kawahara talked about his latest project developing automatic
speech recognition and synthesis technologies to assist the preservation of the Ainu language. Theseminar was chaired by Professor Roger K. Moore.
Conversation with Sachiko Kibata1:

1

Roger:

I have heard that Ainu is a language in danger of disappearing, and that there are barely
any people who speak Ainu as their first language. When you were a child, how muchcontact did you have with the Ainu language? For example, did your father, mother, or
grandmother speak Ainu?

Sachiko:

My parents didn't speak much Ainu. I didn't understand the language well, but because
we were poor, my grandmother looked after me when my parents were out at work.
My grandmother used Ainu, so I heard some of the language when she spoke it.

Roger:

What made you want to learn Ainu, Sachiko-san?

Sachiko:

An Ainu culture festival called the Sisirimuka Culture Festival was held in February 1990,
offering for the first time the opportunity to present Ainu language, dancing and songs.
I was asked to speak there, so I performed the Kamui Yukar that I had heard from mygrandmother. That was the trigger for me to start learning Ainu. I studied by doing mybest to remember what my grandmother had said all those years ago, and writing itdown.

Roger:

Sachiko-san started learning Ainu when she was 59, approximately 30 years ago. How
did you learn the language at the time? How did you become an Ainu teacher?

Sachiko:

I was 59 when I started learning Ainu. There was an Ainu language teacher called Shigeru
Kayano, so I attended his classes for ten years. Later, Mr. Kayano became the first Ainu
member of the House of Councillors (in 1994). He had to move to Tokyo, so he told me"Kibata-san, now you are no longer a student. You are an Ainu language teacher," so I
became an Ainu language teacher. That was more than 20 years ago. I am now 92. I willsoon be 93, so I am pretty old, but I still continue to work as an Ainu language instructor.

Sachiko’s answers as written here are based on the answers she gave during the talk, but are also augmented with some sentences which
provide relevant information that she had given prior to the event taking place.

Roger:Sachiko-san is currently active in many fields as an Ainu language instructor. How manypeople are studying the Ainu language at your classes in the entire town of Biratori,
Hokkaido?

Sachiko:Currently it's around 30 people. Sometime there are as many as 50. People are busy with
their families, so the number of students is not that large. The participants include
university students and elderly people, and I myself am studying, together with them.
We hold Ainu language classes twice a month.

Roger:

Recently, it seems that more and more young people are trying to study Ainu as a second
language. Do you think the environment surrounding Ainu has changed compared to thepast?

Sachiko:

I've been teaching now for more than 20 years, so there may well have been changes
compared with when Mr. Kayano was the teacher. But I studied hard under Mr. Kayano,so I enjoy studying together with everyone.
In the old days the students were mainly elderly, but now young university graduates
are coming to the classes. There are people who come all the way from Yokohama,Kagoshima and Tokyo to learn, which is really fantastic!

Roger:

Professor Kawahara of Kyoto University has developed an AI system, based on
automatic speech recognition and synthesis technologies to assist the preservation ofthe Ainu language for the future. What would you like to see happening to the Ainu
language in 20 or 30 years?

Sachiko:

I am old now, and these days there are university students coming to study the Ainu
language, so I would like those students to study hard and learn the language so thatthey can preserve it.

Roger:

Are there any Ainu traditions other than the language that you would like to pass on tofuture generations? For example, dances, songs, how to make crafts such as wood
carving, etc.

Sachiko:

As a woman, I am training young people to dance Ainu dances so that these can bepreserved into the future. My legs are no good, so I can't dance myself, but I can leadthe singing to accompany the dance, and I do this enthusiastically whenever I have thechance. I enjoy getting involved in the singing and dancing with everyone.

Roger:Finally, do you have a message, in Ainu and then in Japanese, for our participants fromall over the world?2Sachiko:

I am not especially good at Ainu language, but I would like people to understand Ainu,
to study it hard, and to preserve it accurately. I am working as hard as I can to teach it.

Professor Kawahara’s Talk:
Professor Kawahara began his presentation by briefly outlining the history of the Ainu people. It isestimated that in the 18th century, there were around 80,000 Ainu spread across modern-day Japanand Russia, living in Hokkaido, Sakhalin and the Kuril Islands. This number has now dwindled to 25,000,
according to a 2002 Japanese government survey, with a large number living in southern Hokkaido.
Nibutani, the village from which Kibata-san comes, is more than 80% Ainu. About 100km west of
Nibutani lies Shiraoi, where the newly opened National Ainu Museum welcomed the Olympic torch
relay just last month. The museum’s name, Upopoy, means ‘singing together’ in Ainu.
Next, Professor Kawahara elaborated on the current status of the Ainu language. The Ainu people
have experienced a history of hardship, having undergone assimilation to become Japanese. As aresult, Ainu, an orally inherited language, is almost extinct. In fact, it is now classified by UNESCO as a
critically endangered language with only a handful of native speakers left. Despite this, Ainu culture
has recently begun to experience a revival, with last year’s opening of the National Ainu Museum
being proof of this. There also exists a large archive of recordings of Ainu oral folklore that has been
kept since the 1970s. These feature two types of folk tale: yukar, which Kibata-san performed earlier,heroic stories sung in rhythm and uwepeker, stories narrated in prose. This archive has beendeveloped at the museums in Shiraoi and Nibutani. However, since there are just a handful of people
who are able to transcribe Ainu and verify the transcriptions, only thirty out of seven hundred hoursof Ainu speech recordings have been properly archived. It is because of this that Professor Kawahara
decided to find out whether AI speech technology could help remedy the situation.
Professor Kawahara then presented an overview of the Ainu language. Ainu is a language isolate, withno academic consensus as to its origin, though it does share some similarities with Japanese phoneme
sets and word order. It is agglutinative and polysynthetic, meaning a word can consist of multiple
morphemes strung together. For example, the word for ‘turtle’, ‘atuykorkamuy’ is made up of thewords ‘atuy’ (‘sea’), ‘kor’ (‘own’) and ‘kamuy’ (‘god’). In terms of syllabic structure, it is usuallyconsonant-vowel or consonant-vowel-consonant.
Next came an overview of Professor Kawahara’s specialty subject, speech technology. Speechtechnology is made up of three components: speech recognition, alignment and speech synthesis. The
first component recognises speech, converting it into text and adding a timestamp. This timestamp isthen used by the alignment component to synchronise the transcribed text with the speech. Finally,
the speech synthesis component converts text into a speech wave form. Automatic speech recognition(ASR) for major languages is widespread nowadays and is used in smartphones and automatic subtitle
generation. This technology is trained on the huge amounts of available data. However, more than
five thousand languages are designated as ‘low resource languages’, and more than half of these, such
as Maori and Sami, are endangered. Professor Kawahara pointed out that because of the limited
number of speakers and datasets, developing speech recognition technology for these languages is2

Sachiko delivered a speech and message in the Ainu language which was recognised and transcribed automatically by the AI developedby Professor Tatsuya Kawahara. The AI accurately recognized 90% of individual sounds (‘phones’) and 70% of the full words used.

not as straightforward. The conventional framework of ASR requires the system to have all theinformation about a language to hand. Much like a person learning a language, the system has to know
the grammar, rules and phonology of a language before it can ‘understand’ it. In contrast, a recently
developed framework called the ‘end-to-end model’ features a deep neural network that canindependently learn to map speech to text using only data, meaning no prior knowledge of a language
is required. This has enabled Professor Kawahara, a self-professed ‘naïve foreigner’, to develop an ASR
system for Ainu.
The professor and his team have been working on this technology for two years, using speech datasets
provided by two museums. These consist of prose uwepeker stories told by eight speakers, totalling
forty hours of recordings. The speech recognition system has produced encouraging results so far.
When processing speech from speakers included in the training dataset, it achieved a phoneme (thesound of one Latin alphabet letter) accuracy of 94% and a word accuracy of 80%. When the system
was presented with speech from an unknown speaker, phoneme accuracy fell to 86% but this rose to
90% when the speaker’s voice was digitally converted to resemble a voice familiar to the system.
According to Professor Kawahara’s personal assessment of the speech recognition system, the levelof recognition is comparable to that of a graduate student of Ainu. Professor Kawahara hopes it canbe used for generating initial drafts of transcripts in the future.
The system has already been used for aligning transcripts with speech, an important process thatfacilitates searching for a speech segment corresponding to a given word/phrase query. Here,Professor Kawahara outlined how the alignment process works. When the ASR system is fed a speech,
it produces timestamps alongside the text output. Even though the resulting text may contain someerrors, the timestamps will be accurate, and these can then be simply copied over to a manuallytranscribed text. While a human would take a whole day to align a one-hour speech, the ASR system
can do this in one minute. Professor Kawahara also demonstrated the ASR system’s ability to tell thedifference between Ainu and Japanese. This is useful as many speech recordings feature bothlanguages.
Professor Kawahara’s team have also been investigating Ainu speech synthesis - that is, the ability toreproduce speech from a given text. The technology is trained to emulate speakers who have morethan 10 hours of recorded speech in total. Using the aforementioned end-to-end deep neural networkmodel, it converts a sequence of characters or letters into a speech waveform. Professor Kawaharapresented the audience with three audio samples of spoken Ainu, asking them to guess whether thespeech was human or synthesised. Although the audience were able to tell the difference for two ofthe recordings, there was one which elicited more mixed responses, with some believing thesynthesised voice to be human. Professor Kawahara and his team used this technology to generatespeech for a prose story entitled ‘Raijin’s Sister’, transcribed in 1958 and for which no audio recordingexists. He then showed the synthesised audio, adding that staff at the Ainu Museum used therecording as a pronunciation reference during training. Related to this, Professor Kawahara predictedthat speech synthesis technology could be used to create a virtual AI character who could assist in theteaching of Ainu.
There are many legal and ethical issues surrounding the use of speech synthesis and Professor
Kawahara did not shy away from discussing them. For example, manipulating the voice of an unwitting
person, known as ‘fake speech’, violates copyright and personality rights, and must be labelled assynthesised. After summarising his presentation, however, Professor Kawahara concluded byrestating his belief in the potential of speech recognition and speech synthesis technology not only topreserve Ainu language and culture, but also to assist in Ainu language learning.

Q&A with audience:
One of the first questions concerned the possibility of using speech technology to revive ‘dead’languages. Professor Moore answered that this was to some extent possible and that thoughlanguages may be designated ‘zero-resource’, they may still be related to extant languages. In thatcase, one could make use of ‘transfer learning’, a technique where an AI system infers data about one
topic from a separate, but closely related, topic. Professor Kawahara highlighted the limitations ofsuch a use, however, pointing out that without native speakers, it would be impossible to verify theaccuracy of the AI output.
Another question was about the different dialects of Ainu and whether they presented problems for
the ASR system. Professor Kawahara responded by saying that though there are many dialects of Ainu,
his system focuses on the Saru dialect, mainly spoken in Nibutani. One audience member also
highlighted the work of Japanese linguist Kyōsuke Kindaichi, well known for his transcriptions of yukar
in the 1900s. Professor Kawahara added that he had been contacted by the Japanese Ministry ofCulture who enquired as to whether his speech technology could help to make a large stock ofrecordings accessible to the public.