Bio-Medical Information Networks
Professor Tohru Ifukube

Welfare Engineering at the Core
Searching for New Potentials in Science and Industry

Interviewer: Hayato Kuroki, Special Assistant̶̶ First, could you tell us about how this laboratory was

to be dif ferent, aren' t they? It comes down to wanting

established?

to help people on the one hand, and wanting to bringsomething to fruition as an industry on the other. That's

My field is called Rehabilitation Engineering, and it's been

the fundamental difference. So what occurred to me, in thethirty-odd years since I started. In the beginning, I was partend, was rehabilitation engineering. I thought that there

of a medical engineering lab where we were applying themust be a way to use engineering to help people who had

technologies of electronics engineering to medical devices

suffered impairment or loss of function. Or, with the powersuch as artificial hearts and CT scans. That's how I got intoof engineering, if someone were missing a hand, we could

the ﬁeld. At the time, Hokkaido University was the only placebuild an artiﬁcial hand and make it seem just like their own-with a medical engineering field, and I joined their Medical

that's the kind of ﬁeld I was aspiring to.

Electronics Research lab as a Master's student. For the next
two years, I had the extremely rare experience of being able to̶̶In that sense, we could say that you have your roots intake courses in medical subjects like physiology and anatomy

engineering.

while pursuing my own themes.
My roots are in engineering, but I share the values of the̶̶You were involved in a wide variety of areas, weren't you!

medical ﬁeld, in that I want my work to be useful for people.
My research is what they call interdisciplinary. Nobody was

About half of the people in our lab were medical doctors, anddoing this 35 years ago. Maybe it was because there weresince the ﬁeld of medicine is so diverse, that their backgroundalways a lot of composers and musicians in my house, but thealso varied-radiology, internal medicine, surgery, and so on.

ﬁrst thing I looked into was whether there was a ﬁeld called

The other half of the people weren't just from electronics,

rehabilitation engineering where they did research on sound,but also mechanical engineering and materials engineering.

my old friend. I was hoping to revive the pleasure of music forPeople from all different fields were doing research in one

people who couldn't sing because they had lost their hearing.

lab, trying to ﬁgure out how to put technology to work for thesake of medicine.

̶̶Speciﬁcally, what kind of research did you do?.

̶̶Has your work always been tied to medicine?

The first thing I worked on was a device called a tactical
vocoder, which lets the hearing-impaired "hear" voices with

Back then, everybody was doing cardiac research, which istheir fingers just like the blind read Braille. With Braille,

one reason that I was working on measuring strain on the

it's possible to read ten characters per second-fast readers,heart during exercise. But I realized that when it comesanyway. When people speak, they use about ﬁve charactersto doing research like that, you have to enter a medical

per second. Since you can read ten characters per second withschool if you want to follow through to the end. Besides,

Braille, we took five-character speech and converted it into

the values of medicine and engineering are always goinga stimulus that can be felt with the sense of touch, just like1

Braille.We were sure that people would be able to understandif they could read well with their ﬁngers, so we developed a

̶̶The results of their academic work became a product,

device that would let them "hear" voices with their fingers.

and then they created a company to sell it?

This research took a really long time, but it was featured in the
NHK (Japan Broadcasting Corporation) documentary "Hearing

In that company, they started out by devoting themselves

A-I-U-E-O With Fingers" and became quite well-known.

completely to creating assistive devices for the hearing
impaired, but the selling price wound up at ¥1.5 million.

̶̶Is that the research you did as a Master's student?

However, they were surprised when Dai Nippon Printing
unexpectedly showed an interest in the machine. At theI dealt with it in my Master's thesis, but it took about ﬁve years

time, the number of workers who could quickly lay type wasto actually complete. That research provided the opportunity

dwindling, and they ﬁgured that if they could get somewhere

for me to establish myself in this field, but at the samearound 90 percent accuracy even with novices just reading

time it made me realize how terribly difficult rehabilitationout the original documents and converting them into text,

engineering could be. In other words, when you're hearing

then it would come in handy around New Year's or the end ofwith your fingers, it's a question of whether your brain can

the ﬁscal year. They used the machine for seven years. That's

properly process the stimuli as words. The information picked

when we realized that there was a way to make a businessup by your ﬁngertips travels to the brain, then it's converted

out of this: of course, the hearing impaired could still use theinto speech and understood as language. If the information

machine, but we could expand the range of applications as

doesn't pass through that way, then you can't understand it.

well. A device designed under the strict conditions required

Even if vibrations from the ﬁngertips get transmitted, it's up toto allow the hearing impaired to use it can be expanded more

the brain whether they're heard as words. On the other hand,

broadly into the field that we call today "universal design."

it's also possible for the brain to change so that it can accept

We ﬁgured that we could drop the price once the device hadwords like that. Nobody knew anything about the mechanism

come into wider use, and as a result, we would be able toby which such a change might occur.

deliver the devices to the hearing impaired who really are inneed.

Socially as well, it's hard to introduce new technology in afield of education like this. Cost was a major problem, and̶̶ In what ways are you furthering the research on speech

the selling price ended up at ¥740,000 [about $7000]. In therecognition at RCAST?

end, it was too expensive to sell. I thought, "That's the way it's
going to be with this ﬁeld," and I was thinking about quitting.

On the technology side, we're making full use of both theWhat encouraged me to keep at it, though, was when theInternet and speech recognition devices. Personally, having

members of support groups for the hearing impaired told me

worked many years in the field on research to promote thethat if possible, they would really like me to develop a device

well-being of disabled people, I think that we shouldn't

that could convert speech into text. They said, "We might notleave everything up to machines, but rather have people usebe able to understand with our ﬁngers, but if you've come thismachines to help people. It might seem like common sense,

far, surely you can turn speech into letters." So I decided to

but I think that in this field, this is the fundamental way intry my hand at building a machine that could convert spoken

which machines should be used. We should let machines dowords into letters. That was the device that's called a "speechwhat machines are capable of doing, but let humans do thetypewriter." What a tough task-I had to manage everything

things that machines can't do and that humans are particularly

with the microcomputers of the day, just 32 kilobytes! I

skilled at. I'd also been thinking for a long time aboutordered a CPU from America and built my own computer,industrial applications, but after coming to RCAST I received

and I used a language called Assembler to make a device

funding and got support from TBI" (Technology Businessthat could convert voice into text. At a speed of about 0.2

Incubation),*2", and suddenly my ideas started taking shape.

seconds, it had about 96 percent accuracy. The only thing
was, it broke words into syllables: for example, it would turnSpeech recognition devices are extremely difﬁcult to develop

the Japanese word ohayo into o-ha-yo. At the time, Japanese

because they deal with problems that can't be solved withoutword processors had just come out, and in connection withthe kind of high-level processing that the brain performs.

that it became possible to convert syllables into Kanji (Chinese

Today's speech recognition devices are always highly error-

characters) so we made a product out of it. It cost ¥1.5 million

prone, and when you get a different person with a different[about $13,000]!

voice, the machines can't cope. It's impossible to get overthis huge wall with today's technology. Therefore, since we

Before putting it on the market, some students who made thehave achieved about 95 percent accuracy as long as there's adevice started up their own venture company. That was about

designated speaker, we've got to use the system that way. If

25 years ago. This was at a time when the industry-academia

we take something that someone said and have it repeatedcollaboration was completely unheard of, and it was strictly

by a person whom the machine recognizes, then we get 95

forbidden to do any research for one's own proﬁt in academic

percent accuracy. In this way, no matter who's speaking, theinstitutions. Under those conditions, they developed theaccuracy rate remains high. There are several problems that

product secretly in the lab and started up a joint-stock

we have to resolve if we're going to develop this into a bigger

corporation while they were Master's students. That's thebusiness. One of them is the language barrier. For example,

company that became today's "B.U.G. Inc., *1,"the pioneers ofwhen someone speaks in English, how could we have that

venture business.

come out in Japanese? If we could achieve that, surely it
would meet a serious need. Of course, we will anyway need2people who are capable of simultaneous interpretation.

unnatural, so we developed a pitch controlled artiﬁcial larynx.
We made it for people who've lost their voices after having

Using networks over the Internet, regardless of where thelaryngeal cancer removed. It's essentially the same concept as

speaker or the person who converts their speech to text might

the mynah, so the waveform is quite different, but it produces

be, it's possible to send information and have it sent back asa sound that sounds just like a voice to the human ear. Thattext. No matter where a meeting is being held, even if theway, even without vocal cords, you can send sound out of theinterpreter's not present at the meeting, you can transmit thedevice and move your mouth to turn it into a voice. When

voices to the interpreter and have the translated information

research is founded on solving familiar problems, these aresent back. The audience, the speaker, the interpreter, thethe kinds of results that emerge.

person who converts speech to text-it doesn't matter where
anybody is! You can use interpreters at venues where there

After that, we worked on parakeets. I had just happened

aren't any interpreting facilities, and if you add up the costto hear from a TV company about a genius parakeet thatof things like travel, this system will be much cheaper. Once

could do things like recite the Heart Sutra. They wanted me

this service gets established, even people in their homes will

to look into it, so I went all the way to Fukuoka. After doing

be able to work. For example, a visually impaired person willsome research, I found that parakeets and mynahs werebe able to listen to speech and convert it into text. I think thistotally different. That's when another big puzzle appeared.

will make it possible to provide employment opportunities for

Mynahs and parakeets both have voices that resemble thosedisabled people who ﬁnd it difﬁcult to get out of their homes.

of humans, but their vocalization is completely different.

In my lab, this is the project that we're currently tackling with

Moreover, they can make plosive sounds-"pa," "ba," "ma"-the most enthusiasm. Our work is very highly regarded atwith their mouths open. It's impossible for humans to make

RCAST, and we've received generous funding. We're now inthose sounds without closing their mouths. I just couldn't

the middle of setting prices, with plans to make a business out

understand how they could produce those consonants. Iof this in 2005.

hypothesized that they were making the sounds directly fromsomewhere in their throats, but even though I dissected their

Actually, we put this service into practice at the Internationalthroats, I couldn't ﬁgure it out.

Film Festival held in Yubari, Hokkaido, in March 2004. They
needed to translate four languages into Japanese, so first

About 15 years went by, then one day I had a big surprise

we sent the voices down to an interpreting center in Tokyo.

while watching the ventriloquist Ikkokudo on TV. He was

From there, they went to the staff in Sapporo, who converted

saying "pa," "ba," and "ma" without closing his mouth. I

the information into text, then the textual information was

thought that this might help me solve the problem, and since

returned to Yubari, where it was displayed on a screen. ThisI was at Hokkaido University back then, I went to go see one

had a huge impact. Now, a convention planning company,of his shows at a hot springs resort near Sapporo. I went

an interpreting company, and B.U.G., the company started up

backstage afterwards and told him, "This is incredible! Youby my students, have formed a partnership to think of ways tomight tear down what's been considered common sense forpopularize this technology in the future.

all these years." I said, "I could never ﬁgure out how mynahs

Mynahs, Parakeets, and Bats
Practicing Science by Solving Familiar Problems

open, but you can do it!" If we could solve this problem, it

and parakeets can say pa,' ma,' and ba' with their mouths
would be useful in a number of ways. For example, patientswith cleft lip and palate can't close their mouths, and they
̶̶You're making progress in a variety of ways, aren't you?

can't say pa,' ma,' and ba' because they don't know how

Providing barrier-free linguistic services for so-called able-

to vocalize them. When I told him that, Ikkokudo said, "If it

bodied people as well as the hearing impaired, linking that towould be of help, then please study me." It turned out that hethe creation of new employment opportunities, and so on.

didn't know how he was making those sounds himself. So I
asked him to let me do some research on him, and a few days

Since it became a business along those lines, we had achieved

later, I attached sensors to him and recorded his voice andone of the functions of engineering, but the science was still

image. After analyzing his voice, I solved the problem.

lacking. In other words, if we ran into a snag, we would have
nowhere to turn. It wouldn't really be engineering without

In October 2004, the 3rd International Ventriloquism Festivalthat scientiﬁc foundation, so we decided to tackle a variety of

was held in Tokyo, where professional ventriloquists from allscientiﬁc problems. Thinking that working on simple, familiar

over the world gathered. I was asked to make a speech there.

problems would lead to the fundamentals, the first thing

They work hard on "pa," "ba," and "ma" too, don't they? Since

that we worked on was mynah birds. We wanted to ﬁnd out

I understood the mechanism behind it, I gave a demonstration

why mynah voices resemble human voices so much. So wewith videos and so on in front of an audience of about 200.

bought some mynahs, analyzed their mimicry and dissected

They were so delighted at hearing that what they work on

their throats, and we realized that the waveform of theirjust for entertainment might actually help others. I've studied

voices is totally different from humans. Even so, to the humanventriloquism quite extensively, and I learned some incredible

ear it sounds just like they're speaking. Gradually, we learned

things. Actually, it's not that I discovered this mechanism, butthat the human brain processes their voices just like humanrather that others knew about it through experience. There's

voices, even though the waveforms are different.

a part in the book written by world-famous ventriloquist
Valentine Vox that deals with how to make "pa," "ba," andFrom there, we thought that we might be able to create"ma" sounds. It was exactly the same as the hypothesis that I

a "voice" like the mynahs', even if it sounded somewhat

had come up with.

3

would contribute to science as well. With this research as a
̶̶Which was?

foundation, we'll also be able to make a lot of progress in thefield of engineering. I've been told that rehabilitation

If you want to say "pa," first say "ta" with your mouth open,

engineering would never make any money, that it would

then reposition your mouth to make the "a" sound in "pa."never amount to a real academic discipline, and that it wasn't

The changes your mouth undergoes are different between theeven engineering. But at RCAST, there's the view that, to the"a" in "ta" and the "a" in "pa." So if you just imitate that change

contrary, rehabilitation engineering is about removing barriers

that takes place at the end of the sound, even if it starts out as

and that it's going to be the future of industry. I've been herea "ta," it sounds like "pa" to the human ear.

for two and a half years, but the speed at which my research
progresses has totally changed.

̶̶If you don't mind my changing the subject, how did youcome to be involved in Professor Hirose's Sensory Information
and Communication Project?

Tactile Support for High-Speed Aural
Comprehension,
Artiﬁcial Muscles Made with Metal Hydride AlloyE v e r s i n c e I a r r i v e d a t R C A S T, I ' v e b e e n w o r k i n g i ncollaboration with Professor Fukushima who specializes

̶̶In addition to that, are you currently involved in any new

in the barrier-free studies, but I'm also par ticipating incollaboration with companies or other labs?

Professor Hirose's Sensory Information and Communication
Project. It all star ted with bats. I knew what it was like

I've revived "TouchVoice," the device for "hearing" with one's

working with mynahs, but we wondered if it would be

fingers, which transmits the vibrations from speech withpossible to analyze how bats can find prey in the dark bylittle pins. That's another collaboration with a company. In

detecting the reﬂected ultrasonic waves that they emit, and2cv004, a blind man from IBM took his doctorate at RCAST, did

to create a device equipped with the same mechanism.

research on converting online textual information into speech
by well using compensatory functions. The speed at which

Normally, students at schools for the blind use canes, but

you read aloud comfortably is actually quite high. But if you

obstacles that come close to their heads, like tree branches

read too quickly, you can end up losing track of what's mostand eaves, are quite frightening. We wanted to make a device

important in the text as a whole. When a headline or a wordthat would alert them to such things, and the ﬁrst thing thathas turned red, that's quite important information too. So I

came to mind was bats. After conducting various studies, wethought, what if we transmit that information with a tactile

managed to create a device that generates ultrasound waves

supplement device for the hearing impaired? While you're

like bats and converts the reflected sounds into the sounds

listening to the voice at high speed, "TouchVoice" sends a

audible to human ears. We took that device to the blind

little tactile stimulus whenever there's something important,

students, but they had no use for it. They told us, "We don't

and it makes you stop and say, "Hey, what was that?" You canneed anything to do something like that." Actually, they go byalso feel it when lines and fonts have changed. This way, the"feel". So then we thought, "What in the world is feel'?" andreading speed becomes about 2.5 times faster.

we studied it for ﬁve years. "Feel," or perception of obstacles,
is all about sound, subtle changes in the sound ﬁeld. We can't̶̶How about your involvement in the medical ﬁeld?

hear it with our ears, but it's comprehensible through thebrain's compensatory functions. After the loss of sight, theDue in part to my having worked in a medical lab, I've always

sense of hearing begins to play a more active role in spatial

had strong ties to the field of medicine. I went to Stanford

recognition that had not previously been required. People

University for a year to study artiﬁcial inner ears, and I thought

can tell what's on the other side of a wall and, depending on

about a lot of things during that time. Things like differences

the person, some can even tell the texture of clothes people

between Japan and the US, differences between medicine andin front of them are wearing. Some can perceive a 6-cm

welfare, etc. I think that this ﬁeld of rehabilitation engineering

disk that's about 4 meters in front of them. If you use thisis going to gradually overlap with medical engineering.

mechanism well, it's possible to artificially create a sense of"feel" and project imaginary obstacles. If you line up a bunch

For 20 years, I've been working on artiﬁcial muscles for peopleof speakers and control them with a computer, you can feelwith decreased muscle strength of limbs. When you cool or

the presence of obstacles that aren't really there. This ends upheat an alloy called metal hydride alloy, it releases or absorbs

tying into research on virtual reality, in which they're trying tohydrogen 1000 times the volume of the alloy. If you can makemake people recognize imaginary objects. Unlike the normalthat hydrogen go into a container, the container expands andVR method of inserting one image into another, we insert

shrinks, generating incredible energy. With 100 grams of thissounds into a sound ﬁeld and make imaginary obstacles out ofalloy, you can lift a person who weighs 100 kilograms. I've

them. Since I was doing research like that, I ended up working

been making progress on research along these lines, such as

together with Professor Hirose.

developing assistive devices using this alloy, and attachingsuch devices to arms in order to loosen up joints.

The results of this research will lead to new technologies, suchas allowing robots to walk around obstacles in the same way

This ties into robotics as well. Actuators using hydrogen arethat seeing impaired people avoid them. Figuring out what

extremely resilient, so we can use them to make robots thatthe "feel" is is a scientific endeavor. Also, if we can present

won't cause damage if they bump into things. Ultimately,

our hypotheses as cognitive physiology about how the

everything that I've been working on up until now will be

brain's functions change when the sense of sight is lost, thatbrought together in robots. Attaching eyes to robots so they4

can detect obstacles, adding ears so they can comprehend

not watch it.

speech, or enabling them to speak and feel, on and on. What
I have in mind, for the long run, is for my work to come to

The documentary is very interesting to watch even now,

fruition in the shape of a robot that can help human beings.

though. Back then, I spent a lot of time looking for Ainu who
had come to Hokkaido from the Kurils, and I finally found̶̶The potential for developing your work for industry as

three. I asked them to listen to the playbacks, and they would

well as for the disabled has become more of a reality, hasn't

tell me, "Oh, that means such-and-such." I wrote downit?

everything they taught me. At the time, they were all in their
mid-80s, and they all passed away a few years later. So if I

Since there wasn't much of a demand before, up until now

hadn't done that study back then, the mystery never would

it just wasn't possible to make a large industry out of myhave been solved.

work. There were also fundamental systemic problems,
among others, so I've been collaborating with barrier-free labs

̶̶That's research that you probably couldn't have done

and getting them to suggest the necessary social systems.

anywhere else but in Hokkaido, don't you think?

I've also worked with researchers specialize in intellectual
property to think about how patents and laws should be set.

Since I was born out in the provinces, I always wanted to do

At RCAST, these things are all possible. We take our ideas thatthings that weren't possible in big cities. Now I'm at Tokyo

will easily take root in society to the legislature, and then with

University, but that feeling hasn't changed. Everybody thinks

the social systems that the legislature creates, we can makeabout what can only be done in their particular places, don't

an environment in which it's easier for venture companies tothey? For example, I think that we could create a new industry

be born. If this succeeds as an industry in Japan, then in theif we introduced IT into farming and ﬁshery. No one bothers

future it will become an export industry. There's potential forto think about such things in Tokyo. So nobody would everit to become one of Japan's key industries.

imagine that there could be something to learn from the
Ainu language. In that sense, I'd always like to concentrate

I've been working in rehabilitation engineering for overon things that can only be done in the countryside. For

30 years, and I've gathered my thoughts about the field,

example, if we're talking about Hokkaido, if we create originalincluding suggestions about how it should change in thetechnology to solve problems related to snow, the cold, andfuture, in my book Fukushi kogaku no chosen [The Challenge

open spaces, then maybe we could send that technology, as

of Rehabilitation Engineering, on sale 12/20/2004, Chuko

an export industry, to other northern regions that suffer from

Shinsho]. The first half features a lot of stories about mythe same problems. Just like we're using regional "handicaps"

failures. The second half is mainly about how I ﬁnally started

as springboards for the development of new industries within

to believe that rehabilitation engineering could become true

Japan, I think it would be great if we could take Japan's current

engineering after coming to RCAST.

position in the world and turn it into a springboard for creating
new industries, demonstrating Japan's unique strengths.

Overcoming the Barrier of Time
Playing 100-Year-Old Voices from Wax Cylinders

Recent Publications

̶̶Your most unique project is probably your research on

Fukushi kogaku no chosen [The Challenge of Rehabilitation

wax cylinder playback, wouldn't you say?

Engineering]. Tokyo: Chuko Shinsho, 2004.
Jinko genjitsukan no hyoka [Evaluating Artificial Perceptions

There was a time when the ﬁeld of rehabilitation engineeringof Reality]. The author and editor, Tokyo: Baifukan, 2000. Oto

was so difficult that I was thinking about quitting. I thinkno fukushi kogaku [Rehabilitation Engineering with Sound].

it was around seven or eight years after I had become an

Tokyo: Korona, 1997.

associate professor. While I was looking around for my next

Onsei taipuraita no sekkei [Design for a Speech Typewriter].

theme, I got wind of some news that 73 wax cylinders used

Tokyo: CQ Shuppan, 1984.

to record the music and voices of Ainu from the Kuril Islands
had been discovered unexpectedly in Poland. The recordings

Curriculum Vitae

were over 100 years old. The Polish man who made themhad taken a wax cylinder recording device invented by Edison

Born in Hokkaido, 1946

to the Kuril Islands, and there he recorded about 100 voices.

B. S. in Electronics Engineering, Hokkaido University.

After that, though, he committed suicide in Paris, and no one

M. A. in Engineering, Hokkaido University.

knew what had become of the recorded media. Many years
later, someone just found them one day under the roof ofPrior work experience:

the inn where he had been living. At the time, well, there

Assistant, Research Institute for Applied Electronics, Hokkaido

were some political issues involved, but the Japanese Foreign

University.

Ministry and the Polish government negotiated, and a special
JAL jet transported the cylinders to Japan. I worked on playing

Visiting Associate Professor, Stanford University.

back the sounds from the cylinders for six months, but thenProfessor, Research Institute for Electronic Science, Hokkaido

I ended up going to America to work in the artiﬁcial innerear

University.

laboratory. So I was in the US when the NHK documentary

Currently, Professor, Tokyo University Research Center for"Yukar: 80 Years of Silence-The Secret Story of the Kuril Ainu

Advanced Science and Technology (RCAST)

Wax Records" was televised in the "NHK Special" series, I could

Fields of Expertise: bioengineering, audio engineering,5

rehabilitation engineering
PhD, Engineering

Links
RCAST
http://www.rcast.u-tokyo.ac.jp
Ifukube and Ino Laboratory
http://www.human.rcast.u-tokyo.ac.jp/index̲e.html*1:B.U.G. Inc.:
http://www.bug.co.jp/english/index.html=B.U.G. Inc.
*2:(Technology Business Incubation):
http://www.rcast.u-tokyo.ac.jp/en/research/tbi/index.html=(Technology Business Incubation)

6