R&D Spirits
Laying the Foundation for New Communication
Technologies
Dr. Shigeru Katagiri
Director
NTT Communication Science Laboratories
NTT R&D Fellow

NTT Communication Science Laboratories is a place of diverse research activities toward
information-communication that reflects the very essence of human communication. One areaof particular interest is speech recognition. NTT’s speech recognition technology boasts
advanced realtime capabilities and the world’s largest vocabulary. The director of these laboratories, Dr. Shigeru Katagiri, is a recognized authority in speech recognition and an NTT R&DFellow. We asked him about the current state of progress in speech recognition and what benefits it might provide to society.

Pattern recognition research: endowing
computers with intelligence—Dr. Katagiri, could you first tell us about the mission of NTT Communication Science Laboratories?
As the name implies, our mission is to performresearch in the field of communication science. However, as these are research laboratories of a telecommunication carrier, the approach that we take does not
fit the typical “academic” image centered on linguistics, psychology, and the like. Rather, we take a “scientific” approach that aims to clarify human communication functions and develop technologies with theknowledge obtained. Today, “communication” iscoming to include not only communication between
people but also communication between people andthings and between things themselves. In this regard,
we might ask ourselves how society could beimproved for both people and the environment andhow communication technology might be used to thisend. Finding answers to these questions is the missionof NTT Communication Science Laboratories. We
undertake this research with great determination
based not on existing concepts but on this definition ofcommunication science as an evolving field of study.
—What is your personal research theme?
I have been researching pattern recognition from6

the start with the ultimate goal of giving computers
the ability to hear and comprehend on the same levelas human beings. To be more precise, I have been
researching acoustic models to convert speech to text.
In this process, the sound picked up by a computer’s
microphone is compared and judged against an
acoustic model in a database and converted to text.
But since the accuracy of this conversion is only as
good as that of the acoustic model, the problem here
is how to prepare an accurate model. I have been
involved in the construction and development of techniques for this purpose.
—How will such research benefit society in thefuture?
For human beings, speech is a very easy-to-use
medium. If the speech recognition performance ofcomputers can be improved, we can expect various
ripple effects. Take, for example, an NTT call center
that receives telephone inquiries from customers. If
the primary processing of this call center can bemechanized, we can expect significant cost savings toresult. We can also envision a speech recognition system that monitors the handling of customers and collects material for the creation of call-center guidelines. An advanced version of such a system could
issue a warning in real time whenever an inappropriate response is made to a customer. In addition, techniques for comparing and judging entered informaNTT Technical Review

R&D Spirits

The more accurately such boundaries can be determined, the less we have to be concerned with theproblem of whether a model adequately represents acertain class of things. We therefore abandon this ideaof maximum likelihood estimation that assumes themodeling of average patterns for different classes.
Instead, we apply a learning process to input patterns
so that models can grow and more accurate boundaries can be drawn, resulting in a minimization ofclassification errors. This is the idea behind our discriminative learning method.
I first became engaged in the research of discriminative learning methods at the end of the 1980s. At
that time, discriminative learning methods could not
be directly applied to the recognition of speech signals, which are actually time signals. This prompted
us to look for a method that could overcome thisproblem. In the end, we extended a classical learning
method called “probabilistic descent” developed in
the 1960s so that it could be directly applied to hidden Markov models (HMMs), which are suitable forrepresenting time signals, and developed a design
method that could minimize pattern classification
error. This is our GPD method. In other words,
despite the fact that speech signals are very difficult
to process since their patterns have different temporal
lengths, this GPD method can handle them directly. It
also provides a theoretical guarantee that a model canbe trained, or designed, in principle to minimize
recognition error. And though computation is heavy,
the recognition accuracy that the GPD method canachieve is dramatically higher than that of past techniques. I think the IEEE award and the fellowship that
I had the honor of receiving for this achievement
reflect the value of this work.

tion against a model are not limited to speech recognition—they have much in common with various
kinds of pattern recognition. Accordingly, if an effective technique for speech recognition can be established, it should be possible to apply it to image andtext recognition as well. That would mark the beginning of intelligent computers and the coming of dramatic changes to all social systems.
—What are some technical points of interest in thepattern recognition research that you have been
involved with?
The most interesting aspect of this research would
probably be the generalized probabilistic descent(GPD) method, which is one type of discriminative
learning method. This method is somewhat complex,
but I would like to give you an outline of it here.
Assume that the following statement has beenmade: “This is a writing tool. Is it a pen or pencil?”
To answer this question, we could create a model of a
pen and one of a pencil, and then compare and judge
the object against these models using various functions. There are various ways of going about this, but
the most common approach is to create representativemodels of a pen and pencil and measure the likelihood that the input pattern agrees with each of thesemodels as a basis for making a decision. This is called
the “maximum likelihood estimation” method. But
the most important thing in pattern recognition is notmodels per se, but rather the classes that individualobjects belong to, and in particular, where the boundaries of those classes are located. In our example, wewould need to accurately determine where the classof a pen ends and where the class of a pencil ends.
Misclassification measure

1/ µ

 1µ
dk ( x; Λ ) = − gk ( x; Λ ) + g j ( x; Λ ) ∑ M − 1 j , j ≠ k{}

Smooth misclassification losslk ( x; Λ ) =11 + exp( −αdk ( x; Λ ) + β )Probabilistic optimization mechanismδΛ( x (t ), Ck , Λ ) = −ε∇ lk ( x (t ); Λ )Λ(t + 1) = Λ(t ) + δΛ( x (t ), Ck , Λ )L( Λ̇ ) = ∑ ∫ p( x, Ck ) lk ( x; Λ̇ )1( x ∈Ck )dx
k Ω∑ ∫ p( x, Ck )1( x ∈Ck )1( pΛ̇ (Ck x ) ≠ max j pΛ̇ (Ck x ))dxk Ω= ∑ ∫ pΛ̇ ( x, Ck )1( x ∈Ck )dx
k Ωk{}Ω k = x ∈ Ω pΛ̇ (Ck x ) ≠ max j pΛ̇ (C j x )Fig. 1. Essence of the formalization of the generalized probabilistic descent (GPD) method.

Vol. 3 No. 6 June 2005

7

R&D Spirits—How is this research progressing?
Well, we are researching various aspects of speech
recognition, but we are devoting particular effort to a
new formulation of the GPD method. Our past
approach was to achieve a learning process that could
minimize classification error within a framework likethe HMM that can handle time signals, but this was
not very applicable to the control of design tolerance,that is to say, of statistical stability. To solve this problem, we are attempting to redefine the GPD methodwith a new parameter space of fixed dimensions. The
completion of this undertaking should make it easier
to use the GPD method and to make it a more general-purpose discriminative learning method.
And though this is more the work of my colleagues
than my own work, we have been developing a framework for representing models using a finite-state transducer (FST) with the aim of achieving advanced
speech recognition functions. This framework is evolving into a mechanism that can recognize about 2 million words. The current level of large-vocabulary
speech recognition systems in the world ranges from
200,000 to 300,000 words, which makes NTT the only
organization with a system that can recognize 2 million
words. Considering that “Kojien,” Japan’s standard
dictionary, contains about 800,000 words, the recognition of up to 2 million words would enable the handlingof proper nouns such as personal names and place
names that in the past were often treated as “unknownwords.” A model representation on such a huge scaleas this one should be able to express various types ofknowledge in an integrated fashion. We believe that
our research should not only improve the accuracy ofspeech recognition but also bring about major innovations in computer-based communications.

the one organization making steady progress in thisarea is NTT.
By the way, there is a great difference in the waythat discriminative learning methods have been
approached in Japan and overseas. I don’t know thereal reason for this, but I have a feeling that differences in computation power lie at the root of it. In theUnited States, connecting personal computers (PCs)together to perform parallel processing has becomecommonplace, while in Japan, the conditions fordoing that have not yet been established. To be sure,PCs have dropped in price, but Japan is still weak at
putting together such configurations. Perhaps methods like discriminative learning are being avoided
because of the heavy computation that they require.
At any rate, I would speculate that some kind ofrevival in the research of discriminative learning
methods is needed in Japan.
—What kind of response have you received in Japanand overseas to your research?
Since I received the Signal Processing Society
Senior Award from IEEE in 1994, my research has
been receiving a certain amount of attention. However, the path to this recognition was never easy. The
first time I presented a paper on GPD was in 1990 asa joint work with researchers at Bell Laboratories. My
coauthors and I were very confident about our work,
and we submitted it to ICASSP, the most prestigious
forum in this research field. At that time, however,papers were initially evaluated based on a summaryof only a few hundred words, which was hardly
enough to convey the importance and novelty of ourresearch to the reviewers. In the end, the paper wascompletely rejected, and the only presentation that wecould make of the work that year was at the Acousti-International interest in an originallydeveloped GPD method

Research in this field is active overseas, particularly in the United States. The International Conferenceon Acoustics, Speech, and Signal Processing (ICASSP) sponsored by IEEE includes an annual session on
discriminative learning methods, and research that
follows in the footsteps of our work must certainly bepresented there. As for Japan, I must confess thatresearch in this area has dropped off. The last half ofthe 1980s and the first half of the 1990s was a boom
period for our research and for neural network
research, which are closely related, and a good number of sessions on discriminative learning methods
were held during that time. At present, it appears that
8Recognition accuracy (%)—Dr. Katagiri, please tell us about trends in thisresearch field both here in Japan and overseas.

75GPD
70
65

Maximum likelihood method

6055
50

0.5

11.5

22.5(× 104)

Model size (number of Gaussian distributionfunctions for acoustic modeling)

Fig. 2. This figure illustrates that GPD consistently
produces higher recognition rates than its
traditional counterpart, the maximum likelihood
method, in an example word recognition task.

NTT Technical Review

R&D Spirits

cal Society of Japan. But even there, other papers presented at the same time received most of the attention
and there was little response to our GPD method.
After that, we continued to receive little recognition,
so the period was somewhat depressing. Fortunately,
things began to change after the IEEE Award.

Doshisha University since last year.

—Are you involved in any collaboration with outside
institutions?

Well, my undergraduate degree was in electricalengineering, but in graduate school, I was a memberof Professor Ken’ichi Kido’s research laboratory thatwas working on speech-information processing. I
think I became interested in human-oriented researchas opposed to simply electrical technology because ofProfessor Kido’s course related to human-machineinteraction. Many students wanted to join his laboratory, but as enrollment was limited, attendees weredecided by playing “rock paper scissors,” which is anold tradition at the university for situations like this.
If I had lost, perhaps my life thereafter would havebeen different.

One great feature of NTT Laboratories is to be as
open as possible to the outside, and I can say that wehave been involved with much collaboration. For
example, we have a very close cooperative relationship with MIT and Stanford University. We have also
invited a number of renowned researchers to serve as
research professors here. These include Professor
Fumitada Itakura of Meijo University, my former
senior at NTT Laboratories and recipient of the AsahiPrize in January of this year, Professor Nobuo Masataka of Kyoto University Primate Research Instituteand author of “Monkeys with Cell Phones”(Chukoshinsha), and Professor Fred Juang of Georgia
Institute of Technology. In addition, many of ourmembers have presented lectures at various universities. As for myself, I served as a guest professor at theGraduate School of Kyoto University up until theyear before last, and I have been a lecturer at

Acoustic
model

Wordmodel

The excitement of understanding: themotivation behind research—Dr. Katagiri, what was your major at university?

—Please tell us something about your research history after entering NTT.
On entering NTT, I was first assigned to Section
No. 4, Basic Research Division at Musashino Electrical Communications Laboratories, which was aspeech-research group that NTT took great pride inPhrase
modelFig. 3. This figure illustrates the behavior of the finite-state transducer that is currently studied at NTT Communication
Science Laboratories for large vocabulary speech recognition.

Vol. 3 No. 6 June 2005

9R&D Spirits

for its achievements in speech coding and telephonystandardization. The head of the group at that time
was Professor Itakura, who I mentioned earlier. Next,
when the Advanced Telecommunications ResearchInstitute International, or ATR, was established, I wasassigned to hearing research in ATR Auditory andVisual Perception Research Laboratories. There, I
conducted research as the only member of a section
set up through the good offices of Yoh’ichi Tohkura,
who is now the Deputy Director General of theNational Institute of Informatics (NII). After thatassignment, I worked at Bell Laboratories for a ninemonth period beginning in 1989, and then returned toNTT, where I was able to devote myself to researchfor several years. But since becoming an NTT R&D
Fellow, I have worked only in the Research Planning
Section, and I am currently serving as director ofCommunication Science Laboratories. Accordingly, I
can’t really say that I have had many periods in which
I could devote myself to pure research. I sometimes
wonder whether I deserved being named an NTT
R&D Fellow.
—What do you feel is so interesting about researchwork?
The excitement of finally understanding a difficult
problem—when you finally say “I got it!”—is whatmakes research a very interesting endeavor. I haveexperienced that wonderful spine-tingling feeling at
the point of enlightenment any number of times during my research career. For example, I had that exact
sensation when I was thinking about the GPD method
and I suddenly realized that I could turn it into amethodology if I approached it in a certain way. That
was immensely interesting, and experiences like thatprovided me with the motivation to continue myresearch work.
—How do you think the research of speech recognition will develop from here on?
To begin with, we would like to make the GPD
method easier to use. To be more specific, we wouldlike to reduce the amount of computation that it now
requires as much as possible and would like to makeit so that anyone can use it without the need for extensive “tuning” know-how. In speech recognition, however, it is not sufficient to simply improve the performance of an acoustic model no matter how far thatmodel can be improved. In actuality, human beings
understand the meaning of sounds that they hear bytaking into account various limitations possessed bylanguage. Japanese people, for example, listen to andunderstand each other while unconsciously holding
to the rule that a consonant cannot be used by itself in
10

the Japanese language. In addition, connections
based on context are extremely important in speech
recognition. I therefore think that design work mustalso consider how a language model should be created and how tasks should be divided. A part of our
GPD method has been based on this approach and theplan is to expand this line of thinking across the entire
method. If this can be achieved, I think that ourspeech recognition technology cannot help but beincorporated in the outside world.

NTT Laboratories: the last treasure of the20th century—Dr. Katagiri, what are your plans for the future?
For the near future, I think we should be able tosolve some of the crises that we now face using
telecommunications. That might sound like an exaggerated claim, but it’s a sincere feeling. For example,
it should be possible to provide an effective response
to today’s serious energy and environmental problems through the use of videophones and the Internet.
But despite the fact that a broadband infrastructure isspreading rapidly, business trips for the purpose of
attending meetings are still fairly frequent, and thereality is that videoconferencing is not functioning
well. Nevertheless, if telecommunications were to
take root in the real world, there would be no need forpeople to move from one location to another formeetings. This would make a substantial contribution
to alleviating the energy problem. To this end, it isessential that we develop telecommunications to thepoint where people can share each other’s space.
Telecommunication technology should also be useful in the revival of Japan’s industrial sector. Given thatthe economies of neighboring countries are capable ofrapid growth as we have seen, an industrial structurethat depends on cheap overseas labor will reach its
limit sooner or later. To overcome this problem, wemust pursue a knowledge-intensive type of production. I believe that telecommunication technologies
involving pattern recognition, statistical learning, andknowledge processing provide a foundation forknowledge-intensive technologies and hold the key tothe success or failure of new-generation production.
—In your opinion, what is the significance of NTT
Laboratories?
As a researcher, I think it’s the last treasure handed
down from the 20th century. Let me explain. BellLaboratories and NTT Laboratories were the finest
research institutions in the 20th-century world oftelecommunications. And Bell Laboratories can becalled an American treasure of the 20th century. But
NTT Technical Review

R&D Spirits

as time passed, Bell Laboratories went through majorchanges, and the only research treasure left became
NTT Laboratories. Even if we broaden our viewpoint
here to include research institutions that are holding
on to the traditions of the 20th century, I can only
think of the IBM Watson Research Center. Perhaps
researchers who are constantly trying to innovate willfind this emphasis on “tradition” strange, but if we
exchange it with the words “know-how,” its importance suddenly becomes clear. Fortunately, NTT continues to place importance on basic research as a corporate policy. It supports basic research in a variety ofways, starting with the provision of research funds.
From my position as an on-site manager, I consider
the preservation of NTT tradition to be very important work.

—Dr. Katagiri, could you leave us a message foryoung researchers?

Interviewee profile■ Career highlights
Shigeru Katagiri received the B.E. and M.E. degrees
in electrical engineering and Dr. Eng. degree in information engineering from Tohoku University, Sendai,
Japan in 1977, 1979, and 1982. From 1982 to 1986, heworked at the Musashino Electrical Communication
Laboratories, Nippon Telegraph and Telephone Public
Corporation (now NTT), Tokyo, Japan, where he wasengaged in speech recognition research. From 1986 to1998, he was with the Advanced Telecommunications
Research Institute International (ATR), Kyoto, Japan.
At ATR, he worked on various speech-related
research issues that included speech recognition, auditory scene analysis, and spoken language acquisition.
From 1997 to 1998, he headed the Hearing and Speech
Processing Research Department at ATR HumanInformation Processing Research Laboratories. Since
1999, he has been with NTT Communication Science
Laboratories (CS Labs), Kyoto, Japan, where he has
been engaged in a wide range of machine-learning
research. There, he served as Supervisor of theResearch Planning Section for two years. Currently,
he occupies the position of Director at NTT CS Labs.
From 1989 to 1990, he was a visiting researcher at theSpeech Research Department, AT&T Bell Laboratories, Murray Hill, NJ, USA. He has also served as an
Adjunct Professor at the Graduate School of Kyoto
University (1998-2004), and as a lecturer at both theGraduate School of Nagoya University (2001-2003)
and Doshisha University (2004-present). He is anIEEE Fellow and an NTT R&D Fellow.

■ Awards
1. 22nd Sato Paper Award of the Acoustical Societyof Japan (ASJ) (1982)
2. 27th Sato Paper Award of the ASJ (1987)
3. ATR R&D Award (1991)
4. ATR R&D Award Special Prize (1992)
5. IEEE Signal Processing Society 1993 SeniorAward (1994)
6. Best Presentation Award of The Japanese Societyfor AI (2002)■ Major academic society functions
1. Associate Editor of the IEEE Transactions on Signal Processing (1994-1997)
2. Chair (1999-2000) of Technical Committee on
Neural Networks for Signal Processing of theIEEE Signal Processing Society (IEEE-SPS)
3. Vice-Chair (1997-1998) of Tokyo Chapter of theIEEE-SPS
4. Action Editor of Neural Networks (2000-)
5. Associate Editor of the Transactions of IEICE D-II(1997-2000)
6. Senior Editor of EURASIP Journal on Applied
Signal Processing (2001-2002)
7. Member of IEEE Neural Networks Society
Administration Board (1998-2002)
8. Member of Technical Committee on Multimedia
Signal Processing of the IEEE-SPS (1998-2001)
9. IEEE-SPS Board of Governors, Member-at-Large(2003-2005)
10. Member of IEEE Frank Rosenblatt Award Committee (2005)
11. Chair of Kansai Section, Acoustical Society ofJapan (2005)

Vol. 3 No. 6 June 2005

I’d be happy to. Taking my research as an example,
it is more important in research to search out boundaries rather than models that usually represent central
regions in place of boundaries. Many seeds of new
research can be found in boundary regions rather thanin the provision of models. With this in mind, I wrote“Please look around you” in an e-mail that I sent out
to all staff members of the Communication Science
Laboratories on my first day as director. What I meantby this was to look for the seeds of innovation in theboundaries of your own region and in the boundaries
of your colleagues’ regions. NTT Laboratories has awide spectrum of talented people, and I want young
researchers to look at the people around them andconsider how their work might benefit their ownwork. If such a frame of mind can be established, I
think it will have great value in forming teams madeup of professional researchers.

11