Intelligence Science and Technology Course
09

Construction and Elucidation of Intelligence,and Realization of Flexible,
Human-like Information Processing.
In an advanced information-oriented society, information processing
with flexible, human-like information capabilities becomes essential.
Information processing in human and animal organic systems isunparalleled in nature which has developed through structural andfunctional adaptation to the environment in the long process of evolution.
Intelligence Science and Technology is a multidisciplinary field that aims toelucidate the mechanisms of biological-, particularly, human-information
processing for the development of higher-level information processing.

Intelligence Science and Technology Course

Welcome to Intelligence Science andTechnology CourseIntelligence Science and Technology CourseIntelligence Science and Technology is an interdisciplinary research fieldin which we study sophisticated human information processing and develop
new technologies on the basis of an understanding of human intelligence.
By “intelligence,” we do not mean simply “artificial intelligence” but rather
a wider scope embracing intelligent aspects of human, systems, andinformation technology. More specifically, foci of our research andeducational activities include (i) basic mechanisms of human beings, such as
life, brain, cognition, and behavior, (ii) information media related tolanguage, speech/auditory, and visual information, and (iii) a broad range of
topics related to intelligence and informatics, such as interaction between
human and intelligent systems, artificial intelligence and machine learning,and software and networks.
One of our distinguishing features is integration of these topics under aunique discipline, “intelligence science and technology,” notwithstandingthat these topics are also investigated in separate research fields such asneuroscience, psychology, engineering, and computer science. Faculty
members and students coming from a wide variety of academic backgrounds
devote themselves to cooperating in pursuit of this common goal. We hopethat, in this unique academic environment, many young students will
engage in this exciting research field, tackle the mysteries of complex human
intelligence, and develop new ideas for addressing challenging issues
regarding intelligent media and computation.
Our course welcomes outstanding students from various fields besides
those of computer science and engineering. We provide interdisciplinary
curricula and research projects. Join us, and enjoy an intellectually exciting
experience!

Shin’ya Nishida

Professor, Intelligence Science and Technology Course
1990 Research Associate, ATR Auditory and Visual Perception Laboratories
1992 Research Scientist, NTT Basic Research Laboratories
1996 Ph.D., Kyoto University
1999 Research Scientist, NTT Communication Science Laboratories
2019 Professor, Kyoto University

Graduate School of Informatics

10Intelligence Science and Technology Course

Outline■Group and Teaching Staff
Group

Teaching Staff

Neuroinformatics

Yukiyasu Kamitani/Professor Yukiori Goto/Associate Professor Hiroshi Hosokawa/Senior Lecturer
Yoshihiro Nagano/Assistant Professor Shingo Maegawa/Assistant Professor

PsychoinformaticsTakatsune Kumada/Professor Ryoichi Nakashima/Associate Professor

Cognitive Informatics

Shin'ya Nishida/Professor Hiroaki Mizuhara/Associate Professor
Kiyofumi Miyoshi/Assistant Professor

Computational Cognitive
Neuroscience (Adjunct Unit)

Hiroyuki Nakahara/Adjunct Professor Wataru Sato/Adjunct Associate Professor(RIKEN)

Computational Intelligence

Akihiro Yamamoto/Professor Nozomi Akashi/Assistant Professor

Collective IntelligenceHisashi Kashima/Professor Koh Takeuchi/Assistant Professor
Han Bao/Program-SpecificConversational Informatics
Language Media
Processing

Sadao Kurohashi/Program-Specific Professor Yugo Murawaki/Senior Lecturer
Chenhui Chu/Program-Specific Associate Professor Fei Cheng/Program-Specific Assistant Professor
Yin Jou Huang/Program-Specific Assistant Professor

Speech and Audio
Processing

Tatsuya Kawahara/Professor Kazuyoshi Yoshii/Associate Professor
Koji Inoue/Assistant Professor Keiko Ochi/Program-Specific Assistant Professor
Eita Nakamura/Program-Specific Assistant Professor

Computer Vision

Ko Nishino/Professor Shohei Nobuhara/Associate Professor
Marc Aurel Kastner/Assistant Professor

Human Sensing

Yuichi Nakamura/Professor Kazuaki Kondo /Associate Professor
Kei Shimonishi/Assistant Professor

Text Media

Shinsuke Mori/Professor Hirotaka Kameko/Assistant ProfessorBiological Information
Networks

Tatsuya Akutsu/Professor Takeyuki Tamura/Associate Professor
Tomoya Mori/Assistant Professor■Curriculum of Intelligence
Science and Technology Course
CurriculumofIntelligenceScienceandTechnologyCourseDoctoralProgram(Informatics)
DoctoralThesis

rd3 
nd2 
st1 Subjects provided by the Course (total 6 credits including 4 credits from seminars)

Seminar on Intelligence Science and Technology, Adv. E (Mandatory, 2 credits)
Seminar on Brain and Cognitive Sciences, Adv. A, B E,
Seminar on Cognitive System, Adv. A, B E,
Seminar on Intelligence Media, Adv. A, B E,Seminar on Application of Multimedia, A, B Adv. E,Seminar on Bio-system Informatics, A, B Adv. E(2 credits each)ResearchGuidance

Master’sProgram(Informatics)



Master’sThesis
nd- 4 -

2 

SubjectsprovidedbytheCourse(optional6creditsormore)







AdvancedSubjects

Seminars(4credits,Mandatory)

SeminaronCognitiveScience,ComputationalCognitiveNeuroscience,PatternRecognitionAdv.E,�SpeechProcessingAdv.E,Language
InformationProcessingAdv.,ComputerVisionE,VisualInterface,BioinformaticsAdv.(2creditseach)

SeminaronIntelligenceScienceandTechnologyIIE
SeminaronIntelligenceScienceandTechnologyIVE�
(AssignedtoM2,2creditseach)
SeminaronIntelligenceScienceandTechnologyIE,
SeminaronIntelligenceScienceandTechnologyIIIE�
(AssignedtoM1,2creditseach)

BasicSubjects

RecommendedSubjectsProvidedbyOther
CoursesIntroductiontoCognitiveScience,IntroductiontoInformationScience,
(2creditseach)
IntroductiontoBioinformatics st1 Computationallearningtheory
Statisticallearningtheory

GeneralsubjectsprovidedbytheSchool
InterdisciplinarysubjectsofthePerspectivesin
Informatics(Mandatory2creditsormore,upto4
credits)
PerspectivesinInformatics1
PerspectivesinInformatics2�
PerspectivesinInformatics3E PerspectivesinInformatics4E�
PerspectivesinInformatics5E (2creditseach)

Priorto
admission

Bioinformatics

Psychology

PerspectivesinPlatformStudies(2credits),ComputationalScience,Introduction(2credits),ComputationalScience,ExerciseA(1credit),
InformationandIntellectualProperty(2credits),Innovationand
Information(2credits),InformationAnalysisandManagement(2credits),
InformationAnalysisandManagement,Exercise(1credit),Social
ContributionsthroughInformaticsE(1credit)InternshipintheFieldofInformaticsE(1credit)

Computational
ScienceElectricaland
ElectronicEngineeringSeminarsand
exercisesfor
Master’sthesis

(Mandatory8credits)

AdvancedStudyinIntelligenceScienceand
Technology2E
(AssignedtoM2,6credits)
AdvancedStudyinIntelligenceScienceand
Technology1E�
(AssignedtoM1,2credits)
Specific
subjects
provided
bytheschoolRequiredbasicbackgroundofeithersubjectontheleft

Note:�Courses�marked�with�the�letter�“E”�will�be�provided�in�English.

11

Intelligence Science and Technology Course

Brain and Cognitive Sciences
Decoding neural codes
Brain signals can be seen as “codes” that encode ourmental contents. We study methods for modeling brain
functions and representations using information scienceand technology including machine learning and artificial
neural networks. Our approach is based on data-driven
predictive models that relate brain data and mind states
via analysis of massive neural, behavioral, andmultimedia data. Using these models, we aim toIntelligence Science and Technology Course

understand basic principles of neural information
processing, and seek to develop real-life applications
such as brain–machine interfaces that exploit decoded
brain information.
[Yukiyasu Kamitani , Yukiori Goto , Hiroshi Hosokawa , Shingo Maegawa , Yoshihiro Nagano]Psychoinformatics
Toward understanding human cognition and applying it tohuman-machine interface
Human activities in daily life are supported by basic
cognitive functions, such as perception, attention,
memory and high-order executive control. We investigate
the psychological and neuroscientific bases of these
cognitive functions (especially focusing on attention andexecutive function), using psychological experiments,
brain-imaging and computational techniques. We areinterested in human behavior not only in well-controlled
experimental settings in a laboratory, but also in

An experiment examining eye and action coordination

real-world settings such as IT-equipment use and real
car driving. We are also interested in cognitive functions
in a wide range of populations, from healthy young adults
to older or disabled individuals.
[Takatsune Kumada , Ryoichi Nakashima]

Graduate School of Informatics

12

Intelligence Science and Technology Course

OutlineCognitive Informatics
Understanding Human Sensory and Cognitive Information Processing
Our lab studies the computational principle
and the neural mechanisms of human functions
including perception, cognition and language
processing, by means of psychophysics,
computer simulations and EEG recordings. By
comparing human brains with cutting-edge

artificial intelligent systems, we attempt to reveal
the characteristic nature of the human
information processing. We are also interested inleveraging human scientific studies for innovationof information technologies.
[Shin'ya Nishida , Hiroaki Mizuhara , Kiyofumi Miyoshi]

Material perception is one of
our research topics.

Brain researches byEEG measurements

Computational Cognitive Neuroscience (Adjunct Unit)
Neural information processing and computational models
We aim to understand the computational
principles that underlie the way neural systems
realize adaptive behavior and complex minds:e.g., decision-making, reinforcement learning,
motivation and emotion, and social behaviors.
Our approaches are tightly linked to statisticaland information science including machinelearning and neural networks, as well as toquestions in neuroscience and psychology. To
address our questions, we build computational

13and mathematical models, and develop data
analysis methods for linking those behavioral
functions and brain signals throughcomputations. We use human fMRI to examine
neural signals and computations, combined withthose modeling and quantitative methods. We
also seek to use our insights to be applied toconstructing brain-based intelligence.
[RIKEN: Hiroyuki Nakahara , Wataru Sato]Intelligence Science and Technology Course

Computational Intelligence
Formulation of principles to support intelligent information processing
We will formalize the intelligent information
processing seen in human activities and conduct
studies on the basic principles that underlie theseprocesses, as well as realization methods.
Specifically, this will involve education and researchr e l a t i n g t o a r t i fi c i a l i n t e l l i g e n c e i n f o r m a t i o nprocessing such as inductive logic, knowledge
discovery, hypothetical reasoning, and evolvable
computers, using mathematical logic, inference
procedures, machine learning theories andself-organization.
[Akihiro Yamamoto , Nozomi Akashi]

Machine Leaning and Data Structure
Second Principal
Component

Supporting Intelligent Activity
Evidence

Projection

Choices
PCA

Verifying Robustness of NN Models
Minimum Word Replacement Problem

x：an input text
 : a replacement patterny：the correct label of x f : the target NN model  : the size of the replacement pattern Intelligence Science and Technology CourseFirst Principal
Component

Collective Intelligence
Machine Learning That Matters
Our research focus is on the development offundamental technologies for intelligent information
technology, with a focus on machine learning, andtheir real-world applications. While pursuing R&D on
new problems and methods of machine learning anddata analysis, we also tackle various challenges inscience and business. At the same time, we conduct
research on methodologies for solving challenging
problems that are difficult to address by AI or humans
alone, by combining AI with human power.
[Hisashi Kashima , Koh Takeuchi , Han Bao]

⋇ Advanced machine learning algorithms

K1�
K2�
Ky�
A(1)�
�J 1�

X�Jy� Y�A(2)�~=V(y)�
T(y)�

U(y)� +V(1)�
T(y)�

U(1)� +V(2)�
T(2)�

U(y)�

+

⋈ Applied machine learning

V(y)�
T(3)�

U(3)�

A(3)�

J3�
Iy�

I2�

I3�

⋉ Human-in-the-loop AI

Graduate School of Informatics

14

Intelligence Science and Technology Course

OutlineLanguage Media Processing
Making computers that can understand language
Since language lies at the core of human
intellectual activity, the emergence of computerscapable of freely manipulating language will
significantly impact society in a broad variety ofways. With this goal, we are engaged in research
to shed light on how humans utilize language andto enable computers to communicate using
language in human-like ways. We are pursuingboth pure research on language understanding
based on large-scale language models andapplied research for useful real-world purposes,
such as translation, dialogue, and knowledge
structuring.

࠙Speech Recognitionࠚ࠙Structured Knowledgeࠚ

E: I polish it using sandpaper.
I: Howtomoveitwhilepolishing?
E: >ŝŬĞ drawing a circle, …

<replace CHANGE>[PURPOSE] to remove scratches[OBJECT] sandpaper
Q: In what [CONDITION]?

࠙Pre-made Imageࠚ<ƉŽůŝƐŚPOLISH>[INSTRUMENT] sandpaper[MANNER] ůŝŬĞdrawing a circle
Q: How much [TIME]?

DomainExpert

Interviewer[Sadao Kurohashi , Yugo Murawaki , Chenhui Chu , Fei Cheng , Yin Jou Huang]

Speech and Audio Processing
Recognition and understanding of speech, audio and music
Speech communication plays a key role inhuman intelligence. We are studying theintelligent processing of speech, audio and music
as exchanged by human beings for automatic
recognition, understanding and interaction
systems, specifically (1) automatic speechtranscription of meetings and lectures, (2)analysis of audio scenes and music signals
composed of multiple sound sources, and (3)humanoid robots capable of natural interaction
by combining non-verbal information.
[Tatsuya Kawahara , Kazuyoshi Yoshii , Koji Inoue , Keiko Ochi , Eita Nakamura]

15

Intelligence Science and Technology Course

Computer Vision
Making Computers See
Neural substrates dedicated to vision are said
to occupy about 40 percent of the cerebral
cortex. Realizing computer vision as a truly
intelligent perceptual modality is fundamental
for artificial intelligence, and would also inform
our understanding of human visual intelligence.
Towards computational visual intelligence, ourresearch is focused on establishing thet h e o r e t i c a l f o u n d a t i o n s a n d e f fi c i e n t
implementations of computational methods forbetter understanding people, objects and scenes
from their appearance in images and video, as
well as the development of novel computational
imaging systems that can see beyond what wesee.
[Ko Nishino , Shohei Nobuhara , Marc Aurel Kastner]
Intelligence Science and Technology Course

Tracking People in Crowds

Shape from Water

Reflectance and Natural Illumination from a Single Image

Graduate School of Informatics

16

Intelligence Science and Technology Course

OutlineHuman Sensing
Toward human-centered information and machine systems
through visual and embodied interactions
The main purpose of this group is to create
information media and mechanical systems
that support human with respecting theirsubjective actions.
Such media include watching over humans toprovide supports in appropriate way and time,
supplying only deficient force in motions,interfaces that connects humans andcomputers, and assisting human memory.
A common approach in all of these media isthat information or mechanical systems noticehuman’s intention and augment their abilities

for meeting what they want to do.
To realize this support scheme, this group is
engaged in basic researches involving (1)artificial intelligence and knowledge processing,(2) recognition of human physical andphysiological activities using visual, acoustic,and biological measurements, (3) modelinghuman behaviors in motion and communication,
while (4) designing and implementing thehuman-centered supporting media using (5)robotic and artificial agent control technology.

･ Interface of visualizing human intentions
inferred from motions･ Wearable computers for recording andsupporting personal/group experiences･ Motion assist based on behavioral analysis
using electromyography･ Facial expression recognition for healthscience and care of dementia persons･ Joint research using image analysis andAI with other laboratories and research
centers within and outside of Kyoto
University[Yuichi Nakamura , Kazuaki Kondo,
Kei Shimonishi]

Text Media
Speech and Natural Language Processing for Multi-media Archives
Since time immemorial human knowledge
has been recorded as text. The research
activities of this group focus on computerscapable of understanding these texts anddescribing new knowledge. As a basis we arestudying fundamental natural language
processing. And we are studying naturallanguage generation to explain data analysis
and future prediction by computer or todescribe other media such as video and17speech.
Specifically, we deal with real-world media,
including procedural texts such as cooking
recipes with execution videos, academic
knowledge such as history/geography
research, and game/data analysis bycomputers.
We also try to expand human knowledge
based on our research results.
[Shinsuke Mori , Hirotaka Kameko]

Intelligence Science and Technology Course

Biological Information Networks
It for analysis of biological information
We develop algorithms for inferring interactions among genes, proteins and chemical structures,and for analyzing their interactive networks based on mathematical methods. We also develop
algorithms and software tools for other problems in bioinformatics, including sequence analysis andinference of higher-order structures and functions of protein.
[Tatsuya Akutsu , Takeyuki Tamura , Tomoya Mori]Intelligence Science and Technology Course

Analysis of three-dimensional structures and interactions of protein

Graduate School of Informatics

18