Artiﬁcial Intelligence
Associate Professor AKAISHI, Mina

KiSs:An Information Visualization System
That Makes Thinking Fun

Interviewer: Katsuaki Tanaka, Research Assistant̶̶Professor Akaishi, I wonder if you could tell us what you're̶̶Your system doesn't extract "points," does it? In

currently most interested in. I heard that the "Interdisciplinary

information retrieval, it's usually "points" that are generated,

Memory Research Meetings" have all been quite interesting.

don't you think?

Generally speaking, these days I'm interested in highlighting

I started out by paying attention to "points," but currently

the relationships that exist between all sorts of things. It's

I'm looking at how "points" are tied together. At first, I

possible to look at a single thing in a variety of ways, andbegan researching information searching in databases, andrelationships change according to one's perspective. To begin

I proposed two methods: "content designation search" andwith, I'm trying to bring to light the structures within written"context designation search." The former is a method where

texts. For example, if you look at a historical chronology

you extract information by specifying a piece of what you'reusing KiSs, the tool that I created, I can show you a new way

looking for as the search conditions. For example, if you'reof looking at it as something other than a simple chart. Whenlooking for books written by Goethe, you put "author is

I showed this to Professor Mikuriya at one of our campus-

Goethe" as a search condition, you can ﬁnd the books. On thewide, "Interdisciplinary Memory Research Meetings", heother hand, "context designation search" is where the searchcame up with one idea after another about how someone

conditions are taken from the peripheral information. In otherwriting a historical novel could use it to incorporate different

words, it's a method where you look for a piece of information

episodes into their story. This is all to say that, by using ausing the conditions and relationships in which it's placed ascomputer to freely change perspectives and show things fromclues. For example, let's say that you lost your day plannervarious angles, I think that it should be possible to draw out

yesterday. If you have contextual information?such as, thethe knowledge that lies dormant within someone's mind.

last time you took your planner out was when you conﬁrmed

Normally, when we speak of information retrieval, it's mainly

the schedule at the end of your meeting, and you did that ina question of how best to extract necessary information fromsuch-and-such a place?it may be possible to find the target

a computer, but I think that by interacting with computers,

object, in this case the planner that you left at your meetingit should be possible to draw out various knowledge andyesterday. So you see, it's possible to retrieve the planner,

relationships from humans as well. Doesn't it seem thateven if you don't have any information about the item itself,

we should be able to give shape to and clarify the implicit

based on the information surrounding it. In other words, theknowledge that an individual is not directly aware of, andconnecting relationship between two "points" is frequently

based on that, go on to think of one thing after another,signiﬁcant.

causing a kind of chain reaction? In this way, in the field of
human memory, I'm hoping that the KiSs system can be used
as a trigger to help people recall the memories that they

̶̶It's possible to bring out such relationships in subjects

had forgotten. For example, it occurred to me that Professor

other than texts, isn't it?

Mikuriya might be able to use it as a support system while heis interviewing politicians.

Currently, for starters, I'm focusing on extracting the1relationships between words that appear in texts. From

where people could obtain even deeper understanding.

there, once I've developed a method for extracting general
structures, I want to study a wider variety of subjects andWhen I visited a museum on Crete in Greece a while ago, theapply my research to other problems. For example, if we think

chairs for guards just happened to catch my eye. The backsof the people, places, and things that appear in scenes fromof the chairs were decorated with carvings of an ox and an

everyday life as a gathering of objects and make a model, thenaxe, but they looked like chairs that the local people usedusing the same method, I believe that it will be possible toin their daily lives. I was wondering, "I'm seeing oxen andanalyze the dynamic changes those objects undergo and theaxes all over the place, but what do they mean?" Since therelationships between them.

chairs weren't part of an exhibit, however, there wasn't any
information about them. When I asked, it turned out that they
were actually symbols of Minoan religion, and I was struck by

̶̶Is it better to keep as much information as possible in

the way that the traces of Minoan civilization were still a partorder to be able to conduct subsequent analysis from variousof people's lives there. What occurred to me at the time waspoints of view?

that, since the information that museum visitors are looking
for varies from person to person, museums are not always

Rather than concerning about the amount of information, I

providing all the information that people want. If there werethink it's really more a question of what kinds of stories anda technique, like the one I just mentioned, for extracting

relationships you want to retrieve. The storage capacity ofrelationships, then by showing people the information andrecording media has increased, and it's possible to store

stories that lie behind the things they're interested in, at thelots of information in computers. But the amount of stored

same time that they would obtain understanding, they would

information and what you get from them are quite different

gain new interests. That's the kind of thing I want to achieve.

matters. After all, if you can't retrieve the information thatmeets your goals from a massive pool of information, then it's
not really useful.

̶̶When we look at different aspects of a certain thing orview it in a different context, its structure changes, doesn't it?

Interacting with Computers:
E x tr ac ting Infor mation f rom Humans andRetrieving Information from Computers

That's exactly what context is. Take a school-teacher, forexample. He might be called "teacher" while he's at school,"Dad" at home, and "Mister" on the street, but in each of thosesituations, the information related to that person changes.
It changes depending on the circumstances, and it's also aquestion of who's observing. I want to deal well with all these̶̶Is it possible, in addition to retrieving information fromdifferent aspects.

previous data stores, to come up with something like a policy
for what types of data should be stored in the future?
̶̶Even individual units might change depending on our
I'm not sure about a policy. Really, the only choice we have isperspective.

to record the things that we think are necessary.
If we think of our schoolteacher as an object, then at a given
With things like knowledge and culture, new things pile up

time the unit might be "person," but at another time we mighton the foundation of prior investigations and works in thebe talking at the level of bodily organs.

past. You look at something from a certain point of view,
and in response to that you make your own edits, or you
accumulate new work that incorporate your own thoughts.

̶̶I wonder what would happen if we were to input the text

When new elements are added, you will be able to adoptof this interview into your system. Unlike a written text, more

new perspectives, and that knowledge itself develops andthan one person is talking, right? Could the computer still

accumulates. Take math, for example?modern math didn't

identify a structure?well, it might not be a single structure,

just suddenly appear, it resulted from the gradual piling up of

maybe a multiplex structure?out of the whole?

achievements. Lots of things were rejected as well. What tokeep and what to discard, that's a really big problem.

Yes, as long as we speak using a well-developed structure.

̶̶You don't seem to set a limit to the main fields that the̶̶Would it be consistent with what we intended?

tool can be applied, do you?
I think there would be both consistencies and inconsistencies.
There's no single "main field. " What it boils down to is

Out of all the words, the system identifies relationshipsthat we want to discover interesting relationships. If there

according to certain rules, so naturally it will show them inwere a technique for highlighting the various relationships

a state where some of the information in the original text issurrounding a certain object, we could take the things thatmissing. Some parts, a person could understand it and say,people are interested in and show them, "This is the story"Ah, yes yes, that's right," and I think that other parts would

behind that, this is the background knowledge", then thatcome out where that wouldn't happen. But when you see

would bring out new interests and lead to a chain reaction

these results, if you're able to realize "Oh, there's such-and-2

such a relationship there!" then I think that might provide anfor showing data related to laborers, but as soon as we uniﬁedopportunity to lead to new ideas.

the two systems, the relationships between the two things
became apparent. Actually, at that time my goal was not toshow relationships, but rather I was doing research in order

Information Search
Designed to Inspire New Ideas:
Transitioning from Tools
for Searching toward Tools for Thinking

to suggest a system framework and software that could be
widely used.

̶̶What do you ﬁnd most interesting about the work you're
doing now?

̶̶You joined RCAST in spring 2004. What kind of research

The setup behind the system that I currently use to pick out

w e r e y o u d o i n g before when you were at Hokkaido

the relationships between words is extremely simple, but it

University? You gave a talk about how, in the past, the priceseems possible to use this simple thing in more areas thanof rice would go up whenever a feudal lord's procession

I previously imagined. Even though it's simple, it is widely

would pass by, didn't you?

applicable?I ﬁnd that very interesting. From an engineering
standpoint, I'm interested in how widely it can be used,

That was a research on data visualization. Various types ofwhether it is simple, or what its essential qualities are. At thehistorical materials, such as chronologies, ukiyo-e prints, andsame time, I'm also interested in what it can be applied to.

the price of rice were stored up in a database. In order to use
differently structured data in a uniﬁed way, I made it possible
to input historical materials into, and then get output from,

̶̶Up until now, we've mainly been talking about extracting

the database as "meme media" that Professor Yuzuru Tanaka

information, but your system can probably be used to createat Hokkaido University has been advocating. As for the

structures as well.

visualization system, I mapped the attributes of the data on
a two-dimensional axis and made the chronological changes

This is related to what I was saying earlier about what wein those attribute values visible. This made them possible

retrieve from stored information, but when it comes down

to change dynamically as movement of media objects.

to what is and what's not interesting, of course it's up toSpecifically, when we altered the time, the coordinates ofindividual judgment, but I think the key is whether or nota troop's location changed, so we could see the troop's

there's a story involved. I'm also interested in how to deﬁne

movement on a map of Japan. Superimposed over that, wethings like narrativity or the nature of a "story. " I think thatshowed the price of rice and the numbers of laborers, horses,

if it's possible to use my current technique to create stories

and amulets in each region. As a result, it became patently

or structures, then extremely simple and well-designed newclear that in each region that the troops passed through,structures will be produced.

following their movements, the price of rice went up, thenumber of conscripted laborers and horses suddenly rose,and after the troops passed by, a mysterious phenomenon

̶̶It's been about eight months since you joined RCAST. Has

called ofuda-kudari occurred, where amulets fell from theanything about your research mission changed?

sky. I did this research when I was at Shizuoka University. I
was collaborating with a professor of history. Apparently,

When I was at Hokkaido University, I was working on

historians had previously conjectured from historical materials

proposals for software architecture and frameworks,

and documents and put forward a number of doctrines.

suggesting how they could be used for the widest range of

But by presenting how, based on the data, some otherpurposes. Since I came here, I reevaluated my work from theinformation would change in related ways, they could very

perspective of Professor Hori's concept of "Liquidization andclearly support their theories. Even if the original data was theCrystallization of Knowledge," and I became aware of a variety

same that the historians had been using, through a differentof advances and potentials for this technology. Then, by

way of presenting the data, I thought I might be able torethinking the material that I'd been researching until then, I

support them in verifying their theories and coming up withcame up with lots of new things to think about. In the artiﬁcial

new hypotheses.

intelligence lab, each member has certain research topics towork on for each theme, and I'm trying to approach them toﬁnd out if there is an all-purpose, generic way to solve them.

̶̶If I had to put what you do into a word or two, I would saythat you're involved in information visualization, but you deal
with things with somewhat complex structures, don't you?

̶̶What is the research that you've been doing with Professor
Nicolas Spyratos about?

As a framework for using differently structured data in a
unified way, I'm using "meme media." Using media editingThat is also research about how to deal with context. Basedtools like "IntelligentBox*1," and "IntelligentPad*2", we canon the context model that Professor Spyratos has proposed,

use lots of different kinds of data in a standardized way, andwe store information and then do research on techniques

rapidly accumulate both tools and data. So in the previous

for searching information based on context. For example, aexample, there were things that couldn't be seen when thecertain person has a number of different names, depending

system for showing rice prices was separate from the systemon context.

He's called Mr. X, but he also has some3

nicknames, and so on. In other words, different names arerich environment.

given to the same object, depending on context. Also, there
is information about what kind of names the other objects
have in the same context. Using that information, we're doing

̶̶The "Interdisciplinary Memory Research Meeting" that we

research on methods for identifying relationships that aren't

were talking about in the beginning also has scholars fromclearly visible. For example, when we search for a relationshiplots of different ﬁelds participating. When I think about how

between Takeshi Kitano and Akira Kurosawa, we soon realize

you all manage to stay on the same wavelength, even thoughthat they're related because they're both movie directors.

your ﬁelds are different, I ﬁnd it hard to believe!

But what we try to discover through our system is like thefact that the person in charge of costume design for Takeshi

Since everyone speaks about the same topics from their own

Kitano's movie Zatoichi was Akira Kurosawa's eldest daughter,

perspective, it's actually extremely stimulating. Personally,

Kazuko Kurosawa?a relationship that is dependent on context

during the time that I've been participating in this group, I've

surrounding the two objects.

found a variety of new themes. I believe meetings like thiswill play a role in opening up new ﬁelds of research.

̶̶When you say that you "discover" this, is that because allof the information is heavily annotated?

̶̶Well, even if everyone is satisfied during the meetings,
saying "Ah, yes, that's right.", isn't it also the case that

Both a "name" and "reference information" are attached tosometimes what they're actually thinking is totally different

each object. We search for relationships by following up on

from person to person?

names and reference information. Of course, relationshipsthat everyone knows appear, as well as relationships that

I'm sure they're different. But I think it's better that way.

people aren't aware of.

Well, this is my ﬁrst year at RCAST, and this has been such an
enjoyable time for me. I mean, it's fun to talk with different

We thought it might be possible to apply this technique toscholars, and I enjoy having discussions with the students andmuseum information, so we were introduced to the people

other people as well. From now on, I want to try lots of newworking in the Greek research center, FORTH. Zeus, forthings, without being constrained by common sense. And not

example, who appears in Greek myths, turns into a whitejust in my research.

bull and golden rain, transforms himself into a variety ofthings, and does all sorts of other things. We proposed thema framework that visualizes the data automatically that was

̶̶It must be fun to be able to hold your lab's weekly study

inputted in the database based on context modeling. But

meetings in a variety of places too.

with this method, only the relationships in the database
could be visualized, so I was a little dissatisfied. Essentially,

While I was at Hokkaido University, I collaborated worked

relationships that everyone knows well were identified, butwith Professor Sunaga at Tama Art University. At Hokkaido,

unknown or hidden relationships that are hard to see were

we tended to hole ourselves up in the lab, staring at ournot expressed. That made me think that I'd like to focus a little

computers, but the team from Tama considered experience as

more on this thing called a "relationship" in my research, andan important part of the process when working on informationthat led to what I'm working on now.

design. As a result, they placed great emphasis on using theirbodies and being active. So in order to think about what itmeans to design information, they actually made sketches on

Crossing Disciplinary Borders,
Synthesizing Humanities and Sciences:
A Research Environment that
Stimulates the Cycle of Converging
and Diverging Ideas

the Hokkaido University campus and gathered the informationthat would be expressed on the computers by themselves.
Regardless of whether they were art students or students from
Hokkaido University, everyone gathered information fromtheir own perspective. After that, we broke up into groups,
had discussions, and experienced what it's like to design asystem. As a result, I came to believe that, even though it's
important to make our ideas converge through thorough̶̶How do you ﬁnd the research environment at RCAST?

investigation, on the other hand, it's also important to let ourthoughts diverge using some element of play. You can ﬁnd

The first thing that I found interesting after I came to RCAST

pleasure in thinking both ways, of course. The systems that I

was that there are scholars from so many different fields

create are not supposed to rob human beings of the pleasures

here. Lots of new ideas actually occur to me on a daily basisof thinking, but rather, I think that they will make thinking

by having discussions with people who think differently.

more enjoyable for people.

Sometimes I ask Professor Hori or Professor Nakakoji, "Do you
have a second?" and I get them to listen to some idea, give

(October 21, 2004)

me comments or discuss it with me. Occasionally there arequestions that I can't answer or times when I can't put my own
thoughts together, but by thinking about those things later, I
come up with answers in my own way. To have scholars who
are active in a wide range of ﬁelds around me, that in itself is a4

Recent publications
Journal Articles
Mina Akaishi, Nicolas Spyratos, Makoto Ohigashi, Yuzuru
Tanaka: A system for Tracing Implicit Relationships in a Web ofContexts, 14th European-Japanese Conference on Information
Modelling and Knowledge Bases (EJC2004), Vol.1 pp.64-74,
Skovde, Sweden June1-4, 2004
Mina Akaishi and Yoshihiro Okada: Time-Tunnel: Visual
Analysis Tool for Time-Series Numerical Data and its Aspects
as Multimedia Presentation Tool, 8th International Conferenceon Information Visualization (IV04), pp.456-464, London,
England, 14-16 July 2004
Mina Akaishi, Ken Satoh and Yuzuru Tanaka: An Associative
Information Retrieval based on the Dependency of Term
Co-occurrence, 7th International Conference, DS2004,
pp.195-206, Padova, Italy, October 2-5, 2004

Links
RCAST
http://www.rcast.u-tokyo.ac.jp
Artiﬁcial Intelligence Laboratory
http://www.ai.rcast.u-tokyo.ac.jp/index.html*1:IntelligentBox:
http://km.meme.hokudai.ac.jp/cgi-bin/wiki.cgi/English?page=IntelligentBox*2:IntelligentPad:
http://km.meme.hokudai.ac.jp/cgi-bin/wiki.cgi/English?page=IntelligentPad5