Social Intelligence Design and Communicative Intelligence
for Knowledgeable Community
Toyoaki Nishida
The University of Tokyo, Bunkyo-ku, Tokyo, Japannishida@kc.t.u-tokyo.ac.jp
Abstract
In the network age, knowledge evolves at every moment. Community plays a critical role in the sense that itprovides people with a competence of dealing with the complex and dynamic nature of the information world.
In order for a community to function effectively, an effective circulation of information need to be established.
In this talk, I argue for communicative knowledge media that allow for accumulating knowledge in a form
usable in conversational context and show how it is applied to community support systems. I show a research
agenda towards communicative knowledge media.

1. Introduction
Computer-mediated communication over theubiquitous network has drastically changed the natureof knowledge. Knowledge evolves at every moment
by incorporating new thoughts arising in the history.
People or organizations cannot be competent unlessthey continually evolve their knowledge byincorporating new information.
Community, a collection of people who build andexploit the common context, plays a critical role in
the network age in the sense that it provides people
with a competence of dealing with the complex anddynamic nature of the information world.
A
community serves as a ground for creating both tacitand explicit knowledge (Fig. 1). Community ofpractice, a group of people who share common work
practice, develops informal networks of relationships
cutting across organizational boundaries in search forbetter cooperation [1]. Those social networks not
only enable the organization to accomplish tasks
faster or better but also create novel knowledge.
In order for a community to function effectively,
an effective circulation of information need to beestablished.
Unfortunately, there are numerous
factors that may hinder information circulation in acommunity.
Social Intelligence Design is a new research area
aimed at designing a new computer-mediatedcommunication systems for supporting community
knowledge process and human networks based on
in-depth understanding of intelligence as a social
phenomenon, which have been called "social
intelligence" [2].
Conventionally, social intelligence has been
discussed in the context of an individual's ability, e.g.,an ability to be able to manage relationship with otherFig. 1. The role of community in the network society.
agents and act wisely in a situation governed by an
implicit or explicit set of shared rules, based on an
ability of monitoring and understanding other agents'
mental state. It is distinguished from other kinds ofintelligence such as problem solving intelligence(ability to solve logically complex problems) oremotional intelligence (ability to monitor one's own
and others' emotions and to use the information to
guide one's thinking and actions).
Alternatively, social intelligence might be
attributed to a collection of agents and defined as anability to manage complexity and learn fromexperiences as a function of the design of social
structure. This view emphasizes the role of social
rules or culture that constrain the way individual
agents behave. We might attribute a good social
behavior to a good social structure and consider that agood social structure affords the members of thecommunity to learn from each other.
Issues that have been discussed in the context ofSocial Intelligence Design involve:1-methods of establishing the social context
embodied conversational agents for social
intelligence- collaboration design- public discourse- theoretical aspects of social intelligence design- evaluations of social intelligence- applications.
In this paper, I present a suit of communication
support tools that facilitate information circulation in
a community by providing people with sufficient
support of organizing their knowledge in acommunicative fashion. Our tools not only provide
the user a handy means of producing an engaging
presentation but also help the user accumulatematerials using communicative knowledge media
from which a presentation can be readily produced.
In addition, our tools integrate personal andcommunity support. Particular emphasis is placedon the use of conversational media as a means forcommunicating knowledge.
I describe how the above consideration isdesigned and implemented as a framework ofaugmenting the knowledge process of communities,
featuring embodied conversational agents driven byknowledge cards and the knowledge channel for
strategic control of information stream. Then, I
briefly overview an ongoing research project aiming
at developing intelligent media technology forsupporting content production with communicativeknowledge media.

Fig. 2. A framework of communicative knowledge
media.

Fig.3. Landscape as a spatial representation ofknowledge. This figure illustrates an "AI-centeredview" of the research fields. It may lookconsiderably differently when other subjects arechosen as the center.

2. Communicative Knowledge Media
Communicative knowledge medium provides ameans for representing knowledge for use incommunication in the personal or community
context.

Conversation enables an effective presentation ofknowledge in an interactive fashion. Conversations
not only provide the most natural communicationmeans but also facilitate the functions of knowledge
creations, as discussed in the next subsection.

2.1 Components of Communicative Knowledge
Media

2.2 Characteristics
knowledge medium

A communicative knowledge medium comprises
knowledge documents, information landscape, andconversations (Fig. 2).
Documents are the most popular means ofcommunicating ideas over the Internet. Multimedia
hyper documents provide a powerful means forencoding knowledge. WWW enables us to easily
disseminate document world-wide.
The story
structure, which is a causal understanding of theworld is a key to make each document intelligible, assuggested by Schank [3].
Landscapes allow for spatially representing
knowledge. A landscape enables the user to visually
grasp the global nature of knowledge, explore theinformation space, and accommodate new
information at an appropriate place (Fig. 3).

Among others, conversation brings about heuristic
production of stories from different points of view,
tacit-explicit knowledge conversion, and entrainment
to the subject.
(1) Heuristic production of stories from different
points of view
Conversations allow a heuristic composition ofstories in a style agreed by a community. In eachconversation, attempts are made to structure
fragments of knowledge pieces, either tacitly or
explicitly possessed by participants into a larger
pieces of knowledge in search for possible
viewpoints, connections and clusters to encompass
the given collection of pieces.
Conversation is an improvisational social process2ofConversationasa

in which agents from different viewpoints collaborate
and negotiate from time to time in search for thesocial and personal satisfaction.
The flow ofconversation changes dynamically according tomicro-macro interaction of participants' thought.
(2) Tacit-explicit knowledge conversion
Messages communicated with conversations are on
the boundary of tacit and explicit knowledge.
Conversations give a good opportunity andmotivation to externalize tacit knowledge.
Indiscussions, each participant tries to figure out theidea into expressions that have an intended effect,
such as propose, counter propose, support, challenge,
negate, etc, in the conversational discourse. In
debate, participants try to be the most expressive, insearch for new points to win the game.
On the other hand, people can make conversation
or discussion to examine the given subject fromdifferent points of view. Exercises of applying
knowledge may allow people to find and socialize
how pieces of knowledge are associated with eachother and identify a critical cluster of knowledge thatis most relevant to the discussion (Fig. 4).
(3) Entrainment to the subject
In order to understand the given subject deeply, one
should be involved in the subject and capture theproblem as her/his own. Conversations tend toentrain the audience by providing enough sympathy,
facilitating to simulate the role of a playing character(Fig. 5).
Conversations will facilitate reality sharing, forthey reflect how people articulate the world.
Conversations will support social awareness, for they
provide a handy means for communicating references
to other people and events that the speaker consider
worth communicating. It will help context sharing
in a community, for conversations often encourage
people to explicitly think about the issues in acommunity that are tacitly shared in the community
but not written in anywhere.
Even thoughconversations may not directly address dispute
resolution or supporting large scale arguments, they
contribute to establishing a firm ground for mutual
understanding with a dense collection of daily
thoughts.
In the meanwhile, conversations have limitations.
Logically precise arguments or visual information are
not well communicated by spoken language alone.
Furthermore, conversations are volatile. Both thecontent and the context may be lost quite easily, for itis difficult to reproduce the same mental state as thesituation where the conversation was made. Even
though conversations are recorded, it is pretty hard to
recover the implications of conversations after theconversation session or correcting the transcript istime consuming, tedious task.
These weakness
should be compensated by advanced information andcommunication technology if conversation is to be

Fig. 4. Conversation as interface between tacitand explicit knowledge.

Fig. 5. Conversation brings about sympathetic
understanding of the subject.
used as a primary means of communicatingknowledge.
Recent advent of intelligent media technology
makes it feasible to develop a conversation-centricknowledge medium that takes advantage of
conversations while compensate for limitations.
2.3 Conversational Knowledge Process
The conversational knowledge process is a term wecoined to refer to a collective activity for knowledge
creation, management, and application whereconversational communication is used as a primarymeans of interaction among participating agents. It
refers to the collection of community-wide
concurrent processes of conversations that contributeto community knowledge creation.
The
conversational knowledge process serves as a main
thrust for accumulation and use of very high density
information about a given subject which is invaluable
for understanding not only issues related to thesubject but also people in the community.
In the conversational knowledge process,co-evolution of conversation, documents, andlandscape take place, as shown in Fig. 6.
Conversation making is a process for adapting
story pieces into utterances in each scene of aconversation. Various types of transformation areneeded to tailor each piece of information so that it
can be appropriate in the context.
Story making is a process of composing
well-structured stories from a given sequence of3

Fig. 6. The essence of conversational knowledge
process.

Fig. 8. A general framework of the knowledge
channel approach.
(2) Stories
Story is a sequence of knowledge cards, representing
a complete story with a plot. In a story, preceding
and succeeding knowledge cards give a discourse toeach knowledge card.
(3) Embodied Conversational Agents
Embodied conversational agent (ECA) is a device forplaying stories consisting of knowledge cards. An
ECA employs paralinguistic and nonlinguistic
conversational means to control information flow in aconversation. In normal settings, each ECA stands
for an existing real person.
Accordingly, theconversation partners can recognize whom the ECA
talking in front of them stands for. This provides
proper background information for interpretingmessages from the ECA. Meanwhile, we mightwell employ anonymous ECAs in certain situations
where we want to facilitate communication.
(4) Knowledge channel
Knowledge channel is a mechanism for strategicallycontrolling the flow of knowledge cards. This willcontribute to designing long-term interaction bycontrolling the flow of knowledge cards.
Knowledge channel is a conduit connecting a server
and a client where knowledge card flow is controlled
strategically. Both the knowledge card server andthe knowledge card client can control the knowledge
card flow using the knowledge channel control
policy.
(5) Memory Manifold
Memory manifold serves as a container that allows
for spatially accumulating memory items for
producing landscapes.
(6) Knowledge Server.
A knowledge card server stores knowledge cards andprovides them for knowledge card clients.

Fig. 7. Evolution of landscape.
utterances. It involves seeking for an optimal story
structure that can incorporate conversation fragments,changing the tacit conversational knowledge into
explicit representation, transforming components of
information representation into those with a uniform
grain size, and so on.
Story and conversation making are activities ofcreating stories or conversational utterances byarticulating information resulting from exploring one
or more landscape. Landscape making is an activityof producing landscapes from story or conversationarchives.
Landscapes will evolve as new
information is curved into the archive (Fig. 7).

3.
Computational
Model
Conversational Knowledge Processof

I present a computational model of conversational
knowledge process using several ideas including
quantized
conversation,
stories,
embodied
conversational agents, knowledge channel, memory
manifold, and knowledge server.
(1) Quantized Conversation
We handle articulated conversation, called quantized
conversation and use a knowledge card to represent
the content of quantized conversation.
Eachknowledge card is relatively self-contained packageof tacit and explicit knowledge, enabling one to
implement a rather robust conversational system
without introducing a complex discourse generation
algorithm.

This approach approximates the conversational
knowledge process with existing technologies, and it
can be extended by advanced technologies such a
dialogue engine [4].
Tacit knowledge can beembedded into each knowledge card such as visual
images or sound, even though it cannot be encoded
into knowledge representation. The current natural4

Fig. 9. EgoChat: normal presentation mode by a single agent (left),
Question mode where the user's question is read by an avatar (right).
language processing techniques such as documentclassification or summarization may be applied toturn the collection of knowledge cards into a
structured knowledge.

Table 1. Four strategies for the knowledge channel policy

4. EgoChat as a Conversational
Knowledge Process Facilitator
EgoChat is a system for enabling an elaborate
asynchronous communication among community
members
[5].
It
is
based
on
the
talking-virtualized-egos metaphor.
A virtualized
ego is a conversational agent that talks on behalf of
the user. Each virtualized ego stores and maintains
the user's personal memory as a collection ofknowledge cards and presents them on behalf of theuser at appropriate situations. It helps the user not
only develop her/his personal memory but also better
understand other members' interest, belief, opinion,
knowledge, and way of thinking, which is valuable
for building mutual understanding.
We use apowerful dialogue engine that permits a virtualized
ego to answer questions by searching for the best
match from a potentially large list ofquestion-answering pairs prepared in advance (Fig.
9).
EgoChat provides a couple of unique features.
First, it integrates personal and interpersonal
knowledge life cycles. At earlier stages of thelifecycle when knowledge is not well captured, theuser might want to describe her/his idea as one ormore knowledge card and have the virtualized ego
present them for personal review. After a while
when the knowledge becomes clear and evident to theuser, s/he might want to delegate her/his virtualized
ego to her/his colleagues to present the idea and ask
for critiques. The automatic question answering
facility of EgoChat will encourage the audience to
ask questions or give comments. The virtualized
ego can reproduce the question-answering session
with the audience so the owner can review theinteractions. It will highly motivate the owner tosupply more knowledge or improve existingknowledge for better competence.

Fig. 10. Example of dynamic program table.
Second, EgoChat allows for describing a channel
policy that is used to define the control strategies ofthe sender and the receiver. Four types of strategies
are identified depending on whether the strategy is
about the order of programs in a single program
stream or about the way multiple program streams are
mixed, and whether the program scheduling is staticor dynamic, as shown in Table 1. The skeleton of
the actual flow structure of knowledge cards for agiven pair of the sender and receiver is determined byresolving constraints of their channel policies. It
can be visually presented to the user by the dynamic
program table, as shown in Fig. 10.
5

Table 2. Behavior selection rules.

Fig. 11. The framework of SPOC.

Fig. 12. RISA-CAST subsystem for generating ECA
presentation.

5. SPOCSPOC (Stream-oriented Public Opinion Channel) [6]
was designed to serve as an integrated
communication environment for a multi-faceted
community activities such as risk communication.
SPOC supports reality sharing based on visual
images, knowledge sharing, and community-wide
discussion for decision-making (Fig. 11).
The SPOC uses the POC system [7] as aknowledge sharing engine and extends it in several
respects.
First, SPOC allows for sophisticatedpresentation mechanism using an embodied
conversational agent. Second, it supports video
editing and presentation.
Third, it allows acommunity discussion support system to be pluggedin.
(1) The presentation mechanism using ECAThe
RISA-CAST
component
allowsforautomaticallygenerating
from
plain
texts
presentations featured with an animated agent(Fig.12). It selects and generates appropriate gesturesand facial expressions for a human-like animatedagent according to linguistic information in the text.
An agent animation system called RISA can draw
animations of natural human behaviors on web-based
applications. The ECA subsystem called CAST(The Conversational Agent System for neTwork
applications) generates agent’s nonverbal behavior
automatically according to rules based on theories ofhuman communication and observation of humancommunicating behaviors (Table 2). The basic
algorithm is extended that synchronization of verbal

Fig. 13. Highlighting Synchronization of Verbal
and Nonverbal aspects of presentation.

Fig. 14. The overview of the video editor.
and nonverbal behaviors may be enhanced [8].
(2) Video editing and presentation tools
The SPOC system allows for video presentation,using streamed video clips and camera works, such aszoom and pan, applied to graphic images. In orderto help a casual users to create and review the video
contents, the following tools are incorporated.
1. tools for collecting and accumulating materials(e.g. pictures, graphics and videos)
2. tools for creating video contents by assemblingcomponents of various kinds (Fig. 14)
6

Fig. 15. The conceptual framework of research into conversational content.
3. a viewer for converting a knowledge card into a
synchronized presentation with the ECA andmultimedia materials.
(3) Discussion support system
SPOC allows CRANES (Coordinator for Rational
Arguments through Nested Substantiation) [9] to beplugged in, which supports discussions in thecommunity for consensus making. CRANES uses a
probabilistic approach to selectively identify "stirring
arguments" that stimulates the entire group processes.
Two random-tree models are developed for the
purpose of defining an indicator of noteworthiness.
In the perfect random tree, the probability for a newcomment to be associated with a certain existing nodeis assumed to be the same among all existing nodes.
A biased random tree model is a more elaboratemodel that inflates the probability of the last node’s
producing a new child compared to other existingnodes. It reflects an intuition that when readers of a
BBS consider which comment they respond to, they
are more likely to choose the last node in the wholethread or those with some specific features.
CRANES was applied to a community conflict over aregeneration scheme in London, to result in an
augmented social construction of the conflict withmultiple dimensions.

between People" sponsored by Japan Society for the
Promotionof
Science(JSPS)
addresses
communicative intelligence as a generalization of theidea of communicative knowledge media. The term"communicative intelligence" reflects the idea ofcommunicative intelligence implies a view that
communication plays a critical role in both individualand collective intelligence. This project aims at
establishing a technology for creating and utilizing
conversational contents.
It encompasses theenvironmental medium technology for embedding
computational services in the environment, theembodied conversational agent technology for having
computers interact with people in a social context,
and the communication model of conversations thatserves as a theoretical basis of system development(Fig. 15).
The environmental medium technology is basedon the "making-computers-invisible" approach,aiming at embedding computers into the everyday
environment so that they can assist people in pursuit
for their goals without enforcing them to pay special
attention to computer operations.
It addresses
cooperative intelligent activity sensors, automatic
analysis of nonverbal communication withhigh-resolution scene analysis, personalization ofenvironment medium, and intelligent editing ofaudio-visual streams.
A typical subject on the environmental medium
technology is semi-automated tagging to videos using
real time computer vision [10].
Basically, an
intelligent video production system computes the best

5. Towards Communicative Intelligence
A five-year research project "Intelligent Media
Technology for Supporting Natural Communication7

matching between the given index-scenario and video
stream captured by camera to produce an annotatedvideo.
The basic algorithm is scenario-speech
matching where DP is used to calculate the best
match between the given scenario and the sentencesfrom a speech recognition system. The system alsouses computer vision techniques to match the speechclues and motion cues for capturing basic behaviors
such as pointing, holding out, manipulation, andillustration.
Namely, the motion clue of pointing
and holding-out is detected when the speaker's arm isstretched beyond the threshold. In contrast, there isno fixed pattern of manipulation or illustration.
Currently, they are detected in a heuristic fashion bychecking if the both hands are on or above the desk.
In the vision system, information from various typesof cameras is integrated to figure out the mostplausible interpretation. For example, hand regionis the intersection of the in-volume region, themoving region, the skin-temperature region and theskin-color region, while held-object region is theintersection of in-volume region, moving region, but
not in the hand region.
The embodied conversational agent technology isbased on the "making-computers-social-actors"
approach, aiming at developing embodied
conversational agents that can make conversation
with people by following various social conventions.
Embodied conversational agents are characterized as
an interface between people and conversational
content. We attempt at developing virtualized egos
that can talk on behalf of the user, socially intelligent
robots that facilitate conversations among people, andintelligent conversational content management
systems by combining natural language processing
and computer vision.
Research on the communication model ofconversations aims at establishing a theoreticalfoundation of conversational content. We attempt at
developing a unified model of verbal and nonverbal
communication, a conversation corpus that
accumulates conversation records with annotations toverbal and nonverbal events, and a method ofanalyzing conversation behaviors. A typical subject
in this vein is analysis of the interactions of CMC.
In [11], a model is proposed by applying thestructural equation modeling to log data in 5748
message threads taken from 2channel with eightindices. It is suggested that the dynamism may becaused by three latent factors called specificexpression, discussion type, and chitchat type.

content production and entertainment computing thatbrings about user involvement.

References[1] Etienne Wenger. Communities of Practice -Learning Meaning, and Identity, Cambridge
University Press, 1998.
[2] T. Nishida. Social Intelligence Design for Web
Intelligence, Special Issue on Web Intelligence,
IEEE Computer, Vol. 35, No. 11, pp. 37-41,
November, 2002.
[3] Roger Schank. Tell me a story: a new look at real
and artificial memory, John Brockman Associates,
Inc. 1990.
[4] Yoji Kiyota, Sadao Kurohashi, and Fuyuko Kido:"Dialog Navigator": A Question Answering
System based on Large Text Knowledge Base, In
Proceedings of 19th COLING (COLING02), pp.
460-466, 2002.
[5] H. Kubota, T. Nishida: Channel Design for
Strategic Knowledge Interaction, in Proceedings
KES 2003, pp. 1037-1043, 2003.
[6] Y. I. Nakano, T. Murayama, D. Kawahara, S.
Kurohashi,
T.
Nishida:Embodied
Conversational Agents for Presenting IntellectualMultimedia Contents, in Proceedings KES 2003,
pp. 1030-1036, 2003.
[7] Fukuhara, T., Chikama, M., and Nishida, T.:
Supporting an Experiment of a Community
Support System: Community Analysis andMaintenance Functions in the Public Opinion
Channel, in: Proceedings of the FirstInternational Conference on Communities andTechnologies(C&T2003), Kluwer Academic, pp.
347-367, 2003.
[8] T. Murayama, Y. Nakano, and T. Nishida.
Participatory Broadcasting System Using
Interface Agent and Multimedia, presented atSocialIntelligence
DesignInternational
Conference (SID2003), 6-th – 8th July 2003,
Royal Holloway University of London, Egham,
Surrey U.K.
[9] M. Horita and N. Iwahashi. On Discovery of
Stirring Arguments: A Random-Tree Approachto Collaborative Argumentation Support,
presented at Social Intelligence Design
International Conference (SID2003), 6-th – 8thJuly 2003, Royal Holloway University of London,
Egham, Surrey U.K.
[10] M.Ozeki, M.Itoh, Y.Nakamura, Y.Ohta: Tracking
Hands and Objects for an Intelligent Video
Production System, Proc. IAPR 16th Int'l
Conference on Pattern Recognition, Vol.III,
pp.1011-1015, 2002.
[11] Naohiro Matsumura, Asako Miura, Yasufumi
Shibanai, Yukio Ohsawa, and Toyoaki Nishida:
The Dynamism of 2channel, presented at Social
Intelligence Design International Conference(SID2003), 6-th – 8th July 2003, Royal Holloway
University of London, Egham, Surrey U.K.

6. Conclusion
In this paper, I presented information systems that
help communities of people build and evolve
collective knowledge. The future work include
introducing advanced functions to universally support
8