Detection of Assessment Patterns in Ordinary Triadic Conversation
1

2

2

Eiki Fujimoto
Yasuyuki Kono
Katsuya Takanashi
3
3Kazuhiro Takeuchi and Hitoshi Isahara
1 The University of Tokyo, 7-3-1 Hongou, Bunkyou-ku, Tokyo, Japan

2 Nara Institute of Science and Technology, 8916-5 Takayama-cho, Ikoma City, Nara, Japan
3 National Institute of Information and Communications Technology, 3-5 Hikaridai, Seika-cho, Kyoto, Japan

Abstract. This article outlines a three-party conversation corpus built by the National Institute of Information andCommunications Technology (Section 2) and introduces three analyses that contribute to the development ofConversational Informatics: The features of turn-taking in three-person conversation (Section 3); assessment
sequential patterns that appeared in the data (Section 4); and shared knowledge and interpersonal relationships
between participants observable from the assessment sequences in triadic conversation (Section 5).

1 Significance of Triadic Conversation Analysis
The importance of triadic conversation analysis has begun to be recognized in various research disciplines in Japan inrecent years. Here are the following attempts of corpus construction, analysis and implementation:
1. Social psychological analysis of “social skills” in triadic conversation [3] [8]: Clarifying the perceived relationships
between nonverbal behaviors and expressive dimensions or impressions and rapports.
2. Social agents mediating network communication between two users and the analysis of their social influence
power [7] [12].
3. Multimodal humanoid robots coping with “who to whom” problem in human-human conversation [11].
4. Building interaction corpus [18] and capturing the dynamics of participation framework [9] from ubiquitous or
wearable sensor information in poster presentation environment.
5. Systematics of turn-taking and participation roles in triadic conversation [4] [5]: Fine-grained video analysis ofnonverbal behaviors like gaze, gesture and body posture.

2 NICT ThreeThree-Party
Party Conversation Corpus
2.1 Recording Design
The National Institute of Information and Communications Technology (NICT) has developed and analyzed athree-party conversation corpus.
The number of subjects was 45 (15 groups). Each group performed 2 experiments and 30 sets of dialogue wererecorded. The average duration required for each experiment was approximately 20 minutes.
The speech sound was recorded via DAT and the video image of nonverbal information was via mini-DV. Three
DAT decks were utilized: Deck 1 recorded the speech of subjects A+B; D2 recorded that of B+C; and D3 recorded that of
C+A on the L & R tracks, respectively. The images of the three subjects were synthesized through the 4-divider and wasrecorded with one DV deck. The recording by the DV was not for detailed analysis, but for overview of all sounds andimages in a dialogue.
Each subject entered an individual soundproof compartment to clearly record each speech sound on an independent
track. Therefore, subjects were not placed in a face-to-face situation. We installed two small monitors according to thedirection of the input sound so that the members could receive feedback as naturally as possible: A, for example, heard
the voices of B and C through the L and R headphones, respectively and could see an image of B in the left monitor andof C in the right (Fig. 1). This recording environment is like the video telephone conferencing by three persons, where
subjects cannot use eye contact though can obtain speech and nonverbal information. However, the timing ofturn-taking seems to be smooth enough.
All subjects were university students. Each group consists of 3 members of the same sex (besides one group). The
following 4 combinations were prepared: (a) subject A, B and C are friends with each other; (b) A and B are friends + A
and C are friends; (c) only A and B are friends; and (d) everyone is a stranger with each other. Each group went throughthe two tasks consecutively. Experiment 1 was a “photo task” [1], in which subjects are seeking an answer for each of
three questions like in Fig. 2, on which all of them can agree. Experiment 2 was a “free topic conversation,” in which
subjects could use an optional topic list.

Subject A’
A’s compartment
Headphone A
B

B

C

C

Microphone A

Camera A

Fig.
Fig. 1. Soundproof compartment

fig. 2. Photo task: These two figures area. brothersb. husband and wife for four years
c. strangers

2.2 Theoretical FociOne of the focuses of our analysis was to discover various kinds of linguistic devices that speakers use for audience design [2]of their utterances that are utilized for turn-taking [15] and assigning participation roles [6]. In the conversation including
three or more participants, there is no security or obligation for a current non-speaker to become the next speaker, anddetermination of the next speaker therefore becomes an indispensable concern for participants. The audience design islinguistic and nonverbal devices by which the next speaker is determined and at the same time the utterance can beunderstood for all participants. Points we especially tried to clarify for the audience design are:(1) What kind of linguistic devices enable participants to find who, or what kind of participant, should or can become arecipient or the next speaker, besides nonverbal cues like gazes?
(2) What kind of shared knowledge and interpersonal relationships between participants are detectable from a variety ofaudience designs and turn-allocation dependent on them?
For (2) above, it is not task-oriented dialogue data in a toy-world but only natural free topical conversation that enables
us to uncover ordinary ways of using world knowledge in conversation.

3 Characteristics
Characteristics of TurnTurn-Taking in Triadic Conversation
Table 1 shows a list of resources1 for next speaker determination that resulted from the corpus analysis.
Table 1 Resources for speaker determination2
I. Nonverbal devices: gaze, etc.
II. Linguistic devices:- Explicit: vocatives, etc.
- Tacit: references to participants (personal pronouns or names, their grammatical functions are not vocatives but nominatives,
possessives, etc.), choices of addressee honorific/non-honorific particles, selection of specific vocabularies (+III), discourse
markers (+IV), etc.
III. Use of shared knowledge: Mention to shared episodes, information requests from or giving to someone who does not know them.
IV. Use of sequential organization: Second parts of adjacent pairs, continuous questions to the preceding answerer, etc.

Unlike explicit devices like vocatives, tacit resources are not specified for speaker selection but have some propositionalor modal contents. This means that each of them cannot be used for speaker selection in a context-independent manner, but
their functions are implicit triggers for activating shared knowledge or having participants orient to local sequential
organization of conversation. Therefore, it is important to clarify their nature of dependency on the context of knowledge orsequential organization.
Excerpt
Excerpt 1[C questionsAand B: “Have you decided your seminar yet?” ]1 B:A, mada desu ne. [→C] Haitte kara dayo ne. [→A]Ah, not yet.

Will it be determined after the new academic year begins?

1 The term "resource" is suitable both for avoiding strong commitment to the speakers’intentions and for emphasizing the contingency of next speaker determination.
2 This list is similar to that of Lerner [10], in which use of “shared knowledge” and “sequential organization” are called use of “social” and “sequential” identities,

respectively.

Excerpt
Excerpt 2
1 B: Kataku dou natta? [→B]
(How did Kataku go?)2A:Are wa na: ichibu. [→A]
(It was only by some members.)
3 B:Ah.
(Oh.)
4A: Camp de sa: adana ga kettei shiten. [→C] (My nickname was determined during the camp.)
5 C:Ah.
(Oh.)

In Excerpt 1, the former half of the utterance is an answer to C’s question and the latter is a confirmation to 1B who is A’sfriend and belongs to the same university and class as A. These selections of addressees are expressed by the distinction
between honorific post-verbal particle “desu ne” and non-honorific “dayo ne”. In Excerpt 2, the use of the word Kataku in 1B
implies that it is the one sharing the episode about Kataku with B, who is selected as the next speaker. After this exchange,
4A explains this episode to C, who has no knowledge of it yet.

4 Assessment Sequences in Free Topical Conversation
One of the most important natures of information included in conversation is the fusion of objective world knowledge andattitudes or opinions of participants toward things, persons or events. Any attempts to develop technology of conversational
informatics must be concerned with how to utilize and activate such subjective information as attitudes or opinions. Thissection proposes that sequential analysis of assessment pattern appearing in conversation will contribute to this issue.
Though one of the urgent problems for conversational informatics is how to extract subjective information like attitudes
or opinions of participants from conversational data in an effective and reliable manner, it is not easy to do this because
these kinds of information are mingled tightly with more objective propositional information, and often omitted from thesurface forms of utterances. For instance, it could be said that "USJ is interesting" is an expression of the speaker's
subjective judgment, while "USJ is in Osaka" is objective information. However, as clearly seen from the following excerpts,
even "USJ is in Osaka" and "Cockroaches can fly" will not always transmit only objective information but can sometimes
imply the subjective judgment or taste of speakers as well.
ExcerpExcerpt
cerpt 3
1A: USJ tte Osaka shinai desu yone?
2 B: Hai, totemo benri desuyo.

(Is USJ in Osaka?)(Yes, it is very convenient.)

Excerpt
Excerpt 4
1 C: Gokiburi tte tobu yan na, yappa.
2 B: Iya: kowai kowai. Iya ne.
3A: Tobu tobu.
4 C: Mou sore ga kowai nen.
5A: Kowai.

(Cockroaches can fly, can’t they?)(Yeah, it is very very scary. I hate them.)(They can fly.)(That terrifies me.)(Surely.)

In general, some expressions like “The score of the test was 28 points” and “We visited a hot spring after the conference”,
in addition to 1A in Excerpt 3 and 1C in Excerpt 4, regularly evoke subjective or emotional responses from the recipients. In
other words, it is one of the most important features of conversation data that some implicit aspects of prior utterances
become overt by listeners’ responses to them. This nature of conversation is most properly grasped by the doctrine“responses display how the speakers understand prior utterances” proposed by conversation analysts, called “sequential
analysis” [14], which appears most clearly in the formulation of adjacency pairs [16] [17]. From this viewpoint, Pomeranz
analyzed assessment sequences in conversation and found that a first assessment makes relevant the occurrence of thesecond assessment, expressing agreement or disagreement to the first [13]. Based on her formulation, we developed the“assessment sequence schema” (Fig. 3).
A typical assessment sequence consists of four elements: <Introduction of Target>, <1st Assessment>, <2nd
Assessment>, and <Negotiation Pattern>. The procedures and notices for analysts to identify them are the following:
Agreement Demanding Relation. As mentioned above, the targeted parts are only those which include the pairs of <1stAssessment> and <2nd Assessment>. The relations between these elements are called “agreement-demanding” because
the most typical relevant response to the <1st Assessment> is agreement/disagreement.
Commentary Relation. The <1st Assessment> also has “commentary” relations with things or events taken up in the<Introduction of Target>. There are two kinds of <Introduction of Target> according to where and how they are introduced
into the conversation: In pattern A, the <Introduction of Target> elements are introduced by the speaker of the <1stAssessment> in the same utterance, and in pattern B, things or events that have already been the topics of conversation are
retrospectively recognized as the <Introduction of Target> by virtue of the occurrence of a <1st Assessment>, where those
who introduced these things or events for the first time are not necessarily the speakers of the <1st Assessment>; these
introductions can be done through several utterances of several participants. In either case, the criterion of identification is
the occurrence of the <2nd Assessment>.

TURN
0
1

SPEAKER
B
A

2B

PATTERN (A)<Introduction ofTarget>&<1stAssessment><2ndAssessment>

PATTERN (B)<Introduction ofTarget>
COMMENTARY
AGREEMENTDEMANDING<1stAssessment><2ndAssessment><Negotiation Patterns> (agreement / disagreement, praise / accusation / teasing, etc.)

Fig. 3. Assessment sequence schema
Negotiation Pattern. There appear various kinds of <Negotiation Pattern>, according to whether the <2nd Assessment> isagreement or disagreement, whether the type of attitude in the <1st Assessment> is praise, accusation or teasing, etc., andwhat kinds of things or events the <Introduction of Target> includes, etc. For instance, when the <2nd Assessment> isdisagreement, the speaker of the <1st Assessment> often either tries to justify his or her prior judgment or modifies it inpursuit of agreement [13].
Excerpt
Excerpt 5: The number 6 is big3
1A: Doko koukou?
Where is your high school?

2 C: Koukou? Koukou wa Edogawa-Higashi tte iu kore mata Kore *c1 mata miner na un.
High school? My high school is called Edogawa-Higashi. It is not so famous.
*c1Asou nanya:
I see.
4 C: Kouritsu, Kouritsu *b2*c2 :Public, public.
5 B:*b2 He *a1:Oh.
6A:
*c2*a1 He:, K-dai kita hito otta?
Oh:, was there anyone who passed K university?
7 C: K-dai kita hito roku nin kana *c3 :There were six students who passed K university.
⇒ 8 A:*c3 Roku nin *a2 mo orun?
Six persons is a lot.
9 B:
*a2 He *b3 :Oh:.
→10 C:*b3 Un, roku nin, ooi n kana *c4 :Yeah, six persons, is it big number?
11A:*c4 Ooi yo *c5 :It is big.
12 C:
*c5 Nanka
Well,= watashi sukunai*c6 toka omottotta kedo.
I was thinking that it was small.
13A:*c6 Iya, uchi atashi dake damo:n.
No, it was only I who passed from my high school.

3A:

In Excerpt 5, the fact “It was six students who passed K university”, counted as the <Introduction of Target>, appears inlines 1-7. 8A is the <1st Assessment>, in which A declares a subjective judgment that 6 is a big number. This assessment is
not agreed upon by the corresponding <2nd Assessment> in 10C. Thus, the fact that six persons passed K university from ahigh school can be judged as either a big or small number, depending on the knowledge and experiences possessed by eachof the participants. The sequence after 10C, which is engendered by this disagreement, should be regarded as the<Negotiation Pattern>, in which, especially, 13A tries to justify her argument.
Of course, all implicit aspects of participants’ attitudes cannot always be identified by this methodology, and it might
therefore seem to be too strict to restrain our analytic focus only on the parts where both a <1st Assessment> and <2nd
Assessment> do occur. However, as pointed out above, it is difficult for analysts to securely extract subjective information
like participants’ attitudes toward things or events, which are an integrated part of conversational information, andtherefore methods to do this must successfully provide a way to identify evidence that some attitude information is in factused by participants in the conversation. The sequential analysis of assessment patterns proposed above enables analysts
to treat only those subjective information items that become overt (“publicized”) by and for participants themselves in theparticular conversational context.

3 Transcriptional notation: In "*xn", “*” shows the starting points of overlapped speech, “a-c” means between whom they are (a: between participantsAand B; b: B

and C; c: C andA); and “x” are serial numbers in each except for a, b and c, respectively. ":" shows that the vowels are prolonged. "⇒"and "→" at the head of line points<1stAssessment> and <2ndAssessment>, respectively. “=” shows that the line is continuing from the prior line.

5 Shared Knowledge and InterpersonalInterpersonal Relationships between Participants
The combination of two kinds of analyses introduced above, the analysis of turn-taking in triadic conversation in Section 3
and of assessment sequence patterns in Section 4, enables us to discover shared knowledge and interpersonal relationships
between participants. The excerpts for detailed analysis are those in which either speaker or recipient of the <1stAssessment> is the one who has not become either speaker or focal recipient in the exchange immediately before the <1stAssessment>. Here, they are classified as “from or to the side-participant”, correspondingly.
Excerpt
Excerpt 6: Even though you're in the 4th grade[C is explaining the advantage of a part-time job in convenience store]
1 C:A, nanka betsuni isshuukan ni nikai demo ii shi, shiken kikan toka mo meccha raku nan desu yo.
It's no problem if I can work only twice a week, and it's flexible too even during exams.

2 B:Ah *a1 :Oh:*a1 Sore wa ii desu wa *c1 ne.
That’s good.
4 C:*c1 Un.
Yeah.

3A:

5A: Kateikyoushi nante kekkou shiken no jiki mo kasanatte kuru n desu *c2 yo.
6 C:

In case of tutors, exam periods often come simultaneously.
*c2 Hu:n.
Oh.

7A: Demo shiken mae ni wa ma: fudan okane moratte irushi, zettai ika nai to ikkenai tte yuu *c3 ka
But during exams, because I'm ordinarily paid a salary, I must be sure to go.
*c3 U:nOh.
9 B: Tsugi no hi gogaku ga futatsu arou *c4 to *a2 mo.
Even if I have two exams of foreign language classes the next day.
10 C:
*c4 Ha *b1 hahaha.
(laughter)
11 B:*b1*a2 Hahaha hahhahha.
(laughter)
12A: Report ga arou *c5 tomo ikanai *c6 to ikenai
Or even if I have a deadline for my report, I have to go.
13 C:*c5 U:*c6 u:
mhm:
mhm
14A: To iu *a3 no ga arimasu kara ne.
I think so.
15 B:*a3 Hahha.
(laughter)

8 C:

16 C: U:n

Oh:

17A: *a4 Kukuku(laughter)⇒18 B: *a4 Kimi yon kaisei nanoni mada gogaku totteru kara da *a5 me nan *b2 desu yo, *a6 hahahahaha.
You're still taking a foreign language class even though you're in the 4th grade.
So, it’s your own sake, (laughter)*a5 Hu *c6 hu.
*c7*a6 Kukuku.
(laughter)(laughter)*c6*b2 Hahaha *c7 haha.
(laughter)

→19A:
20 C:

Excerpt 6 illustrates the "from side-participant" type. As a response to C’s argument about the advantage of thepart-time job in a convenience store up to 1C, 3-9A compares it with the case of a tutor, and conversation is going on between
A and C. The <1st Assessment> is 18B, noting that (1) B has not become a speaker before it, and (2) B’s assessment resorts
to shared knowledge between only A and B that A is taking a foreign language class despite in the 4th grade. Thus, one who
issues the <1st Assessment> often utilizes an existing interpersonal relationship and shared knowledge between him andthe target person.
Excerpt 7: We are only new acquaintances
1 C: Nande ore shin-san shiri attan darou
How did I get acquainted with you, Ms. Shin?

2A: E?

Pardon?

3 C: Ma, doko de ore no kao mitakke?
Well, where did you meet me?

4A: Hu, ichiban saisho ni atta no wa: test no saishuubi no fuyou-kan ja nai?
The first time was at the Confederate-Rose Pavilion on the final day of exams, wasn’t it?

5 C: *c1 E?

Really?

6A: *c1A, chigau?
Oh, that's not right?

7 C: Sou nano?
Surely?

8A:Atashi hajimete mita no wa, a test chuu kana test *c2 chuu.
The first time I saw you was, oh it might have been during the exam, during the exam.

9 C:
10A: Fuyou-kan, fuyou- *c3 kan
11 C:*c2 Doko nanoka wakaranakute sa.
I can’t remember.

The Confederate-Rose Pavilion, Confederate-Rose Pavilion.
*c3 Un.
I see.

12 C: De koe kakerarete dare daka jitsuwa wakatte nakatta

But actually, when I was called on at that time, I couldn’t find who you were.

⇒13A: So atashi tachi sugoi: tomodachi reki asaku *c4 te→14 C:

So, we are only new acquaintances.

→15 B:
16A:
17 B:*c4*b1Asai.
Recently.
*b1A, sou na *a1 n?
Oh, so.
*a1shichigatsu no owari yone, *a2 shiriatta.
It was at the end of July when we first met, wasn’t it?
*a2Amada kotoshi nan?
Oh, it was this year.

Excerpt 7 is the “to side-participant” type. Participants A and C are jointly recalling the situation of their first encounter.
It does not seem that the conversation between A and C is not co-telling to B though B has not known this episode.
Therefore, the <1st Assessment> in 13A turns the conversation to B in the form of resuming it. It is appropriate for A to
introduce the relationship between A and C to B, because only A is an acquaintance of both B and C. Both 14C and 15B are
two different kinds of <2nd Assessment> responding to 13A. 14C shows agreement from the standpoint of the one who has
shared the episode with Awhile B in 15 shows himself as a person who hears it for the first time.
Thus, shared knowledge and interpersonal relationships between participants appear clearly in the assessment
sequences in the "from side-participant" and "to side-participant" types.

6 Conclusion
In this article, we have insisted on the significance of analysis of triadic conversation to develop conversational informatics,
and proposed a method of analysis of turn-taking in triadic conversation, assessment sequence patterns as the exchange
and negotiation process of attitudes and opinions of participants, and shared knowledge and interpersonal relationships
made explicit through assessment sequences between three participants.

References.
1.Archer, D.: How to ExpandYour S.I.Q. M. Evans and Company, Inc. (1980)
2. Clark, H. H. & Carlson, T. B.: Hearers and speech acts. Language, 58 (1982) 332-373. Also in Clark, H. H.: Arenas of Language Use.
University of Chicago Press ＆ Center for the Study of Language ＆ Information (1992) 205-247
3. Daibo, I., Goto. M. & Miyagi, H.: The perceived relationship between the conversation style and the nonverbal cues in a triadic conversation.
28th International Congress of Psychology (ICP2004). (2004)
4. Den, Y.: Towards a Science of Conversation. Technical Report of the Institute of Electronics, Information and Communication Engineers.
(2004) (in Japanese)
5. Enomoto, M.:An analysis of nonverbal behavior affecting participation-role taking. JSAI SIG-SLUD-A301-02 (2003) 25-30 (in Japanese)
6. Goffman, E.: Forms of Talk. University of Pennsylvania Press (1981)
7. Isbister, K., Nakanishi, H., Ishida, T. & Nass, C.: Helper agent: Designing an assistant for human-human interaction in a virtual meeting
space. International Conference on Human Factors in Computing Systems (CHI2000) (2000) 57-64
8. Iso, Y., Kimura, M. Sakuragi, A. & Daibo, I.: The effects of nonverbal behaviors on impression formation and rapport in a triadic
communication. 28th International Congress of Psychology (ICP2004). (2004)
9. Katagiri, Y., Bono, M. & Suzuki, N.: Capturing conversational participation in a ubiquitous sensor environment. Proceedings of Pervasive
2004 Workshop on Memory and Sharing of Experiences (2004)
10. Lerner,G.H. Selecting next speaker: The context -sensitive operation of a context-free organization. Language in Society.32 (2003) 177-201
11. Matsusaka, Y., Fujie, S. & Kobayashi, T.: Modeling of conversational strategy for the robot participating in the group conversation.
Eurospeech 2001 (2001)
12. Nakanishi, H., Nakazawa, S., Ishida, T., Takanashi, K. & Isbister, K.: Can software agents influence human relations?: Balance theory inagent-mediated communities. International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS2003) (2003)
717-724
13. Pomeranz, A.: Agreeing and disagreeing with assessments: some features of preferred / dispreferred turn shapes. In Atkinson, J. M. &Heritage, J. (eds.) Structures of SocialAction: Studies in ConversationAnalysis. Cambridge University Press (1984) 57-101
14. Psathas, G. : ConversationAnalysis: The Study of Talk-in-Interaction. Sage Publications (1995)
15. Sacks, H., Schegloff, E. A. ＆ Jefferson, G.: A simplest systematics for organization of turn-taking for conversation. Language. 50(4) (1974)
696-735
16. Schegloff, E.A.: Sequencing in conversational openings.AmericanAnthropologist. 70(6) (1968) 1075-1095
17. Schegloff, E.A. & Sacks, H.: Opening up closings. Semiotica. 8 (1973) 289-327
18. Sumi, Y., Ito, S., Matsuguchi, T., Fels, S. & Mase, K.: Collaborative capturing and interpretation of interactions. Proceedings of Pervasive
2004 Workshop on Memory and Sharing of Experiences. (2004)