Influence Relation Estimation based on
Lexical Entrainment in Conversation
Tomoharu Iwata1 , Shinji Watanabe1

Abstract
In conversations, people tend to mimic their companions’ behavior depending on their level of trust. This phenomenon is known as entrainment. We
propose a probabilistic model for estimating inﬂuences among speakers fromconversation data with multiple people by modeling lexical entrainment. The
proposed model estimates the word use as a function of weighted sum of earlier word use of other speakers. The weights represent inﬂuences between
speakers. The inﬂuences can be eﬃciently estimated by using the expectation maximization (EM) algorithm. We also develop its online inference
procedures for sequentially modeling the dynamics of inﬂuence relations. Experiments on two meeting data sets in Japanese and in English demonstrate
the eﬀectiveness of the proposed method.
Keywords: conversation analysis, inﬂuence, latent variable model,
entrainment
1. Introduction
In conversations, people tend to mimic such aspects of their companions’ behavior as postures [1], facial expressions [2], lexicon [3, 4], syntax [5],acoustic, prosodic [6] and amplitude [7]. This phenomenon is known in theliterature as entrainment, accommodation, adaptation, or alignment [8]. Entrainment is said to indicate that people are trusting, accommodating andempathic [1, 9].
Email address: iwata.tomoharu@lab.ntt.co.jp (Tomoharu Iwata)
URL: http://www.kecl.ntt.co.jp/as/members/iwata/index.htm (Tomoharu
Iwata)
1
NTT Communication Science Laboratories

Preprint submitted to Speech Communication

February 20, 2012

This paper focuses on the entrainment of lexicon in polylogue, or how people are inﬂuenced by their companions in terms of word use in conversation
with multiple speakers. The degree to which a person exerts an inﬂuence andis inﬂuenced by others varies from speaker to speaker. A powerful person is
likely to be mimicked by others, and a passive person might often be accommodating to others. The inﬂuences also diﬀer between pairs depending on
their level of trust. For example, Alice might use words spoken by Bob, but
not words spoken by Charlie. The inﬂuences therefore have an asymmetricnature.
We propose a simple and eﬀective probabilistic model for estimating inﬂuence relations among speakers from conversation data with multiple people [10]. With the proposed model, we assume that a speaker’s word use(word distribution) depends on the preceding word use of other speakers
as well as his/her own preceding word use and the general word distribution. We estimate the strength of inﬂuence for each pair of speakers using
the expectation maximization (EM) algorithm [11]. We also develop online
inference procedures for sequentially modeling the dynamics of inﬂuence relations. Note that the proposed model estimates inﬂuences on the word useof a speaker from the word use of other speakers.
The remainder of this paper is organized as follows. In Section 2, we
brieﬂy review related work. In Section 3, we formulate our proposed probabilistic model for inﬂuence estimation. In Section 4, we describe the inference
procedures for the proposed model. In Section 5, we extend the inference with
online version for eﬃciency. In Section 6, we demonstrate the eﬀectiveness ofthe proposed method by analyzing two meeting data sets in Japanese and in
English. Finally, we present concluding remarks and a discussion of future
work in Section 7.
2. Related WorkIn recent years, a huge amount of conversation data have been accumulated due to the improvement of recoding devices and automatic speech
recognition systems, and there has been great interest in the analysis of conversation [12, 13]. For example, [4] investigated the correlation between task
success and similarity of word use, and [9] analyzed the relationship betweensocial game results and word repetition. However, they focused on dyadic
conversations, and did not consider the asymmetric nature of inﬂuences. On
the other hand, we deal with the conversation of multiple people, in which2

inﬂuence and sensitiveness are assumed to depend on the pair of speakers. In
addition, since the proposed method is a probabilistic generative model for
conversations, we can eﬃciently estimate inferences in a principled statisticalframework, and use it for a language model of the conversation.
The proposed method is related to speaker role recognition [14, 15], inwhich each speaker is automatically classiﬁed into a role category, because
inﬂuences depend on the speaker’s role. In the role recognition, roles arepredeﬁned, and classiﬁers are trained using labeled data. On the other hand,the proposed method directly estimate inﬂuences, and it does not requirelabeled data.
A number of language models for conversation have been proposed [16,17]. However, they do not aim to estimate inﬂuences between speakers. By
using online social network data, [18] proposed a probabilistic model for estimating inﬂuences in online behavior, but this is not applied to conversationdata.
3. Proposed Model
With the proposed model, we assume that the word use of a speakerchanges depending on the preceding word use of other speakers as well as
the own preceding word use. Figure 1 shows the dynamics of the word useof each speaker in the proposed model, where λ represents the weight of theinﬂuence.
Let w = {w1 , · · · , wt , · · ·} be a word sequence of a polylogue, where wt
represents the tth word, and let s = {s1 , · · · , st , · · ·} be its speaker sequence,
where st indicates the speaker of the tth word. Here, wt ∈ {1, · · · , W } andst ∈ {1, · · · , M }, where W is the vocabulary size, and M is the number ofparticipants.
In the proposed model, we assume that a speaker’s word use depends on
the preceding word use of other speakers. The preceding word use of speakerm at position t can be modeled as follows:
N (t − 1, τ, w, m) + β,
0w0 N (t − 1, τ, w , m) + βW

t−1, m) = ∑PC (w|wt−τ

(1)

where τ represents the period of the inﬂuence, β is a smoothing parameter,
and N (t−1, τ, w, m) represents the count of word w that is spoken by speakerm from t − τ to t − 1. This probability is proportional to the number of times
3

preceding word use
from time t-τ to t-1

word use
at time t

λ11

speaker 1
word

word

λ12

speaker 2

λ22
λ13

word

word

λ32

speaker 3
λ33

word

word

Figure 1: The word use of a speaker changes depending on the preceding word use of otherspeakers as well as the own preceding word use.

word w is used by speaker m in the preceding period τ . The smoothing
parameter β is introduced to avoid the zero probability problem.
The word use of speaker n at position t is then modeled by a mixture of
the preceding word use of the participants as follows:t−1
P (w|wt−τ, n) =M∑t−1
λnm PC (w|wt−τ, m) + (1 −

m=1

M
∑

λnm )PG (w),

(2)

m=1

where λnm represents the inﬂuence of speaker m on speaker n, 0 ≤ λnm ≤ 1,
and PG (w) is the general word distribution, which does not depend on thepreceding conversation. The general word distribution can be obtained byusing other corpora. Speaker m who inﬂuences the word distribution of
speaker n is not observed, and m is a latent variable.
This proposed model is an extension of cache models [19] for multi-speaker
conversations. The cache-based language model integrates short-term patterns of word use into the word distribution by means of a cache component.
With the proposed model, we build speaker-speciﬁc cache components, andset diﬀerent inﬂuences among pairs of speakers. The proposed model can beeasily extended for n-gram language models by taking a n-gram sequence as
input instead of unigram sequence w. The other approaches to incorporate
sequential information include unigram rescaling [20] and linear interpolationof the proposed model with n-gram language models.
44. Inference
We estimate the inﬂuences λnm based on maximum posterior (MAP)
estimation. For simplicity, we rewrite the proposed word distribution in (2)as follows:
M
∑
t−1
t−1
P (w|wt−τ , n) =λnm PC (w|wt−τ
, m),
(3)
m=0

∑
t−1
where we set PC (w|wt−τ
, m = 0) ≡ PG (w), λn0 ≡ 1 − M
m=1 λnm , in which∑M
λnm ≥ 0 and
m=0 λnm = 1. With this notation, the logarithm of theTposterior probability of parameters given the conversation data {wt=1, sTt=1 },which is to be maximized, is calculated as follows,
L=

T
∑
t=1

log

M
∑

t−1
, m) +
λst m PC (wt |wt−τ

m=0

M
∑

log P (λn |α),

(4)

n=1

where T is the current position, and the second term represents the priorprobability for parameters λn = {λnm }M
m=0 . We use the following Dirichletprior with hyperparameter α:M∑

log P (λn |α) =α log λnm ,

(5)

m=0

because it is conjugate to multinomial parameters λn . The inference is mademore robust by introducing the priors.
We can eﬃciently maximize the posterior (4) by using the EM algorithm [11]. The EM algorithm is commonly used for the inference of mixture
models. Because the proposed model is a mixture model, the standard EM
algorithm can be used for the proposed model. The conditional expectationof the complete-data log likelihood with priors is represented as follows:
Q=

T ∑
M
∑

t−1
P (m|t) log λst m PC (wt |wt−τ, m) +

t=1 m=0M ∑M∑

α log λnm ,(6)

n=1 m=0

where P (m|t) is the posterior probability of selecting speaker m given thetth word. The derivation of Q is described in Appendix Appendix A. It
indicates the probability that the tth word is inﬂuenced by speaker m. In
the E-step, we compute the probability according to the Bayes rule:
P (m|t) = ∑M

t−1
, m)
λst m PC (wt |wt−τt−10
m0 =0 λst m0 PC (wt |wt−τ , m )

5

,

(7)

t−1
where λnm is the prior term, and PC (wt |wt−τ, m) is the likelihood term. In
the M-step, we obtain∑ the next estimate of inﬂuences λn by maximizing Q
w.r.t. λn subject to M
m=0 λnm = 1:
∑T
I(st = n)P (m|t) + α
λnm = ∑M ∑Tt=1,(8)0 |t) + α(M + 1)
I(s
=
n)P
(m
0
t
m =0
t=1

where I(A) represents an indicator function, i.e. I(A) = 1 if A is true,
I(A) = 0 otherwise. Note that the speaker dependent preceding word dist−1
tribution PC (wt |wt−τ, m) can be calculated by (1) independent of estimatingt−1
parameters Λ = {λn }Mn=1 . The general word distribution PC (wt |wt−τ , m = 0)
is assumed to be given in advance. By iterating the E-step and the M-step
until convergence, we obtain a local optimum solution for inﬂuences Λ.
5. Online Inference
In the previous section, we described the inference procedures using all
the data in a conversation session. However, in some real applications, wemight need to estimate inﬂuences in the middle of the conversation session.
Then, we develop online inference procedures for the proposed model forsequentially estimating inﬂuences based on the online EM algorithm [21,22]. With the online inference, the proposed model is sequentially updated
using newly obtained data. This means that past conversation data are notrequired to make the inference, and we can reduce the memory requirement
as well as computational time.
With the online inference, suﬃcient statistics are updated from previous
values using newly obtained data and the current estimated model as follows,(t)λ̄(t+1)
nm = γ λ̄nm + I(st = n)P (m|t),

(9)

(t)

where λ̄nm is a suﬃcient statistic at t and γ is the forgetting factor. P (m|t)(t)
can be calculated by (7). Using the suﬃcient statistics λ̄nm , we can obtain
the estimate of inﬂuence as follows,
(t)

λ̄nm
λ(t)
.
nm = ∑M(t)λ̄00
m =0 nm(10)

The forgetting factor γ represents how likely inﬂuences change over time. By
controlling γ, we can model the dynamics of inﬂuence relations. As an initial(0)value for the suﬃcient statistic, we can use λ̄nm = α.
6We can also update the preceding word use in an online fashion. The
preceding word use of speaker m at position t of (1) can be rewritten as
follows,
N (t − 1, τ, w, m) + β
t−1
PC (w|wt−τ
.
(11), m) = ∑0
w0 N (t − 1, τ, w , m) + βW
This number N (t, τ, w, m) can be calculated using the preceding number
N (t − 1, τ, w, m) and data at t as follows,
N (t, τ, w, m) = N (t−1, τ, w, m)+I(wt = w∧st = m)−I(wt−τ = w∧st−τ = m).
(12)
By updating the model using (10) and (12), we can estimate inﬂuence relations sequentially.
6. Experimental Results
We demonstrate the importance of modeling inﬂuence in conversation
data with experiments. First, we visualize the inﬂuence relations inferred by
the proposed method. Second, we analyze inﬂuential and sensitive speakers,
and show it can be used to identify a chairperson in the conversation. Third,
we compare the proposed method with other methods quantitatively using
perplexity, which represents the performance of predicting word use. We
also analyze perplexities with diﬀerent numbers of training utterances, with
diﬀerent lengths of eﬀective period, and with diﬀerent forgetting factors inonline inference.
We evaluated the proposed method using the following two real meetingtranscription data sets: NTT [23] and RT07 [24]. The NTT data set consistsof six sessions in Japanese. In each meeting, one participant mainly talked
about a technical topic using slides, and the other participants asked questions spontaneously. RT07 data set is an English corpus of eight conference
room meetings, which consists of primarily goal-oriented, decision-making exercises and can vary from moderated meetings to group consensus-building
meetings. Four sites contributed two meeting recordings for eight total meetings. For both of the data sets, speakers in a session are diﬀerent from othersessions. Table 1 shows a summary of the NTT and RT07 data sets, andincludes the number of sessions, vocabulary size, and the minimum and maximum number of speakers and utterances for a session. With the proposedmodel, we used α = 1 and β = 10−8 for the hyperparameters, and modeled the preceding word use by using all preceding utterances in the session,
7

Table 1: Summary of NTT and RT07 meeting data sets.

#session
NTT
RT07

6
8

#speakers
min max
4
4
46#utterance
min max
560
918
337
749

#vocabulary
2,098
3,113

or τ = ∞. The general word distribution PG (w) is learned by using othersessions in each data set.
We estimated the inﬂuences between speakers using the proposed model.
Figure 2 shows the result. Each node represents a speaker, and the widthof the arrow represents the strength of the inﬂuence, where only inﬂuenceswith λnm ≥ 0.1 are shown. The self inﬂuence is generally strong, whichindicates that the word use depends strongly on the speaker’s own preceding
word use. This is an intuitive result. There are also many inﬂuences between
speakers. Some speakers are inﬂuential, e.g. speaker 1 in Session 7 in RT07,
and some speakers are sensitive to other speakers, e.g. speaker 2 in Session
6 in RT07. Most of the inﬂuences are asymmetric. This result indicates that
it is important to model the direction of the inﬂuences.
In all the NTT sessions, speaker 4 was appointed chairperson, and therefore, speaker 4 was inﬂuential and not sensitive. The result obtained withthe proposed model reveals the inﬂuential and non-sensitive characteristics
of speaker 4 as shown in Figure 2 (a), where there are more than three arrows
from speaker 4 in ﬁve out of six sessions, and there is no arrow pointing to
speaker 4 from others in all the sessions. Figure 3 shows its quantitative
analysis. The inﬂuence to other speakers of speaker∑ n in Figures 3 (a) is calculated by summing up the inﬂuences to others m6=n λmn . In the same way,the ∑inﬂuence from other speakers of speaker n in Figures 3 (b) is calculated
by m6=n λnm . The inﬂuences to others of speaker 4 are the highest in ﬁve
sessions, and the inﬂuences from others of speaker 4 are the lowest in all six
sessions. This result represents inﬂuential and non-sensitive characteristics
of speaker 4. In this way, the proposed model can use conversation data to
analyze the inﬂuences between speakers.
For a quantitative evaluation, we compared the following six models:• CC has a common cache that is shared by all speakers, and a common
parameter that control the inﬂuence of the common cache. The word

8

1

2

1

2

1

2

3

4

3

4

3

4

Session 1

Session 2

Session 3

1

2

1

2

1

2

3

4

3

4

3

4

Session 4

Session 5(a) NTT

Session 6

1

2

1

2

1

2

3

4

3

4

3

4

Session 1

Session 2

1

1

2

Session 3
1

2

2

3

3

4

Session 4

Session 5

1
2

4

1

2

3

4

6

3

5
4

Session 6

5
3

4

Session 7

Session 8(b) RT07

Figure 2: Estimated inﬂuences by the proposed model.

distribution is described as follows:t−1
t−1) + (1 − λ)PG (w),, n) = λPC (w|wt−τ
P (w|wt−τ

9

(13)

0.2

0.2

0.2

0.1

0.1

0.1

0

0
1

2

3

4

0
1

Session 1

2

3

4

1

Session 2

0.2

0.2

0.1

0.1

2

3

4

Session 3
0.3

0.2

0.1

0

0
1

2

3

Session 4

4

0
1

2

3

4

1

Session 5(a) Inﬂuence to other speakers.

0.2

0.2

0.2

0.1

0.1

0.1

0

0
1

2

3

4

2

3

4

Session 2
0.2

0.1

0.1

0.1

0
2

3

Session 4

4

2

3

4

Session 3

0.2

1

4

Session 6

1

0.2

0

3

0
1

Session 1

2

0
1

2

3

4

1

2

3

4

Session 5
Session 6(b) Inﬂuence from other speakers.

Figure 3: Quantitative analysis with NTT data. The x-axis represents the speaker.

where

N (t − 1, τ, w) + β,
w0 N (t − 1, τ, w) + βW

t−1
PC (w|wt−τ
)= ∑

10

(14)

is the common cache. Here, N (t − 1, τ, w) represents the number oftimes word w is spoken from t − τ to t − 1. The CC is the same as thestandard cache language model.
• OC has the speaker’s own caches, and a common parameter that controls the inﬂuence of the speaker’s own cache. The word distribution
is as follows:t−1
t−1
P (w|wt−τ, n) = λPC (w|wt−τ, n) + (1 − λ)PG (w).

(15)

This model assumes that the word use depends only on the speaker’s
own preceding word use.
• IC has individual caches for each speaker, and a common parameterthat controls the inﬂuence of the speaker dependent caches. The word
distribution is as follows:t−1
P (w|wt−τ, n) =

M∑t−1
λm PC (w|wt−τ, m) + (1 −

m=1

M
∑

λm )PG (w),

(16)

m=1

where λm represents the inﬂuence of speaker m on all speakers including
speaker m himself/herself. This model assumes that the strength of theinﬂuence depends on the speaker, but the sensitivity does not diﬀer
among speakers.
• CI has a common cache, and individual parameters that control theinﬂuence of the common cache for each speaker as follows:t−1
t−1
P (w|wt−τ, n) = λn PC (w|wt−τ
) + (1 − λn )PG (w).

(17)

This model assumes that the word use depends on all speakers’ word
use and the degree of dependence diﬀers among speakers.
• OI has the speaker’s own caches, and individual parameters that control the inﬂuence depending on the speakers as follows:t−1
t−1
P (w|wt−τ, n) = λn PC (w|wt−τ, n) + (1 − λn )PG (w).

(18)

• II has individual caches for each speaker, and individual inﬂuence parameters. This is our proposed model in (2).
11The ﬁrst letter of a method’s name C/O/I represents common/own/individual
caches, respectively, and the second letter of the method’s name C/I represents common/individual parameters, respectively. Only the proposed model(II) takes the asymmetricity of inﬂuences into account. With all the models,
we used α = 1, β = 10−8 and τ = ∞.
In each session, we used data until the jth word as training data to learn
the parameters, and used words after the (j + 1)th word as test data. We
evaluated the performance of each model using the perplexity of held-out
words:
( ∑T
)
j
log
P
(w
|w
,
s
)
t
i
1
t=j+1
exp −
.
(19)
T −j
A lower perplexity represents higher predictive performance.
Table 2 shows the average perplexities for the NTT and RT07 data sets,in which the number of training utterances ranges from j = 10 to j = 300.
Here, an utterance consists of a set of words that are consecutively spoken
by a speaker. The proposed model achieved the lowest perplexities in all
sessions. This result indicates that it is important to estimate the asymmetric inﬂuences between speakers, which only the proposed model considers.
Figure 4 show the perplexities with diﬀerent numbers of training utterances
for NTT and RT07 data sets. Generally speaking, the perplexity decreased
as the number of training utterances increased because the estimation accuracy of the inﬂuences and preceding word use improves. In some sessions,for example Session 3 in the NTT data set, the perplexity increased becauseof the change of topics. Except when the number of training utterances was
small, the perplexity of the proposed model (II) steadily achieved the lowest perplexities. When the number of training utterances is very small, theperplexity of the proposed model (II) was higher than a few other models
in some sessions because the number of parameters in the proposed model
is more than that of other methods. However, the proposed model achieved
the lowest perplexity with small addition of training utterances.
The average computational time for learning parameters in the proposedmodel with 300 training utterances was 0.01 and 0.02 seconds for the NTT
and RT07 data sets, respectively. The proposed model is very eﬃcient, andit can be used in real time applications [25]. Figure 7 shows the log likelihood
over iterations with the proposed model. The log likelihood, which is to be
maximized, quickly converged.
Figure 8 shows the average perplexities with diﬀerent lengths of eﬀective
12

Table 2: Average perplexities for each session.

session#
1
2
3
4
5
6
averagesession#
1
2
3
4
5
6
7
8
average

CC
259.2
279.1
297.1
321.3
332.8
274.4
294.0

(a) NTT
OC
IC
251.4 256.5
263.2 280.0
287.7 298.9
307.9 314.9
322.6 328.1
260.9 277.7
282.3 292.7

CI
259.4
278.9
298.3
320.8
332.5
275.9
294.3

OI
249.9
264.7
288.1
309.7
320.3
267.5
283.4

II
247.7
261.4
284.4
294.4
313.8
254.9
276.1

CC
395.7
304.3
322.6
373.0
300.5
342.5
340.6
340.7
340.0

(b) RT07
OCIC
411.8 396.7
308.5 308.6
330.6 324.4
386.1 369.0
301.9 299.9
343.7 368.2
350.7 345.4
346.7 345.7
347.5 344.8

CI
397.1
301.6
322.9
377.2
303.2
350.4
355.8
344.9
344.1

OI
412.7
309.4
333.6
390.7
304.4
352.3
357.4
357.9
352.3

II
395.6
296.1
313.2
368.7
293.4
340.4
332.3
340.3
335.0

period τ . As the eﬀective period gets longer, the perplexities decreases. This
result implies that the speakers in theses data sets were inﬂuenced for a longtime.
We evaluated the proposed online inference procedures. Figure 9 shows
the average perplexities with diﬀerent forgetting factors γ. The lowest perplexity was achieved at around γ = 0.1, which indicates that the tuning of γ
is important in the online inference. The perplexities achieved by the online
inference were higher than those by the batch inference in the both data setseven though the online inference is more eﬃcient than the batch inference.
Figure 10 shows the average perplexities with diﬀerent numbers of training utterances in online inference with γ = 0.1. The perplexities decreased
as the training utterances increased because the model can use more training
data for the inference.

13

330
II
OI
CI
IC
OC
CC

320

perplexity

310
CC CI
300
IC
290

OI
OC

280

II
270
260
50

100

150

200

250

300

j

(a) NTT
390
II
OI
CI
IC
OC
CC

380

perplexity

370

OI

360
OC
350

CI

340
IC

CC

330
II
320
310
50

100

150

200

250

300

j

(a) RT07
Figure 4: Average perplexities with diﬀerent numbers of training utterances for NTT and
RT07 data. The horizontal axis represents the number of training utterances.

7. Conclusion
We have proposed a probabilistic model for learning inﬂuences from conversation data with multiple speakers. We have conﬁrmed experimentally
14

290
270

perplexity

perplexity

280
260
250
240
230
220
50

100

150

200

250

274
272
270
268
266
264
262
260
258
256

300

50

100

j

150

300

200

250

200

250

300

200

250

300

330
325
320
315
310
305
300
295
290
285
280
275

300

50

100

j

150
j

Session 3

Session 4

360

280

350

275

340

270

perplexity

perplexity

250

Session 2

perplexity

perplexity

Session 1

100

200

j

310
305
300
295
290
285
280
275
270
50

150

330
320
310
300

265
260
255
250

290

245
50

100

150

200

250

300

j

50

100

150
j

Session 5

Session 6

Figure 5: Perplexities with diﬀerent numbers of training utterances for NTT data. The
horizontal axis represents the number of training utterances.

that the proposed model can extract inﬂuences between speakers and learn
conversation’s word use.
Although our results have been encouraging to date, our model can befurther improved in a number of ways. First, we would like to estimate inﬂuences using other behaviors, such as nonverbal speech acts, posture andeye movement, as well as word use. Second, we would like to extend the
proposed model. The proposed model can be extended so that it can incorporate the dynamics of topics by combining it with dynamic models, suchas dynamic topic models [26] and topic tracking language models [27]. The
15

perplexity

perplexity

420
415
410
405
400
395
390
385
380
50

100

150

200

250

330
325
320
315
310
305
300
295
290
285
280

300

50

100

j

Session 1

perplexity

perplexity

320
315
310
305
150

200

250

300

50

100

300

perplexity

perplexity

305
295
290
285
280
275
200

250

300

50

100

perplexity

perplexity

150

200

250

300

200

250

300

Session 6

440
420
400
380
360
340
320
300
280
260
150

300

j

Session 5

100

250

380
370
360
350
340
330
320
310
300

j

50

200

Session 4

310

150

150
j

Session 3

100

300

430
420
410
400
390
380
370
360
350
340
330

j

50

250

Session 2

325

100

200

j

330

50

150

200

250

300

j

350
348
346
344
342
340
338
336
334
332
330
50

100

150
j

Session 7

Session 8

Figure 6: Perplexities with diﬀerent numbers of training utterances for RT07 data. The
horizontal axis represents the number of training utterances.

16

-14460-8100-14480-8150-14500-8200

log likelihood

log likelihood

-8050

-8250
-8300
-8350

-14520-14540-14560-8400-14580-8450-14600-8500-14620
0

5

10

15

20

25

30

0

5

10

iteration

iteration(a) NTT(b) RT07

15

20

200

250

370

430

360

420

350

410

340

400
perplexity

perplexity

Figure 7: Log likelihoods over iterations.

330
320
310

390
380
370

300

360

290

350

280

340

270

330
50

100

150

200

250

300

50

tau

100

150

300

tau

(a) NTT(b) RT07

Figure 8: Average perplexities with diﬀerent lengths of eﬀective period τ .

proposed model assumes that topics do not change over time in a session.
The word use can be changed by topics as well as inﬂuences. Therefore,
we can estimate inﬂuences more clearly by modeling topic dynamics. The
proposed model can also be extended by using the speaker-speciﬁc generalword distribution, which can help to estimate inﬂuences. Third, we would
like to extend the estimation procedure. We must determine the period of
the inﬂuence automatically. In the proposed model, we used a mixture models with a ﬁxed number of components, where we assumed that all speakers
can inﬂuence on a speaker. We can extend the model by selecting a modelwith an arbitrary number of components using model selection techniques.
Fourth, we would like to quantitatively evaluate the accuracy of estimatedinﬂuences by using a mesure that correlates to the actual inﬂuences such as
17

339

370

338

369

337

368
perplexity

perplexity

336
335
334
333

367
366
365

332

364

331

363

330
0.01

0.1

362
0.01

1

0.1

1

gamma

gamma

(a) NTT(b) RT07

Figure 9: Average perplexities with diﬀerent forgetting factors γ in online inference.
550

520
500

500

480
460
perplexity

perplexity

450
400
350

440
420
400
380

300

360

250

340
50

100

150

200

250

300

j

50

100

150

200

250

300

j

(a) NTT(b) RT07

Figure 10: Average perplexities with diﬀerent numbers of training utterances in online
inference.

the result of questionnaire. We showed that the proposed model achieved
better perplexities than other models. However, the perplexity is a measurefor evaluating language models, and not a measure for evaluating inﬂuence
estimation. Finally, we would like to evaluate the proposed model in an
automatic speech recognition system.

18

Appendix A. Derivation of (6)The lower bound of the posterior probability of parameters given the data
to be maximized can be obtained as follows,
L =

T
∑

log

t=1
T
∑

M
∑

t−1
, m) +
λst m PC (wt |wt−τ

M∑

log P (λn |α)

n=1

m=0
M
∑

M
t−1
λst m PC (wt |wt−τ, m) ∑=log
P (m|t)+log P (λn |α)
P
(m|t)
t=1
m=0
n=1

≥

T ∑
M
∑

P (m|t) log

t=1 m=0
T ∑
M
∑

= Q−

M
t−1
λst m PC (wt |wt−τ, m) ∑+
log P (λn |α)P (m|t)n=1

P (m|t) log P (m|t),(A.1)

t=1 m=0

where we used Jensen’s inequality. Therefore, Q in (6) is the lower bound of
the objective function with respect to parameters λnm .
References[1] T. L. Chartrand, J. A. Bargh, The chameleon eﬀect: the perceptionbehavior link and social interaction, Journal of Personality and Social
Psychology 76 (1999) 893–910.
[2] U. Dimberg, Facial reactions to facial expressions, Psychophysiology 19(1982) 643–647.
[3] S. Brennan, Lexical entrainment in spontaneous dialog, in: ISSD ’96,
pp. 41–44.
[4] A. Nenkova, A. Gravano, J. Hirschberg, High frequency word entrainment in spoken dialogue, in: ACL ’08: HLT, pp. 169–172.
[5] D. Reitter, F. Keller, J. D. Moore, Computational modelling of structural priming in dialogue, in: NAACL ’06, pp. 121–124.
[6] R. Levitan, J. Hirschberg, Measuring acoustic-prosodic entrainment
with respect to multiple levels and dimensions, in: Proceedings of 12th
Annual Conference of the International Speech Communication Association, Interspeech ’11, pp. 3081–3084.
19

[7] R. Coulston, S. Oviatt, C. Darves, Amplitude convergence in children’s
conversational speech with animated personas, in: ICSLP ’02, volume 4,
pp. 2689–2692.
[8] H. Giles, J. Coupland, N. Coupland, Accommodation theory: Communication, context, and consequence, Cambridge University Press, 1991.
[9] L. E. Scissors, A. J. Gill, D. Gergle, Linguistic mimicry and trust in
text-based CMC, in: CSCW ’08, pp. 277–280.
[10] T. Iwata, S. Watanabe, Learning inﬂuences from word use in polylogue,
in: Proceedings of 12th Annual Conference of the International SpeechCommunication Association, Interspeech ’11, pp. 3089–3092.
[11] A. Dempster, N. Laird, D. Rubin, Maximum likelihood from incomplete
data via the EM algorithm, Journal of the Royal Statistical Society,
Series B 39 (1977) 1–38.
[12] S. Renals, T. Hain, , H. Bourlard, Recognition and interpretation of
meetings: The ami and amida projects, in: Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU ’07,
pp. 238–247.
[13] K. Otsuka, S. Araki, K. Ishizuka, M. Fujimoto, M. Heinrich, J. Yamato,
A realtime multimodal system for analyzing group meetings by combining face pose tracking and speaker diarization, in: Proceedings ofthe 10th international conference on Multimodal interfaces, ICMI ’08,
ACM, New York, NY, USA, 2008, pp. 257–264.
[14] A. Vinciarelli, Speakers role recognition in multiparty audio recordings
using social network analysis and duration distribution modeling, IEEE
Transactions on Multimedia 8 (2007) 1215–1226.
[15] F. Valente, A. Vinciarelli, Language-independent socio-emotional role
recognition in the ami meetings corpus, in: Proceedings of 12th Annual
Conference of the International Speech Communication Association, Interspeech ’11, pp. 3077–3080.
[16] G. Ji, J. Bilmes, Multi-speaker language modeling, in: HLT-NAACL’04, pp. 133–136.

20

[17] M. Purver, T. L. Griﬃths, K. P. Körding, J. B. Tenenbaum, Unsupervised topic modelling for multi-party spoken discourse, in: ACL ’06, pp.
17–24.
[18] A. Auyeung, T. Iwata, Capturing implicit user inﬂuence in online social
sharing, in: ACM Conference on Hypertext and Hypermedia (HT ’10),
pp. 245–254.
[19] R. Kuhn, R. D. Mori, A cache-based natural language model for speech
recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1990) 570–583.
[20] D. Gildea, T. Hofmann, Topic-based language models using em, in:Proceedings of Eurospeech ’99, pp. 2167–2170.
[21] R. M. Neal, G. E. Hinton, A view of the em algorithm that justiﬁes incremental, sparse, and other variants, in: Learning in Graphical Models,pp. 355–368.
[22] M. Sato, S. Ishii, On-line em algorithm for the normalized gaussian
network, Neural Computation 12 (2000) 407–432.
[23] T. Hori, S. Araki, T. Yoshioka, M. Fujimoto, S. Watanabe, T. Oba,
A. Ogawa, K. Otsuka, D. Mikami, K. Kinoshita, T. Nakatani, A. Nakamura, J. Yamato, Low-latency real-time meeting recognition and understanding using distant microphones and omni-directional camera, IEEE
Transactions on Audio, Speech, and Language Processing (2011).
[24] J. G. Fiscus, J. Ajot, J. S. Garofolo, The rich transcription 2007 meeting
recognition evaluation, in: Multimodal Technologies for Perception of
Humans, pp. 373–389.
[25] T. Hori, S. Araki, T. Yoshioka, M. Fujimoto, S. Watanabe, T. Oba,
A. Ogawa, K. Otsuka, D. Mikami, K. Kinoshita, T. Nakatani, A. Nakamura, J. Yamato, Real-time meeting recognition and understanding
using distant microphones and omni-directional camera, in: SLT ’10,
pp. 412–417.
[26] D. M. Blei, J. D. Laﬀerty, Dynamic topic models, in: ICML ’06, pp.
113–120.

21

[27] S. Watanabe, T. Iwata, T. Hori, A. Sako, Y. Ariki, Topic tracking
language model for speech recognition, Computer Speech & Language
25 (2011) 440–461.

22