1

Corpus-based Discourse
Understanding in Spoken
Dialogue Systems
Ryuichiro Higashinaka, Mikio Nakano,Kiyoaki Aikawa
NTT Communication ScienceLaboratories

2Overview• A new discourse understanding method inspoken dialogue systems- discourse understanding means utterance
understanding taking the context into
account- retains the ambiguity of a user utterance andresolves it by subsequent utterances- uses statistical information derived from
dialogue corpora

3

Objective• Spoken dialogue systems that can- accurately understand user intention
using the context of a dialogueBenefits:- more efficient dialogue- robust to misrecognitions

4

Discourse Understanding
User Utterance e.g.,”I’d like to go to Tokyo”
Speech Recognition
Recognition hypothesise.g.,”I’d like to go to Tokyo”
Syntactic and Semantic Analysis
Dialogue act e.g.,[refer-destination: Tokyo]
Discourse Understanding
Update e.g.,(before)
Dialogue State(After)

Origin--Destination--

Origin--DestinationTokyo5Problem• Ambiguities in discourse understanding- Speech recognizer outputs multiple
recognition hypotheses (N-best)- Syntactic and semantic analysis produce
multiple parsing results→ multiple dialogue act candidates and thus
multiple dialogue state candidates
are derived from a user utterance
System has to appropriately rank the dialogue statecandidates to obtain the most plausible user intention

Problem: an example

6

Example dialogue:

User : “To Sapporo”(Flight reservation domain) System: “uh-huh”
User : “From Tokyo”“To Sapporo”[refer-origin: Sapporo]
OriginSapporoDestination--“From Tokyo”

Origin--Destination--

Which one ismore plausible?
“uh-huh”“To Sapporo”[refer-destination: Sapporo]
Origin--Destination Sapporo“From Tokyo”[sys:backchannel][refer-origin: Tokyo][refer-origin: Tokyo]

OriginTokyo

Destination--

Which one ismore plausible?

OriginTokyo

Destination Sapporo

7Related Work• ISSS Method (Nakano et al.,1999)- rank multiple dialogue states by handcrafted scoring rules→ creating rules by hand is costly• Estimation of dialogue act type (Nagataet al. 1994)- estimate the most probable dialogue act
from previous dialogue act sequences→ mainly aims at improving recognition
accuracy; not applied to dialogue systems

8Approach• Use of statistical information derived
from dialogue corpora to score thedialogue states• Keep the low-ranked dialogue states toallow possible understanding in thefuture

9Statistical Information• N-gram probability of a dialogue act typesequence (as Nagata et al.)- represents brief (superficial) flow of a dialogue• Collocation probability of a dialogue state andthe next dialogue act- deals with more detailed information about thedialogue- such as dialogue state changes including
grounding information

10Dialogue State Scoring• Update the score of dialogue states by
the following formula
Score of the updated dialogue state =Score of the dialogue state before update＋α・ Score of a dialogue act(from Speech Recognition and the Syntactic and
Semantic Analysis)＋β・ N-gram probability score of dialogue act
type sequences＋γ・ Collocation probability score of a dialogue
state and the next dialogue act(α,β, and γ are weighting factors)

Progress of Understanding

11

Example dialogue:

User : “To Sapporo”(Flight reservation domain) System: “uh-huh”
User : “From Tokyo”“To Sapporo”[refer-origin: Sapporo]

Origin--Destination-->
Score : Y

OriginSapporoScore : X

Destination--←plausible“To Sapporo”[refer-destination: Sapporo]
OriginDestination Sapporo“uh-huh”“From Tokyo”“From Tokyo”[sys:backchannel][refer-origin: Tokyo]
OriginTokyo

Destination--

Score : Z<
Score : W--[refer-origin: Tokyo]
OriginTokyo

plausible→ Destination Sapporo

Data Collection• Corpus- 240 dialogues collected in the meeting room
reservation domain- 26 dialogue act types- Vocabulary of 168 words- All the utterances transcribed and converted todialogue acts

• Extraction of statistical information- Trigram probability of dialogue act types- Collocation probability• Classify the way of collocation into 64 classes• Use occurrence probability of each class• 17 classes found in the corpus

12Implementation

13• Scoring formula
Score of the updated dialogue state =Score of the dialogue state before update+ α ・ log( 1 / N-best-rank )+ β ・ log( dialogue act type trigram probability )+ γ ・ log( collocation probability ) (α=β=γ=1)• Maximum number of dialogue states- Enables real-time processing by avoiding explosion ofdialogue states• Response generation- Rule-based response generation based on the highestranked dialogue stateExperiment (1)14• Verification of our approach- Collected 256 dialogues with the implemented
system- 5-best recognition hypotheses as input- Maximum number of dialogue states: 15- Task completion rate: 88.3%(succeed in reservation within 5 minutes)→ Sufficiently high percentage of task completion
rate suggests that system based on our approachworks sufficiently

Experiment (2)15• Effectiveness of holding multiple dialogue states- System1 (maximum number of dialogue states: 1 )VS.
System30 (maximum number of dialogue states: 30)- 224 dialogues collected with each system- System30 outperformed System1 both in taskcompletion rate and task completion time
- Average task completion time of System30 (95.86 sec.)
was significantly shorter than that of System1 (107.66sec.)→ Holding multiple dialogue states is effective

16Conclusion• A new discourse understanding method that- retains the ambiguity of a user utterance andresolves it by subsequent utterances- uses statistical information derived from
dialogue corpora• Experimental results show the validity of ourapproach