Interspeech 2007 31st Aug.

Effects of Quiz-style
Information Presentation onUser Understanding
Ryuichiro Higashinaka, Kohji Dohsaka,
Shigeaki Amano, and Hideki Isozaki
NTT Communication Science Laboratories
NTT Corporation
1

Overview Introduce the idea of quiz-style

information presentation
 Information is presented as quizzes
 Motivated users would keep theirattention on receiving information
 Lead to better user understanding Useful for educational systems
 Experimental results
 Task is presenting biographical facts
2

Example3

Example

S1
S1 WhoWho isis this?
this?
FirstFirst hint:
hint: He
He mademade contributions
contributions inin thethefield
field ofof natural
natural science.
science.

4

Example

S1
S1 WhoWho isis this?
this?
FirstFirst hint:
hint: He
He mademade contributions
contributions inin the
the
field
field ofof natural
natural science.
science.
U1
U1 Albert
Albert Einstein?
Einstein?

5

Example

S1
S1 WhoWho isis this?
this?
FirstFirst hint:
hint: He
He mademade contributions
contributions inin thethefield
field ofof natural
natural science.
science.
U1
U1 Albert
Albert Einstein?
Einstein?
S2
S2 No,
No, not
not eveneven close!
close!
SecondSecond hint:
hint: HeHe served
served forfor 1010 years
years asasaa government
government minister
minister ofof thethe duchy
duchy ofofWeimar.
Weimar.

6

Example

S1
S1 WhoWho isis this?
this?
FirstFirst hint:
hint: He
He mademade contributions
contributions inin thethefield
field ofof natural
natural science.
science.
U1
U1 Albert
Albert Einstein?
Einstein?
S2
S2 No,No, notnot eveneven close!
close!
SecondSecond hint:
hint: HeHe served
served forfor 1010 years
years asasaa government
government minister
minister ofof thethe duchy
duchy ofofWeimar.
Weimar.
U2
U2 II don’t
don’t know.
know.
7Example

S1
S1 WhoWho isis this?
this?
FirstFirst hint:
hint: He
He mademade contributions
contributions inin thethefield
field ofof natural
natural science.
science.
U1
U1 Albert
Albert Einstein?
Einstein?
S2
S2 No,No, notnot eveneven close!
close!
SecondSecond hint:
hint: HeHe served
served forfor 10
10 years
years asasaa government
government minister
minister ofof thethe duchy
duchy ofofWeimar.
Weimar.
U2
U2 II don’tdon’t know.
know.
S3S3 OK.
OK. Third
Third hint:
hint: He
He waswas aa German
German poetpoetandand writer.
writer.

8

ExampleU3U3 FriedrichFriedrich vonvon Schiller?
Schiller?

9

ExampleU3U3 FriedrichFriedrich vonvon Schiller?
Schiller?
S4S4 Very
Very Close!
Close!
FourthFourth hint:
hint: Inspired
Inspired byby travel
travel toto Italy,Italy, hehecompletedcompleted his
his study
study ofof German
German classicism
classicism
withwith FriedrichFriedrich Schiller.
Schiller.

10

ExampleU3U3 FriedrichFriedrich vonvon Schiller?
Schiller?
S4S4 VeryVery Close!
Close!
FourthFourth hint:
hint: Inspired
Inspired byby travel
travel toto Italy,
Italy, hehecompletedcompleted his
his study
study ofof German
German classicism
classicism
withwith FriedrichFriedrich Schiller.
Schiller.
U4
U4 Hermann
Hermann Hesse?
Hesse?

11Example
U3
U3 FriedrichFriedrich vonvon Schiller?
Schiller?
S4
S4 Very
Very Close!
Close!
FourthFourth hint:
hint: Inspired
Inspired byby travel
travel toto Italy,
Italy, hehecompletedcompleted his
his study
study ofof German
German classicism
classicism
withwith FriedrichFriedrich Schiller.
Schiller.
U4U4 Hermann
Hermann Hesse?
Hesse?
S5
S5 NotNot quite!
quite!
FifthFifth hint:
hint: WithWith The
The Sorrows
Sorrows ofof Young
Young
Werther
Werther andand other
other works,
works, hehe became
became aa
notablenotable proponent
proponent ofof thethe “Storm“Storm andand Stress”.
Stress”.
12Example
U3
U3 FriedrichFriedrich vonvon Schiller?
Schiller?
S4
S4 Very
Very Close!
Close!
FourthFourth hint:
hint: Inspired
Inspired byby travel
travel toto Italy,
Italy, hehecompletedcompleted his
his study
study ofof German
German classicism
classicismwithwith FriedrichFriedrich Schiller.
Schiller.
U4U4 Hermann
Hermann Hesse?
Hesse?
S5
S5 NotNot quite!
quite!
FifthFifth hint:
hint: WithWith The
The Sorrows
Sorrows ofof Young
Young
Werther
Werther andand other
other works,
works, hehe became
became aa
notablenotable proponent
proponent ofof thethe “Storm“Storm andand Stress”.
Stress”.
U5
U5 Johann
Johann Wolfgang
Wolfgang vonvon Goethe?
Goethe?
13

Example
U3
U3 FriedrichFriedrich vonvon Schiller?
Schiller?
S4S4 Very
Very Close!
Close!
FourthFourth hint:
hint: Inspired
Inspired byby traveltravel toto Italy,
Italy, hehecompletedcompleted his
his study
study ofof German
German classicism
classicism
withwith FriedrichFriedrich Schiller.
Schiller.
U4
U4 Hermann
Hermann Hesse?
Hesse?
S5
S5 NotNot quite!
quite!
Fifth
Fifth hint:
hint: With
With TheThe Sorrows
Sorrows ofof Young
Young
Werther
Werther andand other
other works,works, hehe became
became aa
notablenotable proponent
proponent ofof thethe “Storm“Storm andand Stress”.
Stress”.
U5
U5 Johann
Johann Wolfgang
Wolfgang vonvon Goethe?
Goethe?
S6
S6 That’s
That’s right!
right! Congratulations!
Congratulations!
14

Quiz-style information
presentation
 Biographical facts are presented as hints

in a “who is X?” quiz Quiz-style ordering of biographical facts
 Makes it difficult to name the person
 Makes the interaction as long as possible
 Keeps user’s attention for a longer time Feedback utterances to facilitate guessing
by a user
15

Quiz-style ordering

Most
Mostdifficult
difficult(ambiguous)(ambiguous)

Encyclopedic
Encyclopedicorderingordering

HeHemademadecontributions
contributionsininthe
thefieldfieldofofnatural
naturalscience.
science.

1. HeHewas
wasaaGerman
Germanpoet
poetandandwriter.
writer.
2. With
WithTheTheSorrows
Sorrowsof
ofYoungYoung
Werther
Wertherandandotherotherworks,
works,hehebecamebecameaanotable
notableproponent
proponentof
ofthethe“Storm“StormandandStress”.
Stress”.
3. He
Heserved
servedfor
for10
10years
yearsasasaa
government
governmentminister
ministerof
ofthe
theduchy
duchyofofWeimar.
Weimar.
4. Inspired
Inspiredby
bytraveltravelto
toItaly,
Italy,hehecompletedcompletedhis
hisstudy
studyofofGerman
Germanclassicism
classicismwith
withFriedrichFriedrich
Schiller.
Schiller.
5. He
Hemademadecontributions
contributionsininthe
thefieldfieldof
ofnatural
naturalscience.
science.

He
Heserved
servedfor
for10
10years
yearsasasaa
government
governmentminister
ministerof
ofthe
theduchy
duchyofofWeimar.
Weimar.
HeHewaswasaaGerman
Germanpoet
poetandandwriter.
writer.
InspiredInspiredby
bytraveltravelto
toItaly,
Italy,hehecompletedcompletedhis
hisstudy
studyofofGerman
Germanclassicism
classicismwith
withFriedrichFriedrich
Schiller.
Schiller.
With
WithThe
TheSorrows
Sorrowsof
ofYoungYoung
Werther
Wertherandandotherotherworks,
works,hehebecamebecameaanotablenotableproponent
proponentof
ofthethe“Storm“StormandandStress”.
Stress”.

EasiestEasiest(definitive)(definitive)

16

Experiment To verify whether quiz-style information

presentation can effectively convey
information
 Perform comparison of two information
presentation systems
 Read-out system
Reads out the biographical facts in anencyclopedic order Quiz system

Reads out the biographical facts in the quizstyle order with feedback utterances
17

Read-out System
N biographicalfacts about X

Recorded Voice

Biographicalfacts inencyclopedicordering

I’m going to tell
you about X.
Mouse click
He was born in…

SystemOnlineencyclopedia

Mouse click

User

He is famous for...
Continue until all biographicalfacts have been read18

Quiz System Recorded Voice
Let’s start a quiz!
First hint: he wasborn in…

Created manually

N biographicalfacts about X

Biographicalfacts inquiz-styleordering

Type a guess
Close! Second
hint: he isfamous for…

Onlineencyclopedia...

SystemUser

Type a correct guess
Person nameco-occurrence
statistics

DeliverThat’s
all biographicalcorrect! facts
I’ll read theremaining hints..
19

Experimental Setup 40 participants (20 males and 20 females) Procedure Each participant used both systems alternately
 Received biographical facts of 20 people in acontrolled order Evaluation criteria
 User understanding (by a memory test)
 Subjective evaluation (by a questionnaire) Task completion time20

Results (memory test)
Total number ofcorrect answers

Read-out

Answer
Leniency
for grading

808

Quiz

Non-presented

*
723

719

*

640

448

Partial Answers

680
615

410

Stylistic Variations

387

Exact Answers

Quiz system achieves better user understandingin terms of memory recall
21

Results (subjective evaluation)
Scores (1-5) averaged overall 40 participants

Read-out
3.5
3.4

3.4

Usability

5: goodQuiz
3.5

**

2.9

2.9

Willingness

Usefulness 1: poor

Participants seem to be more engaged in theinteraction with the quiz system
22

Results (task completion time)

Seconds

85.3

49.1

Read-out
QuizTime taken to present a set of biographical facts

Overall the quiz system consumes more time(for feedbacks, users’ thinking and typing)23

Conclusion
 Quiz-style information presentation
 Effectively convey informationoutperforming the read-out method High engagement is likely to havecontributed to the improvement in user
understanding Viable alternative when time is not critical Useful for educational systems
 Demo video of our current system24

Demo25