INTERSPEECH 2006 - ICSLPInfluence of Pause Length on Listeners' Impressions
in Simultaneous Interpretation
Hitomi Tohyama†

Shigeki Matsubara*†Graduate School of Information Science, Nagoya University, Japan*

Information Technology Center, Nagoya University, Japan†

hitomi@el.itc.nagoya-u.ac.jp

Abstract

2. Importance of Controlling Pauses

We have been attempting to realize simultaneous machineinterpretation. However, determining the interpreting utterance
timing is as difficult as determining translation units. Thisremains a major concern for the development of such a speechtranslation system. It is also crucial for the system’s users thatthe speech generated by the system is clear and easy to listen to.
In this paper, we focus attention on the pauses that partly
characterize simultaneous interpreters’ utterances. We attempt toanalyze the results of an experiment conducted using 31 subjects
on the relationship between listener-friendliness and the lengthof pauses in speech, using the CIAIR simultaneous interpretation
database as the data source. The results generated someknowledge about listener impressions of simultaneousinterpretation, which will be helpful for realizing simultaneousmachine interpretation.
Index Terms: cross-lingual, simultaneous interpretation,machine translation, bilingual speech corpus

We have conducted an experiment using 31 subjects to clarify
the relationship between listener impressions and pauses insimultaneous interpretation contexts, with the aim of realizing asimultaneous machine interpretation system [3].
Controlling the length of pauses is important in determining
the time required to generate translations and the timing of its
outputs for a simultaneous machine interpretation system. For
example, just at the moment the machine detects a part of anutterance that is possible to translate, it is one of the machine’s
roles to generate and output the translation simultaneously. Yet,a long pause is required for detecting parts of utterances that canbe translated, and such a pause is likely to influence the subjects’
listener impressions. Thus, it is crucial to control the length ofpauses.

3. Experiment
The purpose of our experiment is to investigate therelationship between listener impressions and pauses insimultaneous interpretation contexts. We used the CIAIR
Simultaneous Interpretation Database of Nagoya University [4]for our experiment. The size of the transcribed data is about one
million words, a simultaneous translation corpus that deserves tobe classed as one of the largest in the world. The transcription
was produced based on the standard transcription rules of theCorpus of Spoken Japanese (CSJ) developed by the National
Japanese Language Research Institute [5]. The transcribed dataconsists of utterance units, and each utterance unit is divided bysilences of more than 200 msec. In this paper, we define a periodof continuous silence exceeding 200 msec as a pause. We
visualize the emergence of pauses using a time-charting tool(refer to Fig. 1).

1. Introduction
In speech, a pause is an essential element for producing therhythmic aspect of spoken languages, with the rhythmic aspect
being closely related to humans’ listening skill and ourunderstanding of the semantic contents of the speech [1]. It is
known that humans instinctively feel that phenomena
accompanied by rhythmic aspects are secure [2], and that wehave a nature that is attracted by anything rhythmic andsynchronized with it. However, since simultaneous interpreters
take pauses in order to wait for a speaker’s next input ofnecessary information before starting their interpretation, theirutterances sometimes give us different impressions from usual
utterances, and those pauses tend to influence listeners’impressions. Therefore, in this paper we investigate thecharacteristics of listener-friendly simultaneous interpretation.
This paper is presented as follows: Section 2 describes theimportance of the investigation to realize simultaneous machineinterpretation. Section 3 illustrates the process and results of anexperiment performed using 31 human subjects. In Section 4, weanalyze the relationship between listener impressions and theperiods of pauses in simultaneous interpretation contexts basedon the experimental results, and discuss our observations.
Section 5 includes our conclusions and a brief mention of futurework.

3.1. Outline of Experiment
We extracted twelve interpretation cases from corpus datacomprising freely spoken lectures (hereafter, “A-style lectures”)
and nine interpretation cases from corpus data consisting oflectures read from a script (hereafter, “B-style lectures”),
twenty-one interpretation cases altogether. These cases consistof speech sounds from lectures on various themes such aspolitics, economics, and culture, given in English and those of
their English-Japanese simultaneous interpretations.

893

September 17-21, Pittsburgh, Pennsylvania

INTERSPEECH 2006 - ICSLP

3.2. Experimental Results
We evaluated each interpretation case using a five-point scale.
The highest score was 5, and an interpretation case receiving thisevaluation was easiest to listen to. On the other hand, a casereceiving a score of 1 was considered most difficult to listen to.
The listeners’ evaluations toward both lecture styles A and B
were scored and the mean scores for each interpretation casewere calculated. The graphs in Fig. 3-(1) and 3-(2) show theranking of the mean score of those simultaneous interpretation
cases in descending order.

Figure 1㩷 Time chart tool of CIAIR simultaneousinterpretation database
Figure 3-(1)㩷 A-style lectures: Evaluation result of eachinterpretation case

Figure 3-(2)㩷 B-style lectures: Evaluation result of eachinterpretation case

Figure 2㩷 Sample of text data
(English- Japanese interpreting data)

4. Analysis of the Experimental Results

We added discourse tags to the described speech data and eachutterance was marked with the time when the speech began andthe time when it ended, by which we could calculate the lengthof each pause and the speed of each utterance (Fig. 2).
As the data of A-style lectures were data of simultaneousinterpretation of lectures in which lecturers talked off the top of
their head, the average of their speech speed was 9.1 mora/s. On
the other hand, because the data of B-style lectures were data ofsimultaneous interpretation of lectures in which lecturers gave atalk while referring to their notes, the average of their speech
speed in B-style lectures was 11.2 mora/s. This means the timepressure on interpreters is high in the case of B-style lectures.
We prepared 21 interpretation cases by extracting respectively
speech sound of 60 sec from each speech beginning five minutes
into the speech and ending six minutes into it passed as aninterpretation case. The subjects evaluated the samples
according to a five-grade system: very difficult to listen to,slightly difficult to listen to, undecided, slightly easy to listen to,and easy to listen to. All 31 subjects were native Japanesespeakers.

4.1. Relationship between the Listeners’ Impressions
and the Average Pause TimeWe have investigated the relationship between how thesubjects feel about the interpreter’s utterance -- whether it iscomfortable to listen to -- and the respective average pause timethat appears repetitively in each interpretation case of lecture
styles A and B. In Figs. 4-(1) and 4-(2) below, the rank for eachtranslation case is plotted on the horizontal axis and the averagepause time for each case is on the vertical axis.

4.1.1. A-style lectures: the cases with the slow speech
rateFigure 4-(1) clarifies that the translation cases in which theaverage pause time was short acquired a high evaluation fromthe subjects (the correlation coefficient is shown in the lower
right of the graph). The correlation coefficient of the length of
the average pause time and the ranking of the evaluation is -0.65,
indicating that there is indeed a correlation between those twofactors. It is also clear that the cases with a high evaluation, that894

INTERSPEECH 2006 - ICSLPfrom both lecture styles A and B and observed the respective
cycles of the speak-stop state (See Figs. 5 and 6). The horizontal
axis represents the 60-sec period of the lecture t sampled
between the five- and six-minute marks and the vertical axis
indicates the speak-stop state.

is, interpretations easy to listen to, tended to have a short pause
time.

4.1.2. B-style lecture: the cases with fast speech rate
Figure 4-(2) displays the average pause time of eachtranslation in lecture style B, showing that the length of thepauses of each translation tends to be long. The correlation
coefficient of the evaluation ranking and the average pause time
was -0.30; we could not identify enough evidence for acorrelation between those two factors. Thus, we have found thatthe faster the lecturer speaks, the smaller the influence thespeech rate has on the subjects’ auditory impression. This
indicates how comfortable it was for subjects to listen to theinterpretation.
In this analysis, we used Spearman’s rank correlation
coefficient, which is a special case of the Pearson productmoment coefficient in which the data are converted to ranks
before calculating. The following describes the degree ofcorrelation of the correlation coefficient’s absolute values.
Less than 0.2:
0.2-0.4:
0.4-0.7:
More than 0.7:

4.2.1. The Transition of Interpreters’ Utterances: The
Case where the Lecturers’ Utterance Rate was Slow (Astyle lectures)
Regarding the A-style lectures, the two top-rated
interpretation cases were characterized by the short and stable
length of the pauses between utterance units. This means thecycle of the speak-stop state for the interpreters’ utterances isalso stable. In the two lowest-ranking translation cases, longpauses appear in the middle of utterances, meaning that there
was an uneven distribution of speak- and stop-states.
A-style lectures: the two top-rated interpretation cases㩷

no correlation
weak correlation
moderate correlationstrong correlation㪉㪅㪇㪈㪅㪏㫇㪸㫌㫊㪼㩷㫋㫀㫄㪼㩿㫊㪀

㪈㪅㪍
㪈㪅㪋

Figure 5-(1) The Transition of Interpreters’ Utterance㩷㪈㪅㪉(Interpretation case No.3)

㪈㪅㪇㪇㪅㪏
㪇㪅㪍㪇㪅㪋㪇㪅㪉㪇㪅㪇㪅㪇㪊㪥㫆㪇㪅㪍㪥㫆㪅㪈㪇㪥㫆㪅㪇㪉㪅㪇㪐㪅㪇㪏㪅㪇㪎㪅㪇㪋㪅㪇㪈㪅㪈㪉㪥㫆㪥㫆㪥㫆㪥㫆㪥㫆㪥㫆㪥㫆㫀㫅㫋㪼㫉㫇㫉㪼㫋㪸㫋㫀㫆㫅㩷㪺㪸㫊㪼㩷㪥㫆㪅㩷㩿㪿㫀㪾㪿㩷㫊㪺㫆㫉㪼㩷㫉㪸㫅㫂㫀㫅㪾㩷㪀㪅㪇㪌㪥㫆㪅㪈㪈㪥㫆䌛㪚㫆㫉㫉㪼㫃㪸㫋㫀㫆㫅㩷㪺㫆㪼㪽㪽㫀㪺㫀㪼㫅㫋㩷 㪄㪇㪅㪍㪌䌝㩷

Figure 5-(2) The Transition of Interpreters’ Utterance

Figure 4-(1)㩷 A-style lectures: Listeners’ Impression and theAverage Pause Time

(Interpretation case No.6)

A-style lectures: the two lowest-ranking interpreting cases㩷㪉㪅㪇㪈㪅㪏

㫇㪸㫌㫊㪼㩷㫋㫀㫄㪼㩿㫊㪀㪈㪅㪍㪈㪅㪋㪈㪅㪉㪈㪅㪇㪇㪅㪏㪇㪅㪍㪇㪅㪋

Figure 6-(1) The Transition of Interpreters’ Utterances㪇㪅㪉(Interpretation case No. 11)

㪇㪅㪇
㪅㪉㪈
㪥㫆

㪅㪈㪋
㪥㫆

㪅㪈㪎
㪅㪉㪇
㪅㪈㪌
㪅㪈㪏
㪅㪈㪊
㪥㫆
㪥㫆
㪥㫆㪥㫆
㪥㫆㪠㫅㫋㪼㫉㫇㫉㪼㫋㪸㫋㫀㫆㫅㩷㪺㪸㫊㪼㩷㪠㪛㩷㩿㪿㫀㪾㪿㩷㫊㪺㫆㫉㪼㩷㫉㪸㫅㫂㫀㫅㪾㪀㪅㪈㪐㪥㫆㪅㪈㪍㪥㫆㩷䌛㪚㫆㫉㫉㪼㫃㪸㫋㫀㫆㫅㩷㪺㫆㪼㪽㪽㫀㪺㫀㪼㫅㫋㩷 㪄㪇㪅㪊㪇䌝

Figure 4-(2)㩷 B-style lectures: Listeners’ Impression
and the Average Pause TimeFigure 6-(2) The Transition of Interpreters’ Utterances

4.2. Aspect of the Interpreters’ Utterance(Interpretation case No.5)

To analyze the phenomena observed in Section 4.1, weselected the two highest- and lowest-ranking translation cases895

INTERSPEECH 2006 - ICSLP

timing of simultaneous machine interpretation system being
developed is determined. In the experiment conducted for thisresearch, we used the CIAIR simultaneous interpreting database,which includes two different types of English-Japanese
simultaneous translation (lecture style A and B), and had 31
subjects listen to those translations. The conclusions drawn fromthe results of this experiment are shown below. What we
discovered from this experiment is beneficial toward realizing amachine-based simultaneous interpretation system:a) It was found that, in lecture style A where the speech rateof the lecturers was rather slow, interpretations thatreceived a high evaluation had short pauses between
utterance units. On the other hand, in lecture style B, wherethe speech rate of each lecturer was fast, the influence ofpause length on the listeners’ impression was small.
b) By closely examining the aspects of simultaneousinterpreters’ utterances, we found that the translation cases
receiving a high evaluation in both lecture styles A and B
had the characteristic that the speak-stop state in thosetranslation was stable and rhythmic.
c) What can be derived from (a) and (b) is that pause length
is a crucial factor for listeners’ impression, that is, how
comfortable it is to listen to those interpretations. However,in situations where where the time pressure on thesimultaneous interpreters was high, we have found that theinfluence of pauses on the listeners’ impression is small if
the interval and the distribution of those pauses are stable.

4.2.2. . The Transition of Interpreters’ Utterances: Cases
where the Lecturers’ Utterance Rate was Fast (B-stylelectures)
In lecture style B, where the time pressure on the interpreters
was high, the pause time generally tended to be long. However,when we compare the interpretation cases that received a highevaluation with those receiving a low evaluation, it is clear that
there was a difference in the stability of the speak-stop state
cycles in the interpreters’ utterances. In those two lowestranking cases, long 6-sec pauses appeared, and those pauses
disturbed the stability of the speak-stop state cycle.
B-style lectures: the two top-rated interpretation cases

Figure 7-(1)㩷 The Transition of Interpreters’ Utterances㩷(Interpretation case No. 21)

Future work will involve analyzing fillers (voiced sounds)
such as “ahhh” and “ehhh” (ҁpause in this paper ) so as toclarify the relationship between fillers and listener’s impressions.
Figure 7-(2)㩷 The Transition of Interpreters’ Utterances

6. Acknowledgements(Interpretation case No. 14)

The authors would like to thank Prof. Toyohide Watanabe,
Assoc. Prof. Nobuo Kawaguchi, and all members of Matsubara
Laboratory at Nagoya University for their valuable advice. This
work is partially supported by the Grand-in-Aid for Exploratory
Research (No. 17652040) of JSPS.

B-style lectures: the two lowest-ranking interpretation
cases

7. References[1] Sugito, M., Kunihiro, T., Yokose, H., and Kawano, M.
Accent, Intonation, Rhythm and Pause, SANSEIDO, Japan,
1997.
[2] Sugiura, M. Vom Wesen des Rhythmus (In Japanese),MISUZU SHOBO, Japan, 1994
[3] Matsubara, S., Takagi, A., Kawaguchi, N., and Inagaki, Y.
“Bilingual Spoken Monologue Corpus for Simultaneous
Machine Interpretation Research,” Proceedings of 3rdInternational Conference on Language Resources andEvaluation (LREC-2002), Vol. 1, pp. 153-159, 2002.
[4] Tohyama, H. and Matsubara, S. “Construction andUtilization of Bilingual Speech Corpus for Simultaneous
Machine Interpretation Research,” Proceedings of 9th
European Conference on Speech Communication andTechnology (Eurospeech-2005), pp.1585-1588, 2005.
[5] Maekawa, K. “Corpus of Spontaneous Japanese: Its designand Evaluation,” Proceedings of the ISCA & IEEE
Workshop on Spontaneous Speech Processing andRecognition (SSPR2003), 2003.

Figure 8-(1)㩷 The Transition of Interpreters’ Utterances㩷(Interpretation case No. 16)

Figure 8-(2)㩷 The Transition of Interpreters’ Utterances㩷(Interpretation case No. 19)

5. Conclusion
In this paper we have examined the relationship between thepauses in simultaneous interpretation and listeners’ impressionsof them in order to collect data based on which the translation

896