Towards Speech Dialogue Translation Mediating Speakers of Different Languages
Year: 2023

Abstract:
We present a new task, speech dialogue translation mediating speakers of different languages. We construct the SpeechBSD dataset for the task and conduct baseline experiments. Furthermore, we consider context to be an important aspect that needs to be addressed in this task and propose two ways of utilizing context, namely monolingual context and bilingual context. We conduct cascaded speech translation experiments using Whisper and mBART, and show that bilingual context performs better in our settings.

heading: 1 Introduction
text: In this global era, it is becoming increasingly important for people from different countries/regions to interact with each other and have a mutual understanding. Recent advancements in machine translation (MT) technologies have enabled us to communicate with people worldwide, especially in text. Chat translation or dialogue machine translation (Liu et al., 2021) supports such communications, which enables people who use different languages to have cross-language chats. Speech translation (ST) has also recently shown success (e.g., Chen et al., 2022), especially in monologue translation (e.g., Di Gangi et al., 2019). However, to the best of our knowledge, no study has focused on ST of dialogues, which is an important aspect of language usage.
In this study, we propose a new task: speech dialogue translation (SDT) aiming to mediate speakers of different languages. We consider bilingual dialogues where several people who speak in different languages talk with each other mediated by an ST system.
It is important to consider context in SDT because we need to consider context in different languages, which cannot be readily handled by current ST systems that mainly focus on one translation
direction. Figure 1 shows an example of an STmediated dialogue between an English speaker and a Japanese speaker. They are discussing some ideas, and the English speaker says, “What do you think about it?” The Japanese speaker responds by saying the idea is naive, but without context it can be translated as “I think it’s a bit sweet” because “甘い” has two meanings, sweet and naive. By utilizing dialogue context, the meaning of “甘い” becomes clear so that the utterance can be translated properly.
For the proposed task, we construct the SpeechBSD dataset1 based on an existing text dialogue corpus, BSD (Bussiness Scene Dialogue) corpus (Rikters et al., 2019). We collect audio of the BSD corpus through crowdsourcing along with speaker attributes.
We conduct speech-to-text cascaded ST experiments on the dataset. There are two mainstream methods for ST, the cascade method (Stentiford and Steer, 1988) where automatic speech recognition (ASR) and MT are chained together, and the end-to-end method (Duong et al., 2016; Berard et al., 2016), where translations are directly predicted from speech. Recent study (Bentivogli et al., 2021; Tran et al., 2022) suggests that the two methods are on par. We conduct cascade ST experiments using Whisper (Radford et al., 2022) for ASR and mBART (Liu et al., 2020) for MT.
We consider three settings for translation: without context, with monolingual context, and with bilingual context. The monolingual context is composed in the language the utterance to be translated is spoken, whereas the bilingual context is composed in the original language of the spoken utterances (see examples in Figure 1). We show that translation with bilingual context performs better compared to the one without context by up to 1.9 BLEU points in MT and 1.7 BLEU points in
1The dataset is made public under the CC BY-NC-SA 4.0 license at https://github.com/ku-nlp/speechBSD.
1122
cascade ST with our settings. We also conduct a manual evaluation focusing on zero anaphora, a grammatical phenomenon where arguments of verbs are omitted when they are apparent from the context in Japanese. We show that with bilingual context, the MT models can often predict zero pronouns correctly.

heading: 2 Related Work
text: Although neural MT has greatly improved over the past few years, the translation of dialogues remains a challenging task because of its characteristics. Liu et al. (2021) summarizes the recent progress of dialogue MT and categorizes its issue into four categories, coherence, consistency, cohesion, and personality. The main approaches to address these problems include document MT (e.g., Liu et al., 2021), usage of pretrained models (e.g., Wang et al., 2020), and auxiliary task learning utilizing speaker information (e.g., Liang et al., 2021).
Considering context in ST is recently studied for the end-to-end approach (Zhang et al., 2021). We point out that although not addressed in this work, considering context for ASR is also an active research area (e.g., Inaguma and Kawahara, 2021).
In this work, we focus on the translation of speech dialogue. We use mBART, which performed best in a previous work of chat translation (Liu et al., 2021), and also consider utilizing context.

heading: 3 Speech Dialogue Translation (SDT)
text: In SDT, there are several speakers who speak different languages with the help of a translation system. In this work, we consider M speak-
ers {Sm |m = 1, 2, · · · ,M} and 2 languages {Ln |n = 1, 2}. We consider a dialogue with T utterances D = (U1, · · · , UT ), where an utterance is Ut = (Smt , L n t , Xt). Here, S m t is the speaker, L n t is the language spoken, and Xt is the speech signal of t-th utterance. Let Y nt (n = 1, 2) be text that has the same meaning as Xt in language Ln. The task of SDT is to generate translation Y 2t from speech signal Xt when the source language is L1 (or translation Y 1t from Xt when the source language is L
2) for every utterance Ut.

heading: 4 SpeechBSD Dataset
text: We construct the SpeechBSD dataset to study SDT. It is based on the existing dialogue dataset in text, BSD corpus (Rikters et al., 2019, 2021). We collect audio of all the sentences in the dataset along with speaker attributes (gender and homeplace) through crowdsourcing.

heading: 4.1 BSD Corpus
text: BSD corpus is a parallel corpus of English and Japanese composed of manually designed business scene dialogues. Each dialogue called scenario contains 30 sentences on average spoken by 2-5 speakers. The original language the scenarios were written in is half English and half Japanese so that the expressions are not biased toward one language.

heading: 4.2 Dataset Construction
text: First, we divided each scenario by speaker. For example in Figure 1, the original BSD corpus contains text of Y 11 , Y 2 1 , Y 1 2 , Y 2 2 , Y 1 3 , and Y 2 3 . In this case, we divide the dialogue into four parts: the Japanese speaker part (Y 11 and Y 1 3 ), the English speaker part (Y 22 ), another Japanese speaker part
(Y 12 ), and another English speaker part (Y 2 1 and Y 23 ). In this way, we can compose two crosslanguage dialogues (Y 11 → Y 22 → Y 13 and Y 21 → Y 12 → Y 23 ) from one scenario of the BSD corpus. We collected audio through crowdsourcing so that each part is spoken by a different worker.2 We designed a web application to record audio and collected English speech from the US using Amazon Mechanical Turk3 and Japanese speech from Japan using Yahoo! crowdsourcing.4 We also collected the gender and homeplace (the US states or Japanese prefecture) of the speakers as they may affect translation performance. The instructions given to the workers are shown in Appendix A.1.

heading: 4.3 Statistics of the SpeechBSD Dataset
text: The collected audio was 24.3 hours for English speech and 30.7 hours for Japanese speech in total. Details are provided in Appendix B Table 2. Regarding speaker gender, English speech was balanced, whereas there were more male speakers in Japanese. As for homeplace, in Japanese, the speakers were distributed roughly according to the population distribution. In English, it was less diverse (Appendix B Figure 3).

heading: 5 Considering Context for SDT
text: We propose two ways to consider context in SDT: monolingual context and bilingual context.
First, for every utterance Ut, an ASR system is used to obtain transcripts Y nt .
The monolingual context is composed in the source language of the utterance to be translated. For example, in Figure 1, when translating the third utterance U3 from Japanese to English, as the source language of the utterance is Japanese (L1), the context (Y 11 and Y 1 2 ) is also composed in Japanese. Let the context composed in this way be Y n<t.
For monolingual context experiments, we use two translation models for each translation direction. The training objective of the MT model that translates from L1 to L2 is to maximize the follow-
2Because workers could participate in multiple tasks, it is possible that different parts are actually spoken by the same person.
3https://www.mturk.com/ 4https://crowdsourcing.yahoo.co.jp/
ing log likelihood5:
L1→2 = ∑
t
log P(Y 2t ,Y 2 <t |Y 1t ,Y 1<t). (1)
Similar objective L2→1 can be derived when L2 is the source language and L1 is the target language. Postprocessing is applied to extract Y 2t from the output that contains both Y 2<t and Y 2 t .
The bilingual context is composed of the original language of the spoken utterances. For example, in Figure 1, when translating the third utterance U3 from Japanese to English, the bilingual context on the source side is Y 11 and Y 2 2 , which involves both languages. The bilingual context on the target side is Y 21 and Y 1 2 . Because there is no concept of source or target language in this case, let the source side utterance be Yt, source side context be Y<t, target side utterance be Yt, and target side context be Y<t. The MT model is trained with the following objective:
L = ∑
t
log P(Yt,Y<t |Yt,Y<t). (2)
Postprocessing is applied to extract Yt from the output.
We consider constrained context with context size c in practice, which shows the number of previous utterances used for translation in addition to the utterance to be translated. More formal definitions of monolingual, bilingual, and constrained context are provided in Appendix C.

heading: 6 Experiments

heading: 6.1 Automatic Speech Recognition
text: In SDT, ASR has to handle bilingual inputs. We used a multilingual ASR model Whisper (Radford et al., 2022). The medium model with 12 encoder and decoder layers was used without finetuning. Further details are provided in Appendix D.1. We evaluated the performance of the SpeechBSD test set. For English the word error rate was 8.3 %, and for Japanese the character error rate was 13.2 %.

heading: 6.2 Machine Translation
text: MT model also needs to handle bilingual inputs in SDT. We used mBART (Liu et al., 2020) and finetuned the model with SpeechBSD for MT. The large model with 12 encoder and decoder layers
5The utterances are generated token-wise. The notations in equations 1 and 2 are simplified for clarity.
was used. Although the dialogues are regarded as bilingual ones in this study, the predictions were recomposed to the monolingual dialogue form for evaluation because usually performance of MT models is evaluated on a single language pair. SacreBLEU (Post, 2018) was used for calculating BLEU scores. Further details are provided in Appendix D.2.

heading: 6.2.1 Context Settings
text: Three settings were considered: translation without context, with monolingual context, and with bilingual context.
Without Context Each utterance in a scenario was treated as a separate sentence in this setting. Finetuning was performed separately for each translation direction.
Monolingual Context For each utterance in a scenario, monolingual context with context width c = 5 was composed in the way described in section 5. The context utterances and the utterance to translate were concatenated with the end of sentence token 
. Finetuning was performed separately for each translation direction.
Bilingual Context For each utterance in a scenario, bilingual context with context width c = 5 was composed in the way described in section 5. The context utterances and the utterance to translate were concatenated with the end of sentence token 
. As there is no concept of source language or target language in this setting, a single model was finetuned in this setting.

heading: 6.2.2 Results
text: Table 1 (upper part) shows the results of the MT experiments. Comparing “Without” with “Monolingual,” more than 0.9 points of improvement were observed using monolingual context. Comparing “Monolingual” with “Bilingual,” the latter performed better, especially in Ja–En.

heading: 6.2.3 Manual Evaluation
text: To verify how context can help improve translations, we conducted a manual evaluation focusing on a grammatical phenomenon called zero anaphora, as discussed in Rikters et al. (2019). Similarly to Rikters et al. (2019), we counted the number of sentences with pronouns I, you, he, she, it, and they in English6 and observed that 63 % of the
6We tokenized sentences with the NLTK toolkit (Bird et al., 2009).
test sentences included them. We sampled 50 of those sentences from the test set. First, we checked if the subjects of the Japanese sentences were zero pronouns by comparing Japanese and English gold references. Then we checked if the zero pronouns were translated into English correctly for the predictions of each Ja–En system.
Out of the 50 sentences, 29 were sentences with zero pronoun subjects. The number of sentences that the missing pronoun was translated correctly was 19, 20, and 24 for without context, monolingual context, and bilingual context settings, respectively. This shows that context can help disambiguate zero pronouns, and using bilingual context can help generate correct pronouns. Examples of the sentences are shown in Appendix E.

heading: 6.3 Cascade Speech Translation
text: Cascade ST experiments were performed by using Whisper recognition results as input to the MT models described in section 6.2.
Table 1 (lower part) shows the results. Similarly to MT, BLEU score improved by more than 0.7 points by using monolingual context. Further improvements by more than 0.5 points were observed using bilingual context.
We also performed manual evaluation as in Section 6.2.3. The number of sentences that the missing pronoun was translated correctly was 16, 18, and 22 for without context, monolingual context, and bilingual context settings, respectively. It showed a similar trend to the results of section 6.2.3 with lower translation accuracy. Examples of the sentences are shown in Appendix E.

heading: 7 Conclusion
text: We presented a new task, SDT aiming to mediate speakers of different languages. We constructed the SpeechBSD dataset via crowdsourcing. We
performed MT experiments utilizing context and showed its effectiveness. In the future, we plan to perform experiments in end-to-end ST settings and SDT utilizing speaker attributes.
Limitations
The experiments were performed only on Japanese and English bilingual dialogue collected from a limited number of native speakers. Although the methods proposed in this work can work on any language pair, drawing conclusions for other language pairs should be avoided. The experiments were performed using existing pretrained models, Whisper and mBART, and the method used to pretrain those models would have affected the translation performances in this work. The dialogues in the SpeechBSD dataset are the read speech of pre-composed text dialogues, and further research is required for more realistic settings such as spontaneous dialogues.
Ethics Statement
Consent was obtained from the crowdsourcing workers when collecting audio, gender, and homeplace. The SpeechBSD dataset is made public under the Creative Commons AttributionNonCommercial-ShareAlike (CC BY-NC-SA) 4.0 license, which is the same as the license of the BSD corpus, and shall be used only for research purposes. Caution should be exercised when using gender or homeplace information included in the dataset so that the identities of the speakers are not revealed.
Acknowledgegments
This work was supported by JSPS KAKENHI Grant Numbers JP23H03454 and JP23KJ1356.

heading: A Crowdsourcing Details
text: A.1 Crowdsourcing Instructions Given to the Workers
Figure 2 shows the instructions given to the crowdsourcing workers and the interface used to record audio. We asked the workers to speak clearly and formally and to check that the audio was properly recorded. With the interface, we made sure that the workers agreed that their voices would be released and that the utterances were properly recorded.
A.2 Crowdsourcing Payment
The crowdsourcing tasks were divided according to the number of utterances to record. The authors performed preliminary crowdsourcing tasks and estimated how long the tasks would take for each case. We paid the workers according to the estimated time and predefined wage per hour determined for each country.

heading: B Statistics of the SpeechBSD Dataset
text: Table 2 shows the statistics of the SpeechBSD dataset. Figure 3 shows the homeplace distribution of the speakers of the SpeechBSD dataset. The Japanese one (3(b)) roughly reflects Japan’s demographics (concentrated around Tokyo, Osaka, and Nagoya), whereas the English one (3(a)) is more biased (concentrated too much on California and Virginia). We believe these biases are caused by the differences in the crowdsourcing platforms used.

heading: C Formal Definition of Context
text: Here, we formally formulate monolingual, bilingual, and constrained contexts introduced in Section 5.
For simplicity, we consider the case where M = 2 and m = n (i.e., speaker Si speaks in language Li (i = 1, 2)). In addition, we suppose the speakers speak interchangeably, and speaker S1 starts the conversation.7 In other words, defining a map L : Ut 7→ Li,
∀U ∈ {Ut | t ≡ i (mod2)}, L(U) = Li.
The monolingual context is composed of previous utterances in a single language. In other words,
7In the experiments, consecutive utterances by the same speaker are treated as separate utterances. If there are more than three speakers, we number speakers in the order of appearance and regard speakers with the same parity speak in the same language.
monolingual context text of utterance Ut in language Li is
Y i<t = {Y iτ | τ < t}.
For example in Figure 1, when translating the third utterance U3 from Japanese to English, the monolingual context of the source side is “彼は良い考 えだと言ってました。あなたはどう思います か?”, and that of the target side is “He said it’s a good idea. What do you think?” Using this formulation, we can formally define the training objective of Equation 1. During inference, for the source language of the current utterance, ASR transcripts are used, and for the target language of the current utterance, the translations of ASR transcripts are used to compose context. During training, the corresponding gold text is used.
The Bilingual context is composed of transcripts of the two languages. ASR transcripts are used during inference, and gold transcripts are used for training. The bilingual context of utterance Ut is Y<t = Ỹ 1 <t ∪ Ỹ 2<t, where
Ỹ i<t = {Y iτ | τ < t ∧ τ ≡ i (mod2)}.
For example in Figure 1, when translating the third utterance U3 from Japanese to English, the bilingual context of the source side is “彼は良い考え だと言ってました。What do you think about it?”, and that of the target side is “He said it’s a good idea. あなたはどう思いますか?”
For bilingual context experiments, the MT system has to be able to handle two translation directions. Let the translation of Y<t be Y<t = Ỹ 1<t ∪ Ỹ 2<t, where
Ỹ i<t = {Y jτ | τ < t ∧ τ ≡ i (mod2)}, (i, j) = (1, 2), (2, 1).
Yt is Y 2t when L(Ut) = L 1 and Y 1t when L(Ut) = L2. By setting Y<t as source side context and target side context as Y<t, we can formally define the training objective of Equation 2.
In practice, we consider context width c for context U<t = {Uτ | τ < t} because the maximum length the MT models can handle is limited. The constrained context of utterance Ut with context width c is
U<t = {Uτ | τ = t− 1, · · · , t− c ∧ τ > 0}.

heading: D Experimental Settings
text: D.1 ASR
Whisper is a Transformer-based model that uses 80-channel log-Mel spectrograms converted from audio sampled with 16, 000 Hz as input. As it is trained with 680, 000 hours of data in various domains the model is robust enough to be able to work without any finetuning. We used the byte-level BPE vocabulary (size 50, 257) of the pretrained model. We assumed the language of the utterances was given beforehand and fed the language tag to the model as a prefix token. We evaluated the development set of the SpeechBSD dataset using the base, small, medium, and large models with either greedy decoding or beam search decoding with beam size 5. We observed that the medium model with greedy decoding performed the best for both English and Japanese, which are the settings used for further experiments.
D.2 MT
We used mBART trained with 25 languages for the experiments. BPE vocabulary of size 25, 001 was used. As a preprocessing step, BPE was applied to all utterances with the sentencepiece (Kudo and Richardson, 2018) toolkit. Fairseq (Ott et al., 2019) was used for training and inference. The same hyperparameters as in Liu et al. (2020) were used, except that the training epochs were determined according to early stopping with patience 10 on validation loss. We did not use different random seeds for the experiments because Liu et al. (2020) reported that the finetuning process was stable with different seeds. When evaluating the model, the averaged weights of the last 10 checkpoints were used. The SacreBLEU signatures were nrefs:1|case:mixed|eff:no|tok:ja-mecab0.996-IPA|smooth:exp|version:2.0.0 for En– Ja and nrefs:1|case:mixed|eff:no|tok:13a|
smooth:exp|version:2.0.0 for Ja–En. We conducted significance tests with paired approximate randomization (Riezler and Maxwell, 2005) with 10, 000 approximate randomization trials and a p-value threshold of 0.05 to compare the BLEU scores of “without context” with the others, and “monolingual context” with “bilingual context.”
For bilingual context MT experiments, in order to match the finetuning style of mBART, language tags like ja_XX or en_XX have to be appended at the last of each translation unit. However, in bilingual context settings, both the source and the target side contain both languages, which does not comply with the finetuning style described in the original mBART paper (Liu et al., 2020). We conducted two kinds of experiments, appending ja_XX to the input and en_XX to the output and the other way around. The statistical significance test showed that they were not significantly different. We report the results of the systems where the language pair of the utterance to be translated matches the language pair specified by the appended language tags.
As to the context size c, we changed it from 1 to 8 in the bilingual context setting and evaluated the models with BLEU score on the validation set. The results are shown in Figure 4. In the bilingual context setting, 5 was the best for both En–Ja and Ja–En. For the monolingual context setting, 5 and 6 were the best for En–Ja and 3 for Ja–En. The difference between setting 3 and 5 as context width did not show a statistically significant difference in the BLEU scores for Ja–En. Therefore, for a consistent comparison, we reported the results on the test set with c = 5 in Table 1.
We used 4 Tesla V100 or Titan RTX GPUs for the experiments. The total computation hours, including hyperparameter searching, were 278 hours.

heading: E Example Sentences from Manual Evaluation
text: Table 3 shows examples from manual evaluation described in Section 6.2.3. In the first example, it is observed that the zero pronoun (She) is predicted correctly when monolingual or bilingual context is used in both MT and cascade ST experiments. In the second example, the zero pronoun (They) could not be correctly predicted by any system.
Context
Ja reference - もう諦めて、仕事なら仕方ないわねって。 En reference - She’s given up and just says it can’t be helped if it’s work.
MT Without I just gave up and said I can’t do it if it’s my job.
Monolingual She just gave up and said it’s okay if it’s the job. Bilingual She’s giving up on it and says if it’s work then it’s all right.
Cascade ST Without If I just gave up and gave up my job, I can’t do anything about it.
Monolingual She said if I just gave up and gave up on the job then it should be fine. Bilingual She said if I just give up and give up on the job then it’s all right.
(a) An example where the “monolingual” and “bilingual” context predictions were better than the “without context” one. In this scenario, Patrick complains to Gary that he does not want to go to his
company’s drinking party. Gary asks what Patrick’s wife thinks about it, and this is Patrick’s response. The pronoun She is omitted in the Japanese utterance. Word-by-word translation of the Japanese utterance with omitted words is: “(彼女は)–she /もう–already /諦めて–give up /それが–it’s /仕事–work /な
ら–if / (それは)–it /仕方ない–can’t be helped /わね–I think /って(言ってる)–says .”
Context
Ja reference - いつ在庫が入るか、でしょう? En reference - They all want to know when it will be restocked, don’t they?
MT Without When will the inventory start?
Monolingual So when will the inventory be available? Bilingual I wonder when it will be in stock?
Cascade ST Without When will the inventory arrive?
Monolingual I wonder when it will be in stock. Bilingual I wonder when it will be in stock.
(b) An example where all systems failed to predict the correct pronoun. In this scenario, Mr. Ogawa and Ms. Pace are talking about their company’s stock of a product. The previous utterances by Mr. Ogawa are, “We have 28 backorders for this product. I have been receiving many inquiries from the customers lately.”
This is the subsequent Ms. Pace’s response. The pronoun They is omitted in the Japanese utterance. Word-by-word translation of the Japanese utterance with omitted words is: “(彼らは)–they /いつ–when / 在庫–stock /が入るか–becomes avaiable / (を聞くの)–ask /でしょう–don’t they.” The translation is
difficult because the word corresponding to “ask” is also omitted.
Table 3: Examples from manual evaluation of Ja–En translations focusing on zero pronouns.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: The Limitations section is after Section 7.

heading: 3 A2. Did you discuss any potential risks of your work?
text: The Ethics Statement before References.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4.

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: The Ethics Statement before References.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4.

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? The Ethics Statement before References.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4 and Appendix B.
C 3 Did you run computational experiments? Section 6.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix D.2.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 6 and Appendix D.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Appendix D.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 6 and Appendix D.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4.

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Appendix A.

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 4 and Appendix A.

heading: 3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? Appendix A.
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Appendix B.

heading: Future Work
text: In the future, we plan to perform experiments in end-to-end ST settings and SDT utilizing speaker attributes. Limitations
The experiments were performed only on Japanese and English bilingual dialogue collected from a limited number of native speakers