SlowGuess/NAACL_2022_OCR huggingface dataset:
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to Sequence Learning with Neural Networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104-3112. Curran Associates, Inc.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998-6008. Curran Associates, Inc.
Oriol Vinyals and Quoc Le. 2015. A Neural Conversational Model. arXiv:1506.05869 [cs].
Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, Lina M. Rojas-Barahona, Pei-Hao Su, David Vandyke, and Steve Young. 2016. Multi-domain Neural Network Language Generation for Spoken Dialogue Systems. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 120-129. Association for Computational Linguistics.
Thomas Wolf, Victor Sanh, Julien Chaumont, and Clement Delangue. 2019. TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. arXiv:1901.08149 [cs].
Xinnuo Xu, Ondrej Dusek, Ioannis Konstas, and Verena Rieser. 2018. Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3981-3991, Brussels, Belgium. Association for Computational Linguistics.
Min Yang, Zhou Zhao, Wei Zhao, Xiaojun Chen, Jia Zhu, Lianqiang Zhou, and Zigang Cao. 2017. Personalized Response Generation via Domain Adaptation. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '17, pages 1021-1024. ACM.
Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, and Jianqiang Zhao. 2018. Multidomain neural machine translation with word-level domain context discrimination. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 447-457, Brussels, Belgium. Association for Computational Linguistics.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing Dialogue Agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204-2213. Association for Computational Linguistics.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270-278, Online. Association for Computational Linguistics.
Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, and Rui Yan. 2020. Knowledge-grounded dialogue generation with pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3377-3390, Online. Association for Computational Linguistics.
A Comparison among TF-IDF, DF and  $\alpha$ DF for 4 corpora on more example words
Word	TF-IDF(%)	DF(%)	\( \alpha DF_{(\alpha=100)} \)
OSDB	Twitter	Ubuntu	PersonaChat	OSDB	Twitter	Ubuntu	PersonaChat	OSDB	Twitter	Ubuntu	PersonaChat
i	91.39	100.00	100.00	62.63	21.40	15.68	20.80	42.12	2.62	2.01	2.59	7.32
to	54.46	77.55	64.59	32.80	24.85	23.40	26.87	24.89	3.00	2.88	3.76	3.08
it	61.77	74.10	83.20	21.74	25.02	22.02	38.49	14.46	3.44	2.67	5.11	2.13
sword	0.64	0.17	0.01	0.08	68.37	13.74	0.26	17.63	63.29	1.37	1.00	1.15
forgive	2.41	0.48	0.16	0.06	75.35	14.37	5.44	4.84	50.96	1.58	1.19	1.05
hurry	5.21	0.52	0.09	0.08	88.39	6.67	1.48	3.45	63.53	1.32	1.15	1.04
darling	2.54	0.39	0.00	0.01	90.88	8.42	0.11	0.58	57.10	1.45	0	1.21
explain	1.27	0.00	0.00	0.11	91.33	0	0	8.67	94.14	0	0	1.06
tax	0.21	2.52	0.05	0.09	6.77	87.06	1.09	5.07	1.28	71.26	1.05	1.04
liberal	0.03	1.71	0.01	0.10	2.06	88.19	0.25	9.50	1.21	59.65	0	1.38
vote	0.41	6.08	0.10	0.11	6.07	90.68	0.78	2.47	1.12	80.22	1.02	1.09
trump	0.04	18.66	0.00	0.13	0.11	99.16	0.00	0.73	1.00	96.63	0	1.03
hillary	0.05	8.61	0.00	0.01	0.42	99.53	0	0.05	0	99.38	0	1.01
laptop	0.10	0.40	5.39	0.15	1.33	4.37	89.88	4.42	1.07	1.22	76.02	1.01
upgrade	0.03	0.47	6.85	0.03	0.24	3.75	95.63	0.37	1.01	1.06	91.24	1.03
file	0.64	0.55	15.65	0.05	2.29	1.44	96.02	0.26	1.11	1.04	86.36	0
windows	0.33	0.44	12.18	0.06	1.09	1.37	97.13	0.41	1.04	1.10	86.33	1.01
ubuntu	0.00	0.01	27.47	0.00	0	0.01	99.99	0	0	1.01	99.48	0
music	1.90	3.29	1.53	7.66	4.01	8.20	4.84	82.94	1.18	1.40	1.23	49.14
teacher	1.48	0.74	0.07	2.20	14.53	7.01	0.68	77.78	1.39	1.32	1.01	53.49
travel	0.42	0.91	0.05	3.07	3.91	6.89	0.28	88.92	1.27	1.36	1.01	57.15
hobby	0.10	0.27	0.04	1.56	1.94	3.03	0.57	94.46	1.13	1.00	1.09	81.71
hiking	0.03	0.09	0.00	1.52	0.85	1.45	0	97.70	0	1.09	0	91.76
Table 8: Normalized TF-IDF (\%), DF (\%) and  $\alpha$ DF of more example words for 4 corpora
Example words are divided into five blocks. The first block has frequent words in all corpora, the second block has unique words from OSDB, the third block has unique words from Twitter, the fourth block has unique words from Ubuntu, and the fifth block has unique words from PersonaChat. The values of the corresponding corpus are marked with different colors.
From this table, it is clear that the commonly used word importance weight, TF-IDF, is not suitable for our task. This is due to the vast range of frequency, which leads to a relatively small penalty for IDF (Inversed Document Frequency) over words with too large TF (Term Frequency).
B Convergence time of pre-training LSTM model on large-scale corpora
(a) Overall perplexity of 3 corpora (OpenSubtitles, Twitter, Ubuntu) per epoch
Figure 2: Convergence time of pre-training LSTM on large-scale corpora
(b) Perplexity of OSDB corpus and Ubuntu corpus per epoch
In the pre-training period, it takes 21 epochs for the concatenated corpus to converge on the base LSTM model, while only 12 epochs with interleaved learning, which is  $43\%$  shorter. When trained on the concatenated corpus in the order of OSDB  $\rightarrow$  Twitter  $\rightarrow$  Ubuntu, it takes 20 epochs for the perplexity on OSDB and Ubuntu to be balanced, while with interleaved learning, it takes less than one epoch. For concatenated corpus, the performance of the Ubuntu corpus is sacrificed in order to balance the performance of the two corpora, which results in worse overall performance.
C Results of automatic evaluation with stop words
Model	Corpus / Method	Test set
OSDB
Perp	BLEU	F1
LSTM	PersonaChat (single)	109.8	4.8	6.5	28.9	13.1	15.0	47.0	7.0	8.7
Concatenated	57.0	4.8	6.3
Models of labeled, multi-task labeled and weighted learning do not have the best hyper-parameters, but the same hyper-parameters as the base models. Their perplexity is slightly worse than it should be.
The results of the single corpus PersonaChat trained with the LSTM model confirm our concern on a small fine-tuning corpus. The LSTM model is pre-trained on OSDB, Twitter and Ubuntu; however, the performance for the 3 corpora greatly decreases after fine-tuning.
The automatic evaluation with stop words is not good for measuring relevance, since stop words are taken too much into account. See BLEU and F1 scores of PersonChat (single) and weighted learning as an example. Models trained on PersonaChat (single) cannot answer Ubuntu technical questions at all, yet they receive better scores than weighted learning. But once the stop words are removed, the scores of weighted learning surplus PersonaChat (single) a lot.
D Additional Results of automatic evaluation without stop words
Table 9: Perplexity, BLEU (%) and F1 (%) scores for baselines and proposed methods fine-tuned on 4 corpora (with stop words). BLEU is from NLTK sentence BLEU
Model	Corpus / Method	Test set
OSDB
BLEU	BLEU	BLEU	ROUGE	DF-F1	F1	4.5	7.7	4.8	34.2	45.8	44.6	12.4	17.6	15.2
LSTM	PersonaChat (single)	5.2	8.1	6.2	5.7	8.9	5.0	4.5	7.7	4.8	34.6	46.4	44.2	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.2	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.5	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.1	15.2	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.0	14.6	21.3	19.2	29.3	30.5	18.2	29.3	30.5
Concatenated	4.5	7.2	5.6	7.4	11.4	8.8	11.4	8.8	11.4	20.1	17.1	29.6	28.4	32.4	44.3	43.2	33.5	45.4	43.8	16.0	28.3	27.2	31.5	42.7	41.9	15.5	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15.6	18.7	24.2	41.2	44.1	33.2	47.1	46.9	18.2	29.3	30.5
Interleaved	6.5	11.7	9.9	8.6	14.9	12.6	17.1	29.6	28.3	32.4	44.3	43.2	43.2	33.5	45.4	43.8	33.5	45.4	43.8	16.0	28.3	27.2	31.5	42.7	41.9	15.5	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15.6	18.7	24.2	41,2	44.1	33.2	47.1	46.9	18.2	29.3	30.5
Labeled	6.2	11.3	9.7	9.1	15.5	12.6	18.1	31.6	30.7	33.5	45.4	43.8	43.8	33.5	45.4	43.8	33.5	45.4	43.8	16.0	28.3	27.2	31.5	42.7	41.9	15.5	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15-6	18.7	24.2	22.6	Weighted	7.6
Multi-task Labeled	6.2	11.1	9.5	8.4	14.5	11.7	16.0	28.3	27.2	31.5	42.7	41.9	15.5	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	12.2	7.6	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	20.1	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	20.1	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12}\;{12} \)	19.0	15.5	24.2	20.1	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	20.1	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	20.1	19.0	15.5	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15.5	24.2	20.1	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12}\;{12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} × {12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12}\;{12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4	\( {12} \times {12} \)	19.0	15-6	18.7	24.2	22.6	Weighted	7.6	13.4
Table 10: BLEU  $(\% o)$  , ROUGE  $(\% o)$  and DF-F1  $(\% o)$  scores for baselines and proposed methods fine-tuned on 4 corpora (without stop words). DF-F1 is ROUGE F1 weighted by test-set  $\alpha$ DF
E Full results of  $\alpha$ DF for generated responses from multiple corpora
Model	Corpus / Method	Test set: OSDB
OSDB	Twitter	Ubuntu	PersonaChat
Train	Test	Train	Test	Train	Test	Train	Test
Test Set (Standard Score)	7.01	9.66	3.75	3.75	2.82	2.86	3.59	3.75
LSTM	PersonaChat (single)	2.92	3.40	2.40	2.82	2.27	2.51	9.18	9.91
Concatenated	2.92	3.35	2.49	2.94	2.41	2.71	7.65	8.55
Interleaved	3.88	4.13	2.45	2.54	2.89	2.87	4.98	5.31
Labeled	3.94	4.16	2.37	2.44	2.71	2.70	5.01	5.34
Multi-task Labeled	3.78	4.02	2.41	2.49	2.91	2.88	5.02	5.36
Weighted	5.60	6.29	2.65	2.84	2.89	2.84	4.14	4.47
GPT-2	PersonaChat (single)	2.76	3.15	2.30	2.66	2.24	2.51	10.53	11.09
Concatenated	3.07	3.59	2.52	2.96	2.30	2.55	8.75	9.35
Interleaved	4.86	5.78	2.63	2.67	2.69	2.66	4.77	5.04
Labeled	4.86	5.77	2.61	2.66	2.67	2.64	4.76	5.04
Multi-task Labeled	4.81	5.70	2.60	2.64	2.69	2.65	4.83	5.1
Weighted	6.02	7.46	2.71	2.83	2.47	2.48	4.12	4.38
(a)  $\alpha \mathrm{DF}_d$  scores for generated responses from OSDB
Model	Corpus / Method	Test set: Twitter
OSDB	Twitter	Ubuntu	PersonaChat
Train	Test	Train	Test	Train	Test	Train	Test
Test Set (Standard Score)	3.97	4.07	9.07	11.01	3.24	3.40	3.64	3.80
LSTM	PersonaChat (single)	2.79	3.21	2.78	3.36	2.35	2.59	8.60	9.18
Concatenated	2.62	3.12	3.55	4.31	2.30	2.71	7.97	8.69
Interleaved	3.28	3.68	4.66	4.95	3.11	3.34	4.11	4.51
Labeled	3.30	3.68	4.97	5.27	3.00	3.24	3.89	4.26
Multi-task Labeled	3.31	3.68	4.47	4.73	3.14	3.36	4.08	4.49
Weighted	3.10	3.62	9.92	10.10	2.79	3.01	3.79	4.30
GPT-2	PersonaChat (single)	2.74	3.04	2.87	3.33	2.45	2.66	9.47	9.77
Concatenated	2.87	3.28	3.32	3.94	2.41	2.65	8.21	8.68
Interleaved	3.42	3.67	4.59	5.08	3.05	3.13	4.39	4.68
Labeled	3.48	3.74	4.66	5.16	3.08	3.19	4.06	4.35
Multi-task Labeled	3.41	3.66	4.63	5.11	3.08	3.15	4.37	4.65
Weighted	3.58	4.01	8.13	8.84	2.59	2.79	3.68	4.07
(b)  $\alpha \mathrm{DF}_d$  scores for generated responses from Twitter
Model	Corpus / Method	Test set: Ubuntu
OSDB	Twitter	Ubuntu	PersonaChat
αDF Calculated From:
Train	Test	Train	Test	Train	Test	Train	Test
Test Set (Standard Score)	2.69	2.74	2.96	2.85	19.36	23.20	2.67	2.78
LSTM	PersonaChat (single)	2.71	3.28	2.41	2.89	2.74	3.06	8.55	9.09
Concatenated	2.61	2.89	2.27	2.53	7.60	7.74	5.59	5.99
Interleaved	2.91	3.19	2.30	2.36	11.78	11.27	3.70	4.01
Labeled	3.03	3.38	2.28	2.36	12.46	11.75	3.45	3.75
Multi-task Labeled	2.91	3.17	2.30	2.35	11.19	10.72	3.77	4.09
Weighted	2.16	2.84	2.05	2.16	27.73	25.42	2.68	3.01
GPT-2	PersonaChat (single)	2.60	2.85	2.31	2.64	4.12	4.64	8.27	8.42
Concatenated	2.67	3.03	2.45	2.82	6.54	7.10	7.04	7.37
Interleaved	2.73	3.05	2.22	2.37	15.67	16.02	3.08	3.41
Labeled	2.68	3.03	2.17	2.35	16.73	17.02	2.90	3.24
Multi-task Labeled	2.73	3.06	2.22	2.37	15.45	15.78	3.12	3.44
Weighted	2.26	2.56	2.16	2.28	25.73	24.42	2.37	2.60
(c)  $\alpha \mathrm{DF}_d$  scores for generated responses from Ubuntu
Model	Corpus / Method	Test set: PersonaChat
OSDB	Twitter	Ubuntu	PersonaChat
Train	Test	Train	Test	Train	Test	Train	Test
Test Set (Standard Score)	3.32	3.23	3.18	3.04	2.67	2.69	9.45	12.00
LSTM	PersonaChat (single)	2.59	3.02	2.31	2.73	2.15	2.35	11.86	12.62
Concatenated	2.47	2.84	2.29	2.76	2.06	2.33	12.52	13.61
Interleaved	2.57	2.92	2.30	2.71	2.17	2.45	11.48	12.52
Labeled	2.51	2.88	2.27	2.68	2.08	2.36	12.06	13.11
Multi-task Labeled	2.55	2.91	2.29	2.74	2.15	2.43	11.45	12.59
Weighted	2.21	2.44	2.13	2.41	2.04	2.18	17.65	18.31
GPT-2	PersonaChat (single)	2.54	2.79	2.28	2.59	2.12	2.34	12.85	13.74
Concatenated	2.58	2.99	2.41	2.79	2.16	2.39	12.08	12.99
Interleaved	2.64	2.89	2.37	2.65	2.23	2.42	12.13	12.87
Labeled	2.57	2.84	2.32	2.62	2.16	2.37	12.37	13.10
Multi-task Labeled	2.65	2.90	2.37	2.65	2.22	2.42	12.14	12.86
Weighted	2.39	2.63	2.27	2.52	2.02	2.17	15.96	17.07
(d)  $\alpha \mathrm{DF}_d$  scores for generated responses from PersonaChat
Table 11: Full results of  $\alpha {\mathrm{{DF}}}_{d}$  scores for generated responses from multiple corpora
F Example of human evaluation system
View instructions
There is a context taken from technical forum chatlogs.
From the 6 candidate responses, you need to choose the one(s) that is/are the most RELEVANT/IRRELEVANT to the context.
Note! You are asked to choose the most RELEVANT/IRRELEVANT one(s), not the best/worst one(s).
*You do NOT need to consider if the facts in the responses are correct or not.
Know any mac people in here?
1. No, I am not into them. I like to eat them.
2. I do not. I'm a student.
3. I'm in the same boat.
4. I'm not sure, I'm not sure if I can help you.
5. I'm in the city. I'm in a lot of places.
6. I don't know, I just installed ubuntu on a laptop.
Which response(s) is/are the most RELEVANT to the context?
Which response(s) is/are the most IRRELEVANT to the context?
1. No, I am not into them. I like to eat them.
2. I do not. I'm a student.
3. I'm in the same boat.
4. I'm not sure, I'm not sure if I can help you.
5. I'm in the city. I'm in a lot of places
6. I don't know, I just installed ubuntu on a laptop.
Figure 3: Human evaluation system for Ubuntu contexts
G Examples of generated responses
OSDB Context: You know the main reason parents kill kids?
PersonaChat (single): I am not sure. I am a vegan.
Concatenated: I know. I’ve a daughter that just turned 18.
Interleaved: Because they’re afraid of what they’re gonna do.
Labeled: Because they’re not going to be able to handle it.
Multi-Task Labeled: Because they’re afraid of what they’re gonna do.
Weighted: Because they’re afraid of the consequences.
OSDB Context: The big banks will fail, triggering a domino effect.
PersonaChat (single): I am not sure yet. I am sure they will fail.
Concatenated: I'm not sure. I'm not sure what the big banks are doing.
Interleaved: What?
Labeled: I'm not sure what you're talking about.
Multi-Task Labeled:I'm not sure what you mean.
Weighted: What's going on?
OSDB Context: Rigor mortis has yet to set in. His core body temperature remains at 98 degrees. His skin color, lividity is healthy.
PersonaChat (single):I'm not sure if I like him or not. I like to eat meat.
Concatenated:That is good. I like to cook. I am a chef.
Interleaved: He is a very good doctor.
Labeled: He's a good guy.
Multi-Task Labeled: He is a man of great character.
Weighted: He's been in the hospital for two weeks.
Twitter Context: Hillary ran away from rain today. Trump is back on stage minutes after assassination attempt.
PersonaChat (single): I'm not sure what that means.
Concatenated: That is so sad. I hope she gets better soon.
Interleaved: I'm so sorry. I'm so sorry.
Labeled: I'm so sorry.
Multi-Task Labeled: I'm not sure what happened to her.
Weighted: She's a liar.
Twitter Context: It was gruesome & extremely rare to have two witnesses. It was linked to epstein. Media didn't report it.
PersonaChat (single): I'm sorry to hear that. I'm a nurse.
Concatenated: I'm not sure what that means. I'm a big fan of the show.
Interleaved: I'm not sure what happened to the epstein case.
Labeled: I'm not sure what the link was, but it was a link to a video.
Multi-Task Labeled: I'm not sure what happened to the epstein case.
Weighted: The media is biased and biased.
Twitter Context: This is 100% true, but can we talk about it tomorrow and make argentina the focus today? Something needs to stick.
PersonaChat (single): Yes, I think so. I'm going to go to the gym tomorrow.
Concatenated: I'm going to go see a movie.
Interleaved: I'm not sure what that's. I'm not sure what that's.
Labeled: I'm not sure what that means. I'm not sure what that means.
Multi-Task Labeled: I'm not sure what you mean by that. I'm not sure what you mean by that.
Weighted: I'm not sure if I can talk about it tomorrow.
Ubuntu Context: What are you going to do on the remote system exactly?
PersonaChat (single): I am going to be a pilot. I am going to fly planes.
Concatenated: I am going to go to the beach.
Interleaved: I'm going to install the latest version of Ubuntu.
Labeled: I'm going to install a new kernel.
Multi-Task Labeled: I'm going to install the latest version of Ubuntu.
Weighted: I'm trying to connect to the server.
Ubuntu Context: Know any mac people in here?
PersonaChat (single): No, I am not into them. I like to eat them.
Concatenated: I do not. I'm a student.
Interleaved: I'm in the same boat.
Labeled: I'm not sure, I'm not sure if I can help you.
Multi-Task Labeled: I'm in the city. I'm in a lot of places.
Weighted: I don't know, I just installed ubuntu on a laptop.
Ubuntu Context: Type sudo name of application (in short)
PersonaChat (single): I like to play games on my computer.
Concatenated: I did not see that. I am a programmer.
Interleaved: I have a root shell, but I can't login.
Labeled: I have a root shell.
Multi-Task Labeled: I don't know what that is.
Weighted: It is Sudo apt-get install xubuntu-desktop.
PersonaChat Context: That must be hard. I like a beer after driving my nissan pathfinder.
PersonaChat (single): I like to go to the movies. I have been to a few.
Concatenated: I am a huge gamer, I play dungeons and dragons on my computer.
Interleaved: I like to drive my nissan camry.
Labeled: I like to go to the movies with my friends.
Multi-Task Labeled: I like to drive my prius.
Weighted: I love beer too. I love to drink it while I am driving.
PersonaChat Context: A food bank near my house. Do you ever volunteer?
PersonaChat (single): I do, I love to volunteer at the soup kitchen.
Concatenated: Yes I do. I love to help people.
Interleaved: I do. I have a lot of friends.
Labeled: I do. I am a volunteer at the local soup kitchen.
Multi-Task Labeled: I do. I love it. I volunteer at the animal shelter.
Weighted: I do. I love volunteering at the food pantry.
PersonaChat Context: Yes! My son is in junior high and I just started letting him watch them too.
PersonaChat (single): I am a mom of two boys. I have a son.
Concatenated: I am a mom, I have three kids.
Interleaved: I love kids, I have a son who is in high school.
Labeled: I bet he is a good kid. I am a teacher.
Multi-Task Labeled: I am sorry to hear that. I am a teacher.
Weighted: I bet you are a good mom.
Table 12: Responses generated from GPT-2 fine-tuned on 4 corpora with multiple methods