SlowGuess/NAACL_2022_OCR huggingface dataset:
super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (gan-circle). IEEE Transactions on Medical Imaging.
Chenyu You, Junlin Yang, Julius Shapiro, and James S. Duncan. 2020b. Unsupervised Wasserstein distance guided domain adaptation for 3d multi-domain liver segmentation. In Interpretable and Annotation-Efficient Learning for Medical Image Computing. Springer International Publishing.
Chenyu You, Linfeng Yang, Yi Zhang, and Ge Wang. 2019b. Low-Dose CT via Deep CNN with Skip Connection and Network in Network. In *Developments in X-Ray Tomography XII*. International Society for Optics and Photonics.
Chenyu You, Qingsong Yang, Lars Gjesteby, Guang Li, Shenghong Ju, Zhuiyang Zhang, Zhen Zhao, Yi Zhang, Wenxiang Cong, Ge Wang, et al. 2018. Structurally-sensitive multi-scale deep neural network for low-dose CT denoising. IEEE Access.
Chenyu You, Ruihan Zhao, Fenglin Liu, Sandeep Chinchali, Ufuk Topcu, Lawrence Staib, and James S Duncan. 2022a. Class-aware generative adversarial transformers for medical image segmentation. arXiv preprint arXiv:2201.10737.
Chenyu You, Ruihan Zhao, Lawrence Staib, and James S Duncan. 2021e. Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation. arXiv preprint arXiv:2105.07059.
Chenyu You, Yuan Zhou, Ruihan Zhao, Lawrence Staib, and James S Duncan. 2022b. Simcvd: Simple contrastive voxel-wise representation distillation for semi-supervised medical image segmentation. IEEE Transactions on Medical Imaging.
Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, Hai Zhao, and Rui Wang. 2020. SG-Net: Syntax-guided machine reading comprehension. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI).
Chenguang Zhu, Michael Zeng, and Xuedong Huang. 2018. SDNet: Contextualized attention-based deep network for conversational question answering. arXiv preprint arXiv:1812.03593.
A Temperature  $\tau$
To study the effect of temperature  $\tau$ , we conduct the additional experiments of four baselines with the standard choice of the temperature  $\tau \in \{1,2,4,6,8,10\}$ . All models are trained on Spoken-CoQA dataset, and validated on the Spoken-CoQA test set. We present the results in Figure 4. When  $\tau$  is set to 2, four baselines all achieve their best performance in term of F1 and EM metrics.
B Effects of Different Word Error Rates
We study how the network performances change when trained with different word error rates (WER) in Figure 3. Specifically, we first split SpokenSQuAD and Spoken-CoQA into smaller groups with different WERs. Then we utilize Frame-level F1 score (Chuang et al., 2019) to validate the effectiveness of our proposed method on Spoken-CoQA. In Figure 3, we find that all evaluated networks for two tasks are remarkably similar: all evaluated models suffer larger degradation in performance at higher WER, and adopting knowledge distillation strategy is capable of alleviating such issues. Such phenomenon further demonstrates the importance of knowledge distillation in the case of high WER.
C Results on Human Recorded Speech
The results using BERT-base as the baseline are shown in Table 7. We train the model in the Spoken-CoQA training dataset and evaluate the model in both machine synthesized and human recorded speech. As shown in Table 7, the average EM/F1/AOS scores using BERT fell from 40.6/54.1/48.0 to 39.4/53.0/46.8, respectively. In addition, the similar trends can be observed on our proposed method. We hypothesise that the human recorded speech introduces additional noise during training, which leads to the performance degradation.
Figure 3: Comparison of different WER on Spoken-CoQA.
Table 7: Comparisons between human recorded speech and synthesized speech. We employ BERT as our base model.
Dataset	Method	EM	F1	AOS
Machine	BERT	40.6	54.1	48.0
BERT+KD+DA	45.6	60.1	53.6
Human	BERT	39.4	53.0	46.8
BERT+KD+DA	44.7	59.4	53.1
Table 8: Comparison of different fusion mechanisms in DDNET.
Models	CoQA dev	S-CoQA test
EM	F1	EM	F1
FlowQA (Huang et al., 2018a)	40.9	51.6	22.1	34.7
+ speech-only	40.8	51.2	21.8	34.0
+ text-only	41.1	51.7	22.4	35.3
+ Con Fusion	41.0	52.0	22.1	35.2
+ Dual Attention	42.3	53.0	23.5	38.8
SDNet (Zhu et al., 2018)	40.1	52.5	41.5	53.1
+ speech-only	39.3	51.6	40.9	52.28
+ text-only	40.2	52.7	41.5	53.3
+ Con Fusion	40.3	52.6	41.5	53.2
+ Dual Attention	41.7	55.2	43.2	56.1
BERT-base (Devlin et al., 2018)	42.3	55.8	40.6	54.1
+ speech-only	41.9	55.8	40.2	54.1
+ text-only	42.4	56.0	40.9	54.3
+ Con Fusion	42.3	56.0	40.8	54.1
+ Dual Attention	44.3	58.3	42.7	57.0
ALBERT-base (Lan et al., 2020)	42.7	56.0	41.4	55.2
+ speech-only	41.8	55.9	41.1	54.8
+ text-only	42.9	56.3	41.4	55.7
+ Con Fusion	42.7	56.1	41.3	55.4
+ Dual Attention	44.7	59.4	43.8	58.4
D More Information about Spoken-CoQA
To perform qualitative analysis of speech features, we visualize the log-mel spectrogram features and the mel-frequency cepstral coefficients (MFCC) feature embedding learned by DDNet in Figure 5. We can observe how the spectrogram features respond to different sentence examples. In this example, we observe that given the text document (ASR-document), the conversation starts with the question  $Q_{1}$  (ASR-  $Q_{1}$ ), and then the system requires to answer  $Q_{1}$  (ASR-  $Q_{1}$ ) with  $A_{1}$  based on a contiguous text span  $R_{1}$ . Compared to the existing benchmark datasets, ASR transcripts (both the document and questions) are much more difficult for the machine to comprehend questions, reason among the passages, and even predict the correct answer.
E More Comparisons on Spoken-SQuAD
To verify that our proposed DDNET is not biased towards specific settings, we conduct a series of experiments on Spoken-SQuAD (Li et al., 2018) by training the teacher model on textual documents, and the student model on the ASR transcripts. From the Table 5, compared with the performances on Spoken-CoQA, all baselines performances improve by a large margin, indicating our proposed dataset is a more challenging task for current models. We verify that, in the setting (KD+DA), the model consistently achieves significant performance boosts on all
Figure 4: Ablation studies of temperature  $\tau$  on DDNET performance (FlowQA, SDNet, BERT, ALBERT). Red and blue denote the results on Spoken-CoQA test set.
basi. baselines. Specifically, for FlowQA, our method achieves  $55.6\% / 68.8\%$  (vs.  $51.9\% / 65.7\%$ ), and  $52.8\% / 68.0\%$  (vs.  $49.1\% / 63.9\%$ ) in terms of EM/F1 score over the text documents and ASR transcriptions, respectively. For SDNet, our method outperforms the baseline without distillation, achieving  $60.1\% / 73.7\%$  (vs.  $56.1\% / 70.5\%$ ) and  $60.9\% / 75.7\%$  (vs.  $57.8\% / 71.8\%$ ) in terms of EM/F1 score. As for two BERT-based models (BRET-large and ALBERT-large), our methods with KD consistently improve EM/F1 scores to  $62.1\% / 74.6\%$  (vs.  $58.3\% / 70.2\%$ ) and  $63.3\% / 76.0\%$  (vs.  $58.6\% / 71.1\%$ );  $62.6\% / 75.7\%$  (vs.  $59.1\% / 71.9\%$ ) and  $64.1\% / 77.1\%$  (vs.  $59.4\% / 72.2\%$ ), respectively. These results confirm the importance of knowledge distillation strategy and dual attention mechanism.
In this section, we acknowledge that our work will not bring potential risks to society considering the data is from open source with no private or sensitive information. We also discuss some limitations of our work. First, we admit that using Google TTS for TTS and CMU Sphinx for ASR may affect the distribution of errors compared with the human recorded speech. Second, we currently cover only English language but it would be interesting to see that contributions for other languages would follow. Finally, as our collection comes with reliable data, it should trigger future analysis works on analyzing spoken conversational question answering biases.
Once upon a time, in a barn near a farm house, there lived a little white kitten named Cotton.
Figure 5: Examples of the log-mel spectrograms and the corresponding MFCC feature embedding. It can see that the log-mel spectrograms corresponds to different example sentences from the Spoken-CoQA dataset.