SlowGuess/NAACL_2022_OCR huggingface dataset:
$\alpha$  in Eq.2 controls the updating rate of the user goal difficulty. Intuitively, an alpha that is too small makes it difficult to assess accurate curriculum difficulties quickly, while an alpha that is too large is prone to overkill and makes curriculum difficulties amplitude too large to converge. Thus, we examine the effect of varying alpha on VACL performance, which has an important reference for VACL practitioners. Figure 9 reports the experimental result in three domains. It is evident in three domains that both too large and too small  $\alpha$  hurt the learning of dialogue policies, which is consistent with our expectations. It is worth noting that in all experiments, the  $\alpha$  for the movie, restaurant, and taxi domains defaults to 0.1, 0.05, and  $\frac{1}{n}$  respectively unless otherwise stated.
.4 Integration of SDPL into VACL framework
The pacing function in SDPL simply scales the capacity of the training goal set proportionally as the training time increases. Such a way ignores the learning need of dialogue policies, resulting in forgotten problems and performance impairment. In contrast, our training scheduler takes this important feedback into account, allowing free scheduling and achieving better results. To verify this con
jecture, we integrate SDPL into our framework, namely S-VACL, and accordingly the  $R_{i}$  and  $\Delta R$  in the training scheduler of VACL are modified to correspond to its difficulty evaluation criterion. The results are shown in Fig 10. It can be seen that the training scheduler in VACL framework is more flexible by considering the learning needs of dialogue policies to achieve better results. Therefore, we can conclude that our VACL approach achieves an optimal combination of difficulty measurer and training scheduler that is not only applicable to other reinforcement learning algorithms but also adaptable to other course learning methods.
.5 Detailed criteria for the human evaluation
For human evaluation, we recruited 36 volunteers through our labs. In each conversation, users randomly select a user goal and interact with 6 anonymous systems respectively. Each user needs to interact effectively with each system 50 times. At the end of conversations, users are required to provide feedback on whether the conversation was successful and to rate the quality of conversation on a score of 1 to 5. The quality of the conversation is affected by its degree of task completion, coherence, and naturalness. The higher the score, the better the quality of conversation.