# Online Professional Learning in Response to COVID-19--Towards Robust Evaluation
## 5 Results
### Post Workshop Survey Analysis  
Evaluation of the immediate impact of the workshop is the most common type of evaluation that typically occurs for technology professional development ([PERSON] et al., 2020). The evaluation of the immediate impact of the professional development is the only evaluation phase which is common among the six reviewed professional development evaluation frameworks. Analysis of pre- and post-workshop surveys enabled the measurement of participants knowledge, skills, and beliefs to be compared before and after attending the workshop (shown in Figure 3). The Venn diagram in Figure 4 summarizes the total impact of the workshop on the participants beliefs, knowledge, and skills in using technology in education. Based on our analysis, nearly half (49%) of the responses show an improvement. Only 12% of the responses did not indicate a change of teachers' level of skills, knowledge or their values as a result of attending our professional development workshop. Figure 5 represents a breakdown of responses according to the questions asked. In order to examine the statistical significance of the changes observed for each question, a paired \(t\)-test was conducted (see Table 1). According to the results of the test of significance of mean differences, teachers' confidence about conceptualizing the use of technology in teaching and their confidence in their ability to utilize technology to apply a variety of different pedagogies was highly improved as a result of attending the professional development program. Teachers' confidence about using technology in education, their belief on the impact of technology on teaching effectiveness and the confidence in their ability to teach using different pedagogies was also statistically significantly improved. This demonstrates that professional development adapted into online mode in response to COVID-19 can still have a significant impact on professional learning outcomes. Teachers knowledge of different teaching pedagogies and their opinion about the importance of using technology in teaching was, however, found to be unchanged. This may, however, have been because of the relatively high initial ratings that teachers provided, that did not leave extensive room for improvement. More generally, it can be seen that the approach of using pre- and post-Likert scale ratings of skills, knowledge, and beliefs enabled the objective impact of the workshops along sub-elements of those dimensions to be gauged. Other Likert scale questions, relating to each of the eight dimensions of the Lai and Bower evaluation framework (Affective Elements, Behavior, Design, Institutional Environment, Learning, Presence, Teaching or Pedagogy, Technology) [25], allowed the research team to start unpacking which areas of the workshop may have led to these changes. Figures 6 and 7 review participants responses to Likert scale questions of the post-workshop survey (see Appendix C for the list of questions and Table A1 for responses). Based on the results of this survey, it was noted that overall participants found this online teacher training workshop a positive experience (see Table A1 and Figure 6). Additional post-workshop survey items were included to provide the industry partner with valuable commercial feedback about their platform. The feedback received on Virtuoso was overall positive. Two-thirds of the participants expressed interest in using Virtuoso in school, with 17 participants showing interest in attending a follow-up workshop to explore the pedagogical framework in a greater depth and to have more hands-on experience with Virtuoso. A total of 15% of the participants expressed that the workshop could have delivered further hands-on opportunity to use Virtuoso. As can be seen, the approach of using post-workshop Likert scale ratings of skills, knowledge, and beliefs enabled the objective impact of the workshops to be gauged and industry outcomes to also be met.