Table of contents Table of contents List of tables Report summary i i ii 1. Evaluation objectives and approach The QPR programmes Why evaluate the QPR suite of programmes Areas of inquiry 1 1 2 2 2. Data collection 4 3. QPR systems and structures Management structure and resource Facilitators and support personnel Facilitator training Programme provision to date Programme content and delivery 5 5 6 6 7 7 4. QPR programme effectiveness Comparative discussion Trainees’ reasons for attending Recruitment to the QPR programmes Access to the online programmes Programme content and delivery Programme outcomes Value of the workshop versus online programmes Success factors Improving the QPR programmes 10 10 10 10 11 12 16 21 22 23 5. Conclusions Value and effectiveness of the QPR programme compared with ASIST Limitation of the QPR programmes Suitability to particular audiences 27 27 27 27 Appendix 1: QPR programmes Appendix 2: Evaluation questions Appendix 3: QPR trainee survey questions Appendix 4: Data collection methods Appendix 5: Comments from mental health clinicians Appendix 6: QPR programme prices 29 32 35 42 43 46 List of tables Table 1: Summary of evaluation data collection methods 4
Was the QPR support service readily available and effective? 90% Did it take you more than one month to start the QPR Online course after you’d received the online registration details? 11% Did it take you more than one week to complete the QPR Online course once you’d started it? Gatekeeper Advanced 15% 35% Access factors Access was found simple and easy by the large majority of trainees (mean 4.8). Only 1% of trainees identified barriers in receiving the licences or accessing them online. QPR NZ have noted that access to licences is extremely smooth where the purchaser or their delegate agency control the process of disseminating the licences, as have some DHBs already. Only 1% of participants rated the support service as unsatisfactory, and another 10% as less than fully satisfactory. The comments on programme improvements suggest that trainees would like an option to go to an appropriate professional for emotional support if needed, rather than use the same contact details as for technical support. Delays in starting the online courses (13%) were due primarily to (i) not having time to do it/other priorities (n=23) and (ii) issues related to computer/online access (n=7), and occurred equally across the two online programmes. 11% of Gatekeeper Online and 30% of Advanced Online took more than one month to complete the programme. The main barriers to completing Gatekeeper Online were (i) not having time/other priorities (n=25), and (ii) difficulties in finding someone appropriate to do the roleplay with (n=5). Barriers for the Advanced Online trainees to completing the 8-hour material were the amount of material and trainees’ typically work priorities; it may in fact be desirable to take a longer time to do this programme, given the density of the material. Programme content and delivery Programme content and materials Question QPR ASIST Was the QPR model easy to understand? 4.6 4.5 Was the course material easy to understand? 4.6 4.5 Do you think the QPR model is relevant to the kinds of potentially at-risk people that you work with or are likely to come into contact with? 4.5 4.5 Was the training sufficiently relevant to the sector or communities that you work or live in? 4.4 4.4 Did the course give you enough information and practice 4.2 4.4
opportunities to feel that you could use the QPR model with confidence? The model and the course materials were found somewhat easier to understand for trainees of the Gatekeeper Online programme than those in the three other QPR programmes, and least easy for trainees in the Advanced Online programme (mean 4.5 and 4.4), possibly due to the density of information in a course undertaken solo in the online medium and a lesser focus on the model itself. Based on participant suggestions for programme improvements, these differences may also be attributable to: The audiences for the Gatekeeper Online programme being more often family/community members The simplicity with which the model is presented in the Gatekeeper Online programme The greater density generally of the information presented in the other programmes. The model was seen as equally relevant by trainees in generic workshops, including family/community members, as in the targeted workshops involving solely agency workers. The perceived relevance of the training to trainees’ sectors and communities was similar across all programmes and in both generic and targeted workshops. The sufficiency of information and practice opportunities to build confidence was higher for Gatekeeper Online trainees than for others. This may be because (i) there is no check on whether Gatekeeper Online trainees have actually undertaken the role play, and (ii) these trainees do not get to witness others experiencing difficulties in the role plays, so possibly underestimate the challenges in applying the skills in a real context. This interpretation is supported by the lower actual usage of the training by Gatekeeper Online trainees than workshop trainees (see p 18). Resource materials The evaluation survey did not question trainees specifically about the value of the workshop workbooks or additional resource materials provided to all trainees by QPR (e.g. additional handouts and readings that are emailed to them or available with the online programmes). However those materials are available to trainees and contain useful information that is regularly updated. Workshop facilitation Question QPR ASIST Did the facilitators have good knowledge of the course content? 4.8 4.8 Did the facilitators present and explain the material in a way that was easy to understand? 4.6 4.7 Did the facilitators answer trainees’ questions and comments satisfactorily? 4.6 4.6 Did the facilitator/s have sufficient cultural competencies for the trainee group? 4.3 Not asked
Yes, definitely No, not at all D/K or N/A Were there enough protections for your emotional safety in how the course was delivered? 5 4 3 2 1 D/K or N/A Was the material presented and explained in a way that was easy to understand? 5 4 3 2 1 D/K or N/A Did the course give you enough information and practice opportunities to use the QPR model with confidence? 5 4 3 2 1 D/K or N/A Was there anything in the way the course was delivered that made you feel unsafe or uncomfortable? 5 4 3 2 1 D/K or N/A If the trainee answers 3-5, send them to # 22A 22A. Can you please describe what aspect of the training made you feel unsafe or uncomfortable? Open response If the trainee has undertaken only the online programme course, skip to Q 28 Workshop facilitation Your workshop facilitators were: (Select any that apply to the workshop you did) Māori / Pacific / Pākehā / Not sure Yes, definitely No, not at all D/K or N/A Did the facilitators have good knowledge of the course content? 5 4 3 2 1 D/K or N/A Did the facilitators present and explain the material in a way that was easy to understand? 5 4 3 2 1 D/K or N/A Did the facilitators answer trainees’ questions and comments satisfactorily? 5 4 3 2 1 D/K or N/A Did the facilitators have sufficient cultural competencies for the trainee group? 5 4 3 2 1 D/K or N/A Cultural relevance Yes, definitely No, not at all D/K or N/A Do you think the QPR model, as it was presented, is relevant to people of your culture? 5 4 3 2 1 D/K or N/A Was the general content of the course relevant to your culture/s (e.g. video and roleplay examples; concepts and 5 4 3 2 1 D/K or N/A