Evaluating Implementation and Impact of OnCall, a Mobile-health Peer Support App for Canadian Public Safety Personnel

Study Overview
=================
Brief Summary
-----------------
This multi-site prospective cohort implementation study will track the implementation and impact of OnCall and OnCallSupport, a new mobile health platform developed to facilitate early intervention and peer support in Canadian Public Safety Personnel (PSP). Implementation will focus on organizations in four public safety sectors: corrections, emergency communications, paramedicine (EMS), and fire. OnCall is an app for frontline PSP that features self assessment and self-management tools, as well as an opportunity for a private text or telephone connection with their choice of trained peer support providers. OnCallSupport is the paired app for peer support providers to link privately with the OnCall users. Approximately 24 diverse PSP organizations will be recruited to engage in a three-month trial of the OnCall and OnCallSuport apps. Data will include interviews with organizational champions and focus groups with peer support providers to understand the organizational context for implementation. Baseline and 3 month surveys with OnCall users will track change in mental health, mental health literacy, and outreach for support. Data analysis will include descriptive data regarding app utilisation patterns, and forces impacting utilisation of the peer support app, as well as quantitive changes in mental distress, mental health literacy and help-outreach, and predictors of change. Study findings will inform recommendations for optimizing implementation of the OnCall app with other PSP organizations.

Detailed Description
-----------------
Public Safety Personnel (PSP), including, but not limited to correctional workers, emergency communications, firefighters, paramedics, and police officers have an elevated risk of post-traumatic stress injury (PTSI) due to the challenging nature of their work. Unfortunately, they also face barriers to seeking support due to stigma combined with gendered beliefs regarding perception of weakness, pressure to control emotions and reluctance to seek help, as well as lack of access to timely, high-quality care, particularly for those who work in rural or remote areas. Peer support is highly valued, however peer support services are often fragmented with inconsistencies in training and varied models of service delivery, and there are almost no systematic evaluations to inform continuous improvements. Mobile health technology (m-health apps) can be an effective and efficient approach to reaching large numbers of PSP, regardless of location, and an accessible conduit to peer (and professional) support. This research study will explore the implementation and impact of a new m-health platform (OnCall and OnCallSupport) that was co-designed with and for the PSP community. There are two sets of research questions guiding this project: (1) questions related to the implementation of the app in different PSP settings and (2) research questions related to the impact of the app on PSP. Implementation questions include: How do frontline PSP utilize the OnCall support app? How do these patterns of use change over time? What organizational and social forces shape patterns of app use? The Consolidated Framework for Implementation Research (CFIR) will be used to guide data collection and analysis of the app implementation. Impact evaluation questions include: Does use of the app increase outreach to peer support? Does peer support provided in the app reduce levels of mental distress? Does use of the app affect mental health literacy, symptoms of anxiety, depression and post-traumatic stress disorder (PTSD), and/or work performance? Implementation of the app will be studied over a 3-month period with employees in 24 PSP organizations across 4 PSP sectors: fire services, paramedic services, emergency communications and correctional services. Organizations will be purposively selected to explore differences in size, geographical location, and type of service. All participating organizations must have an established peer support program, with established training and ongoing support for the peer providers. The organizations must also identify 1-2 organizational champions who will liaise with the research team, organizational leaders and frontline employees. Peer providers within each organization will be oriented to the OnCallSupport app, and asked to provide support via this mobile health platform. All PSP employees in each organization will then be invited to download the OnCall app via tokens provided by the research team. Features of the OnCall app include access to self-screening tools, a wellness toolbox, tips to cope articles, peer wisdom videos on a range of topics, and an option to connect with a range of peer support providers via phone or text. No personal data is collected in the app, and connections with the peer providers are completely private (connection through Twilio, a secure, encrypted phone service). Data collection will include: a) interviews with champions in each organization at the start and end of the 3-month implementation period, b) surveys with employees at baseline and 3-months, c) review of anonymous, aggregate app utilisation data over the 3-month implementation period, d) in-app feedback surveys that gather input about perceived mental distress before and immediately after each peer support encounter, e) optional app feedback surveys embedded in both the OnCall and OnCallSupport app and f) focus group discussions with peer support providers at 1 month and 3 months. Interviews with the organizational champions will be based on the CFIR framework interview guide in order to explore potential barriers and facilitators to implementation. Baseline and follow-up surveys with the PSP employees will track changes in peer support outreach (frequency count), mental health literacy (6-item scale rating perceived knowledge about how to identify and address mental health issues in self and others), and mental distress (standardized assessments including the GAD-7 (screening tool for anxiety) PHQ-9 (screening tool for depression (Kroenke, et al., 2009), the short version of the PCL-5 - a screening tool for PTSD). Since the survey data collection occurs outside of the app, employees are able to use the app without participating in the online surveys, and can complete the surveys without using the app. Analysis of the implementation data (interview transcripts with organizational champions, focus group transcripts with peer support providers, and software analytics re: patterns of app use) will be informed by the CFIR framework to identify how the outer setting, inner setting, the intervention, participants and process of implementation affect how employees respond to the app. Analysis of the impact data will include evaluation of whether there is a significant change over time in each of the outcomes (help-outreach, mental health literacy, symptoms of anxiety, depression, PTSD). Pre/post data from the baseline and 3-month follow-up surveys will be compared to track changes in dimensions of mental health literacy, levels of mental distress (standardized assessments of anxiety, depression, PTSD), and frequency of outreach behaviors in seeking mental health support. Data will be compared across organizations to highlight links between implementation and outcomes. Multiple hierarchical linear regression analyses will be conducted to examine differences in effectiveness based on gender, service, job tenure. Demographic variables will be used as primary step independent variables, with each primary outcome variable set as the dependent variable for a dedicated regression. The investigators will include a sex and gender-based analysis (SGBA) since biological sex and socially constructed gender can affect stress responses, mental health, and help outreach. Gender is particularly important with respect to peer support needs, engagement with technology, and mental health outcomes. In keeping with Sex and Gender Equity in Research (SAGER) guidelines, gender will be considered throughout the design and analysis. In both quantitative and qualitative analyses, gender will be considered as a potential mediator. For regression modelling, gender will be included as a covariate, considering interactions between gender and other variables, and where possible disaggregated modelling will be conducted for men and women to determine if differences exist. Analysis from all data sources will be used to generate recommendations for optimizing implementation and impact at both the individual and organization levels.

Official Title
-----------------
Advancing Peer Support Programming to Address PTSD and Trauma Among Canadian Public Safety Personnel and Veterans

Conditions
-----------------
Mental Health Issue, Mental Stress, Post Traumatic Stress Disorder

Intervention / Treatment
-----------------
* Device: OnCall mobile health peer support app


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: Public Safety Personnel working in fire services, paramedic services, emergency communications and correctional services employed (full or part-time) by the participating organizations agree to download the OnCall app on their mobile phone (Android or Apple) Exclusion Criteria: not able to access the app on their mobile health device

Ages Eligible for Study
-----------------
Minimum Age: 18 Years
Maximum Age: 70 Years

Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Arms and Interventions

| Intervention/Treatment |
| --- |
|Device: OnCall mobile health peer support app|Mobile health platform that provides anonymous access to peer support services that has been co-design by and for Canadian public safety personnel. OnCall is for frontline public safety providers; features include links to mental health self-assessment tools, a wellness toolbox, tips to cope articles and peer wisdom videos, and options to reach out to a peer support provider of their choice via telephone or text. The OnCallSupport app is for the peer support providers which includes a list of resources and an opportunity to create post-action reports following the peer support encounter. Connections are made to peer support via a telephone service that disguises the contact information of both parties. The app platform includes options to complete feedback surveys related to the app and to the peer support encounter.|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Accessing peer support via phone or text wia the OnCall app (frequency count) | Number of times OnCall users within an organization initiate a text or phone call to peer support providers via the app platform (software analytics) | Monthly intervals from 1 to 3 months. | 

Secondary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Change in Mental Health Literacy (6 items - 7-point Likert scale rating) | Rating using a 7-point Likert scale of perceived improvement in dimensions of mental health literacy (Promote mental health, knowledge of how to identify and address mental health issues in self and others; how to identify relevant resources). Range 0-42 with higher scores indicating higher levels of perceived literacy. | Baseline, 3 months. | 
| Change in level of distress (Visual analogue scale) | Following a peer support encounter, a pop-up survey in the app asks users to rate their level of distress before and after the encounter using a 100-point Visual Analogue scale. | Immediately following a peer support call or text exchange via the app. | 
| Changes in symptoms of anxiety (GAD-7 screening tool) | The Generalised Anxiety Disorder (GAD-7) scale is a 7-item tool to evaluate anxiety symptoms over the past two weeks (4 point Likert scale ratings). Scores ranges from 0-27 with a score of 10 or more indicative of potential generalized anxiety disorder. | Baseline, 3 months. | 
| Change in symptoms of depression (PHQ-9 screening tool) | The Patient Health Questionnaire (PHQ-9) is a 9-item screening tool for depression. Scores range from 0-27 , with a score of 10 or more indicative of major depression. | Baseline, 3 months. | 
| Change in symptoms of trauma (PTSD Checklist - PCL-5 short version) | The short version of the Post-Traumatic Stress Disorder Checklist (PCL-5) is a four-item PTSD screening checklist exploring responses to a stressful experience in terms of symptoms over the past month. Scores range from 0-16 with higher scores indicating more severe impairment. | Baseline, 3 months. | 
| Change in Help-seeking behaviour | Rating of how likely to reach out for support related to four different types of support (in-person peer support, app-based peer support, professional support and employee assistance program). Ratings on a 5-point scale from not at all likely to extremely likely. | Baseline, 3 months | 
| Perceived value of the app | Questions about perceived impact adapted from the Mobile Rating Application Rating Scale (MARS) asks users to rate (on a 5 point Likert scale) the perceived value of the app on 6 items (from awareness of the importance of addressing mental health to reaching out for help and impact on mental health, perceived support and ability to cope at work). In addition, users are asked to rate the value of 6 different features of the app on 5-point Likert scale from very poor to excellent, plus an overall rating. | 3 months | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
peer support, mobile health