Title: Management by geographic area or management specialised by disorder? An evaluation of effects of an organisational intervention on secondary mental health care for common mental disorder
Funder: Department of Health (England)

Abstract: THEORETICAL/CONCEPTUAL FRAMEWORKDescriptions of the structural changes associated with the CAG program and relevant theories such as product line management provide little clear understanding of the CAG program and how its outcomes will be achieved. In particular they fail to take advantage of the detailed contextual knowledge of stakeholders in the program and the availability of data for quantitative analyses. Our response is to perform a realistic evaluation (Pawson & Tilley 1997). Realistic evaluation aims to establish clear and measurable relationships between an intervention like the MAPCAG program and its outcomes which are explained by an underlying theory comprising a set of context-mechanism-outcome relationships (CMOs). It ”employs no one standard ‘formula’ other than the base strategy of producing a clear theory of program mechanisms contexts and outcomes and then using them to design the appropriate empirical measures and comparisons” (Pawson & Tilley 1997 p.xv). In WP1 logic modelling (Ling & Scoggins 2006; Soper et al. 2008) will be used to “produc[e] a clear theory of program mechanisms contexts and outcomes—the visual representation of key context-mechanism-outcomes functions first to orient qualitative data collection to the central task of developing and refining a theory of the program under evaluation and then provides a visual representation of the results of the evaluation and an interpretative structure for the quantitative results. WP2 (time series analyses) will entail the design and carrying out of the “appropriate empirical measures and comparisons” and the feeding back of these results into the logic modelling process.WP1 – LOGIC MODEL – MONTHS 1-4 10 AND 22 Key context-mechanism-outcomes for the CAG program will be defined by the managers clinicians service users family members and GPs involved using a logic model as a diagrammatic and visual means of eliciting their professional tacit and formal knowledge (see above). WP1 will be co-led by an external expert (Dr Bryony Soper) and by a service user researcher (Dr Diana Rose) and assisted by an additional researcher. The respondent group will comprise at least 6 clinicians managers and GPs involved in / impacted by the MAPCAG 3 service users and 3 relatives of service users. Professional respondents will include research team members indicated in “Team Expertise”. These “insider” team members have already identified a number of innovations of interest within the CAG—see “Background” above. Service user respondents will include and will otherwise be identified by the patient-public involvement representative for the CAG. In months 1 to 4 two user-only meetings two meetings for relatives of service users and two professionals-only meeting will be followed by a first joint meeting (the logic model will combine elements from each group). There will be further joint meetings in Months 10 and 22 where the logic model will be refined in the light of WP2’s results. The lead for WP2 (Dr Tulloch) will provide input regarding data availability. Data collection will be performed by Dr Soper Dr Rose and a further service user researcher. The central output will be the final logic model for the CAG program.WP2 – TIME SERIES ANALYSIS – MONTHS 1-22 SLaM is the only NHS Trust in England that both uses a comprehensive electronic patient record and makes its contents available for research in anonymised form (the BRC Case Register—see Detailed Project Description). Data will come from this unique resource from staff and patient surveys and by content analysis of archive materials especially minutes of borough directorate meetings and CAG board meetings. We will define all episodes of care for depression anxiety and personality disorder from 2009 onwards using activity data and ICD-10 codes as well as additional text-mined diagnostic codes. We shall restrict the study to secondary rather than tertiary services and take a 10% random sample (N~2000). To each episode we will append measures determined by the logic model: outcomes of interest and mediators. From the point of view of dataset construction much important data (e.g clinical activity routine outcome measures such as the Health of the Nation Outcome Scales) already exists in usable structured form. Other variables for example guideline concordance would require manual coding. This can be facilitated as in the Veterans’ Administration (Watkins et al. 2011) by using SAS to generate customised data displays in which the necessary free-text is selected arranged and displayed in order to streamline coding (see the detailed project plan for screenshots from a similar project). We will measure interrater reliability of this “facilitated” coding and its validity compared to full casenote review. Cost-effectiveness is an important outcome; we will measure costs from the provider perspective by combining activity data (inpatient care clinican contacts medication) with appropriate unit costs (reference costs BNF PSSRU). Effectiveness will be measured using routine outcome measures (HoNOS/CORE). We will create a dataset of indicators from January 2009 to December 2013 aggregating data at 208 weekly timepoints. Main analyses will make use of piece-wise regression of each indicator of interest allowing for autocorrelation and testing for an interaction between slope and time-period that would indicate an effect of CAG implementation. We will also estimate the effects of lagged variables that are postulated mediators of the effect of CAG implementation and perform tests of Granger causality. We will test for interactions by borough and by diagnosis. In the analysis of each time-series a sample of 208 timepoints has >99% power (at alpha 0.05) to detect a time x slope interaction that has a medium effect size (0.15). Economic analysis will compare cost-effectiveness (service cost divided by outcome change) pre- and post-CAG. Where HoNOS scores are missing we will use multiple imputation and use the imputed scores in sensitivity analyses. Potential variables included in the imputation will be demographic characteristics previous HoNOS scores and past service history.


Research activity codes: 7.1 INDIVIDUAL CARE NEEDS;8.1 ORGANISATION AND DELIVERY OF SERVICES
Health categories: MENTAL HEALTH