Title: Transforming community health services for children and young people who are ill: a quasi-experimental evaluation
Funder: Department of Health (England)

Abstract: DESIGN AND THEORETICAL FRAMEWORKInterrupted time series (ITS) supported by longitudinal qualitative research and a cost consequence study to address the research aims outlined above. ITS design is appropriate when randomisation is not feasible e.g. when evaluating organizational change and when time series data are available e.g. Health Episode Statistics (HES). The design will support quantitative analysis of change over time in outcomes and costs and qualitative analysis of change over time in processes. This mixed methods approach is appropriate when additional information is needed to interpret quantitative measurements generating understanding of how service redesign has worked for whom in what context and at what cost (i.e. providing a ‘realistic evaluation’). There are two main outcomes for the ITS analysis – hospital admission for the most frequently occuring common childhood conditions presenting for acute non-elective care and length of stay for all conditions in children. These outcomes were chosen based on hypotheses derived from existing knowledge (see full proposal). We will also carry out sub-group analysis on childhood conditions associated with the highest average and highest total length of stay in in-patient care (complex care).SITESThree health communities (currently PCTs) representing different geographies deprivation levels population profiles commissioning models and stages of change where comprehensive children’s community nursing (CCN) coverage is being or has been introduced while redesigning children’s services. We know that point at which these three sites did or will (in 2012) introduce their CCN service thus making it possible to date the implementation of change and determine the before and after periods for the ITS.METHODS AND ANALYSIS1. INTERRUPTED TIME SERIES ANALYSIS of HES data will measure the impact of CCN services on the main outcomes using data for 24 months pre- and 24 months post-implementation of CCN services. A control condition unlikely to be affected by the organizational change will be used to examine threats to internal validity and to enhance the construct validity of any effect (see full proposal). Additional analysis will explore differences between children with and without complex care needs and the impact of demographic and socio-economic factors. After graphical exploration of the interrupted time series formal statistical analysis will include an ARIMA analysis and a time series regression. We will identify the most appropriate model to describe the data. The time series regression will estimate the impact of the service redesign taking into account any underlying trends time lags and serial correlation. The regression coefficients corresponding to a change in outcome after introducing the service and to a change in slope will be calculated. A Durbin-Watson test will check for serial correlation. A non-equivalent but conceptually related dependent variable will be used to enhance internal and construct validity.2. LONGITUDINAL QUALITATIVE RESEARCH including documentary analysis observation of key meetings in-depth interviews with commissioners providers. primary care professionals and service user families and use of service quality tools will track the process and impact of service redesign before during and after it happens. The focus will be on known impediments to progress in community service development and recommended actions from Transforming Community Services. We will also track the influence of changed commissioning arrangements on the service redesign. Narrative analysis of documents and data from observation of meetings will provide contextual understanding of the process of service redesign. Framework analysis of data from in-depth interviews with commissioners and providers will identify key issues related to quality outcomes and costs of CCN services over time. Framework analysis of data from families will similarly identify issues related to outcomes and over time. Data from the validated service quality tools will be compared with qualitative accounts of service quality over time.3. COST CONSEQUENCE STUDY Resource use will be estimated from: a) HES data for all secondary care contacts for frequently occuring common childhood conditions length of stay for all children’s conditions. If possible given the experimental status of HES data for emergency care A&E attendances for common childhood illnesses may also be explored; b) questionnaires for CCN team staff and existing data sources covering grade/scale of staff delivering care number type and duration of contacts equipment travel time and administration enabling an accurate ‘bottom up’ costing of care; c) family questionnaires to assess contacts with other health professionals out-of pocket expenses time lost from work time spent on informal care and contacts with other public sector agencies; d) interviews with local commissioners and providers (see above) to generate alternative ‘top down’ costing of CCN services. Unit cost estimates will be applied to relevant resource use data to generate a total cost per patient. Outcomes generated from the other two parts of the study will be presented in a descriptive way alongside the costs of the alternative methods of service provision. The primary analysis will estimate the cost of the intervention using the ‘bottom up’ approach. The ‘top down’ approach will be used as a sensitivity analysis. The impact of the intervention on other costs will be presented disaggregated into costs falling on the health service other public sector agencies and families. Results of the cost-consequence analysis will be presented in a manner useful for decision makers and will include estimates of the mean incremental costs associated with CCN services and an assessment of the uncertainty around these estimates. Extensive scenario analysis will test the robustness of the results and the generalisability of results to other patient groups will be discussed.TIMETABLEYear 1: set up cost data collection and HES datasets; begin longitudinal qualitative elements. Year 2: continue cost data collection;begin qualitative and service quality tool data analysis. Year 3: HES and cost data analysis complete other analysis draft final report.


Research activity codes: 8.1 ORGANISATION AND DELIVERY OF SERVICES;7.3 MANAGEMENT AND DECISION MAKING
Health categories: GENERIC HEALTH RELEVANCE