# Presentation: 686717

## Australian Injury Indicators

**James Harrison**
**Malinda Steenkamp**
**Research Centre for Injury Studies,**
**Flinders University of South Australia**
**Incorporating the AIHW National Injury Surveillance Unit**

**Notes:**
Madam chair, ladies and gentleman

I would like to begin my presentation by acknowledging my co-author, James Harrison. Much to his regret he could not be here, but he sends his regards.

Much of what I present today is his work.

## Background

**Injury - National Health Priority Area**
**34 Australian Injury Indicators**
**NISU report on injury indicators**
**Update data source developments**
**Provide technical review and specification of indicators**
**Identify actions and processes for further improvements**

- Research Centre for Injury Studies, May 2002

**Notes:**
In Australia, there are six National Health Priority Areas (NHPA). Injury prevention and control is one of these.
There are currently 34 injury indicators which came about through a number of processes, the first of which started in the context of the WHO Health for All by 2000 initiative in the mid-80s.
The work I am reporting on today involved three aspects as was done for the Australian Institute of Health and Welfare and the Department of Health and Ageing. The aim was to contribute to a data development plan for the Injury Prevention and Control NHPA.
As said, the work consisted of three parts:
We reviewed and updated what had occurred re data sources in the country (I will not cover this in this presentation)
We developed a framework for specification of indicators and developed specifications for each of the indicators according to this framework to the extent possible.
We also identified actions and processes required to achieve further improvements in indicators and data sources.

## Australian Injury Indicators

**12 - deaths data**
**Death rate (DR) for injury in total population**
**DRRatio injury status of Indigenous + non-Indigenous populations**
**11 - morbidity data**
**Hospital separation rate (HSR) for falls - persons 65+ yrs**
**HSR for near drowning - children 0-4 yrs**
**Rest - other data sources/not defined**
**% domestic pools with appropriate barriers/fencing**
**Access of injury patients to optimal care**

- Research Centre for Injury Studies, May 2002

**Notes:**
As said there are 34 injury indicators. Twelve of these are based on national deaths data (as provided by the Australian Bureau of Statistics) and eleven are based on national hospital separations data (as sourced from the Australian Institute of Health and Welfare). These indicators were defined in terms of external cause codes and seemed to have a focus of injury occurrence.
The remaining eleven indicators were based on other national data sources (which did not exist at the time the indicators were developed, eg emergency department surveillance data) or have not been properly defined.
The indicators reflect certain constraints at the time of their development, for example:
there was less knowledge about the extent and distribution of injury;
there was limited national data sources and within existing sources there were certain constraints (eg the absence of multi-cause coding in national deaths data);
the lack of a well-defined technical framework for the indicators.

## Useful and valid injury indicators?

**Purposes and topics**
**Specifications**
**Data sources ****(not discussed)**
**Data quality ****(not discussed)**

- Research Centre for Injury Studies, May 2002

**Notes:**
The indicators were reviewed in terms of their purposes, technical specifications, data sources and data quality.
Due to a lack of time, I will only discuss the first two aspects.

## Purpose

**Process - Purpose(s) of indicators?**
**Why specified purpose?**
**To guide development and evaluation**
**Information requirements**
**Current set - no specific purpose**
**Ambiguity in specification**
**e.g. indicators based on hospital data measuring**
** ****population incidence of serious injury?**
** ****burden of injury on acute health care services?**
** ****something else?**

- Research Centre for Injury Studies, May 2002

**Notes:**
We need to take account of the process of how indicators were developed. The main feature of this process was that there were no specified purposes for the 34 indicators.
But why is this important?
From a technical perspective, a necessary condition for the development of good indicators is a clear statement of purpose. Only if the exact purpose is clear, is it possible to design an indicator to achieve the purpose or to evaluate whether and existing indicator is serving its purpose.
Precise statement of purposes and topics guide the development and evaluation of indicators. Another reason is that superficially similar information requirements can have very different implications for the attributes of the data source required.
The current set of indicators lacked sufficiently specific statements of purpose and this was reflected in the ambiguity in specification of some indicators. For example, one-third of the indicators are based on hospital separations data, but does this refer to incidence of serious injury, the burden of injury on acute health services, or something else?

## Australian Injury Indicators

**Injury indicators based on national data**
**12 based on deaths data**** **
**(e.g. Death rate for injury in total population)**
**11 ****based on**** morbidity data**
**(e.g. Hospital separation rate for falls - persons 65+ yrs)**
**Rest - other data sources/not defined**
**(Total number of Australian injury indicators=34)**

- Research Centre for Injury Studies, May 2002

**Notes:**
When we look at the 34 Australian injury indicators, it was stated in an earlier document reporting on the indicators, that their “main focus was to monitor the occurrence of new cases of injury”.
Most of the injury indicators are based on deaths or hospital separations data - as such they refer to relatively serious injury cases.

## Purpose(s) of injury indicators

**Indicators based on national data:**
**To measure, and monitor change in, the population incidence of serious injury in Australia (recommended)**
**Remaining indicators:**
**Purpose not clear ****(not discussed further)**

- Research Centre for Injury Studies, May 2002

**Notes:**
In reviewing the injury indicators, it appeared as if the primary purpose for the current set of those based on deaths and hospital data is:
‘To measure, and monitor change in, the population incidence of serious injury in Australia’

This is a descriptive purpose. The purpose is not framed in terms that refer to a causal model of injury.

This adoption of purpose provides greater clarity regarding indicator specification, as well as data source and quality requirements.

The purpose for the remaining indicators remain unclear and is not discussed further.

## Technical specifications (1)

**Translates purpose and topic - set of definitions and instructions**
**Specification:**
**Definition of a measure**
**Documenting definitions and instructions**
**Criteria ****( after Injury ICE, Cryer, Langley, et al):**
**Define ito anatomical/physiological damage**
**Included cases should be all/well-defined sample**
**Case ascertainment independent **

- Research Centre for Injury Studies, May 2002

**Notes:**
Technical specification translates the purpose and topic of an indicator into a set of definitions and instructions for a measure that is intended to correspond to a purpose and topic.
Indicator specification has two aspects:1) Defining a measure that is a suitable indicator of a topic of interest; and 2) Documenting definitions and instructions for doing the measurement.
What we need is a measure that truly measure what it is thought to measure and does so with attributes (such as precision, reliability, timeliness, cost and acceptability) that are sufficient to serve the purposes for which the indicator is wanted.
We therefore need to consider several technical criteria to ensure that we measure what we want to measure. Formal consideration of how to specify satisfactory indicators of injury incidence has been developed recently and the issue was a theme at the ICE meeting in April 2001. Initial outcome was a draft set of criteria for good indicators of injury incidence. These are:
Case definition should be specified in terms of specified anatomical or physiological damage;
Cases included should be all of those that the indicator aims to reflect or a well defined sample of them; and
Probability of case ascertainment should be independent of extraneous factors.

## Technical specifications (2)

**Defined ito anatomical/physiological damage?**
**NO, based on external cause codes**
**(e.g. Falls in 65+ - ICD-9 E880-E888)**
**Reflects data situation at time of definition**
**Change in situation - deaths and hospital data**
******	**** ****Redefine ‘injury’ ito ICD-10(-AM) S00-T89**
**Includes ‘Medical injury’ (T80-T88)**
**Excludes ‘Sequelae’ (T90-98)**

- Research Centre for Injury Studies, May 2002

**Notes:**
We assessed the Australian indicators against each of these criteria.
Criteria 1:
None of the injury occurrence indicators were specified in terms of anatomical/physiological damage. All were specified in terms of external cause codes (take falls in the elderly as an example).
This reflects the data situation at the time when the immediate ancestors of the NHPA set of injury indicators were written. That is, the national deaths data only contained the underlying cause of death code. Since 1997, the introduction of automated coding software has allowed for addition of multi-cause codes, which now allows for redefinition of the indicators. For hospital separations data, it was more a question of lack of data quality but data from 1993/94 are considered to be of sufficient quality to allow for reporting of the relevant indicators.
We suggest that injury occurrence cases be defined in terms of ICD-10 diagnoses codes, ie codes S00-T89.
We include ‘medical injury’ - we acknowledge the debate around this, but are of the opinion that there are enough similarities with ‘injury’ to warrant inclusion.
We exclude sequelae because our interest is ‘new cases’, not cases that follow from a previous injury - even thought the case numbers in Australia are small.

## Technical specifications (3)

**Included cases all/well defined sample?**
_**National deaths data collection**_
**	****All supposed to be included****No formal evaluations**
_**National Hospital Morbidity Data Base**_
**	****Nearly all cases - small number excluded**
**	****No formal evaluations**
**	****Private hospital coverage - 94.4%**
**	****Not ‘all injuries’ - % of various types not incl.?**
**	****(changes in admission policies, etc.)**

- Research Centre for Injury Studies, May 2002

**Notes:**
Criterion 2:
For deaths data all registered deaths are supposed to be included in the files of the Australian Bureau of Statistics. However, no formal evaluations have been done on the completeness of the process. We know that a few deaths are registered long after they occur and presumably some are never registered. There is no evidence that this is a significant problem.
For hospital separations, nearly all acute hospital cases are included. Although there are a small number of hospitals excluded. These are mostly private hospitals and involve small case numbers. It is estimated that private hospital coverage is just less than 95%.

## Technical specifications (4)

**Ascertainment independent?**
_**National deaths data collection**_
**	****In general - probably YES, some queries**
**Age-related variation in querying External cause**
_**National Hospital Morbidity Data Base**_
**	****Only some injuries result in admission**
**	****Some admitted cases - more than one record**
**	****Some injury occur in hospital**
**	****Deaths in hospital data**
**	**

- Research Centre for Injury Studies, May 2002

**Notes:**
Criterion 3:
For deaths data, the answer is ‘in general - yes’. There are however instances where this is not so.
One such example relates to an age-related variation in querying external cause of death. For injury deaths, the underlying cause of death is coded to an ICD external cause code. In cases where there is insufficient information to code the external cause, the ABS normally queries these. However, relatively few queries are made where the person was aged 75+ years. Therefore, in the absence of more specific information, the proportion of cases assigned to residual categories - such as X59 (exposure to unspecified factor) - increased selectively. The proportion of injury cases correctly allocated to indicator specific external cause codes (eg falls) is likely to decrease with age. The impact is greatest for those types of deaths that are more common in old age, such as fall-related injury.
For hospital separation data, there are a number of issues. The most important are that only some injuries result in admission and this proportion is affected by a number of things, eg admission policies.
Another issue is that some cases result in more than one admission, but we have no validated method of distinguishing repeat cases.
Also, some injuries occur during hospitalisation and deaths appear in hospital data as well.

## Draft Framework (1)

- Research Centre for Injury Studies, May 2002

**Notes:**
As originally stated, the indicators were not completely specified. Technical aspects that were left unstated resulted in the reporting of inconsistent indicator values.
One way we aimed to address this was to develop a framework for specifying the indicators. This framework is intended to include all information about each indicator that is necessary to ensure consistent reporting.

## Draft Framework (2)

- Research Centre for Injury Studies, May 2002

## Two models of specification

_**Minimal change model**_
**	****Better specification based on more comprehensive criteria - replicate explicit and implicit specifications that underlie indicators as originally specified**
_**Technically revised model**_
**	****Recommended revisions - now feasible****will improve national data for purpose of monitoring population incidence of serious injury**
**	**

- Research Centre for Injury Studies, May 2002

**Notes:**
In writing the specifications, we came up with two models:
The first is a Minimal change model which is designed to fully specify and document existing indicators in order to replicate sources and methods used previously when reporting on indicators. This model documented many technical aspects not previously specified. The only real change was the replacement of ICD9(-CM) codes with ICD10(-AM) codes.
The Technically revised model builds on the previous model, but incorporates a set of specific changes to technical specification of indicators based on deaths and hospital data in order to improve the performance of indicators for the purpose of monitoring injury incidence rates. Most of these changes can be implemented without requiring changes to source data. No changes to the topics were suggested as this was outside the scope of the work reported on.
The latter model has been put forward for consideration by relevant stakeholders. Changes proposed in the second model are based on assessment of purposes of the indicators, followed by the specification of indicators in a way designed to improve the capacity of measures to serve these purposes. The process took careful account of availability and quality of current data sources. The process is applied to 24 (71%) of the 34 indicators.
The Technically Revised Model is a step towards a more useful and reliable set of injury indicators.

## Technically revised model

**Specification - set of technical criteria**
**Restriction of cases with anatomical/physiological damage**
**Omitting ‘same day’ cases from hospital data**
**Specifying mortality indicators ito date of death**
**Re-specification of all indicators ito ICD-10 codes**

- Research Centre for Injury Studies, May 2002

**Notes:**
In the second model we suggested a number of steps to improve reporting. These include:
Specification according to a set of technical criteria (ie according to the framework for specification);
Restriction of cases to those with anatomical/physiological damage;
Omitting ‘same day’ cases from hospital data (ie cases admitted and discharged on the same day - there were great variability in the proportion of these cases between states and territories);
Specifying mortality indicators in terms of date of death and not date of registration; and
Re-specification of all indicators in terms of ICD-10 codes.

## Main points

**Part of national process, for review and testing**
**Most indicators refer to injury occurrence**
_**Purpose**_**: Measure and monitor change in population incidence of serious injury in Australia**
**Specific products:**
**Better technical specification of current indicators based on more comprehensive criteria (*****Minimal change***** model)**
**Recommendations for improvements re monitoring population incidence of injury (*****Technically revised***** model)**

- Research Centre for Injury Studies, May 2002

**Notes:**
The work reported here is part of an ongoing process and delivered specific products.

## Further development

**Indicators of injury incidence**
**Implement *****Technically revised***** model after discussion and testing**
**Ensure enough system stability to enable monitoring**
**Implement incremental technical refinement**
**Develop indicators of severe injury incidence**
**Review indicator specification periodically**
**Undertake quality assurance**
**Propose changes to data sources where necessary**
**Remaining indicators**
**Assess and specify purpose(s)**
**Revise, develop, introduce relevant new ones**
**Develop new data sources where possible**

- Research Centre for Injury Studies, May 2002

**Notes:**
A number of other matters are important to the development and maintenance of an effective set of indicators.
We need to maintain consistency of current indicators and associated data sources, for the duration of monitoring. We should think in terms of decades rather than years.
Indicators should not be added or removed on an ad hoc basis, but should be done in the context of health information processes. Stated purposes and topics should inform design of new indicators while assessing feasibility.

This work sets an example of how to go about developing or improving indicators. It is the model that pushed the refinement of indicators for Australia, and is only the continuation of an ongoing process. It can be of great value for work on indicators in other countries.