# Presentation: 185706

## Organizational Readiness to Change Assessment(ORCA)

**National QUERI Meeting**
- December 11, 2008
- Christian D. Helfrich, MPH, PhD
- Implementation Research Coordinator
- IHD-QUERI

## ORCA

- Organizational readiness to change assessment
- 77-item survey instrument
- Fielded among clinicians and staff involved in planned, discrete clinical change projects
- Measures factors that determine effective implementation of a practice change

## ORCA Development

- Developed by Nancy Sharp and Anne Sales, Ischemic Heart Disease Quality Enhancement Research Initiative
- Based on Promoting Action on Research Implementation in Health Services (PARIHS) framework

## ORCA

- Context

- Evidence

- Facilitation

- Organizational to change

- Implementation outcomes

## Evidence Scale

- Practice experience

- Research

- Patient experiences

- Context

- Discord over evidence

- Organizational to change

- Evidence

- Facilitation

## ORCA

- Context

- Organizational to change

- Evidence

- Facilitation

- Opinion leader culture

- Staff culture

- Leadership practice

- Leadership culture

- Evaluation / Accountability

- Slack resources

## ORCA

- Context

- Organizational to change

- Evidence

- Facilitation

- Implementation team roles

- Leadership roles in support

- Assessment

- Project champion roles

- Evaluation

- Implementation plan

- Leadership roles in planning

- Communication

- Project resources

## Scoring the ORCA

- Each subscale measured with between 3 – 6 items
- Items are statements
- Respondents rate level of agreement from 1 (strongly disagree) to 5 (strongly agree)
- Subscales are calculated as average of items; scales as average of subscales
- Higher scores indicate greater likelihood of success.

## Preliminary validation

- Three quality improvement (QI) projects (n = 113):
- 1) The Cardiac Care Initiative (n = 65 from 49 facilities);
- 2) The Lipids Clinical Reminders project (n = 12 from 1 facility)
- 3) An intensive care unit quality improvement project (n = 36 from 9 facilities)
- Identical surveys administered to one or more staff involved in quality improvement efforts;
- Only the initial evidence statement varied, identifying the evidence-based practice change.

## ORCA scale reliability

| Scale | Cronbach’s alpha |  |
| --- | --- | --- |
|  | For overall scale | For subscales |
| Evidence | 0.81 | 0.70 – 0.75 |
| Context  | 0.85 | 0.87 – 0.94 |
| Facilitation | 0.95 | 0.84 – 0.94 |

## Content validation

- Exploratory factor analysis
- 3 retained factors (eigenvalues 7.61, 7.12, 3.23)

## Principal components analysis

|  | Factor 1 | Factor 2 | Factor 3 | Uniqueness |
| --- | --- | --- | --- | --- |
| Evidence scale | -0.10 – 0.06 | -0.24 – 0.11 | 0.62 – 0.83 | 0.27 – 0.67 |
| Context scale | -0.17 -0.07 | 0.67 – 0.88 | -0.08 – 0.26 | 0.18 – 0.48 |
| Slack resources | 0.41 | 0.10 | 0.13 | 0.71 |
| Facilitation scale | 0.65 – 0.92 | -0.14 – 0.34 | -0.10 – 0.07 | 0.13 – 0.34  |
| Leadership roles in planning | 0.24 | 0.74 | -0.02 | 0.19 |
| Project champion roles | 0.49 | 0.35 | 0.15 | 0.34 |

## Validation Conclusions

- Overall good reliability and content validity of the three scales
- Poor reliability of the evidence subscales
- Slack resources may be separate factor
- Facilitation subscale measuring leadership roles in planning, may be part of context scale; may need rewording

## Next Steps

- Submitted grant to conduct further validation
- Collaborating with three implementation projects by Mental Health and Spinal Cord Injury QUERIs to assess predictive validity
- Develop user’s manual and Web-based, customizable version
- Work with partners to produce example cases of how to use organizational readiness assessments to guide implementation projects

## References

- Helfrich C, Li Y-F, Sharp ND, Sales A. Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARiHS) framework. *Implementation Science. *Under review.