# Presentation: 030649

## QII in the Practice Partner Research Network: Group Randomized Trials and Other Designs

- Steven Ornstein, MD
- Professor of Family Medicine and Director
- Practice Partner Research Network
- Medical University of SC
- Charleston, SC, USA
- Ornstesm@musc.edu

## Presentation Goals

- Introduce PPRNet
- Present 2 PPRNet QII studies:
- TRIP-II (GRT)
- A-TRIP (Time-series)
- Discuss strengths and weaknesses of each project and lessons learned from them that may be applicable to other QII in primary care

## PPRNet is...

- ... “a practice-based learning and research organization designed to improve health care in its member practices and elsewhere in the United States.”

## PPRNet is...

- Consultants,
- Collaborators

- Research Offices
- MUSC, Charleston

- Practices

- Vendor, PMSI Seattle

## PPRNet Practice Membership 

- Interested users of Physician Micro Systems Inc (PMSI) Practice Partner Patient Records EMR
- 101 practices (mostly FP, IM)
- 502 clinicians
- 37 U.S. States

## Practice Size

| # of Clinicians | % of practices |
| --- | --- |
| 1 or 2 | 25 |
| 3 or 4 | 35 |
| 5-9 | 30 |
| 10+ |  11
 |

## PPRNet Data Collection/Analyses

- Practice

- PMSI

- PPRNet

- Practice Report

*Q 3 months*

## PPRNet’s Motto

- “Blurring the distinction between quality improvement and research”

## PPRNet Quality Improvement Studies

| Project | Funder | Dates | Design | Practices | Conditions |
| --- | --- | --- | --- | --- | --- |
| TRIP-II | AHRQ | 2000-03 | GRT | 20 | CVD |
| A-TRIP | AHRQ | 2003-06 | TS | >100 | Many |
| AA-TRIP | NIAAA | 2004-07 | GRT | 22 | Alcohol |
| C-TRIP | NCI | 2006-09 | GRT | 30 | CRC screening |

## TRIP-II: Primary and Secondary Prevention of Cardiovascular Disease and Stroke in Small Primary Care Practices

- Funded by AHRQ as part of TRIP-II RFA (Grant No. 1 U18 HS11132-01.)
- Project period 9/30/00 to 2/28/04

## TRIP-II Methods

- Group randomized trial
- 2 years (1/01 to 1/03)
- 20 non-academic FP and IM practices in 14 States
- 61 total HCP (44 MD, 17 mid-level)
- 87,291 patients >= 18 y.o. visit in 3 years

## TRIP II Methods: 10 Control Practices

- Quarterly practice-level performance reports on 21 indicators of care based on guidelines from JNCVI, NCEP ATP-III, ACC/AHA, ADA
- Each practice had EMR and organized approaches to improvement on their own

## TRIP II Methods: 10 Intervention Practices

- Practice performance reports
- 6-7 practice site visits for academic detailing & QI facilitation with participatory planning, use of EMR tools, complexity science, best practice approaches
- 2 network meetings to share “best practices”

## TRIP II Analyses

- Practice-level analyses: randomization test comparing improvement in targets reached between groups, Wilcoxon signed-rank tests to assess improvement in each group
- Patient- level analyses: generalized mixed-regression models with random practice effects
- Qualitative methods (interviews, review of archival documents to identify improvement model)

## TRIP II Results--Quantitative

- Practice-level analyses: improvement in both intervention and control group practices in % of targets reached
- Intervention group had greater improvement in 18 of 21 indicators
- Patient-level analyses: improvement in intervention group > control group for 2 of 21 measures

## TRIP II Results--Qualitative

- The PPRNet QI model that emerged emphasized the importance of “Prioritizing Performance”, “Involving All Staff”, “Delivery System Redesign”, “Patient Activation”, and “EMR Tools”
- Another model focused on “Leading a Practice through QII”

## Nemeth, 2005

## TRIP-II Challenges and Resolutions

## Dealing with “Buy-in”

- Some providers in larger practices did not “buy” in to study (denial, competing obligations, lack of self-efficacy, etc.)
- Response was to focus on more amenable members and have them model changes for others

## Misplaced Focus of Site Visits

- Initial emphases on academic detailing and EMR optimization met reality that providers knew guidelines and wanted to use EMR in their idiosyncratic way
- Response was to change focus of site visits changed focus to QI approach at microsystem level

## Importance of Non-Provider Staff 

- Insufficiently recognized at study initiation.
- Response was to encourage these individuals to play greater role in participatory planning, implementation at later visits.  Also, non-provider staff from each practice participated in 2nd network meeting.

## TRIP-II QII Lessons/Conclusions

## Clinicians will Participate

- PCP with EMR systems that facilitate QI reporting and interventions and that volunteer to participate in QII generally enjoy the work and will participate in an intervention that they deem beneficial to their patients

## The EMR is not the Panacea

- A more robust QII model was needed.  Simply giving practices the information (academic detailing) and the tool (EMR) is insufficient

## One-Size Does Not Fit All

- Given the constraints of practices, intervention approaches and emphases have to be customized at the microsystem-level
- Study sections accustomed to specific protocols that require rigid adherence to assure this requirement may need to appreciate this reality

## Accelerating TRIP (A-TRIP) in a Practice-based Research Network  

- Funded by AHRQ as part of Partnerships for Quality RFA (Grant 1 U18 HS13716)
- Project period 9/30/02 to 9/29/06

##  A-TRIP Goals  

- Recruit 100 primary care practices
- Enhance PPRNet practice performance reports to include ~80 indicators in 8 discrete clinical areas from based on guidelines from JNCVI, NCEP ATP-III, ACC/AHA, ADA, USPSTF, Beer’s criteria.
- Disseminate PPRNet-TRIP QI improvement methods/model to participating practices and assess impact on practice-level performance on ~80 indicators and 3 summary measures

## A-TRIP Methods

- Demonstration project with descriptive and time-series evaluation components
- 2.75 year intervention (10-03 to 6-06)with enrollment until the final six months
- 101 non-academic FP and IM practices in 37 States
- 502 total HCP
- ~500,000 patients >= 18 y.o.

## PPRNet TRIP Intervention Methods

- Practice Performance Reports (all practices)
- Practice Site Visits q 6 months (62 practices to date)
- Network Meetings (~1/2 of practices at one of 3 meetings to date)

- © PPRNet, 2003

**Notes:**
CCI good example of lab management: MAs writing last date/value, docs email next lab needs and staff sets up lab & f/u

##  Clinical Practice Guideline Measures in PPRNet Practice Reports

- Diabetes mellitus (13)
- Heart disease and stroke (21)
- Cancer screening (12)
- Adult immunization (14)
- Respiratory disease/infectious disease (6)
- Mental health/substance abuse (8)
- Nutrition/obesity (3)
- Inappropriate Rx prescribing in the elderly (2)

## Practice Performance Reports

- Distributed Quarterly
- 84 indicators
- SPC methodology (SAS)
- Comparison with PPRNet benchmark (ABC) and median
- Comparison with national benchmarks (where available

## Site Visit Activities

- Review of practice report
- Presentation/discussion/local adaptation of multi-faceted PPRNet-TRIP QI model developed in TRIP-II

## Summer Network Meetings

- One clinician and one staff member invited from each practice
- Best “practice presentations”
- Small group workshops, among participants with similar roles in their practices (e.g., clinicians, nurses, back office staff)

## A-TRIP Results

- Recruitment and reporting goals met
- Greater than expected participation in practice-site visits; less at network meetings
- Impact on practice-level performance reports will be assessed at study conclusion—6-06, preliminary data encouraging

## A-TRIP Challenges and Resolutions

## Need for Enhanced Reporting

- Participants wanted patient-level in addition to practice-level reports to better identify those in need of specific interventions; EMR did not have sufficient native reporting functions
- Response was to develop/enhance this tool...

## Analyses Challenges I

- Practice attrition of ~10% annually (due to turnover in leadership, retirement, relocation, etc.) and continued recruitment of new practices create challenges in time series analyses
- Plan is to include “duration of exposure” as variable in analyses

## Analyses Challenges II

- The absence of a control group will compromise the validity of the findings.  Using a “delayed intervention” approach was not feasible in this setting it was not possible to recruit the entire sample of practices at the beginning of the project
- Looking at a broad set of indicators and for a large effect size as evidence of efficacy may mitigate this limitation

## A-TRIP QII Lessons/Conclusions

## Clinicians will Participate

- A great number of PCP with EMR systems that facilitate QI reporting and interventions will participate in QII, particularly when they receive a tangible benefit (e.g., free practice reports, CME, support to attend meetings), believe that the project is in the best interests of their patients, **and **can titrate their level of involvement to suit their particular needs/level of interest

## Non-Provider Staff Are Key

- Non-provider staff in primary care settings span the spectrum from RNs to individuals whose only training is provided by the practice.  These individuals need proper supervision, focused training, and inclusion as equally respected team members in QI planning activities in the practice setting
- Practice leaders need to be developed so as to better incorporate these individuals in QII; research is needed on best approaches

## Other Relevant Outcomes

- In addition to clinical outcome measures studies need to look for the behavioral/organizational changes that take place as a result of different intervention strategies so approaches can be tailored efficiently to needs of specific practices

## QII in the Practice Partner Research Network: Group Randomized Trials and Other Designs

- Steven Ornstein, MD
- Professor of Family Medicine and Director
- Practice Partner Research Network
- Medical University of SC
- Charleston, SC, USA
- Ornstesm@musc.edu