# Presentation: 331856

## Getting a Clinical Research Protocol Started The CTSA Response

- Daniel E Ford, MD, MPH
- Vice Dean for Clinical Investigation
- Director, Institute for Clinical and Translational Research
- Johns Hopkins School of Medicine

## Current State

- Time to initiate studies longer for US than most countries
- Steps required to start study are increasing (ex, billing for services provided in clinical trial)
- Recent NCI analysis found that for cooperative group phase 3 trials time from original idea to activation of study was 808 days (481 distinct processes)

##  June 22-23, 2008

**Enhancing Clinical Investigation by **
**Improved Management**
**A Workshop Convened by the **
**	****Clinical and Translational Science Award (CTSA) Consortium/**
**    ****Regulatory Knowledge Workgroup/**
**	****Clinical Research Management Taskforce**

## Organizing Committee

- Teshia Johnson, MBA Yale
- Michael Joyner, MD Mayo
- Robert Califf, MD Duke
- Daniel Ford, MD, MPH Johns Hopkins
- Daniel Rosenblum, MD NIH NCRR

## Premeeting Survey

- Three surveys similar
-     to each of 24 CTSA institutions
- CTSA PI (representing researcher perspective)
- IRB Director
- Research Administration (contracts) Director

## PI Survey Results

- 4 Institutions post performance measures for IRB and contracting
- IRB barriers
- Incomplete applications from research teams
- Infrequent meetings of IRB or subcommittees
- IRB committees too picky about wording

## PI Survey Results

- Major barriers for contracting
- Even if Master agreements in place, they are not followed by companies
- Legal review too slow
- Difficult negotiations related to indemnification, publication rights and data transfer

## PI Survey Results

- 84% contracting longer than IRB
-  5% IRB longer than contracting
- 11% equal time for both
- Where delays occur:
- 36% sponsors
- 48% academic centers
- 21% PIs and research teams

## Research Administration Survey

- Huge variation in organizational structure
- 36% of staff have left in past three years
- 78% of contracts tracked electronically
- 34% of those faculty can view status of contracts
- 50% report not meeting with one sponsor in 2007 to discuss contracting

## IRB Survey

- Average of 2025 protocols approved in 2007 (682 convened)
- Average of 18 staff
- 50% turnover since 2005
- 50% of protocols tracked electronically
- 50% of institutions do not pay their IRB members
- 55% met with no sponsors/NIH in last year

## Agenda for Workshop

- Stakeholder perspective on problem
- Walter Koroshetz, MD      NINDS
- Nancy Trapp, Eli Lilly       Sponsor
- Tony Perez, JD                 IRB
- Gary Chadwick, PhD        Alternative IRB
- James Moran, JD             Contracts
- Sara Rockwell, PhD          Faculty

## Agenda for Workshop

- Discussion groups addressing specific questions
- Poster session where everyone could share best practices and recent quality improvement programs

## Why do we need research study initiation performance measures?

- Perception of sponsors and research faculty we are inefficient
- No other accreditation organization has taken on this role
- Association of Human Resource Protection Programs (AHRPP)
- Office of Human Research Protections (OHRP/HHS)
- Federal Drug Administration (FDA)
- Opportunity for “the doers” to be proactive and lead the way

## Why do we need research study initiation performance measures? 

- Measurement is the first step to understanding the process
- Same metrics across multiple academic sites allows comparisons and identification of best processes (ask for more resources if not performing as expected)
- Measurement of performance metrics tends to reduce variability

## Why do we need research study initiation performance measures?

- Posting metrics for sponsors and public demonstrates responsibility and accountability
- Once we demonstrate accountability, can ask for same from sponsors and research teams

## Challenges to Measuring Performance

- Selecting the right performance measures
- Are study protocols included in the performance measure representative or informative?
- Are the performance measures easy to measure?
- Can the performance measures be measured with reliability?
- Does the performance measure vary too much by type of study to be informative? (pediatric vs. adult, first site versus 50th site)
- Do the performance measures provide hints about the process or what the path to improvement might be?

## Challenges to Measuring Performance

- Is speed the only measure of performance? What about quality of IRB review? What about quality of a contract that is clear and covers multiple potential problems?
- Are there ways to “game” the performance measures?

## First Performance Metric

- Duration in days from receipt of first “ready” study protocol/site agreement received to time both the clinical investigational site and sponsor have executed the agreement

## Performance Metric Definitions

- What studies are included?
- Multicenter clinical trials
- Easier to assign date when site was able to start study initiation/approval process
- Include all sponsors (NIH and commercial)
- Dates
- 2007 chosen as most study protocols will have had opportunity to be activated

## Performance Metric Definitions

- When is start date?
- Date agreement sent from sponsor
- Date agreement is received
- Research teams may not have recorded this date

## Performance Metric Definitions

- Time to executed site agreement
- Total days or “business” days?
- Is it date when one party signed agreement or when all parties signed agreement?
- Possible to “game” this time by signing agreements quickly and then need multiple amendments to really have study ready to enroll participants

## Performance Metric Definitions

- Days to execution of agreements
- How to count protocols that never were completed?
- What about outliers?
- Report median or 90th percentile?
- Report 25, 50 and 75 percentile?
- Report percent of protocols that met some performance standard

## First Performance Metric

- Duration in days from receipt of first “ready” study protocol/site agreement received to time both the clinical investigational site and sponsor have executed the agreement

## Slide 24

## Days to Contract Execution by Percentiles (Start date when contract presented to Research Office)

## IRB Performance Metric

- Duration in days from PI receipt of protocol to site IRB approval
- Measure protocol receipt to initial (department or IRB) submission
- Measure protocol receipt to IRB submission complete
- Measure protocol receipt to first IRB review

## IRB Performance Metric

## IRB Outcome Measures

## Insights from Workshop

- Valuable forum to bring together IRB and research administration professionals
- Need to consider the entire study initiation process
- Need to bring general counsels into the quality improvement process
- Agree that establishing metrics will aid in quality improvement
- Not certain if goal is to standardize study management across sites or maximize efficiency within each site with own solutions

## Process For Implementing Study Initiation Metrics

| Item/Month | 11/08 | 12/08 | 1/09 | 2/09 | 3/09 | Sept/09 | Sept/10 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| IRB Task Force  | Propose 3 core and 2 optional metrics by Nov 15th  | IRB TF site – collect data on multi-center studies for itself and 4 other sites |  | 3 CTSA institutions to volunteer to gather data on performance metrics | Report data to CRM TF |  |  |
| Contracts Task Force | Propose 3 core and 2 optional metrics by Nov 15th | Contracts TF site – collect data on multi-center studies for itself and 4 other sites |  | 3 CTSA institutions to volunteer to gather data on performance metrics | Report data to CRM TF |  |  |
| CTSA Clinical Research Management Taskforce  |  |  | Review the recommendations from each task force and propose combined set of metrics by Jan 1st   | Present metrics, request comments from sites, general counsels at sites, NIH and others by Feb 1st  | Report plan to CCOC | Performance report on 2007 multi-center studies, due Sept 1st.   | Post Metrics on Performance on www.CTSAweb.org |