# Presentation: 700144

## ODE and OIVD FY 2004 Premarket Customer PerceptionSurvey Results

**April 2005**
**Office of Management and Operations (OMO)**
**Division of Planning, Analysis, and Finance (DPAF)**
**Analysis Branch (AB)**

## Survey Background and Purpose	

**Second Year of Survey**
**Obtain industry feedback on the premarket review process to:**
**Help CDRH gauge how customers perceive the device review process. **
**Identify areas that may need improvement.**
**Use results in the 2005 Performance Scorecard Key Indicator “Knowledge Mgmt and Stakeholder Collaboration.”**

## Methodology

- A random sample of industry contacts were selected from the FY 2004 premarket applications with final decisions.*
- Sample was proportionally allocated within each Office by application types.
- Respondents were contacted by telephone.   (279 responses.)

- *Based on 4,649 final premarket application decisions (ODE = 3,962: OIVD= 687)
- Note:  Survey approval was under the OMB customer service program

## 	  	   	                      
Percent of responses by document type:
			FY04  (FY03)
510(k)		  	82%   (86%)	  
PMA (Original)	  2%   (1%)	     
PMA Panel Track   	  0%   (1%)	     
PMA Expedited	  0%   (0%) 	     
HDE		    	  0%   (0%)	     
Pre-IDE	12%   (8%)	     
IDE		      	  4%   (4%)	     

**Percent of responses by document type:**
- 			_FY04_  (_FY03)_
- 510(k)		  	82%   (86%)
- PMA (Original)	  2%   (1%)
- PMA Panel Track   	  0%   (1%)
- PMA Expedited	  0%   (0%)
- HDE		    	  0%   (0%)
- Pre-IDE	12%   (8%)
- IDE		      	  4%   (4%)

**Percent of responses by Division: **
- 		_FY04_  (_FY03)_
- DGRND	14%   (14%)
- DCD		13%   (12%)
- DOED		10%   (13%)
- DRARD	14%   (11%)
- DAGID	13%   (11%)
- DCTD		13%   (13%)
- DMD		10%   (14%)
- DIHD		13%   (12%)

**Percent of responses by days to decision:**
**          ****	****	****    **_FY04_**   (**_FY03)_
**            **0-30 days   	      26%   (25%)
-            31-60 days	      18%   (18%)
-            61-90 days	      19%   (26%)
-            91-180 days        18%   (14%)
-          181-365 days        13%   (14%)
-           366-above	         6%   (3%)

**Percent of responses by decision codes:**
**	****	**_FY04_**  **_(FY03)_
-  SE or Approved	60%   (67%)
-  NSE or Not Approved	  3%   (2%)
-  Withdrawn		  2%   (3%)
-  Other		 35%  (28%)

- Percents may not total 100% due to rounding.
- Note:  Response rates – FY 2004 (N=279), FY 2003 (N=274)

**Premarket Customer Perception Survey Demographics**
**FY 2004 vs. FY 2003**

**ODE and OIVD Combined**

## Premarket Customer Perception SurveyFY 2004 vs. FY 2003ODE and OIVD Combined

| 1.  CDRH reviewed and processed your 
     pre-market submission in a timely manner. (N=278 (273))  | 3%       (3%) | 6%       (5%) | 41%    (40%) | 49%   (50%) | 0%        (2%) |
| --- | --- | --- | --- | --- | --- |
| 2.  CDRH staff treated you in a fair, courteous, and professional manner. 
    (N=279 (274))   | 1%       (0%) | 2%       (1%) | 27%    (26%) | 68%   (71%) | 2%        (2%) |
| 3.  CDRH review staff exhibited a level of scientific expertise appropriate to review your submission. (N=279 (273)) | 1%       (0%) | 8%       (5%) | 43%    (38%) | 45%   (52%) | 3%        (4%) |
| 4.  CDRH review staff applied consistent review procedures in processing your submission. (N=279 (271)) | 4%       (2%) | 8%       (7%) | 50%    (43%) | 34%   (41%) | 4%        (7%) |
| 5.  CDRH website information and guidance documents provided adequate direction on the format and recommended contents of your submission. (N=277 (273)) | 4%       (1%) | 13%   (10%) | 43%    (40%) | 29%   (37%) | 11%    (12%) |

**Strongly**
** ****Disagree**

**Disagree**

**Agree**

**Strongly**
** ****Agree**

**N/A**

- Percents may not total 100% due to rounding.
- No significant differences in responses in FY 2004 vs. FY 2003

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

## Premarket Customer Perception SurveyFY 2004 vs. FY 2003ODE and OIVD Combined

| 6.  CDRH guidelines, standards, and policies are adequate to prepare  your submissions on the latest advancement in  technologies.   (N=277 (272)) | 3%       (2%) | 23%   (17%) | 52%    (54%) | 17%   (18%) | 5%        (8%) |
| --- | --- | --- | --- | --- | --- |
| 7.  CDRH demonstrates that it uses the “least burdensome” approach in applying the regulatory process. (N=276 (266)) | 4%       (3%) | 18%   (22%) | 58%    (56%) | 16%   (15%) | 4%        (4%) |
| 8.  CDRH pre-market review meetings have been productive and have met your needs.  (N=278 (272)) | 2%       (0%) | 4%      (3%) | 24%    (28%) | 21%   (16%) | 49%    (53%) |
| 9.  Overall, CDRH is customer service oriented. (N=278 (273)) | 3%      (2%) | 11%   (10%) | 59%    (55%) | 28%   (33%) | 0%        (0%) |

**Strongly**
** ****Disagree**

**Disagree**

**Agree**

**Strongly**
** ****Agree**

**N/A**

- Percents may not total 100% due to rounding.
- No significant differences in responses in FY 2004 vs. FY 2003

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

_**FY04**_**  **_FY03_

**Premarket Customer Perception Survey****FY 2004 vs. FY 2003**ODE and OIVD Combined

## Comments or Suggestions onCDRH’s premarket review process...

- _Top Ten Categories of Responses (323 total responses)_
-      28% pleased with process
-      17% none (no comments or suggestions)
-      13% guidance documents need updating
-       12% problems with communication
-       12% lack of consistency
-         5% website needs improving
-         4% problems with reviewers
-         4% not timely reviews
-         2% fees are excessive or shouldn’t exist
-         2% CDRH’s approach is not “least burdensome”

## Types of Comments Included in eachof the most frequent responses:

_**Pleased with process 28%**_
-      Very satisfied with process, no problems
-      Doing a very good job
-      We are a new company – CDRH is extremely helpful
_**Guidance documents need updating 13%**_
-      Need to finalize draft guidances
-      Outdated guidance documents;  clinical data was based on guidance
-        document but reviewer       said the documents were out of date
-      Need to improve guidance on software for medical devices
_**Problems with communication 12%**_
-       Reviewers should try to return emails/phone calls within 24-48hrs
-       Supervisor and reviewer did not communicate with each other
-       Difficult to find right person to ask questions

## Lack of consistency 12%
Review process differs from product to product, not consistent
Not all branches in CDRH are consistently interactive with sponsors
CDRH needs to work on consistency between divisions and from reviewer to reviewer

Website needs improving 5%
The website needs to be more consistent with the needs of manufacturers submissions, i.e., formats
Simplify the website, it’s difficult to find information
There are different versions of the same document on the website.  Website needs updating

Problems with reviewers 4%
Some reviewers have an adversarial attitude
Occasionally we get a reviewer who asks for an unreasonable request beyond what is required in the guidance document
New reviewers need more training/ ”0lder“ reviewers need re-training

- Review process differs from product to product, not consistent
- Not all branches in CDRH are consistently interactive with sponsors
- CDRH needs to work on consistency between divisions and from reviewer to reviewer
_**Website needs improving 5%**_
- The website needs to be more consistent with the needs of manufacturers submissions, i.e., formats
- Simplify the website, it’s difficult to find information
- There are different versions of the same document on the website.  Website needs updating
_**Problems with reviewers 4%**_
- Some reviewers have an adversarial attitude
- Occasionally we get a reviewer who asks for an unreasonable request beyond what is required in the guidance document
- New reviewers need more training/ ”0lder“ reviewers need re-training

- Types of Comments Included in each  of the most frequent responses: (continued)

## Types of Comments Included in each of the most frequent responses: (continued)

_**Review timeliness 4%**_
-      90 day clock should not be re-started with request for new information
-       Each phase of the process takes a long time
-       Easier if more applications could be processed electronically
-       CDRH does not review the application immediately, my perception is
-         that they wait until mid way to days to respond.
_**Fees are excessive or shouldn’t exist 2%**_
-      Reduce fees
-      Too much for small companies
-      Improve payment process
_**CDRH’s approach not least burdensome 2%**_
-     “Least burdensome” needs some work, too much time spent on
-       non-relevant topics
-      Need to encourage more of the “least burdensome” approach
-      Individual review groups that form their own ways of approaching
-       submissions can be a problem at times.  They are reluctant to
-       accept "least burdensome" approach.

## Three of the Top Ten Categories of Responsesfor FY 2004 had Significantly Different Results from FY 2003

| Categories that were Significantly Different | FY04 | FY03 |
| --- | --- | --- |
| Pleased with Process1 | 28% | 14% |
| Problems with Communication2 | 12% | 7% |
| Guidance Documents Need Updating3 | 13% | 4% |

- Chi square=20.0263, df=1, p is less than or equal to 0.001
- Chi square=5.3993, df=1, p is less than or equal to 0.025
- Chi square=18.9869, df=1, p is less than or equal to 0.001

## Suggestions from RespondentsConsistency and Expertise *

| I suggest there be more consistency across reviewers:  Interpretation of guidance documents does not have a universal interpretation across all the reviewers.

There is a lack of consistency between reviewers.  Similar applications have different requirements.

I suggest that CDRH needs to work on consistency of reviews between divisions and from reviewer to reviewer.

In the PMA process, there is a need for consistent requirements among reviewers and levels of supervisors.

I suggest CDRH needs to increase the level of device specific scientific expertise.
 |
| --- |

- *This section includes only a portion of the suggestions

**I suggest more guidance & consistency across divisions and products. **
**I would like to see better guidelines for contents of pre-IDEs.**
**I suggest that more guidance is needed on hazard analysis and expectations.**
**Guidance documents for “cardio” need updating; I’ve been waiting four years.**
**I suggest that CDRH needs to have new and revised guidance on software and wireless technology.**
**I suggest more guidance is needed in the pediatric area.**
**I suggest that draft guidance documents be finalized.**

- Suggestions from Respondents
- Guidance and Website*

- *This section includes only a portion of the suggestions

## Suggestions from RespondentsGuidance and Website* (continued)

**I suggest that the website list requirements and guidances by product area.**
**The website is impossible to follow and is not user-friendly.  I have to call a person to help me find what I need.**
**I suggest that the website be more consistent with the needs of manufactures such as formats of submissions.**
**In the future, I would like to be able to get the status of my submission, i.e., maybe online by typing in a submission number.**
**I suggest that the website show the format changes for indications for use statement.**

- *This section includes only a portion of the suggestions

## Suggestions from RespondentsCommunications *

| I would like more verbal communication, feedback should be more explanatory since sponsors have different levels of experience and understanding.  Response time should not lag, it is important to make sure that there is closure.

I suggest that if a reviewer has questions, they should not delay in speaking to the sponsor.  Some reviewers are not familiar with  a device which can causes a delay in feedback to the manufacturer.  

I suggest that we are informed when your internal policies are changed.

I suggest that when meetings are held during the review process, it would be helpful if reviewers reviewed all the relevant preparatory material. |
| --- |

- *This section includes only a portion of the suggestions

## What’s Next?Closing the Industry Feedback Loop...

**Suggest the Offices prepare an action plan based on the survey results**
**Provide feedback to Industry through the Performance Scorecard Process  (Phased-In Approach):**
_**Phase I.**_**  Post survey results on the CDRH Internet as OCER did last year**
_**Phase II.**_**  Offices post action plans during the last quarter of FY 2005 on the CDRH internet.**
**Phase III.  Offices implement action plans during FY 2006**