Medical Device Reporting: Improvements Needed in FDA's System for Monitoring Problems With Approved Devices
Released: 1997-01-29T00:00:00

Letter

Background

To protect the public from harmful medical devices, the 1976 Medical Device Amendments to the Federal Food, Drug, and Cosmetic Act expanded FDA’s responsibility for regulating medical devices in the United States. FDA uses three primary mechanisms to ensure that all medical devices are safe and effective for their intended use:

Premarket review is a system of checks, reviews, and controls intended to be applied before new devices are approved and made available to the public.

On-site inspections of device manufacturers’ facilities are intended to ensure compliance with FDA laws and regulations, including good manufacturing practices (GMP), and prevent the production and marketing of defective devices.

The adverse event reporting system is intended to provide FDA with early warning of problems associated with devices after they become available for public use. This monitoring system gathers information about device problems that could necessitate withdrawing a device from the market or taking other corrective actions.

Among other things, the act authorized FDA to establish a reporting system for adverse events associated with the use of medical devices. As a result, FDA established the Mandatory Medical Device Reporting (MDR) system. Under this system, since 1984, manufacturers of medical devices have been required to report to FDA—and maintain records on—device- related deaths, serious injuries, and malfunctions that, should they recur, would be likely to result in death or serious injury. A system called the Medical Device and Laboratory Problem Reporting Program (PRP) was established to receive voluntary reports from health care professionals. FDA’s responsibility is to audit manufacturers’ performance to identify systemic problems and trends associated with an individual firm, a specific device, or the entire device industry, issuing guidance as necessary.

The Office of Surveillance and Biometrics (OSB) within CDRH operates the adverse event reporting system. OSB reviews adverse event reports on medical devices to identify, manage, and resolve public health issues; prepares safety alerts and public health advisories; and provides regulatory guidance on adverse event reporting issues. CDRH estimates that it spent approximately $8 million and used about 80 full-time-equivalent (FTE) positions in support of the medical device reporting program in fiscal year 1996. This represents an increase of about $606,000 and three FTEs over medical device reporting program expenditures in fiscal year 1995.

When FDA receives an adverse event report, OSB health care analysts evaluate and compare the report with other information contained in FDA databases. A decision is then made about whether or not the device problem poses a potential risk to the public health. If so, depending on the nature and severity of the risk, an analyst may initiate a follow-up investigation, which generally involves a written request for additional information or an inspection of the manufacturer or user facility. (App. II provides a detailed description of FDA’s report evaluation process.)

For years, FDA’s implementation and enforcement of the medical device reporting regulation have concerned the Congress. Studies issued by our office, the former Office of Technology Assessment, and the Office of Inspector General of the Department of Health and Human Services (HHS) have found significant weaknesses in FDA’s ability to gather information about medical devices in use. For example, in 1986, we found that less than 1 percent of the device problems occurring in hospitals were reported to FDA and that, the more serious the problem with the device, the less likely it was to be reported to FDA. In a follow-up study, we concluded that the problem still existed despite full-scale implementation of the medical device reporting regulation.

SMDA 90 and Related Regulations

To improve the flow of information about medical device problems to FDA’s adverse event reporting system, the Congress passed SMDA requires the establishment of a network of communication about device-related events among user facilities, distributors, manufacturers, and FDA. The act requires that user facilities report to FDA—and to device manufacturers, if known—when they become aware of information that reasonably suggests that a device caused or contributed to a death. User facilities are also required to report to manufacturers when they become aware of information that reasonably suggests that a device caused or contributed to a serious injury or serious illness. In the event that the manufacturer of the device is unknown, the facility is required to report a serious injury or illness directly to FDA. User facilities are required to submit reports within 10 work days of becoming aware of such situations. The act also requires that user facilities submit to FDA semiannual summaries of all the adverse event reports they submitted to manufacturers and FDA during the previous 6 months. Table 1 summarizes the medical device reporting requirements for user facilities, distributors, and manufacturers.

In addition, SMDA reporting requirements. In addition to the statutory requirement that user facilities and distributors report serious medical device-related events to FDA and manufacturers, the tentative medical device reporting regulation provided nonmandatory guidance. This guidance instructed user facilities to establish and maintain written procedures that include (1) training and educational programs to inform employees about how to identify and report reportable events, (2) internal systems for documenting and reporting adverse events, and (3) documentation and recordkeeping guidelines.

In December 1995, 4 years after the tentative medical device reporting regulation was issued, FDA published the final regulation for user facilities and manufacturers. As of July 31, 1996, they are both required to report device-related events under a uniform system of reporting. Also under the final medical device reporting regulation, manufacturers are required to submit baseline reports and annual certifications to FDA. FDA can issue various types of written notification to inform a user facility that it has not complied with the medical device reporting regulation, which could result in civil monetary penalties or warning letters.

CDRH established the Manufacturer and User Device Experience (MAUDE) system to replace the PRP and MDR systems. From 1992 to 1996, voluntary reports from user facilities, distributors, and health professionals were input into the MAUDE system; as of April 1996, the MAUDE system contained approximately 20,000 such reports.

During this same period, FDA continued to enter manufacturer reports into the MDR system. FDA plans to begin entering into the MAUDE system manufacturers’ reports that give July 31, 1996, or later as the date the manufacturer learned of an adverse event. FDA expects to have moved all the data from the PRP and MDR systems into the MAUDE system by January 1997.

Finally, in 1993, FDA established the Medical Products Reporting Program—called “MedWatch”—to encourage health professionals to report adverse events and product problems to FDA, simplify reporting, and improve the postmarket product information in FDA’s adverse event reporting system. MedWatch consists of two forms used to elicit the information needed to monitor the safety of marketed devices: the form 3500 is completed by health professionals for voluntary reporting, and the form 3500A is completed by user facilities, distributors, and manufacturers for required reporting. Using the 3500A was voluntary until the final medical device reporting regulation became effective in July 1996.

Growth in Reporting Results in Significant Backlogs

Although the flow of information to FDA about medical device problems increased dramatically after SMDA is directly responsible for this growth is unclear because the majority of reports were provided by manufacturers rather than by user facilities. The increased volume posed problems for CDRH’s adverse event reporting system, and CDRH experienced significant delays in processing and reviewing reports of potentially hazardous device problems. These delays could hinder the timely identification and resolution of device problems. Moreover, the annual volume of reporting is expected to rise significantly in fiscal year 1997 due, in part, to increased reporting requirements under the final medical device reporting regulation. To speed processing and analysis of reports, FDA has initiated several changes to its adverse event reporting system.

Manufacturer Reports Have Increased Significantly

Manufacturer reports of medical device problems increased dramatically after SMDA directly accounted for the increased volume of reports because, until recently, FDA did not systematically collect information on the source of complaints. Device manufacturers submitted over 370,400 of the approximately 407,700 (91 percent) adverse event reports that FDA received from all sources during fiscal years 1991 through 1995. A comparison of two 4-year periods, fiscal years 1987 through 1990 and 1991 through 1994, illustrates the rapid growth of FDA officials believe the upward trend in reporting is due to (1) the unanticipated, large volume of problems associated with silicone gel-filled breast implants, which accounted for nearly one-third of all of the manufacturer event reports submitted to FDA during fiscal years 1992 through 1994, and (2) increased compliance by manufacturers with the 1984 medical device reporting regulation and SMDA .

System Could Not Handle Volume of Reports

CDRH has had considerable difficulty processing and reviewing the heavy volume of adverse event reports in a timely manner. Between March 1994 and April 1995, a backlog of about 48,900 malfunction reports from manufacturers accumulated at CDRH. Many of the malfunction reports were not entered into the adverse event reporting system and available for complete review and assessment until 1996. Although FDA assigns malfunction reports a lower priority than reports of death and serious injury, processing malfunction reports quickly is critical because of their potential to alert FDA to device problems that could cause or contribute to a death or serious injury if the malfunction were to recur. Entering all adverse event reports into the system promptly allows FDA analysts to perform more complete reviews and assessments on device problems. Further, entering event reports expeditiously is important because an event report, regardless of whether it requires immediate action, can become part of a group of reports that ultimately stimulates corrective action on a device problem.

Two factors contributed to the backlog, according to FDA. First, FDA received an unanticipated, heavy volume of breast implant adverse event reports. Second, FDA lacked funds to increase hiring for the former medical device reporting contractor (United States Pharmacopeial Convention, Inc.) that performed data processing functions related to the reports. FDA told us that it recognized the potential impact of the increasing backlog and decided to enter death and serious injury reports immediately, leaving less serious injury and malfunction reports to be entered when resources permitted. FDA also emphasized that it screened the malfunction reports for hazards that required immediate attention and that it believes the public health was not compromised by the backlog.

Volume Expected to Continue to Increase

In addition, the volume of reporting from all sources is projected to rise in fiscal year 1997 due, in part, to increased reporting requirements under the final medical device reporting regulation. Additional reports that are required include imminent hazard reports and baseline reports for manufacturers. FDA estimates that its fiscal year 1997 reporting workload will exceed 152,000 reports, up 52 percent from the approximately 100,000 reports received in 1995.

FDA Is Acting to Speed Processing and Improve Analysis

CDRH is taking steps to minimize backlogs and speed report reviews through changes it made to the final medical device reporting regulation and through computer innovations. To minimize the effects of potential backlogs caused by large increases in reporting, a CDRH official told us, FDA wrote the final medical device reporting regulation with the flexibility for FDA to modify the timing and content of adverse event reports. FDA may, upon written request or at its own discretion, grant to user facilities or manufacturers an exemption or variance from any of the reporting requirements or change the frequency of reporting.

Also, CDRH is in the process of developing an electronic data interchange (EDI) system to allow firms to submit their adverse event reports electronically. Currently, most of the reports are manually transmitted and then entered into FDA computers. The EDI system will function as a single system that receives device reports and distributes them for automated data entry into FDA computers. CDRH believes the EDI system, once embraced by the device industry, will reduce the likelihood of a significant backlog in coding and data entry of reports. EDI requirements and standards are currently being developed and will be piloted in fiscal year 1997.

In addition, CDRH is developing alternative methods of analyzing event reports through enhancements to the computerized MAUDE system. For many years, CDRH has used several adverse event databases that are not integrated for analysis of reports. As a result, gaps have existed in the information available to analysts, and this information has also been redundant.

CDRH’s goal, according to officials, is to reduce its dependency on the individual review of reports and move toward aggregate analysis and other methods of statistical review. The MAUDE system is expected to enable analysts to identify generic product problems and problem-reporting trends across industries and product lines, thereby helping analysts to more efficiently determine appropriate courses of action on device problems. MAUDE will also include a trend and statistical analysis subsystem that will allow analysts to compute statistical trends, such as increases in frequency or severity of reported events for a particular model, family, or type of device.

User Facilities Have Not Adequately Reported Adverse Events

Despite the surge in reporting by manufacturers, user facilities’ participation in reporting device-related events to FDA has been significantly lower than expected. Moreover, the quality of the medical device reports that have been received from user facilities has frequently been poor. Our analysis of CDRH data shows, for example, that many user facilities may be underreporting deaths to FDA and that some facilities are reporting events they are not required to report. Further, user facilities’ reports are often late, inaccurate, or incomplete. User facility reports of device problems are important to manufacturers and FDA because only the user facility’s physicians and staff have direct access to the patient and the device, as well as the clinical skills necessary to detect any adverse effects. Thus, the user facility is in the best position to obtain information that manufacturers and FDA need to determine whether a device presents a public health risk. Without sufficient high-quality reports from user facilities, therefore, FDA’s ability to analyze device problems and assess the public health risk is hampered. Finally, FDA attributed user facilities’ inadequate reporting to FDA’s delay in issuing a final medical device reporting regulation to fully enforce the law, a low level of awareness about SMDA During the 4-year period, more than 80 percent of the user facility event reports were for serious injuries and illnesses (52 percent) and malfunctions (29 percent). Death reports accounted for less than 10 percent of the device-related incidents reported. (See fig. 3.)

User Facilities May Be Underreporting Deaths to FDA

Before SMDA ’s reporting requirements became effective in late 1991, user facilities were not required to submit device-related adverse event reports to manufacturers and FDA. Consequently, FDA’s knowledge about unsafe devices involved in adverse events, such as deaths, was generally confined to reports from manufacturers, which were themselves sometimes limited by the degree of their compliance with the 1984 medical device reporting regulation. In contrast, user facility reporting under SMDA requires user facilities to submit a report to FDA and manufacturers, if known, within 10 work days of receiving information that reasonably suggests that a device has caused or contributed to a patient death. However, our analysis of CDRH data revealed that user facilities may have underreported thousands of patient deaths to FDA. Without the user facilities’ perspective on these events, FDA’s adverse event reporting system has significantly less information about medical device problems to use in identifying problems and assessing the public health risk.

We found that during fiscal years 1992 through 1995, manufacturers reported 5,976 deaths to FDA, while user facilities reported only 978 deaths—a difference of 4,998 death reports (see fig. 4). According to CDRH officials, FDA has taken no action to determine the extent to which the discrepancy in reporting is due to underreporting by user facilities to FDA. Nevertheless, a CDRH official told us that the difference in the number of death reports received is a gross estimate of possible underreporting. Accurately determining the number of facilities that actually did not report deaths would require CDRH to use manufacturers’ reports to identify the user facilities that may not have reported a death and then determine if the death was required to be reported—a task the official said would be very time consuming.

The official also stated that the difference in the number of deaths reported could be due, in part, to the different criteria that manufacturers and user facilities were required to use in deciding whether to report an event. For example, under the 1984 medical device reporting regulation, manufacturers were required to submit all event reports from health care professionals, even if the firm believed or knew that the event was not related to its device. In contrast, SMDA Once fully operational, the MAUDE system will be able to match and cross-reference all reports from manufacturers, distributors, user facilities, and health professionals so that an analyst will have a complete picture of the event. Thus, the analyst will be better able to determine, for example, whether or not a death report was submitted by a user facility, as required by law. If a death occurred and the user facility did not report it, the analyst can notify the district offices within the Office of Regional Operations, which is responsible for monitoring compliance with SMDA reporting.

Late and Incomplete Reports Hamper Identification and Assessment of Device Problems

The submission of timely and accurate reports of device problems is important for FDA’s early warning system to operate effectively. Generally, when FDA receives complete reports, they can be reviewed and assessed for public health risks without additional follow-up to obtain information on aspects of the reported incident. However, our review of CDRH statistics shows that many user facility reports are not submitted to FDA in a timely manner and that they frequently lack essential information on the device problem. Untimely and poor quality reports limit FDA’s ability to identify device problems and assess the risk to the public health. Several examples of such problems with reports follow.

First, SMDA incomplete reports and that many of the user facility reports go directly to FDA instead of to the manufacturer.

Finally, most of the adverse event reports that FDA has received from user facilities are not even required by SMDA . By increasing the agency’s workload, extraneous reports can slow CDRH’s review of adverse events that truly merit the agency’s attention. For example, the act requires user facilities to submit event reports to manufacturers when they receive information that suggests a device may have caused or contributed to a serious injury or illness. Only in instances in which the manufacturer of the device is unknown is a serious injury or illness to be reported to FDA. However, of the 5,199 serious injury and illness reports FDA had received from user facilities as of June 1995, 3,588 (69 percent) did not need to have been sent to FDA because the user facility knew the manufacturer and identified it in the report. Moreover, during fiscal years 1992 through 1995, user facilities submitted 3,678 malfunction reports that were not required by SMDA consistent with those in the tentative final regulation (that is, they provided definitions of event terminology, descriptions of educational programs to teach employees how to identify and report device-related events, internal systems for documenting and reporting device-related events, and separate medical device reporting incident files). CDRH concluded that many facilities would benefit from additional educational efforts.

FDA attributed user facilities’ lack of adherence to FDA’s tentative final regulation to several additional factors. First, a CDRH official told us that before the final medical device reporting regulation became effective in July 1996, the only requirements that were legally enforceable were the requirements of SMDA , which do not provide sufficient detail on medical device reporting. As a result, user facilities had to rely on guidance in the tentative final medical device reporting regulation to establish the program’s requirements. This guidance was not enforceable, and, as the official pointed out, user facilities did not have to follow it because doing so was voluntary.

In addition, FDA officials said that planning and performing compliance inspections are difficult because FDA has a limited number of investigators available to conduct the inspections that are required at over 70,000 regulated facilities. However, FDA believes that, by working closely with the Joint Commission on Accreditation of Healthcare Organizations and the Health Care Financing Administration, it will be better able to monitor user facility compliance with SMDA computer. In addition, CDRH plans to continue to publish a quarterly User Facility Reporting Bulletin that provides information to user facilities on various medical device reporting issues. Finally, CDRH held a teleconference with user facilities in May 1996 to discuss matters concerning the final medical device reporting regulation.

FDA Does Not Document Corrective Actions Adequately

For its early warning system to be effective, FDA must be able to rapidly process, review, and assess the potential risk of device problems to the public health. These assessments must result in an appropriate course of action to resolve device problems. Our review of a sample of adverse event reports and discussions with CDRH officials revealed that for many years FDA has not routinely documented the final corrective actions taken by manufacturers and FDA to resolve the problems identified in these reports. As a result, we were not able to determine how manufacturers and FDA have responded to user facility adverse event reports. In cases in which resolutions of problems are undocumented, FDA has little assurance that problems with devices are being corrected. FDA also has limited information both to share with other reporters who have encountered difficulties with similar devices about preventing potentially hazardous situations and for its own use in analyzing subsequent problems with similar devices. Moreover, without adequate documentation, FDA has little assurance that the many reports received are, in fact, useful and result in better protection of the public health.

CDRH attributed its lack of documentation to the large volume of adverse event reports and limited staff resources. In spite of not having systematically documented the actions taken in response to reports received, CDRH did provide us with several examples of corrective actions it has taken. In addition, FDA said it plans to maintain more complete data on corrective actions that are taken in response to event reports submitted after July 31, 1996. Finally, FDA cannot assess its performance as an early warning system for device problems because it does not keep track of the length of time that it takes to review and resolve serious device problems.

FDA Does Not Routinely Document Resolutions of Reported Problems

In previous work, we found that over two-thirds of the adverse event reports received by FDA from 1985 through 1987 lacked a clear-cut determination, such as a recall, a voluntary action by the manufacturer, or even an indication that the information submitted by the manufacturer was insufficient to process the case. To encourage greater use of medical device reporting data, we recommended that FDA fully document its use of these data in acting to correct device problems, especially by ensuring that such actions were recorded in the database. FDA agreed with our recommendation and reported that changes were under way to improve its handling of event reports. However, we found during our recent analysis of adverse event reports and discussions with CDRH officials that CDRH has not begun routinely documenting the final dispositions of device problems in individual event reports.

To determine how manufacturers and FDA have responded to user facility reports, we reviewed 30 reports of events that manufacturers became aware of through user facility reports from 1992 through 1995. The reports documented 1 death, 20 injuries, and 9 malfunctions. Our initial review showed that 29 of the 30 reports contained neither the final determination of the cause of the device problem reported nor a description of the corrective action taken by the manufacturer or FDA. The remaining report described a serious injury that was allegedly caused by a faulty ventilator. FDA later found the injury to have been unrelated to the device.

We reviewed the 30 reports with an FDA analyst to obtain an update on each adverse event. In 16 out of 30 cases, the cause of the event had not yet been identified, and FDA planned to continue to monitor the devices.Twelve reports were reviewed by analysts upon initial receipt, but no further action was documented. Two malfunction reports were assigned to ad hoc groups to determine whether the devices posed an immediate health risk to the public; the groups determined that no additional action was needed.

A CDRH official explained that, under FDA’s medical device reporting system, the decisions that analysts make on adverse event reports are not routinely documented, and statistics are not maintained on the types of corrective actions taken by FDA and manufacturers. The official recalled that during the early 1980s, CDRH closed each case with a conclusion about the probable cause and effect of the event but that this was too resource intensive. When the number of reports increased significantly, CDRH chose to focus on reviewing the reports rather than on documenting how they were resolved.

The official also emphasized that documenting problem resolutions is only an issue with regard to reports that can be or are investigated, which is a relatively small subset of all reports received. Moreover, complete documentation of the resolution of a problem is dependent upon a number of factors that FDA cannot control. According to the official, in some cases FDA has great difficulty obtaining required information from user facilities on device-related events so that it can assess the public health risks. As an example, he cited the following scenario: A manufacturer receives a report that one of its disposable devices has failed. However, the device in question has been discarded by the user facility before the manufacturer can examine it and perform tests on it. Consequently, the manufacturer is unable to adequately report pertinent information, and FDA’s ability to ascertain the problem, its causes, and any resolutions is hampered.

Despite the problems associated with receiving complete information on device problems, the official acknowledged that providing a close-out record for each report could, among other things, provide reporters with feedback to encourage reporting and provide FDA with a pool of information that could be used to ascertain the impact or probable outcome of any subsequent reports on similar problems with the device.

FDA Has Documented Some Corrective Actions

Although CDRH does not routinely link corrective actions to reported medical device problems, it provided us examples of how it has nevertheless identified and acted on serious problems with devices in several instances. The following examples show that well documented events, analysis, and corrective actions are invaluable in responding to reported problems.

Hospital Bed Side Rails

Between January 1990 and June 1995, FDA received 102 reports of head and body entrapment incidents involving hospital bed side rails. These reports indicated that 68 deaths, 22 injuries, and 12 entrapments without injury had occurred in hospitals, long-term care facilities, and private homes. FDA’s analysis of the reports showed that the entrapments actually involved side rails, headboards, footboards, and mattresses. The majority of these adverse events involved elderly patients who were suffering from confusion, restlessness, lack of muscle control, or a combination of these conditions.

In July 1995, a CDRH ad hoc group decided that a safety alert should be issued to apprise health care professionals of the entrapment hazards associated with hospital beds and of ways to prevent them. CDRH consulted with the Consumer Product Safety Commission and the Canadian government, as well as with numerous professional organizations and manufacturers, to obtain their comments as it developed the alert. CDRH reported that 94,000 copies of the alert were distributed to the user facility community.

Dialysate Solution Storage Containers

A dialysis nurse reported an incident involving eight patients who exhibited several symptoms, including severe hypotension, during dialysis. Three of these patients died. A top priority inspection revealed the patients had high serum aluminum levels. After further investigation, it was found that the dialysis solution was being stored in a pump containing aluminum, metered to the patients through such a pump, or both, and that the aluminum had leached into the dialysate concentrate. A safety alert that discussed the potential hazard created when patients are exposed to dialysate with excessive aluminum levels was issued in May 1992.

Apnea Monitor

In February 1985, an infant was disconnected from its apnea monitor but the electrodes with the lead wires were left attached to the infant. A sibling plugged the lead into the power cord, electrocuting the infant. CDRH assembled a committee of experts to investigate the hazard. At that time, about 50,000 apnea monitors—of 20 different models—were on the market. The committee decided that a voluntary plan of action would be most effective because of the speed with which it could be implemented.

In June 1985, CDRH issued a safety alert summarizing the reported events and outlining steps to guard against future incidents. CDRH also sent letters to 31 manufacturers of breathing frequency monitors and heart monitors for home use requesting that each firm evaluate its device for the electrode problem and, when necessary, consider design changes to prevent insertion of the lead connectors into AC power cords or outlets. All 31 device firms either changed their lead designs or explained to CDRH why their device did not present a hazard.

An FDA analysis also shows that many other adverse event reports were used to resolve device problems. For example, FDA reported that from 1985 through 1995, 1,099 of its 4,365 classified recalls (25 percent) were associated with event reports. FDA also reports using information in adverse event reports for other health-related purposes, including identifying areas in which user education can be improved, analyzing premarket approval applications, developing medical device standards, and monitoring device problems in foreign countries.

In the near future, CDRH expects to have better data on corrective actions taken by manufacturers in response to user facility reports. The final medical device reporting regulation requires manufacturers to list on the form 3500A a corrective action taken—such as a recall, repair, relabeling, or other modification—for each device associated with an adverse event report. According to a CDRH official, manufacturers are now also required to refer to the user facility’s unique reporting number when submitting an adverse event report.

FDA Does Not Record How Long It Takes to Respond to Reported Device Problems

Maintaining reliable productivity indicators on the length of time FDA takes to process, review, and initiate action on device problems and on the time that passes before manufacturers correct the problems would better ensure that serious device problems are receiving prompt attention. It would also allow FDA to better measure its adverse event reporting system’s performance as an early warning system. However, CDRH does not have statistics available on these areas of performance. Thus, FDA has no reliable way of knowing how long it takes to respond to reported device problems.

According to FDA, once the enhancements to the MAUDE system are fully operational, it will be able to provide FDA with better information on the time FDA takes to process and review reports. However, FDA officials told us that the MAUDE system will not be able to provide meaningful statistics on the time FDA and manufacturers require to take action on reports. According to FDA, given the complexity of the issues involved and the volume of reports received, any performance indicator based on the length of time needed to correct reported problems would not provide statistics that are reliable. FDA contends that this is an inherent attribute of any system that builds on reports accumulated over many years.

While FDA may not be able to develop reliable measures of performance based on the time taken to correct device problems, it can track the time needed by FDA analysts to initiate action on reports. As a result of report reviews, FDA analysts may request additional information from manufacturers and user facilities about the severity of the event and any corrective action taken, request an inspection of the manufacturer’s facility, request ad hoc meetings to discuss issues pertaining to a problem device, or determine that a report requires no further action and archive the report for future research and monitoring. Once one of these initial courses of action is taken by an analyst, FDA can document the date the action was taken in the adverse event reporting system, thereby providing FDA with a more complete measure of its response time.

FDA Provides Limited Feedback on Adverse Event Trends and Corrective Actions

Another important facet of the adverse event reporting system is the communication of trends in device-related problems and corrective actions taken to ensure the safety and efficacy of medical device products. Feedback to the medical device community on reported problems and corrective actions could increase the knowledge of user facilities and manufacturers about the performance of devices, assist users in making device purchase decisions, and improve awareness of patient safety. However, CDRH does not have a system for routinely communicating adverse event reporting trends and remedial actions taken on device problems to specific reporters of device problems and the medical device community at large. Although FDA occasionally issues safety alerts and public health advisories, which inform the medical device community about health risks associated with devices, it does not provide user facilities and other medical device reporters with direct feedback on the status and outcomes of individual device investigations. Nor does it publish composite statistics on event trends and problem resolutions. Consequently, many adverse event reporters are unaware of the dispositions of their complaints and have little assurance that the agency is taking action on the device problems reported. Both user facilities and the industry have suggested that FDA provide more feedback on how it uses adverse event reports.

A CDRH official told us that FDA does not have a notification system for advising individual user facilities and other reporters about the status and outcomes of reports because significant staff time would be required to provide feedback to reporters on the more than 100,000 event reports received annually. The official said further that often no feedback is available because FDA does not follow up on each event that is reported. Voluntary reporters receive an acknowledgment letter, but it does not provide specific feedback about their report. FDA does, however, disseminate safety alerts and public health advisories regarding medical device problems to user facilities; it also periodically features alerts in its User Facility Reporting Bulletin. From the beginning of fiscal year 1994 through June 1996, FDA issued three safety alerts and seven public health advisories.

User facility and industry officials believe that FDA could better protect the public health by publishing statistics on adverse event trends and corrective actions that result from medical device reports. For example, a senior official at a teaching hospital in Baltimore, Maryland, told us that although the adverse event reporting system has received an increased quantity of data, he has seen no analysis of this information that would provide device users with suggestions on courses of action that could be taken to prevent incidents from recurring. Similarly, representatives of the Medical Device Manufacturers Association (MDMA) told us that since the early phases of medical device reporting, their members have derived little benefit from the hundreds of thousands of reports stored within the agency’s databases. MDMA representatives believe the public should know what benefits have been derived from reporting. In addition, a senior executive of a New Jersey hospital said he was disappointed that a database has not been developed that can be searched for information about problem medical devices to help his hospital learn more about the performance of devices and assist in device purchase decisions. Similar views were repeatedly expressed in responses to a 1994 FDA questionnaire sent to user facilities: Over 642 of the 4,419 participating user facility officials—or 15 percent—indicated that they wanted more feedback from FDA on adverse event reports.

According to FDA, the MAUDE system will be used to work closely with the device community to identify safety problems; explore the most effective strategies for resolving problems; and provide feedback to user facilities, manufacturers, and the public.

Medical Device Community Suggests Ways to Improve Reporting

Some representatives of the medical device community believe that reducing the volume of adverse event reports could help FDA more effectively manage the reporting system. Suggestions for improving reporting include (1) requiring a statistical sample of user facilities, rather than all user facilities, to report device-related events to FDA and manufacturers and (2) eliminating malfunction reporting by manufacturers to FDA.

Require Only a Statistical Sample of User Facilities to Report

According to the President of the Emergency Care Research Institute (ECRI), a nonprofit research agency that operates a worldwide medical device reporting system, FDA’s adverse event reporting system is hampered by prolific and duplicative data that divert resources away from analysis to data entry. ECRI contends that the adverse event reporting system, as currently configured, makes it difficult for FDA to effectively recognize, track, cross reference, investigate, and follow up on significant medical device problems.

ECRI believes that FDA should free all user facilities but hospitals from the reporting requirement and employ statistical sampling techniques to limit even the number of hospitals required to report device-related problems to manufacturers and FDA. Sampling, according to ECRI, would reduce duplication, speed processing and analysis, and free up resources to devote to analysis and educating hospitals about medical device reporting.

NEMA members, however, believe that developing a user facility reporting program predicated on reports from only a small number of entities would pose at least two implementation problems. First, NEMA members believe, in general, that statistical sampling is beneficial only when it pertains to identical devices from one manufacturer. If statistical sampling is used in the device industry, they contend, the possibility exists that problems identified with a given type of device could be incorrectly generalized to apply to devices that do not possess the same attributes. Such a generalization could impose additional costs on the manufacturer if it had to repeatedly spend resources on investigating incident reports that turned out to be groundless.

Second, statistical sampling is also problematic from the perspective of the user facility, in that not all user facilities are alike. In the opinion of NEMA members, the only useful approach to a statistical sampling program would be to confine the sample size to a small number of user facilities that share similar patient populations and operating characteristics, such as Public Health System facilities or Veterans Administration facilities.

According to the safety coordinator at a hospital in Boston, Massachusetts, statistical sampling would not be a good idea because, given the demands on the user facility community to provide quality health care, if reporting were not required, facilities would not report medical device-related problems. He believes that medical device reporting is an important prerequisite for good patient care and that user facilities must be required by law to report medical device-related incidents in order for them to consistently do so.

CDRH’s director told us that CDRH frequently receives adverse event reports from user facilities that are not very helpful in identifying and resolving device problems. He went on to say that CDRH would prefer to receive fewer, but more helpful reports. As a result, CDRH is planning a pilot study to determine the feasibility of adopting the statistical sampling approach to medical device reporting. Under this approach, FDA would educate a smaller population of user facilities in reporting device problems, which might improve the quality of reporting and increase manufacturers’ and FDA’s knowledge about device problems.

CDRH recently issued a “Request for Proposals for a Sentinel System” that asked prospective bidders to develop a design for a pilot study involving about 10 to 20 user facilities. Bidders are required to recruit and negotiate with user facilities interested in participating in the pilot study. Staff of the user facilities that are chosen to participate will be trained in the medical device reporting requirements and user facility responsibilities. Participating facilities will be monitored, and the adverse events reported will be qualitatively evaluated for 12 months. User facilities will receive feedback and refresher training as needed. Depending on the results of the pilot study, CDRH envisions that a representative sample of user facilities could be used either to supplement the existing system or as a replacement for the universe of user facilities, if approved by the Congress.

Eliminate the Requirement That Manufacturers Report Malfunctions

Another industry suggestion for improving FDA’s adverse event reporting system is to eliminate malfunction reporting. Currently, manufacturers are required to report to FDA malfunctions that are likely to cause or contribute to a death or serious injury should they recur. During fiscal years 1991 through 1995, reports of device malfunctions accounted for about one-third of all reports submitted by manufacturers. Industry officials contend that the medical device reporting system is so overburdened that FDA does not have an opportunity to evaluate the reports of device malfunction and that such reporting is costly and burdensome. Thus, these officials believe malfunctions should no longer be considered reportable events. They contend that manufacturers have systems in place to review and evaluate product malfunctions and that these evaluations are available to FDA during the routine inspections that FDA makes of their facilities.

The purpose of the 1984 medical device regulation was to provide FDA with more information about device problems, such as malfunctions, so that the agency could learn about potential device problems before they endangered the lives of people. A CDRH official told us that deciding whether to eliminate malfunction reporting is the choice between having an adverse event reporting system that is proactive and one that is reactive. Further, he stated that if the objective of medical device reporting is to find out that a device is dangerous after it has caused or contributed to a death or serious injury, then malfunction reporting can be eliminated. However, if the objective is to become aware of problems before they have caused a death or serious injury, then malfunction reporting should not be eliminated. CDRH, he said, would prefer to retain malfunction reporting so that FDA can continue to learn of potential public health problems and resolve them before a serious injury or a death occurs.

Conclusions

The quantity of information reported to FDA about medical device problems has increased dramatically since SMDA ’s reporting requirements became effective in 1991. Because FDA has not ensured that reported device problems receive prompt attention and appropriate resolution, however, its adverse event reporting system is not providing an early warning about device problems. Without reliable statistics that measure the length of time that FDA takes to review and initiate action on device problems and systematic documentation of how all reported problems are resolved, FDA cannot ensure that the system is serving as an early warning system for unsafe and ineffective devices. Moreover, because FDA has not identified and followed up with user facilities that may have underreported thousands of patient deaths, its ability to analyze device problems and assess the public health risk may be significantly hampered. Finally, without feedback from FDA to user facilities about device problem trends and corrective actions taken, user facilities do not receive information that could help them determine which devices to purchase, and they have little assurance that FDA is taking action on their adverse event reports.

FDA’s new MAUDE system is now providing opportunities for FDA to correct weaknesses in its adverse event reporting system. In July 1996, FDA began requiring manufacturers to list on the form 3500A corrective actions that are initiated to resolve medical device adverse events. This improvement will provide the MAUDE system with the information that FDA needs to systematically document solutions to device problems and help it communicate more effectively with user facilities. In addition, once the MAUDE system is fully operational, it will be able to generate productivity data on the time FDA takes to process and review adverse event reports. If FDA expands this capability to include the time it requires to initiate action on reports, it will be able to more easily measure its performance as an early warning system. The MAUDE system will also enable FDA to identify and follow up with user facilities that do not submit reports of device-related deaths to FDA. If FDA uses the MAUDE system in this way, FDA will be able to ensure that user facilities are complying with the law and also obtain the user facilities’ perspectives on these events.

Limiting the number of user facilities required to report medical device problems could improve the quality of data FDA receives without jeopardizing its ability to identify device problems. On the other hand, fewer reporters could mean that problems would be less likely to be identified and resolved, especially for devices with relatively low usage rates. An evaluation of whether the identification and correction of device problems would better be accomplished through the current system or through a smaller user facility reporting program may help FDA decide which approach is better for protecting the public health. Furthermore, although FDA’s initiatives may improve its reporting system, they do not address its difficulty in ensuring prompt resolution of device problems, compliance with SMDA document corrective actions on adverse event reports that result from analysis and investigations of device problems; and collect and disseminate adverse event trend analysis and corrective actions taken by manufacturers and FDA to the medical device community.

Finally, we recommend that FDA’s study of an adverse event reporting system based on a representative sample of user facilities focus on whether this approach can provide manufacturers and FDA with the quantity and quality of information needed to rapidly identify and correct problems with devices that have varying usage rates.

Agency Comments and Our Response

FDA agreed with our recommendations for improving its adverse event reporting system. However, FDA believes that the three components of its postmarket surveillance system together provide an adequate early warning system for device problems and help to ensure that the public is protected. Specifically, FDA believes that (1) medical device reports are supplemental in nature and that its entire postmarket surveillance system, its GMP regulations, and other related activities combined constitute the most comprehensive source of information regarding marketed medical devices; (2) the deficiencies in the adverse event reporting system cited in our report are being corrected; and (3) adverse event reports from all sources are promptly and thoroughly reviewed, and appropriate actions are taken to protect the public health.

While we agree that FDA uses several sources of information to monitor marketed devices, we disagree with their suggestion that the adverse event reporting system serves mainly as a supplement to other components of the postmarket surveillance system. This suggestion conflicts with FDA’s response to a recent congressional inquiry in which FDA characterized the adverse event reporting system as “critically” important to its postmarket surveillance system because it provides a balance with its premarket evaluation of new medical devices. FDA also characterized the adverse event reporting system as a “safety net” that allows the agency to move new technologies into the marketplace more rapidly because problems with devices that were not detected at the premarketing stage can be identified and corrected through the adverse event reporting system. Moreover, FDA’s implication that it relies more heavily on its GMP compliance program than on the medical device reporting system to help protect the public health is worrisome. The GMP program is intended to assess the safety and effectiveness of devices before, rather than after, they reach users. Yet in an earlier study of the program, we documented that FDA did not inspect many manufacturers of medium- and high-risk devices at least once every 2 years, as required by law; the quality of the inspections that were performed was frequently poor; and the inspections did not detect quality assurance problems.

While we acknowledge that FDA has recognized weaknesses in the adverse event reporting system and has instituted improvements to correct them, none of these improvements has been tested and proven effective. For example, although the form 3500A may provide FDA with the information it needs to better protect the public health, voluntary use of the form by user facilities since 1992 has shown that many data quality problems exist. Further, it is too early to tell if mandatory use of the form 3500A will result in improved reporting.

Although FDA has provided some information that associates medical device reports with recalls and other administrative and regulatory activities, the adverse event reporting system does not routinely identify the actions that were initiated to correct the problems detailed in event reports. Nor does the system provide data on how long it takes the agency to respond to serious device problems. In our view, an effective adverse event reporting system should permit FDA to readily determine how each event report was handled and that all problems reported received prompt attention and resolution.

FDA’s written comments on a draft of this report are reproduced in appendix III. FDA also provided technical comments clarifying aspects of its adverse event reporting system, which we have incorporated in this report where appropriate.

We are sending copies of this report to other congressional committees and Members with an interest in this matter, and we will make this report available to others upon request. If you or your staff have any questions about this report, please call me on (202) 512-7119 or John C. Hansen, Assistant Director, on (202) 512-7105. Others who contributed to this report are Darryl Joyce, Lynn Filla-Clark, Barbara Mulliken, and Joan Vogel.

Scope and Methodology

We conducted our review of the Food and Drug Administration’s (FDA) implementation of the Safe Medical Devices Act of 1990 (SMDA and to publish the final medical device reporting regulation required by the act.

We analyzed computerized data from FDA’s adverse event reporting system on trends in device-related events from fiscal years 1987 through 1995 to determine whether the volume of reporting of device problems had changed since the user facility reporting requirement became effective in 1991. We also reviewed statistics compiled by CDRH on the extent to which user facilities had filed adverse event reports with FDA during fiscal years 1992 through 1995 in accordance with the act. Specifically, we examined whether user facilities had done the following: reported deaths to FDA and manufacturers, reported serious injuries and serious illnesses to FDA only when the manufacturer’s name was unknown, reported these deaths and serious injuries and illnesses to FDA within 10 work days of becoming aware of them, and submitted a semiannual report to FDA summarizing device-related events that took place during the previous 6 months.

In addition, we reviewed FDA field office inspections of user facilities’ compliance with the SMDA health care providers, two nonprofit organizations, one nursing home association, and three device manufacturer associations). We discussed with them their experiences with user facility reporting and whether changes to SMDA and FDA’s adverse event reporting system were needed to improve reporting. We identified these organizations through a review of the literature and through a list of contacts obtained from FDA. Table I.1 provides the name, location, and a brief description of each organization interviewed.

Beth Israel Hospital, Boston, Mass.

Columbia/HCA, Health Care Corporation, Nashville, Tenn.

Johns Hopkins University Hospital, Baltimore, Md.

Rahway Hospital, Rahway, N.J.

New Britain General Hospital, New Britain, Conn.

American Health Care Association, Washington, D.C.

Mayo Foundation, Rochester, Minn.

Public Citizen, Washington, D.C.

Emergency Care Research Institute, Plymouth Meeting, Penn.

A nonprofit health services research agency whose mission is to improve patient care, ECRI operates a Consumer Reports-like medical product evaluation program and a broader health care technology assessment program that encompasses devices, drugs, biotechnologies, and medical and surgical procedures.

Medical Device Manufacturers Association, Washington, D.C.

Health Industry Manufacturers Association, Washington, D.C.

National Electrical Manufacturers Association, Rosslyn, Va.

FDA’s Adverse Event Report Evaluation Process

FDA receives adverse event and product problem reports from user facilities, manufacturers, distributors, and health care professionals. User facilities, manufacturers, and distributors use a standard form 3500A for reporting device-related events. User facilities and distributors must report patient information, the type of adverse event, a description of the event, relevant laboratory data and patient history, the name of the manufacturer, and certain other information about the device. Manufacturers must report such information as the source of reported information; the type of event reported to them; whether the device was returned to the manufacturer for evaluation; methods of evaluation and results of manufacturer review; and, if applicable, the type of remedial action taken (recall, repair, relabeling, or inspection). Health professionals report adverse events and product problems voluntarily on a form 3500.

Event reports are entered into an adverse event database, with the appropriate quality control checks, by FDA’s data entry contractor, Logistics Applications, Inc. Thereafter, OSB evaluation teams consisting of 15 specialized health care professionals (14 nurses and 1 nuclear medicine technologist) retrieve the reports from the database via the computer for review. Analysts may each have responsibility for all products associated with an assigned medical specialty area, such as radiology, ophthalmology, or orthopedics, or they may be responsible for specific types of products associated with larger clinical areas, such as general hospital, general surgery, and cardiology. Analysts’ reviews focus on identifying the device problems that pose the greatest risk to the public health. Deaths are given first priority, but serious injuries and illnesses are also important concerns, unless they are not considered life threatening.

To assess the nature and magnitude of a problem, the analysts use their clinical expertise to evaluate the data submitted in the adverse event report, as well as other information, such as premarket submissions on similar devices, recall information, and literature reviews on adverse events or reported problems. Known complications and problems associated with a particular device are screened by a computer and a sample of these reports is assigned to the appropriate analyst for review. Reports of particular importance, such as pediatric deaths, are routinely sent to the appropriate analyst either prior to or concurrent with the data entry process. This provides the analyst an opportunity to commence an investigation while the actual report is undergoing data entry.

On the basis of their evaluation of reported events, the analysts determine if any follow-up investigation is needed. These investigations generally involve written requests for additional information from a manufacturer, user facility, or voluntary reporter on aspects of a reported incident or an inspection of either the manufacturer or the user facility.

Written requests may ask for such patient data as an evaluation of the reported incident by a health care practitioner; a patient history, underlying diagnosis, or both; and autopsy results relevant to the reported event. Similarly, requests for device data may include product identifiers such as model, lot, and catalog numbers of a device; the length of time a device has been in use; the disposition of a device involved in a reported event; results of any manufacturer failure testing; and copies of device labeling and instructions for use. This information enables the analyst to better assess the cause of a reported problem. In addition, information may be requested on the number of devices manufactured, distributed, and in use, which assists in determining the exposure of the population at risk. The analyst may also request that a manufacturer provide information on the frequency and severity of a reported event, as well as any corrective action taken by a manufacturer to address a reported problem.

In addition, the analyst may request an inspection of the manufacturer’s facility when immediate follow-up on a reported event is necessary or when the reported event suggests the need for an FDA investigation. Upon receipt of the results of the investigation, the analyst evaluates the data and assesses whether or not further FDA action is needed. If warranted, the adverse event report and any related information are forwarded to the appropriate group within FDA for consideration of regulatory action, such as recall or device user notification through safety alerts, public health advisories, a press release warning the public of potential hazards associated with the device in use, or all three. If the analyst determines that no further action is needed, the adverse event report is archived for future research and monitoring.

Related GAO Products

Medical Device Regulation: Too Early to Assess European System’s Value as Model for FDA (GAO/HEHS-96-65, Mar. 6, 1996).

Medical Technology: Quality Assurance Needs Stronger Management Emphasis and Higher Priority (GAO/PEMD-92-10, Feb. 13, 1992).

Medical Device Problem Reporting: A Case Study of a Home Apnea Monitor (GAO/T-PEMD-90-10, July 17, 1990).

Medical Devices: Underreporting of Serious Problems With a Home Apnea Monitor (GAO/PEMD-90-17, May 31, 1990).

Medical Devices: Underreporting of Problems, Backlogged Systems, and Weak Statutory Support (GAO/T-PEMD-90-2, Nov. 6, 1989).

Medical Devices: The Public Health at Risk (GAO/T-PEMD-90-2, Nov. 6, 1989).

Medical Device Recalls: Examination of Selected Cases (GAO/PEMD-90-6, Oct. 19, 1989).

Medical Device Recalls: An Overview and Analysis 1983-88 (GAO/PEMD-89-15BR, Aug. 30, 1989).

Medical Devices: FDA’s Implementation of the Medical Device Reporting Regulation (GAO/PEMD-89-10, Feb. 17, 1989).

Medical Devices: FDA’s 510(k) Operations Could Be Improved (GAO/PEMD-88-30, July 11, 1988).

Medical Devices: FDA’s Forecasts of Problem Reports and FTEs Under H.R. 4640 (GAO/PEMD-88-30, July 11, 1988).

Medical Devices: Early Warning of Problems Is Hampered by Severe Underreporting (GAO/T-PEMD-87-4, May 4, 1987).

Medical Devices: Early Warning of Problems Is Hampered by Severe Underreporting (GAO/PEMD-87-1, Dec. 19, 1986).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.