Consumer Product Safety Commission: Better Data Needed to Help Identify and Analyze Potential Hazards
Released: 1997-10-23T00:00:00

Letter

Background

CPSC was created in 1972 under the Consumer Product Safety Act (P.L. 92-573) to regulate consumer products that pose an unreasonable risk of injury, to assist consumers in using products safely, and to promote research and investigation into product-related deaths, injuries, and illnesses. CPSC currently has three commissioners, who are responsible for establishing agency policy. One of these commissioners is designated as the chairman, who directs all the executive and administrative functions of the agency.

In fiscal year 1997, CPSC carried out its broad mission with a budget of about $42.5 million and a full-time-equivalent staff of 480. After adjusting for inflation, the agency’s budget has decreased by about 60 percent since 1974. Similarly, CPSC’s current staffing level represents 43 percent fewer positions as compared with the agency’s 1974 staff. voluntary product safety standards. CPSC also addresses product hazards by providing information to consumers on safety practices that can help prevent product-related accidents. In addition to its own efforts to disseminate information, CPSC provides considerable amounts of information in response to requests from the public.

CPSC Has Limited Information Available to Assist in Project Selection

CPSC’s Project Selection Process

CPSC’s resource base and extensive jurisdiction require the agency to select among potential product hazards. New agency initiatives may come to CPSC in several ways. First, any person may file a petition requesting CPSC to issue, amend, or revoke a regulation. For example, CPSC’s cigarette lighter project, which resulted in a new mandatory safety standard, originated with a petition from an emergency room nurse. Second, CPSC can receive a product hazard project from the Congress. The Congress may require CPSC to study a wide-ranging product area (such as indoor air quality) or impose a specific regulation (such as a mandatory safety standard for garage door openers). Third, CPSC commissioners and agency staff can initiate projects or suggest areas to address.

CPSC has wide latitude over which potential product hazards it targets for regulatory and nonregulatory action. Although the agency has little or no discretion over projects mandated by the Congress, it can accept or reject suggestions submitted by petition or proposed by agency staff. Of the 115 projects the agency worked on from January 1, 1990, to September 30, 1996, 59 percent were initiated by CPSC, 30 percent originated from a petition, and about 11 percent resulted from congressional directives. the frequency of injuries and deaths resulting from the hazard; the severity of the injuries resulting from the hazard; addressability—that is, the extent to which the hazard is likely to be reduced through CPSC action—agency regulations note that the cause of the hazard should be analyzed to help determine the extent to which injuries can reasonably be expected to be reduced or eliminated through CPSC action; the number of chronic illnesses and future injuries predicted to result from the hazard; preliminary estimates of the costs and benefits to society resulting from CPSC action; the unforeseen nature of the risk—that is, the degree to which consumers are aware of the hazard and its consequences; the vulnerability of the population at risk—whether some individuals (such as children) may be less able to recognize or escape from potential hazards and therefore may require a relatively higher degree of protection; the probability of exposure to the product hazard—that is, the number of consumers exposed to the potential hazard, or how likely it is that typical consumers would be exposed to the hazard; and additional criteria to be considered at the discretion of CPSC.

Commissioners and staff may select projects on the basis of what they believe are the most important factors. For example, the regulations do not specify whether any criterion should be given more weight than the others, nor that all criteria must be applied to every potential project. Our interviews with present and former commissioners and our review of CPSC briefing packages showed that three criteria—the number of deaths and injuries, the cause of injuries, and the vulnerability of the population at risk—were more strongly emphasized than the others. However, although the commissioners and former commissioners we interviewed generally agreed about which criteria they emphasized for project selection, they expressed very different views on how some of these criteria should be interpreted. For example, their opinions differed about choosing projects on the basis of the cause of injuries. A major issue in this regard concerned the appropriate level of protection the agency should be responsible for providing when a product hazard results, at least in part, from consumer behavior. Some current and former commissioners argued that no intervention was warranted when consumer behavior contributed to injuries; others were more willing to consider a regulatory approach in these situations.

Project Management Information Was Incomplete or Unavailable

Although CPSC conducts a number of projects annually, staff were unable to give us a comprehensive list of projects the agency had worked on in the 6-year period we examined. CPSC was also unable to verify the completeness of the project list that we compiled from agency documents and interviews with staff. According to CPSC staff, internal management systems do not generally contain this information because most projects are accounted for under either broad codes such as “children’s products” or activity codes such as “investigations,” “product safety assessment,” and “emerging problems.” In addition, CPSC staff told us that reliable inferences about the characteristics of individual projects, their outcomes, and the resources spent on them cannot be drawn from management information systems because of limitations in the computer system and because no consistent rule exists about how staff time in different directorates is recorded to project codes. Without systematic and comprehensive information on its past efforts, CPSC cannot fully assess whether its projects overrepresent some hazard areas and therefore agency resources might be more efficiently employed. In our report, we recommend that the Chairman of CPSC direct agency staff to develop and implement a project management tracking system to compile information on current agency projects.

Significant Gaps Exist in CPSC’s Data on All Selection Criteria, Including Product-Related Injuries and Deaths

CPSC has developed a patchwork of independent data systems to provide information on deaths and injuries associated with consumer products. To estimate the number of injuries associated with specific consumer products, CPSC gathers information from the emergency room records of a nationally representative sample of 101 hospitals. CPSC also obtains information on fatalities by purchasing a selected group of death certificates from the states. Because neither emergency room nor death certificate data provide detailed information on hazard patterns or causes of injuries, CPSC also investigates selected incidents to obtain more detailed information.

CPSC’s data give the agency only limited assistance in applying its project selection criteria. Data on all CPSC’s project selection criteria suffer from major limitations, as shown in table 1. In fact, none of the criteria are supported by complete data that are available for most projects at the time the project is selected.

CPSC staff identified four data-gathering areas as key concerns: (1) lack of data on injuries treated in physicians’ offices and other settings outside the emergency room; (2) lack of data that would identify chronic illnesses that may be associated with consumer products; (3) sketchy information about accident victims, which limits the ability to assess which hazards disproportionately affect vulnerable populations; and (4) lack of data on exposure to consumer products. information is needed not only because it is a criterion for project selection but also because it is important in evaluating the success of CPSC’s injury reduction efforts and determining the need for possible follow-up actions.

According to CPSC staff, identifying chronic illnesses associated with consumer products is nearly impossible with CPSC’s current data. CPSC staff stated that little is known about many chronic illness hazards that may be associated with potentially dangerous substances, and even less information is available about which consumer products may contain these ingredients. Chronic illnesses are likely to be especially underestimated in CPSC’s emergency room data, because they are underrepresented among emergency room visits and because product involvement is more difficult to ascertain. Similarly, consumer product involvement is seldom recorded on death certificates in the case of chronic illnesses.

Sketchy information about accident victims also limits CPSC’s ability to assess which consumer product hazards have a disproportionate impact on vulnerable populations. CPSC’s surveillance data systems provide information only on the age of the victim; no systematic or comprehensive information is available to determine whether a given hazard has a special impact on other vulnerable populations such as people with disabilities. A former commissioner told us that the lack of other demographic information (such as race, income, and disability status) made it difficult to know which subpopulations were predominantly affected by a particular hazard. Another commissioner echoed this concern, adding that such information would be useful in targeting public information campaigns on certain hazards to those groups that need the information most. testing of the product, and recreations of the incident. As with exposure data, these investigations are not conducted for every project and are done only after a project has been established. Thus, assessment of causation at the project selection stage is unavoidably speculative.

We believe that improved information on each of these four areas is necessary for CPSC to make informed decisions on potential agency projects. However, we also recognize that such information may be costly to obtain. In our report, we recommend that the Chairman of CPSC consult with experts both within and outside the agency to prioritize CPSC’s needs for additional data, investigate the feasibility and cost of alternative means of obtaining these data, and design systems to collect and analyze this information.

Better Data and Methodology Are Needed to Improve CPSC’s Cost-Benefit Analysis and Risk Assessment

CPSC uses two analytical tools—risk assessment and cost-benefit analysis—to assist in making decisions on regulatory and nonregulatory methods to address potential hazards. Risk assessment involves estimating the likelihood of an adverse event, such as injury or death. Cost-benefit analysis details and compares the expected effects of a proposed regulation or policy, including both the positive results (benefits) and the negative consequences (costs). The Congress requires CPSC to perform cost-benefit analyses before issuing certain regulations, and CPSC has conducted cost-benefit analyses for these regulations and in other situations in which such an analysis was not required by law. Because most of the agency’s projects do not involve regulation, relatively few CPSC projects conducted between January 1, 1990, and September 30, 1996, were subject to these requirements. We identified 8 cost-benefit analyses that CPSC performed in accordance with these requirements and an additional 21 analyses that it conducted when it was not required. Before issuing certain regulations, CPSC is required to consider the degree and nature of the risk of injury the regulation is designed to eliminate or reduce. However, CPSC usually does not conduct a formal, numerical risk assessment before issuing a regulation, and the law does not require it to do so. We determined that CPSC conducted 24 risk assessments between January 1, 1990, and September 30, 1996; only 4 of these were associated with regulatory action. demands for information posed by risk assessment and cost-benefit analysis. As a result, the agency’s estimates of risks, costs, and benefits are less accurate because they reflect the substantial limitations of the underlying data. For example, because CPSC’s data undercount the deaths and injuries associated with particular consumer products, estimates of risk—and the potential benefits of reducing that risk—appear smaller than they actually are. However, CPSC’s data provide information only on whether a product was involved in an accident, not whether the product caused the accident. This can sometimes make the risks assessed by CPSC—and the benefits of reducing those risks—appear greater.

The methodology used to conduct a cost-benefit analysis frequently depends on the circumstances and the context of the analysis. For this reason, there is no complete set of standards for evaluating the quality of an individual cost-benefit analysis. However, the professional literature offers some guidance for analysts, and certain specific elements are frequently used to determine whether a given analysis meets a minimum threshold of comprehensiveness and openness. For example, analysts generally agree that all methodological choices and assumptions should be detailed, all limitations pertaining to the data should be revealed, and measures of uncertainty should be provided to allow the reader to take into account the precision of the underlying data. Similarly, practitioners generally call for sensitivity analysis, which enables the reader to determine which assumptions, values, and parameters of the cost-benefit analysis are most important to the conclusions. the time. Furthermore, some of CPSC’s data sets have a known upward or downward bias because of the way the data were constructed. For example, when estimates of incidents are based only on investigated or reported cases, two potential biases are likely to be introduced into the analysis: (1) the estimates are likely to be biased downward by nonreporting and (2) the incidents reported tend to be the more severe ones. In only 53 percent of applicable cases did CPSC’s analysis inform the reader of known limitations inherent in the data being used for cost-benefit analysis. fewer precautions in response to a change in a product’s safety features. For example, in establishing a standard for child-resistant packaging that was also “senior-friendly,” CPSC considered that because child-resistant medicine bottles can be difficult to open, a grandparent might leave the cover off the bottle, creating an even greater risk than would exist with the original cap. Although CPSC considered such factors in some cases, only 49 percent of its analyses reflected potential risk-risk trade-offs.

CPSC has not established internal procedures that require analysts to conduct comprehensive analyses and report them in sufficient detail. For example, according to CPSC staff, the agency has little written guidance about what factors should be included in cost-benefit analyses, what methodology should be used to incorporate these factors, and how the results should be presented. Staff also told us that CPSC analyses are not generally subject to external peer review. Such reviews can serve as an important mechanism for enhancing the quality and credibility of the analyses that are used to help make key agency decisions. In our report, we recommend that the Chairman direct agency staff to develop and implement procedures to ensure that all cost-benefit analyses performed on behalf of CPSC are comprehensive and reported in sufficient detail, including providing measures of precision for underlying data, incorporating information on all important costs and benefits, and performing sensitivity analysis.

CPSC Has Established Procedures for Complying With Statutory Requirements for Releasing Manufacturer-Specific Information

To help minimize the possibility that a product might be unfairly disparaged, in section 6(b) of the Consumer Product Safety Act, the Congress imposed restrictions on CPSC’s disclosure of manufacturer-specific information. Before CPSC can release any information that identifies a manufacturer, it must take “reasonable steps” to verify the accuracy of the information and to ensure that disclosure is fair; notify the manufacturer that the information is subject to release; and give the manufacturer an opportunity to comment on the information.

These restrictions apply not only to information the agency issues on its own—such as a press release—but also to information disclosed in response to a request under the Freedom of Information Act. Section 6(b) also requires CPSC to establish procedures to ensure that releases of information that reflect on the safety of a consumer product or class of products are accurate and not misleading, regardless of whether the information disclosed identifies a specific manufacturer.

In implementing section 6(b), CPSC established several procedures designed to ensure compliance with these statutory requirements. These include obtaining written verification from individuals of the information they report to the agency, notifying manufacturers by certified mail when manufacturer-specific information has been requested, and giving manufacturers the option to have their comments published with any information disclosed. For example, CPSC has issued clearance procedures for situations when commissioners and staff initiate public disclosures—for example, when CPSC publishes the results of agency research. Under CPSC’s guidelines, each assistant or associate executive director whose area of responsibility is involved must review the information and indicate approval for the release in writing. After all other reviews have been completed, the Office of the General Counsel must also review and approve the release.

Information from three sources—industry sources, published legal cases, and data on retractions—suggests that CPSC complies with its statutory requirements concerning information release. Industry sources, even those otherwise critical of the agency, told us that CPSC generally keeps proprietary information confidential as required by law. Our review of published legal decisions found no rulings that CPSC violated its statutory requirements concerning the release of information. Retractions by CPSC are also rare—only three retractions have been issued by CPSC since the agency was established. suggested possible changes. Although these individuals raised issues about the extent of the protection afforded to manufacturers and the resources necessary to ensure compliance, we did not assess whether the specific suggestions were necessary or feasible.

CPSC’s chairman, other CPSC officials, former commissioners, and the representative of a consumer advocacy group stated that compliance with 6(b) is costly for CPSC and delays the agency in getting information out to the public. To reduce the burden of complying with these requirements, CPSC staff have suggested that the notification requirement that gives manufacturers 20 days in which to comment should apply only to the first time information is released and that, instead of requiring CPSC to verify information from consumer complaints, the agency should be allowed to issue such information with an explicit disclaimer that CPSC has not verified the consumer’s report.

Instead of reducing CPSC’s verification requirements, some industry representatives suggested expanding them. These manufacturers stated that before CPSC releases incident information, the agency should substantiate it, rather than relying on a consumer’s testimony. Industry representatives stated—and CPSC staff confirmed—that many of the requests for CPSC information come from attorneys for plaintiffs in product liability suits. As a result, some industry representatives expressed concern that unsubstantiated consumer complaints could be used against them in product liability litigation. They suggested that 6(b) should require CPSC to substantiate all incident reports by investigating them before they can be disclosed, instead of merely checking with the consumer. However, CPSC officials told us that, because of limited resources, investigations—which are time consuming and costly—can be conducted only on a small proportion of specially selected cases. association said they were trying to work out a more satisfactory arrangement.

Mr. Chairman, that concludes my prepared statement. I would be happy to answer any questions you or Members of the Subcommittee might have.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.