Consumer Product Safety Commission: Awareness, Use, and Usefulness of SaferProducts.gov
Released: 2013-03-11T00:00:00

Letter

Background

CPSC was created in 1972 under the Consumer Product Safety Act to regulate certain consumer products and address those that pose an unreasonable risk of injury; assist consumers in evaluating the comparative safety of consumer products; and promote research and investigation into the causes and prevention of product-related deaths, injuries, and illnesses. CPSC’s jurisdiction is broad, covering thousands of types of manufacturers and consumer products used in and around the home and in sports, recreation, and schools. CPSC does not have jurisdiction over some categories of products, including automobiles and other on-road vehicles, tires, boats, alcohol, tobacco, firearms, food, drugs, cosmetics, medical devices, and pesticides. Other federal agencies—including the National Highway Traffic Safety Administration; Coast Guard; Bureau of Alcohol, Firearms, Tobacco, and Explosives; Department of Agriculture; Food and Drug Administration; and Environmental Protection Agency—have jurisdiction over these products.

Consumers and others previously were able to report safety problems or concerns about consumer products through CPSC’s toll-free hotline; the U.S. mail, or a form on CPSC’s website submitted through e-mail and they can continue to use these methods in lieu of submitting reports through SaferProducts.gov. CPSIA and CPSC define “harm” as injury, illness, or death or risk of injury, illness or death. 15 U.S.C. § 2055a(g); 16 C.F.R. § 1102.6(b)(4). be published. As required by statute, CPSC disclaims any responsibility to guarantee the accuracy of a report.

The submitter of an incident report on SaferProducts.gov must fit into one of five categories: (1) consumers; (2) local, state, and federal government agencies; (3) health care professionals; (4) child service providers; and (5) public safety entities. CPSC regulations specify that “consumers” include, but not be limited to, users of consumer products, family members, relatives, parents, guardians, friends, attorneys, investigators, professional engineers, agents of a user of a consumer product, and observers of the consumer products being used.

CPSIA requires the following information when submitting a report of harm: (1) description of the consumer product sufficient to distinguish the product as a product or component part regulated by CPSC; (2) identity of the manufacturer or private labeler by name; (3) description of the harm related to use of the consumer product; (4) approximate or actual date of the incident; (5) category of submitter; (6) submitter’s contact information; (7) submitter’s verification that the information contained therein is true and accurate; and (8) consent to publication of the report of harm. 15 U.S.C. § 2055a(b)(2)(B); C.F.R. § 1102.10(d). Subject to §§ 1102.24 and 1102.26, CPSC will publish reports of harm containing all the required information. required to contact the submitters for further information.transmits a copy to manufacturers, importers, and private labelers identified in the reports, to provide them with the opportunity to comment. Qualifying reports and manufacturer comments submitted for publication are then available on SaferProducts.gov (see fig. 1).

CPSC Has Used Various Approaches to Inform the Public about SaferProducts.gov, but Has Not Established Metrics to Assess Its Efforts

CPSC’s efforts to promote SaferProducts.gov formed part of a larger effort to increase the public’s awareness of the agency. CPSC has taken a variety of approaches to inform the public about SaferProducts.gov, many of which are consistent with key practices for consumer education planning. However, CPSC has not established metrics for its efforts. As a result, the agency does not know which of its efforts have had the most impact on increasing awareness and use of SaferProducts.gov.

Outreach about SaferProducts.gov Has Been Part of an Effort to Increase Awareness of CPSC

CPSC’s efforts to inform the public about SaferProducts.gov have been part of a larger effort to increase the public’s awareness of the agency. According to CPSC officials, certain segments of the public may not be aware of the agency or its mission in product safety, much less be aware of SaferProducts.gov. Likewise, roughly one-third of the 37 consumers who participated in our website usability tests were aware of CPSC or its mission. To promote awareness of CPSC, officials have conducted public information campaigns related to various product safety hazards such as fire hazards and those involving children’s products, issued press releases about product recalls, and used social media. Officials said that media stories promoting the use of SaferProducts.gov have had the benefit of promoting CPSC as a resource not only for information about product recalls (for which the agency is most commonly known), but also as a place where consumers can raise concerns about the safety of consumer products.

In addition to the outreach efforts noted above, CPSC has planned initiatives to assess the public’s awareness of the agency as a whole. In fiscal year 2011, CPSC’s Office of Communications received funds to award a contract to plan and conduct field surveys to assess consumer awareness of the agency. CPSC and a contractor are developing the survey tool. These surveys are to cover such areas as the public’s knowledge and awareness of the safety issues for which CPSC is responsible, how the agency’s work affects consumers, and how the public responds to product recalls and other safety hazards that CPSC communicates. CPSC officials told us that they plan to administer the survey in 2013, but have been awaiting approval of the survey from the Office of Management and Budget. CPSC also recently redesigned its main website, CPSC.gov, based on feedback from the public. According to CPSC officials, this redesign allowed the agency to provide a more visible link to SaferProducts.gov.

CPSC Has Used a Variety of Approaches to Inform the Public about SaferProducts.gov

As it has for publicizing the agency, CPSC has used a variety of approaches to inform the public about SaferProducts.gov, including the use of social and other media. Before launching SaferProducts.gov, CPSC hosted a web conference on January 11, 2011, to inform interested stakeholders such as consumer groups and the public about the site’s search function and the information required to submit an incident report.2011, CPSC promoted the new website through print and other media. According to CPSC officials, the agency’s promotional strategy emphasized both the public’s ability to search SaferProducts.gov for reports and submit such reports. In addition, near the 1-year anniversary of SaferProducts.gov, CPSC launched three public service Around the time it launched SaferProducts.gov in March announcements (PSAs) about SaferProducts.gov, sending these PSAs to local and national media and making them available on online media channels, such as YouTube (http://www.youtube.com). According to CPSC officials, the agency has a contract with a video production company to produce and distribute the videos. CPSC officials said that the PSAs have been among the 10 most-viewed videos on CPSC’s YouTube channel. They added that it was difficult to attract extensive television coverage or the best airtime slots given CPSC’s PSA budget of about $50,000 for fiscal year 2012. Further, the officials said that PSAs can cost from $700,000 to $1 million to produce, distribute, and air during prime viewing or listening times.

The agency also has distributed informational materials to target audiences at conferences and community events; referenced the site in speeches and presentations by the Chairman, Commissioners, and staff; and held press interviews to promote the site, according to CPSC officials. For example, CPSC developed a series of brochures, including some tailored to specific professional sectors, such as health care, child care, public safety, and government. CPSC officials noted that they have mentioned the site at conferences, particularly those aimed at minority populations and professional groups. The agency also has made a data feed of the incident reports available to third-party software developers to create mobile applications and provided information for developers in a frequently asked questions page on SaferProducts.gov.

In conducting its public information efforts, CPSC has employed a number of strategies consistent with key practices for consumer education planning that we identified in a prior report. For example, CPSC has worked with stakeholders such as consumer groups (a key practice) to promote SaferProducts.gov, and used a variety of media (another key practice) to promote the site. CPSC also has identified “messengers” such as consumer groups and state attorneys general to assist with publicity, and identified the resources needed for publicity (other key practices).

Most of the consumer product safety experts we interviewed from nine groups representing consumers, researchers, and various industries stated that CPSC has been taking appropriate measures to promote the site. However, some also suggested that CPSC could conduct more targeted outreach to other professional groups, such as those in health care, and other populations, such as parents.

CPSC Has Not Established Metrics to Evaluate Its Outreach on SaferProducts.gov

While CPSC has employed many of the key practices for consumer education planning as described previously, it has not employed one of the key practices that could further improve the efficacy of its outreach for SaferProducts.gov. Specifically, CPSC has not established metrics, such as process and outcome metrics, to measure the success of its outreach In its 2013 performance budget request, as part of an effort to efforts.increase awareness of the agency, CPSC has a goal for the number of visits to CPSC.gov. However, CPSC does not have a similar goal for the number of visits to SaferProducts.gov, although it collects such data (as discussed in the next section of this report). Similarly, CPSC has not determined whether its efforts to publicize SaferProducts.gov at conferences or through PSAs have led to increased use of SaferProducts.gov after the events. CPSC also has not incorporated tools or features on the site (such as a drop-down menu on the homepage that would ask users to select an option such as “conferences,” “PSAs,” “printed materials,” or “media”) to identify how the user learned about and arrived at the site. The information generated by such tools also may provide CPSC with ideas for additional metrics to measure awareness and use of the site.

CPSC has not established metrics to evaluate its outreach efforts for SaferProducts.gov because the agency has been focused on increasing awareness of CPSC and improving the functionality of CPSC.gov. CPSC officials said that in comparison with SaferProducts.gov, CPSC.gov received almost 10 times as many visits each month. Officials have said they may focus on evaluating outreach efforts for SaferProducts.gov in the future. However, without current metrics to assess the efficacy of its outreach for SaferProducts.gov, CPSC will not know which of its efforts— for instance, promoting the site at conferences and using PSAs—have had the most impact on increasing awareness and use of SaferProducts.gov, or be able to best target its limited resources to increase use of the site.

CPSC Collects Limited Data about the Use of SaferProducts.gov

CPSC collects limited data about the use of SaferProducts.gov. To track use, CPSC collects data on the number of visitors, most frequently visited pages, and number of reports received, among other metrics. CPSC also collects some data about the category of person who is submitting a report. However, CPSC does not collect any data about who is using the site to search for information. In particular, CPSC has not sought to collect demographic data, such as age, gender, or income. In mandating this report, Congress required us to assess whether a broad range of the public uses the site. However, CPSC’s limited data collection related to use of the site made it difficult to conduct such an assessment.

Website Use Remained Relatively Constant, with Searches and Submission of Reports Constituting Frequent Uses

According to CPSC officials, the agency’s primary measure of the extent of use of SaferProducts.gov is the number of visitors each month. CPSC collects these data through web analytics software. According to CPSC’s data, visits to SaferProducts.gov exceeded 100,000 each month since June 2011, a few months after the launch of the site (see fig. 2), peaking at about 238,000 in November 2012. CPSC officials have not been able to identify the reasons for the increase in visits.

CPSC also collects data on the most frequently visited pages each month (see fig. 3). These data show that users frequently used the site to search for information—for example, to search for recalled products or incident reports submitted by other users of the site.

CPSC also collects data on the number of reports received each month through SaferProducts.gov, as well as by phone, e-mail, postal mail, and fax. These data show that users submitted more than 1,000 reports from all sources each month from March 2011 through December 2012 (see fig. 4).

CPSC collects some data about the categories of persons using SaferProducts.gov to submit incident reports but does not collect additional data such as age, gender, or income level of the submitters or others who use the site to search for information. When completing a report, CPSC requires submitters to state whether they are consumers, represent a government agency, or are health care or other professionals, among other categories of user. As shown in table 1, our analysis of more than 12,000 reports posted on SaferProducts.gov as of January 2013 found that most report submitters—about 97 percent—identified themselves as consumers, results consistent with our prior reporting. As stated previously, “consumers” include, but are not limited to, users of consumer products, family members, relatives, friends, attorneys, investigators, and others. Representatives of government agencies and public safety entities, as well as health care professionals, child service providers, and medical examiners and coroners also submitted reports.

CPSC also asks report submitters to state their relationship to the victim of the incident (such as self, parent, or spouse). As shown in table 2, of those who identified themselves as consumers, most identified themselves as the victims of an incident. However, many submitters did not specify a relationship.

Of those who did specify a relationship to the victim, 4,463, or 60 percent, reported that they were the victims, and 1,867, or 25 percent, reported that their child was the victim (see table 3).

CPSC asks that submitters specify the location of the reported incident, including the country and state. Most submitters providing this information—about 90 percent—reported that the incident took place in the United States (see table 4). Submitters also reported that incidents took place in other countries or did not specify where the incident took place.

In addition, states with the highest population—such as California, Texas, New York, Florida, and Illinois—had the most reported incidents (see fig. 5).

Beyond these data, CPSC does not request or obtain additional details about the users of SaferProducts.gov. According to CPSC officials, the agency also cannot distinguish new users from returning users because CPSC’s web analytics software has not been configured with “cookies” to capture these data. Officials have cited resource and privacy concerns as reasons for not collecting these data, although they said they have been considering using cookies in the future. In addition, CPSC does not collect more specific demographic information such as age, gender, or income level from the submitters of reports or other site users, citing an interest in minimizing the reporting burden on users. As an example, CPSC has not requested that site users voluntarily provide this information during the report submission process or after submitting a report. Congress required us to assess whether a broad range of the public uses SaferProducts.gov, but CPSC’s limited data collection made it difficult to conduct such an assessment. In addition, standards for internal control in the federal government state that agencies should have timely, relevant information for management decision-making purposes.result of its limited data collection about users of the site, CPSC has been limited in its ability to target its marketing and outreach efforts on specific groups, populations, or areas to achieve the goal of increasing use of the site.

Consumer Testing Showed SaferProducts.gov to Be Generally Useful, but Identified Areas for Improvement

As discussed earlier in this report, our website usability tests focused on asking consumers in our testing sessions to judge if SaferProducts.gov was easy to use. We had the consumers perform various tasks (such as searching for recalled products and submitting mock incident reports) and asked for opinions about the site’s usefulness. A moderator facilitated the sessions and we elicited feedback from participants. In addition, a GSA official with expertise in website usability assessed SaferProducts.gov, and another GSA official reviewed the site for website accessibility.

Consumers Said the Site Was Easy to Use, but Noted a Few Search Functions and a Registration Page as Problematic

Many consumers in our testing sessions generally found SaferProducts.gov easy to use, but they encountered difficulties with certain aspects of the two main functions: searching for information and Of the 37 consumers who participated in our submitting incident reports.testing sessions, 20 found SaferProducts.gov easy to use as indicated by their responses to a questionnaire we administered following each test session. For example, almost all the consumers were easily able to determine what initial steps to take to search for or report a product that may be unsafe. In addition, the expert evaluator reviewing the site at our request described the site as clean and easy to navigate.

In conducting the search tasks, consumers generally were able to find recalled products using basic key word searches. But some search functions, including those that required more complicated searches such as use of an advanced search function to narrow results, posed challenges. For example, in one testing session, no consumers were able to complete a task that required them to narrow their search by injury, time period, and location. In another session, the calendar function, which filters the results by time period, posed particular challenges. Five of the eight consumers in that session experienced difficulties in having to enter and, when seeking to make one change, re-enter all the dates to focus their search on products recalled within a particular time period. The expert evaluator from GSA experienced similar challenges in using the calendar function.

In addition, when asked to search for and compare safety information for two products—one for which there were search results and one for which there were none—almost all the testers had difficulty interpreting the lack of search results for the latter product. For example, while some testers assumed that a search for a product that produced no results indicated that the product was safe, others did not make this presumption. In our testing sessions, most consumers were not sure which product to purchase based on their searches and roughly a quarter indicated that they would leave SaferProducts.gov to search other websites if they found no results on SaferProducts.gov. In contrast to SaferProducts.gov, other websites inform users of a possibly incorrect search term, such as a typographical error, which helps users interpret the results of their searches and identify potential errors.

During the usability tests, consumers experienced fewer challenges using the reporting function than the search function. To submit an incident report, consumers must enter information on a series of pages that include a combination of required and optional fields. During our testing, consumers found the instructions for submitting a report to be generally clear. For example, almost all the testers thought the instructions for submitting information about the incident, product, and victims were clear.

However, 15 of the 37 consumers in our test sessions expressed concern about apparently needing to register before submitting a report and generally did not notice that they could continue without registering (see fig. 6). By registering on SaferProducts.gov, site users can save their reports to complete at a later time and receive updates on the status of reports. When reaching the registration page, over a third of the consumers in our focus group sessions said that they would not be inclined to register. In one session, seven of nine consumers said they would not be inclined to register and thought that having to register was a deterrent to completing a report. In another session, none of the participants noticed the option to skip registration. Some of those who noticed that they could skip registration emphasized that the option should be more prominent—for example, placed alongside the registration box rather than below it where it might not be immediately visible. Likewise, as an issue of website usability, the expert evaluator from GSA reviewing the site on our behalf noted that the “continue without registering” option was not prominent enough and stated that registration may deter users from continuing the report submission process.

In addition, some consumers in our testing sessions said that the reporting pages contained too many questions and described the submission process as cumbersome, particularly for busy individuals such as parents. To address this, one consumer suggested grouping all of the required fields on one page. The expert evaluator from GSA also suggested that all the questions in the reporting process should be reviewed to determine if each was necessary.

When CPSC first developed SaferProducts.gov, the agency conducted three focus groups—one with consumers and one with professionals—to test the site and assess users’ experience with it.testing only addressed the incident reporting function, not the search function, and focused on (1) awareness of how and where to submit a safety complaint and (2) general reactions to the site. CPSC has not conducted additional usability testing since launching SaferProducts.gov in March 2011. As mentioned previously, CPSC officials have said that issues such as assessing the level of awareness of CPSC and redesigning CPSC.gov were higher priorities than assessing and improving SaferProducts.gov.

CPSC’s focus group A number of resources across the federal government are available to help agencies in making their websites more usable. For example, as cited previously, GSA’s First Fridays Usability Testing Program is designed to teach agency officials how to find and fix usability problems at no cost to the agency. The program’s services are (1) formal tests, (2) quick tests, (3) mobile tests, (4) observation, and (5) expert evaluation.GSA also offers DigitalGov University, which includes courses in web design and usability best practices. In addition, the Department of Health and Human Services (HHS) operates two usability labs, both of which are free of charge to other federal agencies, to evaluate websites to ensure that they are easy-to-use and useful. Furthermore, GSA and HHS maintain HowTo.gov and Usability.gov, respectively, to provide guidance and resources to help agencies create websites that are usable, useful, and accessible.

Because of the usability issues in the areas we identified, consumers may not take advantage of all the features of SaferProducts.gov, and consumers may be dissuaded from completing and submitting incident reports. As a result, CPSC may not be obtaining all possible information from consumers that can help inform its safety assessments and other regulatory efforts.

Consumers Initially Were Not Aware of SaferProducts.gov and Thought the Site’s Purpose Was Unclear, but Found the Site Generally Useful after Using It

None of the consumers in our test sessions previously had heard of SaferProducts.gov, although a few were familiar with CPSC as an agency involved in recalling certain products. In addition, 5 of the 37 consumers who participated in our tests said that the purpose of SaferProducts.gov was not clear based on its name and the initial information on the home page. In our testing sessions, roughly a third of the consumers commented that the name of the website—SaferProducts.gov—and the home page did not accurately convey what consumers could and could not do on the site. For example, when asked about their impressions of SaferProducts.gov, over a quarter of the testers thought that they would find information about safe products, such as a list of products that meet certain standards or a rating of products. These consumers did not appear to notice information on the home page indicating that they would only find information on unsafe products (see fig. 7). In our testing sessions, several consumers commented that the website would be more aptly named UnsafeProducts.gov. In addition, during our one-on-one testing sessions in Washington, D.C., two of the testers had difficulty distinguishing between recall notices and incident reports, which serve Likewise, although the expert evaluator from GSA different purposes.was able to obtain a general sense of the purpose of the site, he noted that a tagline (brief text that gives users an immediate idea of what the site does) would help reinforce the site’s purpose.

In one testing session, a few consumers also said that it was not apparent from looking at the home page that CPSC did not regulate certain categories of products, such as automobiles and medications, although more than half of the consumers in our testing sessions said at the outset that they routinely searched online for safety information on particular products. Similarly, none of the consumers in our testing sessions noticed that they could be directed to the agency’s main site, CPSC.gov, by clicking on certain links in SaferProducts.gov. Only the expert evaluator noticed that by opening a recall notice, the website user would leave SaferProducts.gov and go to CPSC.gov.

However, as consumers completed the various tasks in the testing sessions, they better understood the website’s features and functions. In responding to our closing questions about their overall experiences in using the site, most said that they would use SaferProducts.gov again now that they were aware of it. For example, some consumers found information about product recalls the most useful component of the site and said they would give more weight to this information. In our testing sessions, about one-quarter of the consumers also found value in the incident reports, noting that they helped website users understand whether products that had not yet been recalled had safety issues. Two of the consumers commented that they found the content of the reports to be more credible than other websites that provide a forum for consumer complaints. In addition, a few consumers pointed to the amount of detail in the reports, such as the incident description, location, and date as particularly helpful.

Nevertheless, because of the usability issues in the areas we identified (for example, not having a clear and “up-front” statement of what the site contains and how it can be used), consumers may not use all of the site’s available features and be dissuaded from completing and submitting reports. CPSC officials also acknowledged that awareness of the agency could be heightened if consumers were informed about CPSC while using or searching SaferProducts.gov.

Conclusions

CPSC has used many approaches to inform the public about SaferProducts.gov, employing many key practices of consumer education planning in the process. Incorporating the promotion of SaferProducts.gov into its broader effort to increase awareness of the agency has represented a logical approach that has prevented duplication. However, our work confirms CPSC’s perception that public awareness of SaferProducts.gov is likely low. For example, none of the participants in our usability tests had heard of SaferProducts.gov prior to the testing. Although CPSC has employed many of the key practices for consumer education planning, it has not established metrics to measure the success of its efforts. By establishing such metrics, the agency would be better able to determine which of its outreach efforts had the most impact on increasing awareness and use of the site and thus could more effectively target its limited resources to increase use of the site.

In addition to establishing and using metrics, more data about the use of SaferProducts.gov could help CPSC target its marketing and outreach. Currently, CPSC collects limited data about the use of SaferProducts.gov. For example, it collects data on the number of visitors, but not whether they are using the site to search for information—one of the main functions of the site. It also does not collect demographic data about the users’ age, gender, or income. These types of data could help CPSC identify groups, populations, or areas on which to focus to further increase use of the site.

Our usability testing with consumers identified other ways in which CPSC may increase the use of SaferProducts.gov. Although our testing revealed that many consumers found the site generally easy to use, it also revealed that certain search functions, site registration, and lack of a clear statement of purpose posed challenges for some users. By improving the site in these areas, CPSC could help ensure that consumers take advantage of all the features of the site and are able to search for and report information in an easy and convenient manner. Making these improvements also may provide CPSC with additional reports from consumers to inform its safety assessments and other regulatory efforts.

Recommendations for Executive Action

To improve the awareness, use, and usefulness of SaferProducts.gov, CPSC should take the following three actions: establish and incorporate metrics to assess efforts to increase awareness and use of SaferProducts.gov, look for cost-effective ways of gathering additional data about the users and their use of SaferProducts.gov, and implement cost-effective usability improvements to SaferProducts.gov, taking into account the results of any existing usability testing or any new testing CPSC may choose to conduct.

Agency Comments and Our Evaluation

We provided a draft of this report to CPSC for review and comment. In commenting on the draft report, the Chairman and Commissioners stated that they support the report’s recommendations. Specifically, they stated that CPSC staff will look for cost-effective ways to improve awareness of SaferProducts.gov, improve the usability of the site based on research on best practices in web design, and gather additional metrics about users. The Chairman and Commissioners’ comments are reprinted in appendix III. CPSC also provided technical comments that we incorporated in the report as appropriate.

We are sending copies of this report to interested congressional committees and to the Chairman and Commissioners of CPSC. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-8678 or cackleya@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

The objectives of our report were to examine (1) the Consumer Product Safety Commission’s (CPSC) efforts to inform the public about SaferProducts.gov, (2) who has been using the website and to what extent, and (3) the extent to which consumers have found the website to be useful.

For the first objective, we reviewed CPSC marketing, budget, evaluation, and planning documents to determine the status of the agency’s public information efforts related to SaferProducts.gov. We sought to determine whom CPSC had been targeting; how, if at all, the agency was evaluating the outcomes of its efforts; and any future plans to promote awareness and use of the site. We compared CPSC’s efforts with criteria on key practices for consumer education planning. We interviewed consumer product safety experts from nine groups representing consumers, researchers, and various industries to determine what additional steps, if any, CPSC could take to better inform the public about SaferProducts.gov. We identified these experts through our prior work or based on recommendations from those we interviewed. For context, we interviewed an official in CPSC’s Office of Communications and reviewed CPSC’s strategies to increase awareness of the agency as a whole.

See GAO-12-30. reliability and determined that, for the purposes of this report, the data were sufficiently reliable. We did not review specific incident descriptions that individuals filed and do not attest to the reliability of that information.

For the third objective, we conducted website usability tests with 37 consumers—who represented a mix of demographic characteristics in terms of age, gender, and educational level—to obtain their views on how easy it was to use SaferProducts.gov and how useful they found the website. We conducted the tests in Washington, D.C., Dallas, Texas, and San Francisco, California. We chose these locations for geographic dispersion and ease of testing. We followed the protocols and used the Washington, D.C. facilities of the General Services Administration (GSA) for our testing conducted through the First Fridays Usability Testing Program. At this location, GSA recruited three volunteer testers on our behalf. Consistent with the GSA program protocols, a moderator facilitated the testers’ execution of various website tasks, such as searching for recalled products and submitting mock incident reports. We followed similar protocols in San Francisco and Dallas. To identify the participants in San Francisco and Dallas, we worked with a contractor to recruit prospective testers who had a mix of demographic characteristics. We held two focus groups in each location, for a total of four groups. Two groups had eight participants per group and the other two groups had nine participants per group. In all four groups, a moderator facilitated the testers’ execution of various tasks, as was done in Washington, D.C. Although the results of our usability tests are not generalizable to all U.S. consumers, they provided us with in-depth, interactive feedback and detailed perspectives from a range of website users about the usability challenges associated with SaferProducts.gov.

To supplement our approach, we requested and reviewed an expert evaluation conducted by the First Fridays program manager. The GSA official evaluated SaferProducts.gov based on the following criteria: (1) accessibility—the ability of people with physical or mental disabilities to use the site; (2) identity and purpose—whether the site clearly presents its purpose, including what the site offers and what a user can do on it; (3) clarity—the ability to read and digest content; (4) navigation—how easily users can find information; and (5) design and content—focusing on the layout, headers, and design. Another GSA official provided a more in- depth accessibility review of SaferProducts.gov to identify issues that users with disabilities might encounter when navigating the site. According to GSA, although an expert evaluation can be a useful starting point for determining a website’s usability strengths and weaknesses, the expert evaluation emphasizes the importance of the user experience. In addition, we reviewed various other website usability resources and criteria, including Usability.gov, to understand the key practices for making websites easy to use and helpful.

We conducted this performance audit from July 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: GAO Script, Including Questions and Tasks, for Usability Testing with Consumers

Appendix III: Comments from the U.S. Consumer Product Safety Commission

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Debra Johnson, Assistant Director; Meghana Acharya; Mark Bird; William Carrigg; Jeremy Cluchey; Meredith Graves; Ronald Ito; Sarah Kaczmarek; May Lee; Marc Molino; Patricia Moye; Barbara Roesmann; Andrew Stavisky; and Julie Trinder made key contributions to this report.