Managing for Results: Leading Practices Should Guide the Continued Development of Performance.gov
Published: 2013-06-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Congress took steps to improve federal performance reporting through GPRAMA by requiring that OMB provide performance information via a publicly-available central website, Performance.gov. GAO is mandated to review GPRAMA's implementation at several junctures; this report is part of a series doing so. The report examines the extent to which Performance.gov incorporates leading practices for the development of federal websites. To address this objective, GAO compared the design of Performance.gov to GSA's Top 10 Best Practices for federal websites on HowTo.gov; reviewed performance reporting literature and OMB guidance; collected information from 13 national, state, and local performance reporting website practitioners; and interviewed federal and nonfederal groups most likely to use the information on the website because of their management, oversight, advocacy, or academic interests. These groups included officials from five selected agencies, staff from 13 U.S. Senate and House of Representatives congressional committees, and representatives from 10 transparency organizations and academic institutions.
section_title: What GAO Found
paragraphs: The GPRA Modernization Act of 2010 (GPRAMA) requires Performance.gov to provide program and performance information accessible to the public and members and committees of Congress. GAO used leading practices from HowTo.gov, a key source of guidance for federal website development and management, to assess the website and found that although Performance.gov incorporates some leading practices, opportunities exist to further incorporate them through continued development. For example, consistent with leading practices, the Office of Management and Budget (OMB), working with the General Services Administration (GSA) and the Performance Improvement Council (PIC), provided information about the purposes and audiences for the website, but they have made limited efforts to clarify how audiences can use the information provided. If the specific uses of Performance.gov are not clarified, while taking into consideration what the law requires, it could lead to varying ideas and expectations for how Performance.gov should be developed.
Leading practices also recommend that developers engage potential users through focus groups and other outreach and regularly conduct usability tests to gather insight into areas such as navigation and the organization of website content. Efforts to collect input and feedback from potential audiences of Performance.gov, however, have been limited to the collection of suggestions through the website's "Feedback" page and briefings for selected audiences. Similarly, OMB has not yet conducted any usability tests of the website, although staff said that usability testing is being planned for September 2013. Without this information, the needs of the audiences and how they are using or want to use the website cannot guide further improvements.
In addition, leading practices recommend that agencies collect, analyze, and report on a baseline set of performance, customer satisfaction, and other metrics. Of the 24 recommended metrics, 15 are currently tracked for Performance.gov. Leading practices also recommend setting goals for metrics and making sure these align with the website's objectives to help prioritize and guide design changes. These goals can be identified based on prevailing practices or the desire to improve a particular metric over time. Except for the area of customer satisfaction, OMB has not established performance metric goals, which may make it more difficult to analyze the effectiveness of the website.
OMB staff stated that, thus far, the specific legal requirements of GPRAMA have been the primary framework used to guide efforts to develop Performance.gov. They said they have been focused on compliance with these requirements by providing information on agency and cross-agency priority goals and by establishing a phased development plan for integrating additional information from agency strategic plans, performance plans, and performance reports. OMB and GSA staff members said, however, that the leading practices provided by HowTo.gov will help guide the development of Performance.gov. They also noted that as the phased development of Performance.gov unfolds, they expect to use broader outreach to a wider audience, including members of the public, to make Performance.gov more "public-facing" and "citizen-centric."
section_title: What GAO Recommends
paragraphs: GAO recommends that OMB should work with GSA and the PIC to (1) clarify specific ways that intended audiences could use Performance.gov and specify changes to support these uses; (2) systematically collect information on the needs of intended audiences; and (3) collect recommended performance metrics and, as appropriate, create goals for those metrics. OMB staff agreed with these recommendations.

section_title: Letter
paragraphs: In increasing recognition of the value of open government and the need to  make performance information more transparent, Congress took steps to  improve federal performance management and reporting by updating the  Government Performance and Results Act (GPRA) through the GPRA  Modernization Act of 2010 (GPRAMA). GPRAMA established important  changes to modernize and refine existing federal agency reporting  requirements and to ensure more frequent, relevant data are available to  inform decision making and the public. The act included a requirement  that the Office of Management and Budget (OMB) provide quarterly  updates on agency and cross-agency priority goals via a central,  government-wide website, Performance.gov. In March 2013, federal  agencies added the first performance updates for cross-agency and  agency priority goals to the website. OMB stated that over time it plans to  add more information from agency strategic plans, performance plans,  and reports and produce the information in formats that allow users to see  trends, look at goals contributing to common themes, see programs  contributing to common goals, and cross reference other related data.
GPRAMA required OMB to establish a single, performance-related  website by October 1, 2012, to provide program and performance  information, which would be accessible to members and committees of  Congress and the public. First developed by OMB in 2010 for executive  branch use, Performance.gov was made available to the public in August  2011. OMB’s stated goals for Performance.gov include providing both a  public view into government performance to support transparency as well  as providing executive branch management capabilities to enhance  senior leadership decision making. In addition to linking agency programs  and their relationships and contributions to strategic objectives, another  OMB goal of the website, as a repository of federal government  performance information, is to increase the utility of this information  through enhanced agency comparisons, supporting both benchmarking  and best practice identification.
This report is part of our response to a mandate to assess initial  implementation of the GPRA Modernization Act of 2010. Specifically, this  report examines the extent to which Performance.gov incorporates  leading practices for the development of federal websites. This report is  the fifth in a series that looks at how agencies are implementing various  GPRAMA requirements.
To address our objective, we analyzed information from the  Performance.gov website and the requirements from GPRAMA regarding  the website. We also examined related guidance, such as OMB’s Circular  No. A-11, which lays out expectations for the design, development, and  implementation of Performance.gov. We reviewed relevant academic  and policy literature on performance management and reporting,  including our previous reports on performance management, and  conducted interviews with staff at OMB and the General Services  Administration (GSA), who worked on the design and development of the  website. We reviewed OMB and GSA staff’s description of the  development of Performance.gov from our interviews and compared it  against criteria established by HowTo.gov, a source of guidance and  leading practices for government websites. We also conducted an  analysis of the ease with which commercial search engines, and  Performance.gov’s internal site search engine, can be used to find  relevant information on Performance.gov. Lastly, we reviewed design  features from a subset of other federal open government websites,  including Recovery.gov, Data.gov, and USASpending.gov, which like  Performance.gov are used to make government-wide information and  data accessible.
To further address our objective, we interviewed potential users of  Performance.gov, including executive branch and congressional staff and  a variety of nonfederal stakeholders. We reached out to groups most  likely to use the information on Performance.gov because of their  management, oversight, advocacy, or academic interests and asked them  to review the website prior to our interviews. We interviewed staff from  five agencies—the U.S. Department of Agriculture, Department of the  Interior, Department of Labor, Department of State, and the  Environmental Protection Agency—responsible for reporting performance  information to gather their perceptions on the utility of Performance.gov.  We selected from agencies that have not been the subject of our  GPRAMA mandate work and grouped them together according to the  number of their priority goals. We then selected the agency in each group  that used the largest number of tools of government—such as direct  service, regulations, grants, and loans—to achieve their performance  goals. This process allowed us to consider a mix of agency priority goals,  an example of each of the tools of government, and performance  improvement officers under both career and political appointments. We  interviewed staff from 13 different majority and minority congressional  oversight and authorizing committee offices of the U.S. Senate and  House of Representatives to gather their views on Performance.gov and  any suggestions they had for improving it. See appendix I for a full list of  the 13 committees represented in our interviews.
We also interviewed a variety of nonfederal stakeholders including  representatives from 10 academic, advocacy, transparency, and public  policy organizations as potential users to solicit feedback about the  website and any suggestions they had for improving it. We selected  these individuals on the basis of our literature review and  recommendations from other performance management and reporting  academics and practitioners. The results of these interviews are not  generalizable to all nonfederal stakeholders but provided insights into  perceptions of Performance.gov.
In addition, based on a review of government performance reporting  websites, we selected a group of 12 websites from state and local levels,  as well as one from Canada, to gather lessons learned from  representatives of these performance reporting websites that were  relevant to the design and development of Performance.gov. To focus  this review on performance reporting websites, we selected only sites that  presented performance information through web pages; we did not select  transparency websites that are designed to make available information on  state finances and the distribution of state expenditures. Table 1 provides  a list of the state, local, and Canadian performance reporting websites we  selected to review.
We conducted our work from July 2012 to June 2013 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives. More detailed information on our scope and  methodology appears in appendix I.
subsections: section_title: Background
paragraphs: In recent years, significant improvements in data management and  presentation technologies have made it possible for governments to  report performance information in new ways. Prior to the development of  these technologies, performance information was generally reported in  static reports, such as printed annual reports. Technology now allows  data to be collected from different operational units and then presented  through interactive, web-based dashboards and performance-reporting  websites to provide an integrated, multidimensional view of a  government’s performance. This advantage can give web-based  performance reporting the ability to more efficiently convey  understandable information to stakeholders and citizens.
Performance reporting websites are dynamic and can be updated on a  more frequent basis, or in real-time if technology allows and data are  available, ensuring that information is timely. They can also be flexible  enough to meet the needs of different audiences. For example, they can  be used to combine and organize, or layer, different levels of performance  information, allowing users to quickly access information from an overall  organizational perspective and to drill down to get a more detailed  perspective at the sub-unit level. In addition, performance reporting  websites can be made more engaging for users through the use of tables  and interactive graphics and maps that allow users to visualize  performance information in different ways.
Governments at the local and state level in the United States and in other  countries have developed performance reporting websites. The purposes  of these websites include, for example, compliance with statutory  reporting requirements and making information on the goals and  performance of government more widely accessible and transparent for  those inside and outside of government. Officials from other state  governments, who have developed performance reporting websites,  emphasized that making regularly-updated and understandable  information on goals and performance widely accessible for potential  users of information on the site, such as interested members of the  general public, legislators, executive branch leaders, delivery partners,  the media, key stakeholders, advocacy groups, and researchers can  increase oversight and lead to a greater focus within government on the  activities and efforts necessary to improve performance. In this way, the  websites can also enhance accountability within government and support  a culture of performance. These websites can also be used to facilitate  communication by providing a shared source of information on goals and  performance.
According to OMB, the Performance.gov website is intended to highlight  the administration’s management initiatives and address the performance  reporting requirements of GPRAMA. Figure 1 displays the  Performance.gov home page.
To comply with the requirements of GPRAMA, the current version of  Performance.gov also provides information, which is to be updated on a  quarterly basis, on agency and cross-agency priority goals. The pages  with information on agency and cross-agency priority goals can be  accessed either through individual agency pages on Performance.gov or  through the website’s “Clear Goals” page. The information available on  agency priority goals includes the strategies being used to achieve each  goal, next steps and milestones, performance indicators used to track  progress, and contributing programs. Figure 2 provides an example of  how performance trends are presented graphically through an agency  priority goal page.
We recently examined the extent to which 24 agencies implemented  selected requirements related to 102 agency priority goals and  commented on the 21 agency priority goals of 5 selected agencies. We  recommended that OMB could improve agency priority goal  implementation by revising its guidance to better reflect interim target,  milestone, and cross-agency priority goal alignment requirements and by  ensuring that agencies provide complete information about external  contributors to their agency priority goals and describe congressional  input on the goal’s development. OMB concurred with the  recommendations.
In addition, GPRAMA requires federal government performance plan  information; agency-level strategic plans, agency performance plans, and  agency performance updates; and detailed information about each  program identified by agencies to be posted on the website. According to  OMB’s guidance, these additional features will be added to the website  over time. implementation of Performance.gov, including OMB’s projected dates for  the inclusion of additional information.
OMB Circular No. A-11, Preparation, Submission, and Execution of the Budget, pt. 6  (August 2012).
section_title: Leading Practices for Federal Websites Should Guide the Continued Development of Performance.gov in a Number of Areas
paragraphs: HowTo.gov is a key source of leading practices for federal website  development and management. The website is managed by GSA and  designed as a resource to improve how agencies communicate and  interact with customers and use innovative tools and technologies to  provide services and information. HowTo.gov offers best practices,  guidance, and training on strategic planning; federal web requirements  and policies; applications, data, and web infrastructure tools; web content  management, usability, and design; and performance metrics. According  to GSA staff, to provide access to key industry practices HowTo.gov  makes available a list of the “Top 10 Best Practices” for federal websites  and provides detailed information on how to implement each practice (see  table 2). OMB has also encouraged the more widespread application of  many of these practices across the federal government in its Digital  Government Strategy.
In addition to these leading practices for the development of federal  websites, lessons learned from the experiences of those who have  developed other government performance reporting websites and views  of potential users provide useful insights that could help inform the  continued development of Performance.gov.
OMB staff stated that, thus far, the specific legal requirements of  GPRAMA have been the primary framework used to guide efforts to  develop Performance.gov. They have been focused on working to comply  with these requirements by providing information on agency and cross- agency priority goals and by establishing a phased development plan for  integrating additional information from agency strategic plans,  performance plans, and performance reports and the inventory of federal  programs. OMB and GSA staff members have said, however, that the  leading practices provided by HowTo.gov will help guide the future  development of Performance.gov.
subsections: section_title: OMB Provides Little Direction on How Different Audiences Could Use Performance.gov’s Information
paragraphs: Leading practices for the development of federal websites from  HowTo.gov recommend identifying the purposes of a website and the  ways in which specific audiences could use a website to accomplish  various tasks, from finding relevant information to commenting on a  government regulation to conducting a transaction, and then structuring  information and navigation to help visitors quickly complete these tasks.  According to HowTo.gov, this is important because people often visit  government websites with a specific task in mind, and if it is not easy to  find the information they want to quickly complete that task, they will leave  the site. Providing guidance about the tasks that can be accomplished on  a website, along with explanations and navigation assistance, can help  website users successfully achieve their objectives.
Similarly, some officials we interviewed from other governments with  experience developing performance reporting websites emphasized the  importance of understanding and articulating the purposes that a  performance reporting website is designed to achieve. They noted that  the audiences the website is designed to serve, along with the intended  uses of a site, should influence its design and content. For example, the  main purpose of the state of Michigan’s dashboards is to provide citizens  with a quick reference to see how the state is performing in key areas and  whether trends are moving in the right direction. To fulfill this purpose,  according to Michigan officials, the dashboards were designed to be  simple, easy to access and navigate, and provide the public with an at-a- glance overview of the state’s progress. Users interested in more  information on a specific performance measure can click on the link for  that measure and will be directed to a separate page with trend data and  information on why the measure is significant. A prominent link at the  bottom of the page also allows them to provide direct feedback on the  dashboard. The dashboards are supplemented by departmental  “scorecards” that include relevant performance measures to inform  internal departmental management and decision making. An example of  the Michigan Education Dashboard is shown in figure 4.
GPRAMA provides direction on the purposes and audiences for  Performance.gov. In addition to specifying the types of information to be  made available on the website, the act states that information on the  website “shall be readily accessible and easily found on the Internet by  the public and members and committees of Congress.” OMB’s written  guidance states that Performance.gov “serves as the public window on  the Federal Government’s goals and performance in key areas of focus,”  and will be “the single, government-wide performance website required  under the GPRA Modernization Act.” OMB’s written guidance and  information on the website also say that Performance.gov will make  information about cross-agency and agency-specific goals and  performance easier to find for the public and Congress, as required by  GPRAMA, as well as for delivery partners, agency employees, the media,  and other stakeholders. Lastly, the guidance specifies that the website  will “support coordination and decision-making to advance shared  goals.”
An analysis of statements from OMB and GSA staff, agency officials, and  feedback we obtained from potential users, however, indicates that there  are varying expectations regarding the primary audiences and uses of  Performance.gov. For example, OMB and GSA staff emphasized that  they have viewed Performance.gov as a tool for agencies to support  cross-agency coordination and efforts to achieve agency goals.  Consistent with this, OMB staff said that Performance.gov has been used  to facilitate conversations between OMB examiners and agency  managers about progress on agency priority goals. While officials we  interviewed from the five agencies said that OMB has collected feedback  from agencies in the development of Performance.gov, officials from four  of these agencies also said, however, that Performance.gov is not being  used as a resource by agency leadership or other staff. According to  agency officials, agencies have information sources tailored to meet their  needs, and Performance.gov does not contain critical indicators or the  ability to display some visualizations used for internal agency  performance reviews. Agency officials stressed instead that they view  Performance.gov primarily as an external reporting tool so that the public  and other external stakeholders can get a sense for how agencies are  performing on key priorities. For example, an agency official noted that a  key function of Performance.gov is to make performance data more  readily available in one place so that members of the public and external  stakeholders do not have to go to individual agency sites to collect  information on agency goals and performance.
Several potential users and a practitioner we interviewed also indicated  that it was unclear to them who the audience of Performance.gov is, and  how audiences would use the site, given its content and design. For  example, some said that, while the public is listed as an audience of the  website, the detailed, technical nature of the website seemed primarily  oriented toward a government, rather than a public audience. Some also  commented that developers need clarity about how different audiences  should be able to use a website, as developers can then ensure it is  designed to present information in a way that addresses the needs of  those audiences. OMB and GSA staff members have acknowledged that,  given the requirements of GPRAMA, Performance.gov will need to shift to  become a more publicly-oriented website in the future, which could  involve a more thematic presentation to engage the public.
While the “Areas of Focus” section of the Performance.gov home page  directs visitors to pages that allow them to explore information and  metrics in specific areas, OMB has not yet articulated various ways that  intended audiences, such as interested members of the public, Congress,  experts and researchers, agency staff, and delivery partners could use  the website or the information available through it to accomplish other  specific tasks. For example, as mentioned previously, OMB has stated  that a purpose of Performance.gov is to support coordination and  decision making to advance shared goals. While OMB said that this  intended use will help provide direction as they take a phased approach  to development, the current version of the website gives no indication or  examples of the ways that agency staff, or others, such as delivery  partners or congressional staff, could use Performance.gov to facilitate  coordination or communication about goals, activities, and performance  between agencies and with interested stakeholders from other sectors. It  is also unclear whether Performance.gov includes all the information and  design elements necessary to support coordination across agencies. The  President’s budget for fiscal year 2014 indicates that, in the future, efforts  will be undertaken to test the potential use of Performance.gov to  facilitate coordination among goal allies, enlist ideas and assistance to  accelerate progress on goals, and enhance public understanding of the  work of the federal government.
Other federal open government websites have indicated how visitors can  use them to accomplish specific tasks. For instance, Recovery.gov  contains an entire page that outlines what users can do on the site,  including how to use the raw data available through the website, report  waste, fraud, and abuse, or find job and grant opportunities. Data.gov  states that it is designed to enable “the public to participate in government  by providing downloadable Federal datasets to build applications, conduct  analyses, and perform research,” and offers an introductory video that  provides information on the purposes of the website and instructions on  how to use it.
These websites have also integrated Web 2.0 technologies, such as links  to social media feeds, to help people share and use of the information  available through these sites. For example, Recovery.gov and Data.gov  have integrated social-networking tools, such as Facebook and Twitter,  blogs, and online discussion forums to facilitate the sharing of  information, provide new venues for communication and participation, and  enable collaboration and discussions between stakeholders and web- based communities of interest. Recovery.gov and Data.gov also have  dedicated pages for different audiences that compile and organize  relevant resources according to the needs and interests of those  audiences. Data.gov, for instance, contains individual pages that allow  users interested in specific areas, such as education, health, or energy, to  find data and resources on those specific topics and to communicate with  others who share those interests. Figure 5 provides a page for the energy  community. To comply with the recommendations of the Digital  Government Strategy, the Digital Services Advisory Group and the  Federal Web Managers Council have also encouraged the use of social  media to distribute content to large and diverse audiences and more  effectively engage with customers.
OMB and GSA have been working to ensure that Performance.gov  complies with the reporting requirements of GPRAMA. However, if the  specific intended uses of Performance.gov are not clarified, while taking  into consideration what the law requires, it could lead to varying ideas and  expectations for how Performance.gov should be developed and  designed and the audiences it should serve. The website’s accessibility to  visitors and its relevance to them could be increased by additional  direction on how various audiences could use the information available  through the website, such as explanations or navigation assistance, or by  tools, such as Web 2.0 technologies, that could facilitate that use.
section_title: OMB Collaborated with Agency Officials and Collected Feedback from Some Congressional Staff and Stakeholders, but Has Not Indicated How It Will Use Outreach to Other Audiences to Inform Future Changes
paragraphs: Leading practices from HowTo.gov on the development of federal  websites recommend that developers collaborate with officials from other  agencies to avoid duplication of effort, use content that already exists,  and let organizations with the greatest expertise on a topic create content.  According to HowTo.gov, this is important because allowing the  organization with the greatest expertise to create content can help ensure  its accuracy and quality, and linking to existing content can help save time  and resources. HowTo.gov also recommends that developers engage  potential users through focus groups and other outreach; regularly  conduct usability tests to gather insight into navigation, the organization of  content, and the ease with which different types of users can complete  specific tasks; and collect and analyze performance, customer  satisfaction, and other metrics. HowTo.gov states that these efforts are  important for collecting and analyzing information about audiences, their  needs, and how they are using, or want to use, the website. This  information is also critical to help inform the development of a useful and  usable website that meets the needs of intended audiences.
Similarly, some developers of other government performance reporting  websites who we interviewed reported that it is important to use a  developmental approach focused on continuous improvement, providing  users with opportunities to give input and feedback, so that websites can  be adjusted and improved to meet their needs. For example, following the  initial creation of the state’s performance reporting website, officials in  Maryland analyzed the website’s performance metrics to get a sense for  how people were accessing the site, as well as the information they were  searching for. They also employed usability testing to collect insight into  the navigation and content of the website. From these insights, they  identified the need to make information on related state programs and  resources more easily accessible through the website, which is now  reflected in its design.
According to OMB and agency officials we interviewed, to leverage the  expertise of agencies in the development of content for Performance.gov,  OMB established a content management and review process by which  each agency shared information and data on their own goals and  activities with OMB. OMB also developed the Performance Reporting  Entry Portal (PREP), a data input system, which allowed agencies to  directly input their information and data on agency priority goals into  Performance.gov. For many of the pages on cross-agency management  initiatives, they also leveraged information from agencies and other  federal websites, linking to existing resources and reports.
OMB used other strategies to help maintain a collaborative environment  and open lines of communication with federal agencies. According to  OMB staff, during the initial development of Performance.gov, OMB used  a pilot project involving several agencies to model different iterations of  the website and collect feedback on potential designs from agency staff.  OMB staff members also continue to hold biweekly conference calls with  agency staff to provide status updates on the development of  Performance.gov, answer questions, and address concerns. A GSA staff  member supporting the PMLOB also recently met individually with staff  from 24 agencies to discuss the data requirements for Performance.gov  and the best way for agencies to provide information to be made available  through the website. According to OMB, GSA, and agency staff, they  have been able to use this ongoing communication to collect feedback on  Performance.gov and ideas to gradually improve the design and usability  of the system agencies use to input data on agency priority goals. For  instance, the PREP system has been changed in response to agency  requests so that data from prior time periods will be pre-populated in the  system, an improvement that was made to help reduce the burden of data  collection on agencies.
According to OMB and GSA staff, to gather input from other audiences as  they developed Performance.gov, they also held briefings on the website  for congressional staff and government transparency organizations and  performance experts based in Washington, D.C. Since 2010, OMB staff  said that they met several times with staff from the Senate Homeland  Security and Governmental Affairs Committee, House Oversight and  Government Reform, and the Senate Budget Committee to discuss the  development of Performance.gov. They noted that they used this  outreach to stakeholders to identify several specific website modifications.  For example, OMB staff heard from stakeholders, including congressional  staff, that it was important for Performance.gov visitors to be able to  understand how agency priority goals relate to broader agency strategic  goals and objectives, so OMB nested the priority goals within the larger  agency goal framework. OMB staff also said that several stakeholders  noted that they appreciated the ability to filter goals by different themes,  so OMB included the capability to apply this feature to all categories of  goals and objectives. Of the three congressional staff we spoke with who  said they had received briefings on the development of Performance.gov,  however, only one felt she had been consulted on website input. Also,  since 2010, OMB staff reported holding no meetings on the development  of Performance.gov with staff from other House or Senate committees  who might use the website to inform their federal agency oversight.
Furthermore, while a focus of GPRAMA is to make federal performance  information more accessible to the public, efforts to collect input and  feedback from interested members of the public and other potential  audiences have thus far been limited to the collection of suggestions  through the website’s “Feedback” page. According to OMB staff, the  submissions received through this portal are generally on topics unrelated  to the website, although they do receive questions about the website or  suggestions for minor corrections to content. They also said that the  suggestions received through this portal have not yet had a significant  impact on the design of the site. GSA staff said that usability testing of  Performance.gov is planned for September 2013 and that the tests will be  managed by usability experts from GSA and will include a mixture of  participants from within government and from the general public. From  this testing, they hope to inform new development and interface and  navigation improvements.
Other federal open government websites have used broader outreach  efforts to inform their development and design. For instance, the  developers of Recovery.gov used both focus groups and usability testing  with interested citizens to collect feedback and recommendations and  inform the development of the website from its initial stages. The  developers of Data.gov also used a technology service called IdeaScale  to collect feedback and ideas from hundreds of users during the initial  development of the site.
OMB and GSA monitor visitors’ use of Performance.gov using a variety of  metrics. For example, one metric that is tracked is the total number of  visits to Performance.gov, by month, as seen in figure 6.
HowTo.gov recommends that agencies collect, analyze, and report on a  minimum baseline set of performance, search, customer satisfaction, and  other metrics, which allow officials to get a holistic view of how well online  information and services are delivered. Of the 24 metrics recommended  by Howto.gov, 15 are currently tracked for Performance.gov. See table 3  for the list of recommended metrics and whether they are collected for  Performance.gov.
HowTo.gov also recommends setting goals for metrics, and making sure  that these align with the objectives of a website, to help prioritize and  guide design changes for improved performance and usability. These  goals can be identified based on prevailing practices or the desire to  improve a particular metric over time. Except for customer satisfaction,  which is discussed later in this report, OMB has not yet established goals  for any recommended metrics.
GSA staff acknowledged the need to collect additional recommended  metrics for the website. They said that as they expand the list of metrics  that are collected, they are planning usability testing to identify the most  appropriate metrics for analyzing the website’s use and identifying  improvements that will increase the usability of Performance.gov. OMB  staff also said that right now their primary goal is releasing the legally  required information on Performance.gov, but they will consider setting  targets for certain metrics in the future.
In addition to performance and search metrics, HowTo.gov recommends  that agencies collect customer satisfaction metrics, which include  measures of overall customer experience, the completion rate of intended  tasks, and the percentage of visitors likely to return and recommend the  website. The Digital Government Strategy requires that agencies use  customer satisfaction measurement tools for all .gov websites. While  OMB has established a numerical customer satisfaction target for  Performance.gov, the agency has not established tools on the website,  such as online customer satisfaction surveys, that would allow them to  collect the data necessary to track the recommended customer service  metrics. GSA staff said that there are no public-focused customer  satisfaction tools for Performance.gov, outside of the “Feedback” page,  on the website because, at this stage in the website’s lifecycle, the  primary focus has been on collecting feedback from Congress and  agency management. They said, however, that they have used feedback  collected through congressional meetings and feedback from members of  the PIC, as well as comments from the “Feedback” page of the website,  to get a general sense of how satisfied some users are with the website.  The “Feedback” page, however, does not ask users to rate their overall  satisfaction or the quality of their experience.
As stated earlier, according to OMB and GSA staff, complying with the  time frames and requirements for public reporting in GPRAMA has been  the focus of development, although they used outreach to collect  feedback from some congressional staff, government transparency  organizations, and performance experts to inform changes to the website.  A lack of outreach to other potential audiences and user testing, however,  means that developers have not yet had an opportunity to systematically  collect input on the information needs and preferences of potential  audiences of the website, including interested members of the public,  other congressional committees, delivery partners, or others; whether the  content on the website meets the needs of different audiences; or if it is  presented or organized in a way that makes it easily accessible.  Furthermore, without collecting all recommended search, and customer  satisfaction metrics, and establishing goals or targets for metrics, it may  be more difficult for the developers of Performance.gov to get a holistic  picture of how they are delivering information and to identify and prioritize  potential improvements.
OMB and GSA staff have said that as the phased development of  Performance.gov takes place, they expect to use outreach to a broader  set of audiences, including members of the public, and usability testing  with these audiences to make Performance.gov more “public-facing” and  “citizen-centric.” While they have developed a general time frame for  conducting usability tests, they have not yet established a specific plan, a  timeline for other forms of outreach, or indicated the specific audiences  they plan to target for greater outreach.
section_title: Performance.gov Was Designed with a Consistent Structure and Navigation, but Users Reported Some Issues with Accessibility, Complexity of Navigation, and Search Capabilities
paragraphs: Leading practices from HowTo.gov recommend organizing website  content based on audience needs and using common terminology and  placement, consistent navigation, and plain language. HowTo.gov states  this is important because providing consistent terminology and placement  across a website helps users easily find what they need as they will know  what labels to look for. Consistent navigation makes websites easier to  use because users are more likely to find what they need from a website  if they are familiar with its navigation scheme. Using plain language also  makes the content of a website more accessible by using words that the  website’s typical user can understand the first time. Lastly, leading  practices recommend organizing and categorizing website content to help  ensure that it can be found using commercial search engines or a  website’s internal search engine. All of these elements factor into the  overall usability of a site, which is key to ensuring an effective user  experience.
Similarly, developers of some government performance reporting  websites we interviewed emphasized the importance of designing these  websites in a way that makes them easy to access, use, and navigate.  Several states have tried to increase the visibility of their performance  reporting websites by integrating them with other transparency and open  data websites and by including prominent links on other state government  websites. A number of state and local officials also emphasized the  importance of making the information and data on a performance  reporting website engaging and easily understandable. They suggested  doing this by organizing information so users can quickly locate what is of  interest to them through visual displays of data, minimizing the amount of  text, avoiding the use of jargon, and ensuring that data is current and  timely. For example, City of Boston staff said that visitors generally stay  on these sites for limited amounts of time, so they designed the Boston  About Results website to present performance information in a way that  can be easily consumed and understood, using brief descriptions of key  strategies and graphics to depict performance trends. Figure 7 shows an  example of a screen shot from the Boston About Results website.
Performance.gov is accessible from some other federal websites, a key  source of potential traffic. For example, USASpending.gov includes a link  to Performance.gov on its home page and the “Government  Performance” link on the “topics” page of USA.gov includes  Performance.gov at the top of a list of links. Other federal websites,  however, including Data.gov, the White House and OMB home pages,  and the websites of the five agencies included in our study do not have  links to Performance.gov. As one potential user from a government  transparency organization pointed out, if Performance.gov is not  connected in any meaningful way with other federal information and data  websites, it may not be a site that government data users would naturally  go to. Furthermore, a lack of social media integration on Performance.gov  could increase the effort required for users to share content from the  website. A lack of other communication tools, such as RSS feeds and  social media, which can be used to alert users to new or important  content and enable website administrators to push out timely and relevant  information, could also limit the website’s ability to keep interested users  informed about new content.
The various sections of Performance.gov were designed with a consistent  organization and navigation. This can be seen particularly in the structure  of individual agency pages. As shown in the U.S. Department of  Agriculture example in figure 8, each page includes a brief overview of  the agency’s mission, includes links to the agencies existing performance  reports, and has three tabs with information on the agency, the agency’s  goals, and the agency’s efforts on government-wide management  initiatives. The “agency goals” tabs also use a consistent, layered  structure to provide lists of agency strategic goals, objectives, and priority  goals. According to GSA staff, this platform will also be used to organize  the Performance.gov information on agency strategic goals and  objectives that will be added to comply with the GPRAMA requirement  that the website include agency strategic plans, annual performance  plans, and annual performance reports.
The use of layering is a common practice in the design of performance  reporting websites, as it allows developers to structure the presentation of  information in a way that meets the needs of different audiences by  allowing them to access the level of information that is most relevant for  them. For example, the approach begins with the presentation of higher- level goals, highlights, or more aggregated information, while allowing  those users interested in progressively more detailed goals or information  to drill down and find the level of information most appropriate for their  needs.
Several individuals we interviewed had positive comments on the layered  structure used to organize the information on agency strategic goals,  objectives, and priority goals within Performance.gov and said that using  the same format across goals and agencies facilitated navigation.  However, others we spoke with, including some agency officials, raised  concerns that the complexity of the website’s navigation contributes to the  difficulty of finding critical pieces of information on the site, particularly  information on individual agency priority goals. For example, if a user  chooses to access information on agency priority goals through the page  of an individual agency, the user would need to click through multiple  drop downs and tabs, and navigate to a new page, before reaching the  tab that has a graphic depicting a specific agency priority goal’s progress.  If a user is unaware that this information exists, the user may not be able  to easily find it, and Performance.gov does not include instructions on  how to locate this information. To facilitate access to information on  agency and cross-agency goals, a number of experts and potential users  suggested that the Performance.gov home page include a thematic index,  similar to the filter on the “Clear Goals” page that currently allows users to  search for goals by different themes, like agriculture, energy, or natural  resources and environment. Other suggestions for the pages on  individual agency priority goals were to make the goals and visualizations  of performance trends a more central part of the presentation.
OMB staff said that lessons learned through the federal government’s  past experience with performance reporting have emphasized presenting  performance information in an appropriate context with relevant  information on agency missions, goals, activities, and factors that  influence performance. According to OMB staff, the use of plain language  was a focus in trying to make Performance.gov easy to understand, and  OMB held a briefing on plain language for agencies developing content  for Performance.gov. However, several potential users we spoke with  also noted that some pages on Performance.gov include jargon and  technical detail that may not be understandable or relevant to certain  audiences, including members of the general public. Concerns were also  expressed that some pages on Performance.gov contain too much text,  which can limit the accessibility of information.
Our analysis of the ease with which commercial search engines, and  Performance.gov’s internal site search engine, can be used to find  relevant information on the website produced mixed results. When using  a commercial search engine to search for several general, government- wide terms, including “federal government goals,” “agency performance,”  and “federal agency performance measures,” we found that a link to  Performance.gov was listed on the first page of results for all of these  searches. We also found that a link to Performance.gov was included on  the first page of results when using a variety of agency-specific search  terms, including “Department of Agriculture performance,” “EPA goals,”  “Department of Labor goals,” “Department of Interior goals,” and  “Department of State performance.”
The results of using Performance.gov’s internal search engine to find  relevant information on the website were mixed, however. To analyze the  ease with which a user could find relevant agency priority goal pages  using the website’s search engine, we used topic-focused search terms  taken from specific agency priority goals of the five agencies included in  our study, including “rural communities,” “greenhouse gas,” “climate  change,” “training programs,” and “Afghanistan.” For only two of these  five searches—on “rural communities” and “Afghanistan”—however, did  the results include a link to the relevant agency priority goal page. The  results sometimes included links to other potentially relevant pages on  Performance.gov, such as those with information on related cross-agency  priority goals, cross-government management initiatives, or featured  stories. Many of the links included in the results direct users to web pages  that include agency overviews or lists of agency strategic goals, but which  do not appear to be part of the normal Performance.gov structure. Figure  9 is an example of one of these pages.
While the opinions of potential users on the organization and navigation  of Performance.gov were mixed, comments and our analysis of  Performance.gov’s internal search engine indicated the site’s overall  usability could be limited for potential audiences by several factors: the  complexity of the website’s navigation, difficulty accessing some  information, the lack of an effective internal search engine, and the level  of detail and technical information provided. Actions OMB and GSA are  taking to develop a more public-oriented and usability-focused  perspective for Performance.gov are discussed later in the report.
section_title: The Proposed Performance Management Line of Business Offers a Potential Structure for Addressing Identified Issues and Applying Leading Practices
paragraphs: Leading practices for the development of federal websites recommend  defining and documenting how a website will be governed, including the  structure of the team managing the website, the roles and responsibilities  of different stakeholders, and the policies and procedures that will be  used to govern the creation and management of content. HowTo.gov  states this is important because codifying how a website will be governed,  as well as its organizational structure, policies, and procedures provide  involved staff with a common understanding of how things are supposed  to work.
As noted previously, staff from OMB, GSA, and members of the PIC are  considering whether to create a Performance Management Line of  Business (PMLOB). According to OMB and GSA staff, in an effort to  document the potential governance structure of the PMLOB and  Performance.gov, as well as the roles and responsibilities of various key  stakeholders and the processes that will be used for major decisions, a  draft charter was developed. OMB staff said the charter was completed in  May 2013.
The proposed creation of the PMLOB has also offered an opportunity to  define the purposes and scope of the system that will be used to  centralize the collection of performance data from agencies, as well as  how it will be displayed on Performance.gov. For instance, an executive  steering committee made up of staff from OMB, GSA, and several  members and staff from the PIC, has been created. OMB staff said that  the intent of the committee is to provide a forum for the discussion of  potential options and to help determine the requirements that a future  iteration of Performance.gov and its underlying data collection system  must fulfill and what purposes they should be designed to accomplish.
According to OMB and GSA staff, they are also developing working  groups to examine various issues. For example, a working group on  “business intelligence” has been established through the PIC to help  determine how to more effectively use the website’s performance  information to support internal management and coordination. They are  also starting a working group on the presentation and display of  performance information through Performance.gov. GSA staff said that  members of the PIC have been asked to collect agency input on the  current version of Performance.gov, the reporting burden it presents,  what they like and do not like about it, and what they would like to see in  future iterations of the website. Potential solutions to address identified  issues will then be developed and considered.
OMB and GSA staff also said that the knowledge and capacity of the  Center for Excellence in Digital Government, a component within GSA  that provides government-wide support on the use of the Web, social  media, and other technologies to improve customer service, will be used  to help bring a more public-oriented and usability-focused perspective to  Performance.gov.
A GSA representative emphasized that the development of  Performance.gov will happen in phases as funding becomes available.  Performance.gov has been funded through the E-Government fund,  which consists of funding appropriated by Congress every year.staff said that while funding for the development of the website has been  consistent over the past 2 years, it has varied over time. The staff added  that at times funding was only sufficient to cover operational expenses  with no funding available for new development. The GSA representative  also noted that while the focus of the PMLOB is on making performance  information more visible and transparent for Congress and the public by  replacing printed reports with a publicly accessible website, there is hope  that the shift to the PMLOB will provide improved, steady governance of  performance management across government and greater stability from  one administration to another.
section_title: Conclusions
paragraphs: OMB’s and GSA’s development efforts for Performance.gov have been  focused on ensuring that information required by GPRAMA is available on  the website. Leading practices for the development of federal websites,  however, offer insights into approaches that could be used to guide the  development of Performance.gov going forward. For example, clarity  about the intended uses of Performance.gov could provide additional  direction to ensure that the website is developed as a valuable and  relevant tool for its audiences, including interested members of the public  and Congress, agency staff, delivery partners, and other potential  audiences. The practices could also help determine any design changes,  such as navigation assistance or the integration of Web 2.0 technologies,  to facilitate those intended uses.
While OMB collected input from some congressional staff, government  transparency organizations, and performance experts, the limited  outreach to a broader set of potential audiences and the lack of usability  testing to date means that OMB does not know whether Performance.gov  meets users needs or how the website could be further developed. Not  tracking all recommended performance metrics, particularly those  measuring user satisfaction or establishing appropriate goals for certain  metrics, may also make it more difficult to analyze the effectiveness of the  website and to identify and prioritize potential improvements.
section_title: Recommendations for Executive Action
paragraphs: To enhance the value of Performance.gov for intended audiences and  improve the ability to identify and prioritize potential improvements, we  are recommending that the Director of the Office of Management and  Budget—working with the Performance Improvement Council and the  General Services Administration—take the following three actions:
Clarify the ways that intended audiences could use the information on  the Performance.gov website to accomplish specific tasks and specify  the design changes that would be required to facilitate that use.
Seek to more systematically collect information on the needs of a  broader audience, including through the use of customer satisfaction  surveys and other approaches recommended by HowTo.gov.
Seek to ensure that all performance, search, and customer  satisfaction metrics, consistent with leading practices outlined in  HowTo.gov, are tracked for the website, and, where appropriate,  create goals for those metrics to help identify and prioritize potential  improvements to Performance.gov.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Director of OMB and the Acting  Administrator of GSA for review and comment. OMB staff provided oral  comments, and we made technical changes as appropriate. OMB staff  agreed with our recommendations. GSA did not have comments on the  report. We also provided the draft for review and comment to the five  agencies with officials we interviewed regarding their perspectives on  Performance.gov. None of the five agencies had comments.
We are sending copies of this report to the Director of OMB and the  Acting Administrator of GSA as well as appropriate congressional  committees and other interested parties. The report is also available at no  charge on the GAO website at http://www.gao.gov. If you have any  questions concerning this report, please contact me at (202) 512-6806 or  mihmj@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  Key contributors to this report are listed in appendix II.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: This report is part of our response to a mandate to assess initial  implementation of the GPRA Modernization Act of 2010. Specifically, this  report examines the extent to which Performance.gov incorporates  leading practices for the development of federal websites. This report is  the fifth in a series that looks at how agencies are implementing various  GPRAMA requirements.
To address our objective, we analyzed information from the  Performance.gov website and the requirements from GPRAMA regarding  the website. We also examined related guidance, such as the Office of  Management and Budget’s (OMB) Circular No. A-11, which lays out  expectations on the design, development, and implementation of  Performance.gov. We reviewed relevant academic and policy literature on performance management and reporting, including our previous  reports on performance management. The literature was selected based  on whether it provided relevant information on the reporting requirements  in GPRAMA, lessons learned from the development of other federal  reporting systems, or insights into performance reporting best practices or  lessons learned.
We conducted interviews with staff at OMB and the General Services  Administration (GSA), who worked on the design and development of the  website. We reviewed OMB and GSA staff’s description of the  development of Performance.gov from our interviews and compared it  against criteria established by HowTo.gov, a source of guidance and  leading practices for government websites. We also conducted an  analysis of the ease with which commercial search engines and  Performance.gov’s internal site search engine can be used to find  relevant information on Perfomance.gov. Lastly, we reviewed a subset of  federal open government websites, including Recovery.gov, Data.gov,  and USASpending.gov. These websites were selected because, like  Performance.gov, they are used to make government-wide information  more transparent through publicly-accessible websites. We reviewed  design features of each, including whether they identified their purposes,  audiences, and potential uses, and approaches they used to facilitate  access to information.
To further address our objective, we obtained the views and suggestions  of potential users of Performance.gov. We interviewed executive branch  and congressional staff and a variety of nonfederal stakeholders. We  reached out to groups most likely to use the information on  Performance.gov because of their management, oversight, advocacy, or  academic interest and asked them to review the website prior to our  interviews. We selected five agencies to gather their perceptions on the  utility of Performance.gov—the U.S. Department of Agriculture,  Department of the Interior, Department of Labor, Department of State,  and the Environmental Protection Agency. We selected these agencies  from those that have not previously been the subject of our GPRAMA  mandate work. We grouped agencies with the same number of priority  goals together, and selected the agency in each group that uses the  largest number of tools of government—such as direct service,  regulations, grants, and loans—to achieve their performance goals. This  process allowed us to consider a mix of agency priority goals, an example  of each of the tools of government, and performance improvement  officers under both career and political appointments.
We also interviewed staff from 13 different majority and minority  congressional oversight and authorizing committee offices of the U.S.  Senate and House of Representatives to gather their views on  Performance.gov and any suggestions they had for improving it. We  interviewed congressional staff from the committees shown in table 4.
We also interviewed a variety of nonfederal stakeholders including  representatives from 10 academic, advocacy, transparency, and public  policy organizations to solicit feedback about the use of performance  reports, their use of the website, and suggestions they had for improving  it. We interviewed academics and practitioners from the California State  University San Bernardino; Government Finance Officers Association of  the United States and Canada; Government Performance Coalition;  Socrata; University of Victoria; and the Wilfrid Laurier University. Our  advocacy and transparency group representatives were from the Center  for Effective Government; Coalition for Evidence-Based Policy;  OpentheGovernment.org; and the Project on Government Oversight. We  selected these academics and practitioners on the basis of our literature  review and recommendations from other performance management and  reporting academics and practitioners. The results of these interviews are  not generalizable to all nonfederal stakeholders but provided insights into  perceptions of Performance.gov.
In addition, based on a review of government performance reporting  websites, we selected a group of 12 websites from state and local levels,  as well as one from Canada, to gather lessons learned and best practices  from representatives of these performance reporting websites that were  relevant to the design and development of Performance.gov. To identify  examples of web-based performance reporting by state governments, we  visited the main government websites of all 50 states, or their governor’s  website, and searched each one using the search term “performance” to  identify any relevant agency or government-wide websites providing  performance information in a web-based format. We selected only sites  that presented performance information through web pages; we did not  select transparency websites that are generally designed to make  available information on state finances and the distribution of state  expenditures. To identify relevant examples of web-based performance  reporting by local governments, we conducted a search similar to the  state sites and talked to representatives from the local websites who  agreed to be interviewed. To identify international performance reporting  websites, we visited the websites of several national governments of  English-speaking countries recognized for performance reporting— Canada, the United Kingdom, Australia, and New Zealand—to examine if  these governments had developed government-wide performance  reporting websites. Through this approach, we identified the Canadian  government’s “Planning and Performance Gateway,” which consists of  the Canada’s Performance and Overview of Government Spending and  Performance websites. None of the other countries had government-wide  performance reporting websites. Table 5 provides a list of the state, local,  and Canadian performance reporting systems we selected to review with  representatives who agreed to be interviewed.
We conducted our work from July 2012 to June 2013 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Elizabeth Curda, Assistant  Director; Judith Kordahl; and Adam Miles made key contributions to the  report.