Managing for Results: Leading Practices Should Guide the Continued Development of Performance.gov
Released: 2013-06-06T00:00:00

Letter

Background

In recent years, significant improvements in data management and presentation technologies have made it possible for governments to report performance information in new ways. Prior to the development of these technologies, performance information was generally reported in static reports, such as printed annual reports. Technology now allows data to be collected from different operational units and then presented through interactive, web-based dashboards and performance-reporting websites to provide an integrated, multidimensional view of a government’s performance. This advantage can give web-based performance reporting the ability to more efficiently convey understandable information to stakeholders and citizens.

Performance reporting websites are dynamic and can be updated on a more frequent basis, or in real-time if technology allows and data are available, ensuring that information is timely. They can also be flexible enough to meet the needs of different audiences. For example, they can be used to combine and organize, or layer, different levels of performance information, allowing users to quickly access information from an overall organizational perspective and to drill down to get a more detailed perspective at the sub-unit level. In addition, performance reporting websites can be made more engaging for users through the use of tables and interactive graphics and maps that allow users to visualize performance information in different ways.

Governments at the local and state level in the United States and in other countries have developed performance reporting websites. The purposes of these websites include, for example, compliance with statutory reporting requirements and making information on the goals and performance of government more widely accessible and transparent for those inside and outside of government. Officials from other state governments, who have developed performance reporting websites, emphasized that making regularly-updated and understandable information on goals and performance widely accessible for potential users of information on the site, such as interested members of the general public, legislators, executive branch leaders, delivery partners, the media, key stakeholders, advocacy groups, and researchers can increase oversight and lead to a greater focus within government on the activities and efforts necessary to improve performance. In this way, the websites can also enhance accountability within government and support a culture of performance. These websites can also be used to facilitate communication by providing a shared source of information on goals and performance.

According to OMB, the Performance.gov website is intended to highlight the administration’s management initiatives and address the performance reporting requirements of GPRAMA. Figure 1 displays the Performance.gov home page.

To comply with the requirements of GPRAMA, the current version of Performance.gov also provides information, which is to be updated on a quarterly basis, on agency and cross-agency priority goals. The pages with information on agency and cross-agency priority goals can be accessed either through individual agency pages on Performance.gov or through the website’s “Clear Goals” page. The information available on agency priority goals includes the strategies being used to achieve each goal, next steps and milestones, performance indicators used to track progress, and contributing programs. Figure 2 provides an example of how performance trends are presented graphically through an agency priority goal page.

We recently examined the extent to which 24 agencies implemented selected requirements related to 102 agency priority goals and commented on the 21 agency priority goals of 5 selected agencies. We recommended that OMB could improve agency priority goal implementation by revising its guidance to better reflect interim target, milestone, and cross-agency priority goal alignment requirements and by ensuring that agencies provide complete information about external contributors to their agency priority goals and describe congressional input on the goal’s development. OMB concurred with the recommendations.

In addition, GPRAMA requires federal government performance plan information; agency-level strategic plans, agency performance plans, and agency performance updates; and detailed information about each program identified by agencies to be posted on the website. According to OMB’s guidance, these additional features will be added to the website over time. implementation of Performance.gov, including OMB’s projected dates for the inclusion of additional information.

OMB Circular No. A-11, Preparation, Submission, and Execution of the Budget, pt. 6 (August 2012).

Leading Practices for Federal Websites Should Guide the Continued Development of Performance.gov in a Number of Areas

HowTo.gov is a key source of leading practices for federal website development and management. The website is managed by GSA and designed as a resource to improve how agencies communicate and interact with customers and use innovative tools and technologies to provide services and information. HowTo.gov offers best practices, guidance, and training on strategic planning; federal web requirements and policies; applications, data, and web infrastructure tools; web content management, usability, and design; and performance metrics. According to GSA staff, to provide access to key industry practices HowTo.gov makes available a list of the “Top 10 Best Practices” for federal websites and provides detailed information on how to implement each practice (see table 2). OMB has also encouraged the more widespread application of many of these practices across the federal government in its Digital Government Strategy.

In addition to these leading practices for the development of federal websites, lessons learned from the experiences of those who have developed other government performance reporting websites and views of potential users provide useful insights that could help inform the continued development of Performance.gov.

OMB staff stated that, thus far, the specific legal requirements of GPRAMA have been the primary framework used to guide efforts to develop Performance.gov. They have been focused on working to comply with these requirements by providing information on agency and cross- agency priority goals and by establishing a phased development plan for integrating additional information from agency strategic plans, performance plans, and performance reports and the inventory of federal programs. OMB and GSA staff members have said, however, that the leading practices provided by HowTo.gov will help guide the future development of Performance.gov.

OMB Provides Little Direction on How Different Audiences Could Use Performance.gov’s Information

Leading practices for the development of federal websites from HowTo.gov recommend identifying the purposes of a website and the ways in which specific audiences could use a website to accomplish various tasks, from finding relevant information to commenting on a government regulation to conducting a transaction, and then structuring information and navigation to help visitors quickly complete these tasks. According to HowTo.gov, this is important because people often visit government websites with a specific task in mind, and if it is not easy to find the information they want to quickly complete that task, they will leave the site. Providing guidance about the tasks that can be accomplished on a website, along with explanations and navigation assistance, can help website users successfully achieve their objectives.

Similarly, some officials we interviewed from other governments with experience developing performance reporting websites emphasized the importance of understanding and articulating the purposes that a performance reporting website is designed to achieve. They noted that the audiences the website is designed to serve, along with the intended uses of a site, should influence its design and content. For example, the main purpose of the state of Michigan’s dashboards is to provide citizens with a quick reference to see how the state is performing in key areas and whether trends are moving in the right direction. To fulfill this purpose, according to Michigan officials, the dashboards were designed to be simple, easy to access and navigate, and provide the public with an at-a- glance overview of the state’s progress. Users interested in more information on a specific performance measure can click on the link for that measure and will be directed to a separate page with trend data and information on why the measure is significant. A prominent link at the bottom of the page also allows them to provide direct feedback on the dashboard. The dashboards are supplemented by departmental “scorecards” that include relevant performance measures to inform internal departmental management and decision making. An example of the Michigan Education Dashboard is shown in figure 4.

GPRAMA provides direction on the purposes and audiences for Performance.gov. In addition to specifying the types of information to be made available on the website, the act states that information on the website “shall be readily accessible and easily found on the Internet by the public and members and committees of Congress.” OMB’s written guidance states that Performance.gov “serves as the public window on the Federal Government’s goals and performance in key areas of focus,” and will be “the single, government-wide performance website required under the GPRA Modernization Act.” OMB’s written guidance and information on the website also say that Performance.gov will make information about cross-agency and agency-specific goals and performance easier to find for the public and Congress, as required by GPRAMA, as well as for delivery partners, agency employees, the media, and other stakeholders. Lastly, the guidance specifies that the website will “support coordination and decision-making to advance shared goals.”

An analysis of statements from OMB and GSA staff, agency officials, and feedback we obtained from potential users, however, indicates that there are varying expectations regarding the primary audiences and uses of Performance.gov. For example, OMB and GSA staff emphasized that they have viewed Performance.gov as a tool for agencies to support cross-agency coordination and efforts to achieve agency goals. Consistent with this, OMB staff said that Performance.gov has been used to facilitate conversations between OMB examiners and agency managers about progress on agency priority goals. While officials we interviewed from the five agencies said that OMB has collected feedback from agencies in the development of Performance.gov, officials from four of these agencies also said, however, that Performance.gov is not being used as a resource by agency leadership or other staff. According to agency officials, agencies have information sources tailored to meet their needs, and Performance.gov does not contain critical indicators or the ability to display some visualizations used for internal agency performance reviews. Agency officials stressed instead that they view Performance.gov primarily as an external reporting tool so that the public and other external stakeholders can get a sense for how agencies are performing on key priorities. For example, an agency official noted that a key function of Performance.gov is to make performance data more readily available in one place so that members of the public and external stakeholders do not have to go to individual agency sites to collect information on agency goals and performance.

Several potential users and a practitioner we interviewed also indicated that it was unclear to them who the audience of Performance.gov is, and how audiences would use the site, given its content and design. For example, some said that, while the public is listed as an audience of the website, the detailed, technical nature of the website seemed primarily oriented toward a government, rather than a public audience. Some also commented that developers need clarity about how different audiences should be able to use a website, as developers can then ensure it is designed to present information in a way that addresses the needs of those audiences. OMB and GSA staff members have acknowledged that, given the requirements of GPRAMA, Performance.gov will need to shift to become a more publicly-oriented website in the future, which could involve a more thematic presentation to engage the public.

While the “Areas of Focus” section of the Performance.gov home page directs visitors to pages that allow them to explore information and metrics in specific areas, OMB has not yet articulated various ways that intended audiences, such as interested members of the public, Congress, experts and researchers, agency staff, and delivery partners could use the website or the information available through it to accomplish other specific tasks. For example, as mentioned previously, OMB has stated that a purpose of Performance.gov is to support coordination and decision making to advance shared goals. While OMB said that this intended use will help provide direction as they take a phased approach to development, the current version of the website gives no indication or examples of the ways that agency staff, or others, such as delivery partners or congressional staff, could use Performance.gov to facilitate coordination or communication about goals, activities, and performance between agencies and with interested stakeholders from other sectors. It is also unclear whether Performance.gov includes all the information and design elements necessary to support coordination across agencies. The President’s budget for fiscal year 2014 indicates that, in the future, efforts will be undertaken to test the potential use of Performance.gov to facilitate coordination among goal allies, enlist ideas and assistance to accelerate progress on goals, and enhance public understanding of the work of the federal government.

Other federal open government websites have indicated how visitors can use them to accomplish specific tasks. For instance, Recovery.gov contains an entire page that outlines what users can do on the site, including how to use the raw data available through the website, report waste, fraud, and abuse, or find job and grant opportunities. Data.gov states that it is designed to enable “the public to participate in government by providing downloadable Federal datasets to build applications, conduct analyses, and perform research,” and offers an introductory video that provides information on the purposes of the website and instructions on how to use it.

These websites have also integrated Web 2.0 technologies, such as links to social media feeds, to help people share and use of the information available through these sites. For example, Recovery.gov and Data.gov have integrated social-networking tools, such as Facebook and Twitter, blogs, and online discussion forums to facilitate the sharing of information, provide new venues for communication and participation, and enable collaboration and discussions between stakeholders and web- based communities of interest. Recovery.gov and Data.gov also have dedicated pages for different audiences that compile and organize relevant resources according to the needs and interests of those audiences. Data.gov, for instance, contains individual pages that allow users interested in specific areas, such as education, health, or energy, to find data and resources on those specific topics and to communicate with others who share those interests. Figure 5 provides a page for the energy community. To comply with the recommendations of the Digital Government Strategy, the Digital Services Advisory Group and the Federal Web Managers Council have also encouraged the use of social media to distribute content to large and diverse audiences and more effectively engage with customers.

OMB and GSA have been working to ensure that Performance.gov complies with the reporting requirements of GPRAMA. However, if the specific intended uses of Performance.gov are not clarified, while taking into consideration what the law requires, it could lead to varying ideas and expectations for how Performance.gov should be developed and designed and the audiences it should serve. The website’s accessibility to visitors and its relevance to them could be increased by additional direction on how various audiences could use the information available through the website, such as explanations or navigation assistance, or by tools, such as Web 2.0 technologies, that could facilitate that use.

OMB Collaborated with Agency Officials and Collected Feedback from Some Congressional Staff and Stakeholders, but Has Not Indicated How It Will Use Outreach to Other Audiences to Inform Future Changes

Leading practices from HowTo.gov on the development of federal websites recommend that developers collaborate with officials from other agencies to avoid duplication of effort, use content that already exists, and let organizations with the greatest expertise on a topic create content. According to HowTo.gov, this is important because allowing the organization with the greatest expertise to create content can help ensure its accuracy and quality, and linking to existing content can help save time and resources. HowTo.gov also recommends that developers engage potential users through focus groups and other outreach; regularly conduct usability tests to gather insight into navigation, the organization of content, and the ease with which different types of users can complete specific tasks; and collect and analyze performance, customer satisfaction, and other metrics. HowTo.gov states that these efforts are important for collecting and analyzing information about audiences, their needs, and how they are using, or want to use, the website. This information is also critical to help inform the development of a useful and usable website that meets the needs of intended audiences.

Similarly, some developers of other government performance reporting websites who we interviewed reported that it is important to use a developmental approach focused on continuous improvement, providing users with opportunities to give input and feedback, so that websites can be adjusted and improved to meet their needs. For example, following the initial creation of the state’s performance reporting website, officials in Maryland analyzed the website’s performance metrics to get a sense for how people were accessing the site, as well as the information they were searching for. They also employed usability testing to collect insight into the navigation and content of the website. From these insights, they identified the need to make information on related state programs and resources more easily accessible through the website, which is now reflected in its design.

According to OMB and agency officials we interviewed, to leverage the expertise of agencies in the development of content for Performance.gov, OMB established a content management and review process by which each agency shared information and data on their own goals and activities with OMB. OMB also developed the Performance Reporting Entry Portal (PREP), a data input system, which allowed agencies to directly input their information and data on agency priority goals into Performance.gov. For many of the pages on cross-agency management initiatives, they also leveraged information from agencies and other federal websites, linking to existing resources and reports.

OMB used other strategies to help maintain a collaborative environment and open lines of communication with federal agencies. According to OMB staff, during the initial development of Performance.gov, OMB used a pilot project involving several agencies to model different iterations of the website and collect feedback on potential designs from agency staff. OMB staff members also continue to hold biweekly conference calls with agency staff to provide status updates on the development of Performance.gov, answer questions, and address concerns. A GSA staff member supporting the PMLOB also recently met individually with staff from 24 agencies to discuss the data requirements for Performance.gov and the best way for agencies to provide information to be made available through the website. According to OMB, GSA, and agency staff, they have been able to use this ongoing communication to collect feedback on Performance.gov and ideas to gradually improve the design and usability of the system agencies use to input data on agency priority goals. For instance, the PREP system has been changed in response to agency requests so that data from prior time periods will be pre-populated in the system, an improvement that was made to help reduce the burden of data collection on agencies.

According to OMB and GSA staff, to gather input from other audiences as they developed Performance.gov, they also held briefings on the website for congressional staff and government transparency organizations and performance experts based in Washington, D.C. Since 2010, OMB staff said that they met several times with staff from the Senate Homeland Security and Governmental Affairs Committee, House Oversight and Government Reform, and the Senate Budget Committee to discuss the development of Performance.gov. They noted that they used this outreach to stakeholders to identify several specific website modifications. For example, OMB staff heard from stakeholders, including congressional staff, that it was important for Performance.gov visitors to be able to understand how agency priority goals relate to broader agency strategic goals and objectives, so OMB nested the priority goals within the larger agency goal framework. OMB staff also said that several stakeholders noted that they appreciated the ability to filter goals by different themes, so OMB included the capability to apply this feature to all categories of goals and objectives. Of the three congressional staff we spoke with who said they had received briefings on the development of Performance.gov, however, only one felt she had been consulted on website input. Also, since 2010, OMB staff reported holding no meetings on the development of Performance.gov with staff from other House or Senate committees who might use the website to inform their federal agency oversight.

Furthermore, while a focus of GPRAMA is to make federal performance information more accessible to the public, efforts to collect input and feedback from interested members of the public and other potential audiences have thus far been limited to the collection of suggestions through the website’s “Feedback” page. According to OMB staff, the submissions received through this portal are generally on topics unrelated to the website, although they do receive questions about the website or suggestions for minor corrections to content. They also said that the suggestions received through this portal have not yet had a significant impact on the design of the site. GSA staff said that usability testing of Performance.gov is planned for September 2013 and that the tests will be managed by usability experts from GSA and will include a mixture of participants from within government and from the general public. From this testing, they hope to inform new development and interface and navigation improvements.

Other federal open government websites have used broader outreach efforts to inform their development and design. For instance, the developers of Recovery.gov used both focus groups and usability testing with interested citizens to collect feedback and recommendations and inform the development of the website from its initial stages. The developers of Data.gov also used a technology service called IdeaScale to collect feedback and ideas from hundreds of users during the initial development of the site.

OMB and GSA monitor visitors’ use of Performance.gov using a variety of metrics. For example, one metric that is tracked is the total number of visits to Performance.gov, by month, as seen in figure 6.

HowTo.gov recommends that agencies collect, analyze, and report on a minimum baseline set of performance, search, customer satisfaction, and other metrics, which allow officials to get a holistic view of how well online information and services are delivered. Of the 24 metrics recommended by Howto.gov, 15 are currently tracked for Performance.gov. See table 3 for the list of recommended metrics and whether they are collected for Performance.gov.

HowTo.gov also recommends setting goals for metrics, and making sure that these align with the objectives of a website, to help prioritize and guide design changes for improved performance and usability. These goals can be identified based on prevailing practices or the desire to improve a particular metric over time. Except for customer satisfaction, which is discussed later in this report, OMB has not yet established goals for any recommended metrics.

GSA staff acknowledged the need to collect additional recommended metrics for the website. They said that as they expand the list of metrics that are collected, they are planning usability testing to identify the most appropriate metrics for analyzing the website’s use and identifying improvements that will increase the usability of Performance.gov. OMB staff also said that right now their primary goal is releasing the legally required information on Performance.gov, but they will consider setting targets for certain metrics in the future.

In addition to performance and search metrics, HowTo.gov recommends that agencies collect customer satisfaction metrics, which include measures of overall customer experience, the completion rate of intended tasks, and the percentage of visitors likely to return and recommend the website. The Digital Government Strategy requires that agencies use customer satisfaction measurement tools for all .gov websites. While OMB has established a numerical customer satisfaction target for Performance.gov, the agency has not established tools on the website, such as online customer satisfaction surveys, that would allow them to collect the data necessary to track the recommended customer service metrics. GSA staff said that there are no public-focused customer satisfaction tools for Performance.gov, outside of the “Feedback” page, on the website because, at this stage in the website’s lifecycle, the primary focus has been on collecting feedback from Congress and agency management. They said, however, that they have used feedback collected through congressional meetings and feedback from members of the PIC, as well as comments from the “Feedback” page of the website, to get a general sense of how satisfied some users are with the website. The “Feedback” page, however, does not ask users to rate their overall satisfaction or the quality of their experience.

As stated earlier, according to OMB and GSA staff, complying with the time frames and requirements for public reporting in GPRAMA has been the focus of development, although they used outreach to collect feedback from some congressional staff, government transparency organizations, and performance experts to inform changes to the website. A lack of outreach to other potential audiences and user testing, however, means that developers have not yet had an opportunity to systematically collect input on the information needs and preferences of potential audiences of the website, including interested members of the public, other congressional committees, delivery partners, or others; whether the content on the website meets the needs of different audiences; or if it is presented or organized in a way that makes it easily accessible. Furthermore, without collecting all recommended search, and customer satisfaction metrics, and establishing goals or targets for metrics, it may be more difficult for the developers of Performance.gov to get a holistic picture of how they are delivering information and to identify and prioritize potential improvements.

OMB and GSA staff have said that as the phased development of Performance.gov takes place, they expect to use outreach to a broader set of audiences, including members of the public, and usability testing with these audiences to make Performance.gov more “public-facing” and “citizen-centric.” While they have developed a general time frame for conducting usability tests, they have not yet established a specific plan, a timeline for other forms of outreach, or indicated the specific audiences they plan to target for greater outreach.

Performance.gov Was Designed with a Consistent Structure and Navigation, but Users Reported Some Issues with Accessibility, Complexity of Navigation, and Search Capabilities

Leading practices from HowTo.gov recommend organizing website content based on audience needs and using common terminology and placement, consistent navigation, and plain language. HowTo.gov states this is important because providing consistent terminology and placement across a website helps users easily find what they need as they will know what labels to look for. Consistent navigation makes websites easier to use because users are more likely to find what they need from a website if they are familiar with its navigation scheme. Using plain language also makes the content of a website more accessible by using words that the website’s typical user can understand the first time. Lastly, leading practices recommend organizing and categorizing website content to help ensure that it can be found using commercial search engines or a website’s internal search engine. All of these elements factor into the overall usability of a site, which is key to ensuring an effective user experience.

Similarly, developers of some government performance reporting websites we interviewed emphasized the importance of designing these websites in a way that makes them easy to access, use, and navigate. Several states have tried to increase the visibility of their performance reporting websites by integrating them with other transparency and open data websites and by including prominent links on other state government websites. A number of state and local officials also emphasized the importance of making the information and data on a performance reporting website engaging and easily understandable. They suggested doing this by organizing information so users can quickly locate what is of interest to them through visual displays of data, minimizing the amount of text, avoiding the use of jargon, and ensuring that data is current and timely. For example, City of Boston staff said that visitors generally stay on these sites for limited amounts of time, so they designed the Boston About Results website to present performance information in a way that can be easily consumed and understood, using brief descriptions of key strategies and graphics to depict performance trends. Figure 7 shows an example of a screen shot from the Boston About Results website.

Performance.gov is accessible from some other federal websites, a key source of potential traffic. For example, USASpending.gov includes a link to Performance.gov on its home page and the “Government Performance” link on the “topics” page of USA.gov includes Performance.gov at the top of a list of links. Other federal websites, however, including Data.gov, the White House and OMB home pages, and the websites of the five agencies included in our study do not have links to Performance.gov. As one potential user from a government transparency organization pointed out, if Performance.gov is not connected in any meaningful way with other federal information and data websites, it may not be a site that government data users would naturally go to. Furthermore, a lack of social media integration on Performance.gov could increase the effort required for users to share content from the website. A lack of other communication tools, such as RSS feeds and social media, which can be used to alert users to new or important content and enable website administrators to push out timely and relevant information, could also limit the website’s ability to keep interested users informed about new content.

The various sections of Performance.gov were designed with a consistent organization and navigation. This can be seen particularly in the structure of individual agency pages. As shown in the U.S. Department of Agriculture example in figure 8, each page includes a brief overview of the agency’s mission, includes links to the agencies existing performance reports, and has three tabs with information on the agency, the agency’s goals, and the agency’s efforts on government-wide management initiatives. The “agency goals” tabs also use a consistent, layered structure to provide lists of agency strategic goals, objectives, and priority goals. According to GSA staff, this platform will also be used to organize the Performance.gov information on agency strategic goals and objectives that will be added to comply with the GPRAMA requirement that the website include agency strategic plans, annual performance plans, and annual performance reports.

The use of layering is a common practice in the design of performance reporting websites, as it allows developers to structure the presentation of information in a way that meets the needs of different audiences by allowing them to access the level of information that is most relevant for them. For example, the approach begins with the presentation of higher- level goals, highlights, or more aggregated information, while allowing those users interested in progressively more detailed goals or information to drill down and find the level of information most appropriate for their needs.

Several individuals we interviewed had positive comments on the layered structure used to organize the information on agency strategic goals, objectives, and priority goals within Performance.gov and said that using the same format across goals and agencies facilitated navigation. However, others we spoke with, including some agency officials, raised concerns that the complexity of the website’s navigation contributes to the difficulty of finding critical pieces of information on the site, particularly information on individual agency priority goals. For example, if a user chooses to access information on agency priority goals through the page of an individual agency, the user would need to click through multiple drop downs and tabs, and navigate to a new page, before reaching the tab that has a graphic depicting a specific agency priority goal’s progress. If a user is unaware that this information exists, the user may not be able to easily find it, and Performance.gov does not include instructions on how to locate this information. To facilitate access to information on agency and cross-agency goals, a number of experts and potential users suggested that the Performance.gov home page include a thematic index, similar to the filter on the “Clear Goals” page that currently allows users to search for goals by different themes, like agriculture, energy, or natural resources and environment. Other suggestions for the pages on individual agency priority goals were to make the goals and visualizations of performance trends a more central part of the presentation.

OMB staff said that lessons learned through the federal government’s past experience with performance reporting have emphasized presenting performance information in an appropriate context with relevant information on agency missions, goals, activities, and factors that influence performance. According to OMB staff, the use of plain language was a focus in trying to make Performance.gov easy to understand, and OMB held a briefing on plain language for agencies developing content for Performance.gov. However, several potential users we spoke with also noted that some pages on Performance.gov include jargon and technical detail that may not be understandable or relevant to certain audiences, including members of the general public. Concerns were also expressed that some pages on Performance.gov contain too much text, which can limit the accessibility of information.

Our analysis of the ease with which commercial search engines, and Performance.gov’s internal site search engine, can be used to find relevant information on the website produced mixed results. When using a commercial search engine to search for several general, government- wide terms, including “federal government goals,” “agency performance,” and “federal agency performance measures,” we found that a link to Performance.gov was listed on the first page of results for all of these searches. We also found that a link to Performance.gov was included on the first page of results when using a variety of agency-specific search terms, including “Department of Agriculture performance,” “EPA goals,” “Department of Labor goals,” “Department of Interior goals,” and “Department of State performance.”

The results of using Performance.gov’s internal search engine to find relevant information on the website were mixed, however. To analyze the ease with which a user could find relevant agency priority goal pages using the website’s search engine, we used topic-focused search terms taken from specific agency priority goals of the five agencies included in our study, including “rural communities,” “greenhouse gas,” “climate change,” “training programs,” and “Afghanistan.” For only two of these five searches—on “rural communities” and “Afghanistan”—however, did the results include a link to the relevant agency priority goal page. The results sometimes included links to other potentially relevant pages on Performance.gov, such as those with information on related cross-agency priority goals, cross-government management initiatives, or featured stories. Many of the links included in the results direct users to web pages that include agency overviews or lists of agency strategic goals, but which do not appear to be part of the normal Performance.gov structure. Figure 9 is an example of one of these pages.

While the opinions of potential users on the organization and navigation of Performance.gov were mixed, comments and our analysis of Performance.gov’s internal search engine indicated the site’s overall usability could be limited for potential audiences by several factors: the complexity of the website’s navigation, difficulty accessing some information, the lack of an effective internal search engine, and the level of detail and technical information provided. Actions OMB and GSA are taking to develop a more public-oriented and usability-focused perspective for Performance.gov are discussed later in the report.

The Proposed Performance Management Line of Business Offers a Potential Structure for Addressing Identified Issues and Applying Leading Practices

Leading practices for the development of federal websites recommend defining and documenting how a website will be governed, including the structure of the team managing the website, the roles and responsibilities of different stakeholders, and the policies and procedures that will be used to govern the creation and management of content. HowTo.gov states this is important because codifying how a website will be governed, as well as its organizational structure, policies, and procedures provide involved staff with a common understanding of how things are supposed to work.

As noted previously, staff from OMB, GSA, and members of the PIC are considering whether to create a Performance Management Line of Business (PMLOB). According to OMB and GSA staff, in an effort to document the potential governance structure of the PMLOB and Performance.gov, as well as the roles and responsibilities of various key stakeholders and the processes that will be used for major decisions, a draft charter was developed. OMB staff said the charter was completed in May 2013.

The proposed creation of the PMLOB has also offered an opportunity to define the purposes and scope of the system that will be used to centralize the collection of performance data from agencies, as well as how it will be displayed on Performance.gov. For instance, an executive steering committee made up of staff from OMB, GSA, and several members and staff from the PIC, has been created. OMB staff said that the intent of the committee is to provide a forum for the discussion of potential options and to help determine the requirements that a future iteration of Performance.gov and its underlying data collection system must fulfill and what purposes they should be designed to accomplish.

According to OMB and GSA staff, they are also developing working groups to examine various issues. For example, a working group on “business intelligence” has been established through the PIC to help determine how to more effectively use the website’s performance information to support internal management and coordination. They are also starting a working group on the presentation and display of performance information through Performance.gov. GSA staff said that members of the PIC have been asked to collect agency input on the current version of Performance.gov, the reporting burden it presents, what they like and do not like about it, and what they would like to see in future iterations of the website. Potential solutions to address identified issues will then be developed and considered.

OMB and GSA staff also said that the knowledge and capacity of the Center for Excellence in Digital Government, a component within GSA that provides government-wide support on the use of the Web, social media, and other technologies to improve customer service, will be used to help bring a more public-oriented and usability-focused perspective to Performance.gov.

A GSA representative emphasized that the development of Performance.gov will happen in phases as funding becomes available. Performance.gov has been funded through the E-Government fund, which consists of funding appropriated by Congress every year.staff said that while funding for the development of the website has been consistent over the past 2 years, it has varied over time. The staff added that at times funding was only sufficient to cover operational expenses with no funding available for new development. The GSA representative also noted that while the focus of the PMLOB is on making performance information more visible and transparent for Congress and the public by replacing printed reports with a publicly accessible website, there is hope that the shift to the PMLOB will provide improved, steady governance of performance management across government and greater stability from one administration to another.

Conclusions

OMB’s and GSA’s development efforts for Performance.gov have been focused on ensuring that information required by GPRAMA is available on the website. Leading practices for the development of federal websites, however, offer insights into approaches that could be used to guide the development of Performance.gov going forward. For example, clarity about the intended uses of Performance.gov could provide additional direction to ensure that the website is developed as a valuable and relevant tool for its audiences, including interested members of the public and Congress, agency staff, delivery partners, and other potential audiences. The practices could also help determine any design changes, such as navigation assistance or the integration of Web 2.0 technologies, to facilitate those intended uses.

While OMB collected input from some congressional staff, government transparency organizations, and performance experts, the limited outreach to a broader set of potential audiences and the lack of usability testing to date means that OMB does not know whether Performance.gov meets users needs or how the website could be further developed. Not tracking all recommended performance metrics, particularly those measuring user satisfaction or establishing appropriate goals for certain metrics, may also make it more difficult to analyze the effectiveness of the website and to identify and prioritize potential improvements.

Recommendations for Executive Action

To enhance the value of Performance.gov for intended audiences and improve the ability to identify and prioritize potential improvements, we are recommending that the Director of the Office of Management and Budget—working with the Performance Improvement Council and the General Services Administration—take the following three actions:

Clarify the ways that intended audiences could use the information on the Performance.gov website to accomplish specific tasks and specify the design changes that would be required to facilitate that use.

Seek to more systematically collect information on the needs of a broader audience, including through the use of customer satisfaction surveys and other approaches recommended by HowTo.gov.

Seek to ensure that all performance, search, and customer satisfaction metrics, consistent with leading practices outlined in HowTo.gov, are tracked for the website, and, where appropriate, create goals for those metrics to help identify and prioritize potential improvements to Performance.gov.

Agency Comments

We provided a draft of this report to the Director of OMB and the Acting Administrator of GSA for review and comment. OMB staff provided oral comments, and we made technical changes as appropriate. OMB staff agreed with our recommendations. GSA did not have comments on the report. We also provided the draft for review and comment to the five agencies with officials we interviewed regarding their perspectives on Performance.gov. None of the five agencies had comments.

We are sending copies of this report to the Director of OMB and the Acting Administrator of GSA as well as appropriate congressional committees and other interested parties. The report is also available at no charge on the GAO website at http://www.gao.gov. If you have any questions concerning this report, please contact me at (202) 512-6806 or mihmj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix II.

Appendix I: Objective, Scope, and Methodology

This report is part of our response to a mandate to assess initial implementation of the GPRA Modernization Act of 2010. Specifically, this report examines the extent to which Performance.gov incorporates leading practices for the development of federal websites. This report is the fifth in a series that looks at how agencies are implementing various GPRAMA requirements.

To address our objective, we analyzed information from the Performance.gov website and the requirements from GPRAMA regarding the website. We also examined related guidance, such as the Office of Management and Budget’s (OMB) Circular No. A-11, which lays out expectations on the design, development, and implementation of Performance.gov. We reviewed relevant academic and policy literature on performance management and reporting, including our previous reports on performance management. The literature was selected based on whether it provided relevant information on the reporting requirements in GPRAMA, lessons learned from the development of other federal reporting systems, or insights into performance reporting best practices or lessons learned.

We conducted interviews with staff at OMB and the General Services Administration (GSA), who worked on the design and development of the website. We reviewed OMB and GSA staff’s description of the development of Performance.gov from our interviews and compared it against criteria established by HowTo.gov, a source of guidance and leading practices for government websites. We also conducted an analysis of the ease with which commercial search engines and Performance.gov’s internal site search engine can be used to find relevant information on Perfomance.gov. Lastly, we reviewed a subset of federal open government websites, including Recovery.gov, Data.gov, and USASpending.gov. These websites were selected because, like Performance.gov, they are used to make government-wide information more transparent through publicly-accessible websites. We reviewed design features of each, including whether they identified their purposes, audiences, and potential uses, and approaches they used to facilitate access to information.

To further address our objective, we obtained the views and suggestions of potential users of Performance.gov. We interviewed executive branch and congressional staff and a variety of nonfederal stakeholders. We reached out to groups most likely to use the information on Performance.gov because of their management, oversight, advocacy, or academic interest and asked them to review the website prior to our interviews. We selected five agencies to gather their perceptions on the utility of Performance.gov—the U.S. Department of Agriculture, Department of the Interior, Department of Labor, Department of State, and the Environmental Protection Agency. We selected these agencies from those that have not previously been the subject of our GPRAMA mandate work. We grouped agencies with the same number of priority goals together, and selected the agency in each group that uses the largest number of tools of government—such as direct service, regulations, grants, and loans—to achieve their performance goals. This process allowed us to consider a mix of agency priority goals, an example of each of the tools of government, and performance improvement officers under both career and political appointments.

We also interviewed staff from 13 different majority and minority congressional oversight and authorizing committee offices of the U.S. Senate and House of Representatives to gather their views on Performance.gov and any suggestions they had for improving it. We interviewed congressional staff from the committees shown in table 4.

We also interviewed a variety of nonfederal stakeholders including representatives from 10 academic, advocacy, transparency, and public policy organizations to solicit feedback about the use of performance reports, their use of the website, and suggestions they had for improving it. We interviewed academics and practitioners from the California State University San Bernardino; Government Finance Officers Association of the United States and Canada; Government Performance Coalition; Socrata; University of Victoria; and the Wilfrid Laurier University. Our advocacy and transparency group representatives were from the Center for Effective Government; Coalition for Evidence-Based Policy; OpentheGovernment.org; and the Project on Government Oversight. We selected these academics and practitioners on the basis of our literature review and recommendations from other performance management and reporting academics and practitioners. The results of these interviews are not generalizable to all nonfederal stakeholders but provided insights into perceptions of Performance.gov.

In addition, based on a review of government performance reporting websites, we selected a group of 12 websites from state and local levels, as well as one from Canada, to gather lessons learned and best practices from representatives of these performance reporting websites that were relevant to the design and development of Performance.gov. To identify examples of web-based performance reporting by state governments, we visited the main government websites of all 50 states, or their governor’s website, and searched each one using the search term “performance” to identify any relevant agency or government-wide websites providing performance information in a web-based format. We selected only sites that presented performance information through web pages; we did not select transparency websites that are generally designed to make available information on state finances and the distribution of state expenditures. To identify relevant examples of web-based performance reporting by local governments, we conducted a search similar to the state sites and talked to representatives from the local websites who agreed to be interviewed. To identify international performance reporting websites, we visited the websites of several national governments of English-speaking countries recognized for performance reporting— Canada, the United Kingdom, Australia, and New Zealand—to examine if these governments had developed government-wide performance reporting websites. Through this approach, we identified the Canadian government’s “Planning and Performance Gateway,” which consists of the Canada’s Performance and Overview of Government Spending and Performance websites. None of the other countries had government-wide performance reporting websites. Table 5 provides a list of the state, local, and Canadian performance reporting systems we selected to review with representatives who agreed to be interviewed.

We conducted our work from July 2012 to June 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Elizabeth Curda, Assistant Director; Judith Kordahl; and Adam Miles made key contributions to the report.