Managing For Results: Agencies Should More Fully Develop Priority Goals under the GPRA Modernization Act
Released: 2013-04-19T00:00:00

Letter

Background

GPRAMA specifies that APGs are to have ambitious targets that can be achieved within a 2-year period; a clearly identified agency official, known as a goal leader, who is responsible for achieving the goal; clearly defined quarterly milestones; and interim quarterly targets for performance measures, if more frequent updates of actual performance provide data of significant value.

Other GPRAMA requirements provide additional information and context for the priority goals. For example, agencies are to describe how their APGs contribute to the agency’s long-term strategic goals, as well as any of the CAP goals developed by OMB, as applicable. This information can help illustrate how an agency’s efforts to achieve its priority goals fit within a broader, crosscutting context—both within the agency and across the federal government. In addition, agencies are to describe how they incorporated any input on their priority goals received during consultations with relevant congressional committees.

GPRAMA also lays out a schedule for gradual implementation of its provisions, with a 3-year period of interim implementation following enactment in January 2011. It required agencies to identify their APGs and related information in their strategic plans and performance plans, published concurrently with the President’s Budget in February 2012. Agencies also were to provide information about their APGs for OMB to publish on Performance.gov by October 1, 2012, and agencies are to OMB provided update this information on at least a quarterly basis. guidance to agencies on implementing the act’s provisions, including those related to APGs, in several memorandums and its annual Circular No. A-11 in both 2011 and 2012.

In addition to OMB’s guidance, the Performance Improvement Council (PIC) shared practices related to developing and implementing APGs with agencies in 2011 and 2012. The PIC established the Goal Setting Working Group in May 2011 to assist agencies in setting their 2012 to 2013 APGs. The group produced a draft guide to goal setting, which included criteria for selecting priority goals as well as elements and examples of effective goal statements. In September 2012, the PIC also produced a draft best practices guide for developing milestones; the guide described the characteristics of milestones and provided several examples.

Agencies Have Made Progress Implementing Key Requirements for Priority Goals, but Some Information Is Incomplete

Agencies Have Implemented Several Requirements Related to Priority Goals

For each APG, agencies were required, by GPRAMA or OMB guidance, to make available to OMB for publication on Performance.gov and in their strategic plans or performance plans (1) a performance goal with a target level of performance to be achieved in a 2-year time frame; (2) an explanation of how the goal contributes to agency strategic goals; and (3) the identification of an agency official as the goal leader responsible for achieving the goal. Agencies provided information about each of these requirements for all of the 102 APGs on Performance.gov included in our assessment—which represents an important accomplishment in the development of priority goals.

Figure 1 illustrates how information on Performance.gov for one of OPM’s priority goals meets the three requirements. The full goal statement for the APG provides a targeted level of performance to achieve (“participation of at least 2 multi-state health plans in State Affordable Insurance Exchanges”) within a 2-year timeframe (“by October 1, 2013”). The layout of information on Performance.gov shows that this APG supports OPM’s strategic goal to “Improve Access to Health Insurance,” as part of its strategic objective to “contract with multi-state health plans to be offered on affordable insurance exchanges.” Finally, information on the site identifies OPM’s Director of Healthcare and Insurance as the goal leader for this APG.

As an additional example, the Social Security Administration (SSA) included information about each of its APGs in an appendix of its fiscal year 2013 performance plan. As shown in figure 2, for SSA’s priority goal to ensure faster hearing decisions, the plan provides the targeted level of performance (“reduce the average time…to 270 days”) and the timeframe (“by the end of fiscal year 2013”) in the “Priority Goals” column. In the same column, SSA indicated that the goal is linked to performance measure 1.1c, which supports the agency’s strategic goal “Deliver Quality Disability Decisions and Services.” In the “Goal Leaders(s)” column, SSA identifies the Executive Coordinator for Backlog Initiatives in the Office of Disability Adjudication and Review.

SSA’s strategic plan for fiscal years 2013 to 2016 also identifies its APGs and how each supports an agency strategic goal. Figure 3 provides a table from the strategic plan that presents a list of goals that support each of its strategic goals and denotes those that are APGs with an asterisk.

Our past work has shown that although the federal government faces a series of challenges that in many instances are not possible for any single agency to address alone, agencies often face a range of challenges and barriers when they attempt to work collaboratively. Our annual reports on duplication, overlap, and fragmentation highlight a number of areas where a more crosscutting approach is needed—both across agencies and within a specific agency.

We found that duplication and overlap occur because programs have been added incrementally over time to respond to new needs and challenges, without a strategy to minimize duplication, overlap, and fragmentation among them. Also, there are not always interagency mechanisms or strategies in place to coordinate programs that address crosscutting issues, which can lead to potentially duplicative, overlapping, and fragmented efforts. GPRAMA establishes a new framework for taking a crosscutting and integrated approach to improving government performance, and effective implementation of that framework could play an important role in clarifying desired outcomes, addressing performance that spans multiple organizations, and facilitating actions to reduce unnecessary overlap, duplication, and fragmentation.

Two provisions in GPRAMA, in particular, direct agencies to link their APGs with crosscutting federal efforts. First, the act requires agencies to identify federal organizations, program activities, regulations, policies, and other activities—both internal and external to the agency—that contribute to each of their APGs and include this information in their performance plans and provide it to OMB for publication on Performance.gov. In addition, OMB’s 2012 guidance directs agencies to include tax expenditures in their identification of organizations and programs that contribute to their APGs, as part of their updates to Performance.gov. Since 1994, we have recommended greater scrutiny of tax expenditures, as periodic reviews could help determine how well specific tax expenditures work to achieve their goals and how their benefits and costs compare to those of programs with similar goals. Second, APGs are to be informed by the CAP goals. The act also requires agencies to demonstrate in their performance plans any alignment between their performance goals—including their APGs—and the CAP goals. Both of these provisions are important because they show how agencies are coordinating efforts toward a common crosscutting issue. As we have previously reported, uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort.

Agencies identified at least one internal contributor for each of their APGs, though agencies differed in the amount of detail they provided. For example, the National Science Foundation (NSF), as shown in figure 4, identified its Directorate of Engineering as a lead organization and its Innovation Corps activities and programs as contributing programs to its APG to increase the number of entrepreneurs emerging from university laboratories.

DOT, in its fiscal year 2013 performance plan, organized the descriptions of its planned performance—including its APGs—into broad themes under its strategic goals. As shown in figure 5, DOT identified the operating administrations, activities, enabling legislation, regulations, and other resources that contribute to each theme.

Similarly, DOT identified on Performance.gov a range of contributing programs to its APG to reduce the risk of aviation accidents (see figure 6).

Agencies identified external contributors for 73 of the 102 APGs. When they did identify external contributors, agencies differed in the amount of detail they provided. The Department of State (State), for instance, identified in its fiscal year 2013 congressional budget justification external contributors for six of the eight APGs it jointly developed with the U.S. Agency for International Development (USAID). These external contributors are generally at the department/agency, component, or program level, such as the Department of Defense (DOD), the Department of Justice’s Office of Overseas Prosecutorial Development Assistance and Training, and the Department of Justice’s International Criminal Investigative Training Assistance Program, respectively (see figure 7).

Similarly, State and USAID identified on Performance.gov external contributors to their APG to advance low emissions, climate-resilient development, such as the Department of Agriculture (USDA), the Forest Service (a USDA component), the Environmental Protection Agency, and international governmental and nongovernmental organizations (see figure 8).

We did not verify that agencies included all relevant internal and external federal contributors to their APGs. However, it was not always clear why external contributors were not identified for 29 of the 102 APGs. In some instances this could be explained by the goal being internally focused. For example, the Department of the Interior listed no external contributors to its internally-focused APG to “build the next generation of conservation and community leaders by supporting youth employment” at the department. However, our analysis indicates that 8 of the 29 APGs that lack external contributors are related to crosscutting areas that we have identified as at risk of potential fragmentation, overlap, or duplication. For example, NSF did not list any external contributors to its APG to develop a diverse and highly qualified science and technology workforce by having 80 percent of institutions funded through NSF’s undergraduate programs document the extent of use of proven instructional practices by September 30, 2013. Our past work has identified 209 programs across 13 federal agencies that are focused on science, technology, engineering, and mathematics education, some which may have efforts related to those NSF is undertaking for this goal.

In addition, our in-depth examination of a sample of 21 APGs indentified several APGs related to our work on fragmentation, overlap, and duplication, including one where not all relevant contributors were identified. As we have previously reported, HUD, USDA, and the Department of the Treasury (Treasury) operate rental housing programs with overlapping purposes, although the products, areas served, and delivery methods differed, and recommended that further collaboration be undertaken and documented in strategic plans and performance plans by these agencies. HUD and USDA generally agreed with the recommendations; Treasury did not provide comments. As illustrated in figure 9, HUD identified two tax expenditures (Treasury) as contributors to its APG targeted at preserving affordable rental housing—the only APG out of all 102 to have tax expenditures identified as external contributors. However, HUD did not identify USDA or its rental housing programs.

Most Agencies Did Not State Whether Their Priority Goals Support Any CAP Goals

The Department of Commerce (Commerce) and State both have export- related APGs, and noted on Performance.gov that their APGs contribute to the broader CAP goal to double U.S. exports by the end of 2014. However, our analysis indicates that 27 additional APGs appear to support at least one of the 14 interim CAP goals, but agencies did not describe this connection. In part, this could be a result of OMB’s guidance, which does not state the requirement for agencies to show the alignment between their performance goals—including their APGs—and the CAP goals. Instead the guidance directs agencies to refer to Performance.gov, where the quarterly updates for the CAP goals will describe how the agency’s goals contribute to the CAP goal. While in a few instances CAP goals identified contributing APGs, this alignment was not also provided in the corresponding APG information on the site. For example, in the quarterly update published in December 2012, the export CAP goal identified the export-related APGs of Commerce and State—as well as that of USDA—as supporting the CAP goal’s strategies. Unlike Commerce and State, USDA did not describe how its export-related APG supports the broader export CAP goal. According to OMB staff, as the information presented on Performance.gov and its functionality is expanded and enhanced, they expect to cross-reference related pieces of information, which they stated would include the connections between APGs and any related CAP goals.

We have reported that communicating the relationship between individual agency goals and outcomes that cut across federal agencies provides an opportunity to clearly relate and address the contributions of alternative federal strategies. In addition, as mentioned above, it is important for agencies to identify areas in which they should be coordinating efforts to meet crosscutting goals, and we have reported that strategic plans and performance plans can be tools for doing so.

Without OMB guidance directing agencies to describe how their performance goals—including APGs—support any relevant CAP goals, agencies may not understand the importance of examining how their efforts contribute to broader federal outcomes and planning for those contributions. Similarly, although we did not analyze whether agencies included all relevant internal and external contributors for their APGs, our work on potential areas of fragmentation, overlap, and duplication helped identify several examples where agencies did not list relevant external contributors. In addition, OMB’s review process does not systematically check whether agencies have identified all relevant contributors. This raises questions as to whether larger issues exist with the completeness of agencies’ listings of APG contributors. More importantly, without complete information related to both of these requirements, it is unclear whether agencies have properly planned to coordinate their efforts. As we noted earlier, uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort.

Agencies Generally Identified Performance Measures for Their Priority Goals, but Many Measures Did Not Have Related Targets for Measuring Interim Progress

As noted earlier, agencies are to define a target level of performance to be achieved within a 2-year timeframe for each APG. GPRAMA requires agencies to establish a set of performance measures (called performance indicators in the act), which are used to assess progress toward each goal, at least annually. The act also requires agencies to review and report on progress toward their APGs on at least a quarterly basis. One way agencies can gauge progress this frequently is through the development of interim quarterly performance targets—that is, targets for each quarter that falls within the 2-year period.interim targets for performance measures when more frequent updates of actual performance would provide data of significant value to the federal government, Congress, or program partners at a reasonable level of administrative burden.

The Senate committee report that accompanied the bill that ultimately was enacted states that the quarterly performance review requirement for APGs is intended to increase the use of performance information to improve performance and results. Our past work has shown that although agencies collect a significant amount of performance information, they have not consistently used that information to improve management and results. We have previously identified practices for enhancing agency use of performance information, one of which is to communicate performance information, including performance against targets, frequently and effectively. Frequent, regular communication can help managers to inform staff and other stakeholders of their commitment to achieve the agency’s goals and to keep these goals in mind as they pursue their day-to-day activities. Frequently reporting progress toward achieving performance targets also allows managers to review the information in time to make improvements. Without related targets, agencies may be unable to demonstrate to key stakeholders, including Congress, program partners, and the public, that they are tracking progress frequently enough to address any performance issues related to their APGs as they arise.

In the December 2012 update to Performance.gov, agencies identified 241 performance measures for gauging progress toward 91 of the 102 APGs. For the 11 APGs without performance measures, the agencies stated that the goals are more appropriately measured by milestones. Although OMB’s guidance strongly encourages agencies to use quantitative measures, it allows agencies the flexibility to develop qualitative goal statements that are supported by milestones to assess progress. All 24 agencies have at least one APG with an accompanying performance measure.

The frequency with which agencies collect performance information for the measures varies, as illustrated in table 1. Agencies collect and report results on a majority of the measures (166 out of 241, or 69 percent) on at least a quarterly basis. Measuring and reporting results this frequently represents substantial progress in agencies’ ability to use performance information in a timelier manner to pinpoint and act on improvement opportunities. Previously, GPRA required agencies to report their performance on an annual basis.

Figure 10 illustrates one of the two measures the Department of Health and Human Services (HHS) identified for its APG to increase the number of health centers certified as Patient Centered Medical Homes. The measure is the percent of health centers with at least one site recognized as a Patient Centered Medical Home. HHS provided interim performance targets for each quarter of the goal period, beginning with an interim target of 4 percent in the first quarter of fiscal year 2012, with subsequent targets increasing toward the final target of 25 percent by the fourth quarter of fiscal year 2013. In addition, HHS reported its progress toward those interim targets on a quarterly basis.

In other cases, agencies did not provide interim targets to show the level of performance expected for each underlying measure. For a majority of their measures (136 or 56 percent), agencies provided interim targets that align with their measures (e.g. quarterly targets for quarterly measures for each quarter during the 2-year period of the goal). For example, similar to the HHS example above, for 90 measures agencies provided quarterly targets to be achieved through the end of the goal period (fourth quarter of fiscal year 2013) for each measure. But for 77 measures (32 percent), agencies provided interim targets that align with their measures for only a portion of the 2-year timeframe. Finally, for 28 measures (12 percent), agencies did not provide interim targets that align with their measures for any portion of the 2-year timeframe.

As previously stated, GPRAMA requires agencies to develop interim quarterly performance targets for their measures if more frequent updates of actual performance would provide data of significant value to the federal government, Congress, or program partners at a reasonable level of administrative burden. While OMB’s 2012 A-11 guidance provides a definition of “reasonable administrative burden,” it does not define what constitutes “data of significant value.” Therefore it may be unclear to agencies when it would be appropriate to develop these targets. Furthermore, the guidance does not mention the interim quarterly performance target requirement. OMB staff told us that they expect agencies to provide such targets, and that they have communicated this expectation to agencies. OMB staff shared with us the user guide they developed for agencies to input data for publication on Performance.gov. According to the guide, indicators should include a target for each reporting period.

Agencies Did Not Provide Milestones with Scheduled Completion Dates for Many Priority Goals

The act requires each APG to have clearly defined quarterly milestones— scheduled events signifying the completion of a major deliverable or a set of related deliverables or a phase of work. Similar to performance measures, OMB’s guidance states that milestones will follow fiscal year quarters and notes that agencies may choose monthly milestones if preferred. In addition, a draft guide developed by the PIC describes characteristics of a good milestone, such as articulating concrete actions to be taken and being time-bound.

Milestones can help agencies demonstrate that they have clear and fully developed strategies and are tracking progress to accomplish their goals. Such strategies, as identified in our past work, should (1) identify specific actions agencies are taking or plan to take to carry out their missions, (2) outline planned accomplishments, and (3) provide a schedule for their completion. Milestones can help show the connection between agencies’ day-to-day activities and their goals. In addition, by describing the strategies to be used to achieve results, including clearly defined milestones, and the resources to be applied to those strategies, agencies can provide information that would help key stakeholders, including Congress, better understand the relationship between resources and results. Without clearly defined milestones, agencies may have difficulty demonstrating that they have properly planned the actions needed, and are tracking progress, to accomplish their APGs.

For 63 of the 102 APGs, agencies identified on Performance.gov clearly- defined milestones for both the near term (presented as “Next Steps,” with a scheduled completion date in the next fiscal quarter) and longer term (presented as “Future Actions,” covering the remainder of the goal period). Figure 11 provides an illustrative example of an APG with both near-term and longer-term milestones. For its goal to improve awareness of VA services and benefits by increasing the timeliness and relevance of online information available to veterans, service members, and eligible beneficiaries, VA provided milestones scheduled for completion in the second quarter of fiscal year 2013 (near term) and the fourth quarter of fiscal year 2013 (the end of the goal period).

Agencies did not always identify the quarterly milestones they planned to accomplish in order to achieve their APGs during the 2-year goal period. Furthermore, the presentation of information about milestones on Performance.gov does not always convey the time frames for expected action. For the remaining 39 goals, agencies did not provide specific completion dates in discussions of near-term or longer-term plans (or in some cases both) for accomplishing the goal. As figure 12 illustrates, the Small Business Administration (SBA) provided planned actions it intends to take in the near term and longer term to help accomplish its APG to process disaster applications efficiently. However, it is unclear when SBA intends to complete these actions.

OMB’s 2012 A-11 guidance does not adequately reflect that clearly- defined milestones should have scheduled completion dates and be publicly reported. The guidance states that APGs must have quarterly milestones to track progress, and it outlines the time frames that near- term and longer-term milestones should cover. However, the guidance does not state that agencies should provide specific completion dates for their milestones. In addition, contrary to GPRAMA, the guidance also states that agencies’ presentations of near-term milestones in the quarterly updates on Performance.gov are optional. When we asked OMB staff about this, they agreed that the designation of near-term milestones as optional for the quarterly updates was an error in A-11 guidance; it should have been required. They told us they intend to correct this error in the 2013 A-11 guidance. OMB staff further stated that OMB has communicated to agencies that near-term milestones are to be included in quarterly updates to Performance.gov in other ways. For example, the Performance.gov user guide states that “agencies will summarize how they plan to improve progress…and will include key milestones planned” for the near term, as part of the “Next Steps” portion of the APG information. Without clear and consistent guidance about developing and publishing milestones with clear completion dates, agencies may continue to omit key information about the actions they plan to undertake to accomplish their goals.

Most Agencies Did Not Describe How Their Priority Goals Reflect Congressional Input

Only 1 of the 24 agencies that developed APGs described how those goals reflect input from congressional consultations. GPRAMA states that APGs are to reflect the highest priorities of the agency as determined by the head of the agency and informed by consultations with Congress. Agencies are to consult with their relevant appropriations, authorization, and oversight committees when developing or making adjustments to their strategic plans, including their APGs, at least once every 2 years. Regarding this requirement, OMB’s guidance highlights that agencies should specifically consult with Congress on priority goal issue areas, and suggests agencies could start discussions of their next set of priority goals in the context of providing Congress an update on progress on the current APGs.

The act also requires agencies to describe how input provided during congressional consultations was incorporated for each agency priority goal on Performance.gov. In addition, agencies are to similarly describe in their strategic plans how input from congressional consultations was incorporated into their goals. Without this information, it will be difficult to know whether an agency’s goals reflect congressional input, and therefore if the goals will provide useful information for congressional decision making.

In the December 2012 update to Performance.gov, agencies provided information about how they engaged stakeholders during their goal development processes. Although 19 agencies stated that they included Congress as part of their stakeholder engagement, only SBA provided information about the input it received on its APGs from those consultations, as shown in figure 13.

Two agencies, DOD and DOT, broadly mentioned that congressional input on agency goals was incorporated as appropriate. Education took a different approach and provided information about how it engaged stakeholders, including Congress in several instances, for each of its APGs. However, none of these agencies provided specific information on the input that was received or how it was incorporated.

Several agencies also provided broad descriptions of their consultations in their strategic plans. For example, VA states in its plan that in November 2011 it initiated the process for consulting with Congress regarding the development of its agency priority goals and the VA strategic plan. Additionally, in the SSA strategic plan, the agency mentioned developing its plan in consultation with employees, stakeholders, advisory groups, and Congress. However, in none of these instances did agencies provide any further details about how these consultations influenced their strategic plans, including their APGs.

OMB’s 2011 guidance, which covered the development of APGs for 2012 to 2013, stated that agencies should consult with Congress on priority OMB staff told goal issue areas, prior to submitting draft goals to OMB.us that agencies formed a working group on consultations and OMB staff worked with agencies on how to do the consultations well. However, OMB staff also told us that agencies were generally not comfortable publishing the input they received from Congress during their consultation for a variety of reasons, such as a reluctance to characterize competing or conflicting congressional interests.

However, without such information, it is unclear that agencies have adequately engaged Congress and appropriately incorporated congressional feedback into their APGs. The consultation process was established by GPRA in 1993 so that agencies could take congressional views into account as appropriate. But as noted in the Senate committee report that accompanied the bill that ultimately became GPRAMA, little evidence existed that agencies had formally or significantly considered the input of key stakeholders when developing goals. The requirement for agencies to describe how congressional input was incorporated into their goals was intended to strengthen the consultation process.

Our past work has noted the importance of considering Congress a partner in shaping agency goals. Successful consultations can create a basic understanding among stakeholders of the competing demands that confront most agencies, the limited resources available to them, and how those demands and resources require careful and continuous balancing. We have also reported that agency consultations with Congress on the identification of priority goals presents an opportunity to develop such an understanding, especially given Congress’s role in setting national Although priorities and allocating the resources to achieve them.constructive communication across the branches of government can prove difficult, it is essential for sustaining federal performance improvement efforts. Consultations provide an important opportunity for Congress and the executive branch to work together to ensure that agency missions are focused; goals are specific, results-oriented, and address congressional concerns about performance; and strategies and funding expectations are appropriate and reasonable.

Past Audit Work Provides Perspective on Selected Agencies’ Capacities to Achieve Their Priority Goals

Our prior audit work and in some instances that of relevant agency inspectors general provide additional perspective on the capacity of DHS, DOT, HUD, OPM, and VA to achieve their APGs. Our comments on each of these goals can be found in Appendixes II (DHS), III (HUD), IV (DOT), V (VA), and VI (OPM). Given the breadth of issues dealt with by these goals, our comments for each goal cover a range of topics.

Despite this variation, several goals suggested that agencies continue to grapple with a common challenge from our past work related to measuring progress toward their goals and ensuring that the related performance data are accurate. For example, related to both HUD’s and VA’s APGs to assist in housing and reducing the number of homeless veterans, we have previously reported that HUD and VA lack key data on the population of homeless women veterans, including their characteristics and needs. This hampers VA’s ability to plan services effectively. In December 2011, we recommended that HUD and VA should collaborate to ensure appropriate data are collected and use these data to strategically plan for services. VA concurred with this recommendation and, in April 2013, stated that it had taken additional actions to inform policy and operational decisions about homeless and at- risk women veterans. For example, VA stated that it worked with HUD to ensure that gender specific data were collected during the 2013 Point in Time count of homeless persons. In another example related to DOT’s APG to reduce roadway fatalities, our past work has indicated that the quality of state traffic safety data systems varied across the six data systems maintained by states. In April 2010, we recommended that the National Highway Traffic Safety Administration (NHTSA) take steps to ensure that traffic records assessments provide an in-depth evaluation that is complete and consistent in addressing quality across all state traffic safety data systems. In response, NHTSA has taken a number of steps intended to improve the quality of the assessments and the data systems, and as of spring 2013, these efforts continue.

Conclusions

Many of the meaningful results that the federal government seeks to achieve cannot be realized without effective coordination and collaboration both within and across agencies. Recognizing this, Congress and the executive branch established a new crosscutting and integrated approach for focusing on results and improving government performance with the passage and enactment of GPRAMA. The act’s requirements related to the development of APGs, along with more frequent reviewing and reporting of progress towards them, have the potential to address crosscutting and other federal performance management challenges our past work has identified.

OMB and the PIC provided significant support to agencies during their development of APGs. For example, in 2011 and 2012, OMB developed and provided to agencies detailed guidance and memorandums to explain GPRAMA’s requirements and OMB’s expectations for APGs. In addition, the PIC formed the Goal Setting Working Group to assist agencies in setting their 2012 to 2013 APGs, and developed draft guides related to selecting APGs and developing milestones. Given these past efforts, both OMB and the PIC will have an important role moving forward to help ensure that agencies fully develop their APGs.

Agencies have implemented key provisions related to their APGs. However, they have not always provided information about coordination and collaboration for crosscutting efforts. Agencies also did not always identify external contributors to their APGs. OMB’s review process for publishing APG information on Performance.gov checks to make sure agencies have identified at least one contributor, but it does not verify that agencies have identified all appropriate contributors. In addition, most agencies did not describe how their APGs contribute to CAP goals. OMB’s guidance does not adequately reflect that agencies should describe this linkage. Revised guidance could help ensure that agencies are aware of this requirement and provide information accordingly. Further, without providing information about external contributors or how APGs contribute to CAP goals, it is unclear whether agencies have adequately planned to address performance that spans multiple organizations, thereby putting these efforts at risk for duplication, overlap, and fragmentation and potentially wasting scarce funds and limiting the effectiveness of federal efforts.

The requirement for agencies to review progress made toward their APGs on a quarterly basis is intended to increase agencies’ use of the significant amount of performance information they collect. This, in turn, can help agencies to improve their performance and results in a more timely manner—a challenge our work has previously highlighted. Agencies generally developed performance measures, and collect information more frequently than in the past. This shows promise for their ability to use this information to support more timely decision making, especially when improvements are needed. However, agencies did not always identify related interim performance targets. This could be because OMB’s A-11 guidance does not mention the interim quarterly performance target requirement or define when it would provide data of significant value and therefore be required, although other OMB guidance directs agencies to develop these targets. By revising its guidance documents to consistently include this information, OMB could help ensure that agencies are meeting its expectation (and the requirement) that agencies identify interim performance targets for their APGs when doing so would provide data of significant value. Without clear targets, which enable a comparison of results against planned performance, it is unclear if agency managers have the information they need to determine if they are making sufficient progress toward each APG—a practice our past work has shown can actually lead to increased use of information and improved results. In addition, by not providing targets, key stakeholders have little assurance that an agency is actively managing its performance to make progress towards its APGs on a quarterly basis, thereby limiting oversight and accountability opportunities.

GPRAMA requires agencies to develop clearly defined quarterly milestones for their APGs, which can help demonstrate that agencies have identified concrete actions needed to accomplish their goals and when those actions should be taken. However, agencies did not consistently publish milestones with scheduled completion dates, thereby missing an opportunity to assure the public and key stakeholders that they have appropriate strategies in place to achieve their APGs. Although OMB’s 2012 A-11 guidance directs agencies to develop quarterly milestones for their APGs and outlines near-term and longer-term timeframes those milestones should cover, the guidance does not state that agencies should provide specific completion dates for their milestones. In addition, the guidance does not adequately reflect that GPRAMA requires these milestones to be published on Performance.gov due to an error. Although OMB has provided additional direction to agencies about publishing milestones, revising its A-11 guidance to correct the error would ensure that its direction to agencies is consistent and clear.

Agencies should consult with Congress as a partner in developing their goals, in part to ensure that the resulting performance information is useful for congressional and executive branch decision making. Agencies’ consultations with Congress on their APGs provide opportunities for both parties to gain a better understanding of the competing demands that both confront and how those demands and limited resources require careful and continuous balancing. However, most agencies did not provide information about how they incorporated any views or suggestions obtained through congressional consultations when developing their goals. This lack of information leaves it unclear as to whether agencies made serious attempts to engage with Congress on identifying the agencies’ highest priorities for improved performance.

Recommendations for Executive Action

To ensure that agencies fully develop their APGs, we make the following seven recommendations to the Director of OMB.

To ensure that agencies can (1) compare actual results to planned performance on a more frequent basis, as appropriate, and (2) demonstrate how they plan to accomplish their goals as well as contribute to the accomplishment of broader federal efforts, we recommend the Director of OMB revise relevant guidance documents to provide a definition of what constitutes “data of significant value;” direct agencies to develop and publish on Performance.gov interim quarterly performance targets for their APG performance measures when the above definition applies; direct agencies to provide and publish on Performance.gov completion dates, both in the near term and longer term, for their milestones; and direct agencies to describe in their performance plans how the agency’s performance goals—including APGs—contribute to any of the CAP goals.

When such revisions are made, we recommend the Director of OMB work with the PIC to test and implement these provisions.

In addition, as OMB works with agencies to enhance Performance.gov to include additional information about APGs, we recommend that the Director of OMB ensure that agencies adhere to OMB’s guidance for website updates by providing complete information about the organizations, program activities, regulations, tax expenditures, policies, and other activities—both within and external to the agency—that contribute to each APG; and a description of how input from congressional consultations was incorporated into each APG.

Agency Comments and Our Evaluation

We provided a draft of this report for review and comment to the Director of OMB and the five agencies covered by our in-depth review of APGs (DHS, DOT, HUD, VA, and OPM). All six agencies provided technical comments, which we incorporated as appropriate.

In oral comments, staff from OMB’s Office of Performance and Personnel Management agreed with the recommendations in our report.

In its written comments, reproduced in appendix VII, VA concurred with the conclusions of our report and provided additional information about its strategic plan, and related performance measurement efforts, to reduce its backlog of compensation claims. However, as VA acknowledges in its comments, the plan does not provide individual performance goals and metrics for all initiatives, which we believe are necessary for VA to ensure it is spending its limited resources on proven methods to speed up disability claims and appeals processes.

We also sought comments from relevant agencies covered by the illustrative examples used in this report. We received such comments from Commerce, NSF, SBA, State, and USAID, and incorporated them as appropriate.

We are sending copies of this report to the Acting Director of OMB and the heads of the 24 agencies that developed APGs as well as interested congressional committees and other interested parties. This report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6806, or mihmj@gao.gov. Specific questions about our comments on the sample of APGs contained in appendixes II through VI may be directed to the contact listed for each goal. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix VIII.

Appendix I: Objectives, Scope & Methodology

The GPRA Modernization Act of 2010 (GPRAMA) requires us to review the act’s implementation at several critical junctures, and this report is part of a series of reviews planned around the requirement. Our specific objectives for this report were to (1) examine the extent to which agencies have implemented selected planning and reporting requirements and leading practices related to agency priority goals (APG); and (2) comment on the priority goals of several selected agencies based on our prior work and that of relevant agency inspectors general (IGs) and identify our relevant open recommendations and matters for congressional consideration.

To address both objectives, we reviewed information about the APGs published on Performance.gov in February 2012 and updated in December 2012, as well as the updated strategic plans and performance plans agencies published in 2012 to reflect GPRAMA requirements. To assess the reliability of information presented on Performance.gov we reviewed relevant documentation and interviewed Office of Management and Budget (OMB) staff about data quality control procedures. We determined that the data were sufficiently reliable for the purposes of this report.

In addition, for the first objective, we reviewed and assessed the implementation of selected planning and reporting requirements for 102 of the 103 APGs developed by the 24 agencies selected by OMB and that were released concurrently with the President’s fiscal year 2013 budget on Performance.gov.requirements we used to assess implementation included whether the goal: (1) supports a federal government priority goal (also known as cross-agency priority or CAP goals); (2) contributes to agency strategic goals; (3) reflects input from congressional consultations; (4) identifies the federal organizations, program activities, regulations, policies, and other activities—both within and external to the agency—that contribute to the APG; (5) has a clearly identified agency official as the goal leader; (6) has targets for a 2-year timeframe; (7) has interim quarterly targets; and (8)

The key GPRAMA planning and reporting has clearly defined quarterly milestones. In addition to the requirements of GPRAMA, our assessment of the extent of implementation was also informed by the Senate committee report accompanying GPRAMA, relevant OMB guidance, and our past work on how to effectively implement GPRA.

To address the second objective, we selected 5 of the 24 agencies that developed APGs—the Departments of Homeland Security (DHS), Housing and Urban Development (HUD), Transportation (DOT), and Veterans Affairs (VA), and the Office of Personnel Management (OPM)— based on several factors, including the number and variety of types of federal programs involved in achieving the goals, such as direct service, grant, and regulatory programs, and whether the APGs were related to any of the CAP goals. We then reviewed the work that we and relevant IGs have conducted over a number of years related to each of the 21 APGs developed by the 5 agencies. Because the 21 APGs are a non- generalizable sample of all APGs, our views on those APGs cannot be generalized to the entire universe but provided insights about each of the 21 APGs, as well as a theme common to several APGs. We also updated the status of related key open recommendations and matters for congressional consideration We conducted our performance audit from July 2012 to April 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments on the Department of Homeland Security’s Priority Goals

Based on our past work, as well as that of the DHS IG, we commented on each of DHS’s three priority goals for 2012 to 2013: 1. Ensure resilience to disasters by strengthening disaster preparedness and response capabilities. By September 30, 2013, every state will have a current, DHS-certified threat, hazard, identification and risk assessment (THIRA). 2. Improve the efficiency of the process to detain and remove criminal aliens from the United States. By September 30, 2013, reduce the average length of stay in immigration detention of all convicted criminal aliens prior to their removal from the country by 5 percent. 3. Strengthen aviation security counterterrorism capabilities by using intelligence driven information and risk-based decisions. By September 30, 2013, the Transportation Security Administration will expand the use of risk-based security initiatives to double the number of passengers going through expedited screening at airports, thereby enhancing the passenger experience.

For each goal, we also identify our related past reports and provide an update on the status of any open recommendations and matters for congressional consideration that we previously made related to the goal. We also identify a GAO contact for our work related to each goal.

Goal Statement

Ensure resilience to disasters by strengthening disaster preparedness and response capabilities. By September 30, 2013, every state will have a current, DHS-certified threat, hazard, identification, and risk assessment.

GAO Comments

Our past work has identified a number of challenges DHS faces in achieving its goal of strengthening disaster preparedness and response capabilities including challenges associated with efforts to measure national preparedness capabilities and assess the impact of preparedness grant funding. These efforts involve the Federal Emergency Management Agency’s (FEMA) preparedness grants.

FEMA provides state and local governments with funding in the form of grants to enhance the capacity of state and local emergency responders to prevent where possible, respond to, and recover from natural disasters and terrorism incidents involving chemical, biological, radiological, nuclear, or explosive devices, or cyber attacks. States and urban areas are required to conduct a THIRA as a condition of receiving preparedness grant funding under programs including the State Homeland Security Program, Emergency Management Performance Grant Program and Urban Area Security Initiative grant program.

Department of Homeland Security, Federal Emergency Management Agency, Threat and Hazard Identification and Risk Assessment Guide Comprehensive Preparedness Guide (CPG) 201 First Edition, (Washington, D.C., April 2012.) completed their THIRAs. FEMA granted 6-month extensions to the December 31, 2012, deadline for five states and three local urban areas affected by Hurricane Sandy in late October 2012.

Status of Key Open Recommendations and Matters for Congressional Consideration

In March 2011, we suggested that Congress may wish to consider limiting preparedness grant funding to maintaining existing capabilities (as determined by FEMA) until FEMA completes a national preparedness assessment of capability gaps at each level of government based on tiered, capability-specific performance objectives to enable prioritization of grant funding. In April 2011, Congress passed the Continuing Appropriations Act that reduced funding for FEMA preparedness grants by $875 million from the amount requested in the President’s fiscal year 2011 budget. In December 2011, Congress passed the Consolidated Appropriations Act for fiscal year 2012 that reduced funding for FEMA preparedness grants by $1.28 billion from the amount requested in the President’s fiscal year 2012 budget.

In March 2011, we also suggested that FEMA should complete a national preparedness assessment of capability gaps at each level based on tiered, capability-specific performance objectives to enable prioritization of grant funding, and FEMA could identify the potential costs for establishing and maintaining those capabilities at each level and determine what capabilities federal agencies should provide. In June 2012, the DHS OIG reported that FEMA did not have a system in place to determine the extent that Homeland Security Grant Program funds enhanced the states’ capabilities to prevent, deter, respond to, and recover from terrorist attacks, major disasters, and other emergencies before awarding more funds to the states. As of March 2013, FEMA has not yet completed a national preparedness assessment of capability gaps at each level. According to FEMA officials, the urban area, state, territorial, and tribal nation THIRAs that were due December 31, 2012 will serve as the basis for assessing national preparedness capabilities and gaps. FEMA will coordinate the review and analysis by a THIRA Analysis and Review Team. The team has begun meetings to discuss the common themes and findings and develop an initial proposed list of priorities for building and sustaining the core capabilities and update the proposed list of priorities as needed. These actions are part of the overall process for THIRA analysis and review, which FEMA officials said will help them develop guidance for developing capabilities to meet national priorities.

In July 2009, we recommended that the FEMA Administrator should develop and implement measures to assess how regional collaboration efforts funded by Urban Area Security Initiative grants build preparedness capabilities. FEMA contracted the National Academy of Public Administration to provide recommendations for quantifiable performance measures to assess the effectiveness of the State Homeland Security Program and Urban Area Security Initiative grants. The National Academy of Public Administration issued its report in October 2011 and FEMA released the report in April 2012. The report recommends that FEMA conduct an assessment of collaborative approaches, in coordination with local jurisdictions, states, regions, and urban areas, and use the results to develop a scoring system for future quantitative or qualitative performance measures on collaboration. As of March 2013, FEMA has not yet taken action in response to this recommendation. However, according to FEMA officials, the THIRA process, along with planned coordination meetings with urban area, state, tribal, and territorial officials will likely result in data that they can use to develop collaboration- related performance metrics.

FEMA indentified the “percent of high priority core planning capabilities rated as proficient by states and territories” as a measure of the agency’s progress in achieving its priority goal. This measure reports the percent of high priority core capabilities related to planning that states and territories rate as proficient. According to FEMA, this information is gathered from the SPRs (annual self-assessments by states and territories of their levels of preparedness.) However, as we reported in October 2010, FEMA officials stated, while the SPRs had enabled FEMA to gather data on the progress, capabilities, and accomplishments of a state’s, the District of Columbia’s, or a territory’s preparedness program, these reports include self-reported data that may be subject to interpretation by the reporting organizations in each state and not be readily comparable to other states’ data. The officials also stated that they had taken steps to address these limitations, for example by creating a web-based survey tool to provide a more standardized way of collecting state preparedness information that will help them validate the information by comparing it across states. However, since April 2009, FEMA has made limited progress in assessing preparedness and capabilities and has not yet developed national preparedness capability requirements based on established metrics to provide a framework for these assessments, as we reported in March 2012.

Selected GAO Reports

National Preparedness: FEMA Has Made Progress in Improving Grant Management and Assessing Capabilities, but Challenges Remain. GAO- 13-456T. Washington, D.C.: March 19, 2013.

Managing Preparedness Grants and Assessing National Capabilities: Continuing Challenges Impede FEMA’s Progress. GAO-12-526T. Washington, D.C.: March 20, 2012.

Follow-up on 2011 Report: Status of Actions Taken to Reduce Duplication, Overlap, and Fragmentation, Save Tax Dollars, and Enhance Revenue. GAO-12-453SP. Washington, D.C.: February 28, 2012.

Government Operations: Opportunities to Reduce Potential Duplication in Government Programs, Save Tax Dollars, and Enhance Revenue. GAO-11-318SP. Washington, D.C.: March 1, 2011.

FEMA Has Made Limited Progress in Efforts to Develop and Implement a System to Assess National Preparedness Capabilities. GAO-11-51R. Washington, D.C.: October 29, 2010.

Urban Area Security Initiative: FEMA Lacks Measures to Assess How Regional Collaboration Efforts Build Preparedness Capabilities. GAO-09-651. Washington, D.C.: July 2, 2009.

National Preparedness: FEMA Has Made Progress, but Needs to Complete and Integrate Planning, Exercise, and Assessment Efforts. GAO-09-369. Washington, D.C.: April 30, 2009.

GAO Contact

Stephen L. Caldwell, Director, Homeland Security and Justice Issues, caldwells@gao.gov, (202) 512-8777.

Goal Statement

Improve the efficiency of the process to detain and remove criminal aliens from the United States. By September 30, 2013, reduce the average length of stay in immigration detention of all convicted criminal aliens prior to their removal from the country by 5 percent.

GAO Comments

Our past work does not provide a basis to assess DHS’s ability to improve the efficiency of the process to detain and remove criminal aliens from the United States. DHS has reported progress toward achieving this priority goal since fiscal year 2010. The DHS annual performance report for fiscal years 2011 to 2013—which also serves as the agency’s annual performance plan—showed that the agency reduced the length of stay in detention of all convicted criminal aliens prior to removal from the United States from 37 days in fiscal year 2010 to 34.7 days in fiscal year 2011. This represents a decline of over 6 percent compared to the priority goal target to reduce the length of stay by 5 percent. DHS attributed this decrease in part to expanded detention capacity in locations where detainee transfers occur most often, as the need to transfer a detainee from a facility in one location to another location increased the average length of stay by approximately 14 days. The DHS strategic plan for fiscal years 2012 to 2016 shows planned targets to maintain the same number of days (35) in fiscal years 2012, 2013, and 2016. Similarly, the DHS annual performance report for fiscal years 2011 to 2013 shows the same planned target of less than or equal to 35 days for both fiscal year 2012 and fiscal year 2013. DHS reported that it continues to focus on the development of a detention system that has the right number and type of facilities in the right locations to align with enforcement and removal activities. DHS reported making improvements in prior years, but noted that various challenges, such as case backlogs, could inhibit success in achieving further improvements in length of stay requirements. For example, maintaining the average length of stay for criminal aliens at, or slightly below, 35 days in the long term may require the hiring of additional immigration judges, according to DHS.

Status of Key Open Recommendations and Matters for Congressional Consideration

We currently have no open recommendations or matters for congressional consideration related to this priority goal.

Selected GAO Reports

We currently have no reports related to this priority goal.

GAO Contact

Rebecca Gambler, Director, Homeland Security and Justice Issues, gamblerr@gao.gov, (202) 512-6912.

Goal Statement

Strengthen aviation security counterterrorism capabilities by using intelligence driven information and risk-based decisions. By September 30, 2013, the Transportation Security Administration (TSA) will expand the use of risk-based security initiatives to double the number of passengers going through expedited screening at airports, thereby enhancing the passenger experience.

GAO Comments

TSA’s goal is stated in broad terms; consequently, quantitatively measuring progress toward meeting the goal of strengthening aviation security counterterrorism capabilities will be a challenge. Further, the stated performance measure, with its focus on expedited passenger screening, will not allow TSA to assess its progress in using intelligence driven information and risk-based decisions to meet this goal in other related areas, such as in screening checked baggage or air cargo. TSA, as a component of DHS, relies upon multiple layers of security to deter, detect, and disrupt persons posing a potential risk to aviation security. These layers focus on screening millions of passengers and pieces of carry-on and checked baggage, as well as tons of air cargo, on a daily basis. Our past work has analyzed TSA’s progress in implementing these security measures and identified challenges it has encountered in implementing cost-effective aviation security programs and measuring performance.

To help achieve its priority goal of strengthening aviation security counterterrorism capabilities by using intelligence driven information and risk-based decisions, TSA officials stated that the agency will, among other steps, expand the use of its new “TSA Pre✓™” program to double the number of passengers going through expedited screening at airports by September 30, 2013. TSA introduced TSA Pre✓™ in October 2011, and plans on expanding it to 40 airports by March 2013. Based on current participation, frequent flyers of five airlines as well as individuals enrolled in other departmental trusted traveler programs—where passengers are pre-vetted and deemed a trusted traveler—are eligible to be screened on an expedited basis. This program is intended to allow TSA to focus its resources on higher risk travelers. Agency officials have reported that with the deployment of this program and other risk-based security initiatives, such as modifying screening procedures for passengers 75 and over and active duty service members, TSA has achieved its stated goal of doubling the number of passengers going through expedited screening. According to TSA, by the end of calendar year 2013, TSA will provide expedited screening to 25 percent of the individuals currently processed through security screening. Achieving this target will mean that approximately 450,000 of the 1.8 million passengers who travel on average each day from the nation’s airports will undergo some form of expedited screening. However, since this goal is focused on passenger screening, it will not allow TSA to assess its progress in using intelligence driven information and risk-based decisions in other areas to achieve the broader outcome of strengthening aviation security counterterrorism capabilities, such as in screening checked baggage or air cargo. We plan to initiate a review of TSA’s progress in implementing TSA Pre✓™ in 2013.

In our past work, we found that TSA has taken steps to implement aviation security mechanisms that are more intelligence-driven and risk- based. For example, TSA implemented the Secure Flight program to allow it to focus resources on high risk passengers by vetting passengers’ names, dates of birth, and other information against terrorist watch lists. In May 2009, we reported that TSA had made significant progress in developing the Secure Flight program but also noted that it faced challenges in identifying passengers who might use false identifying information. We also assessed TSA’s efforts to implement a behavior detection program that seeks to selectively identify potentially high-risk passengers for additional screening. Our May 2010 report found that while TSA has taken actions to validate the science underlying the program and improve performance measurement, among other actions, more work remains to ensure the program’s effectiveness, such as developing comprehensive program performance measures. In March 2012, we reported that questions related to the program will remain until TSA demonstrates that using behavior detection techniques can help secure the aviation system against terrorist threats. TSA plans to or is currently implementing a number of other behavior based programs that we plan to report on in 2013.

Status of Key Open Recommendations and Matters for Congressional Consideration

In November 2012, we recommended that TSA take steps to improve its oversight of air passenger screening complaint processes, by establishing (1) consistent policies for receiving complaints and informing passengers about complaint processes, (2) a process to systematically analyze information on complaints, and (3) a focal point to coordinate these efforts. In its comments on this report, DHS concurred with the recommendations and stated that TSA is taking steps to implement them.

In May 2012, we recommended that to help DHS address challenges in meeting the air cargo screening mandate as it applies to air cargo carried on passenger flights inbound to the United States, mitigate potential air cargo security vulnerabilities, and enhance overall efforts to screen and secure inbound air cargo, the Secretary of Homeland Security should direct the Administrator of TSA to assess the costs and benefits of requiring all-cargo carriers to report data on the amount of inbound air cargo screening being conducted. In comments on the May 2012 report, DHS concurred with the recommendation and stated that TSA was working on developing a system that will provide the capability for all- cargo carriers to report data on screened high-risk inbound air cargo shipments. In April 2013, TSA reported that once this system becomes fully operational, these data will be available for each all-cargo carrier.

In July 2011, we recommended that TSA develop a plan to deploy explosives detection systems (EDS) that meet the most recent explosives-detection requirements and ensure that new machines, as well as machines deployed in airports, will be operated at the levels established in those requirements. This plan should include the estimated costs for new machines and upgrading deployed machines, and the time frames for procuring and deploying new machines. In commenting on this report, DHS concurred with the recommendation. As of March 2013, TSA has a plan in place to evaluate and implement the most recent certified algorithms on the existing fleet of deployed EDSs. However, our recommendation calls for a plan to deploy new EDSs as well as to upgrade existing EDSs in airports to meet the 2010 EDS explosives detection requirements. Our recommendation was intended to ensure that all EDSs operating in airports meet the most recent requirements, which are currently the 2010 requirements. Consequently, we continue to believe that a plan is needed that describes the approach that TSA will use to deploy EDSs that meet the most recent explosives detection requirements and ensure that all deployed machines will be operated at the levels established in the latest requirements.

In May 2010, we recommended that TSA perform a cost benefit analysis of TSA’s behavior detection program known as the Screening of Passengers by Observation Techniques (SPOT), including a comparison of the program with other security screening programs, such as random screening, or other already existing security measures. In commenting on this report, DHS concurred with the recommendation and TSA completed the analysis in December 2012. We are evaluating the cost-benefit analysis as part of ongoing work that we will report on in 2013.

In May 2010, we recommended that TSA take steps to better measure the effectiveness of the SPOT program and evaluate the performance of TSA’s behavior detection officers, who implement the program at TSA- regulated airports. We also recommended that TSA establish a plan that includes objectives, milestones, and time frames to develop outcome- oriented performance measures to help refine the current methods used by behavior detection officers for identifying individuals who may pose a risk to the aviation system. In commenting on this report, DHS concurred with the recommendation and completed its plan in November 2012. We are evaluating the plan as part of ongoing work that we will report on in 2013.

Selected GAO Reports

Air Passenger Screening: Transportation Security Administration Could Improve Complaint Processes. GAO-13-43. Washington, D.C.: November 15, 2012.

Aviation Security: Actions Needed to Address Challenges and Potential Vulnerabilities Related to Securing Inbound Air Cargo. GAO-12-632. Washington, D.C.: May 10, 2012.

Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for Checked Baggage, but Additional Screening Actions Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011.

Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges. GAO-10-763. Washington, D.C.: May 20, 2010.

Aviation Security: TSA Has Completed Key Activities Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate Risks. GAO-09-292. Washington, D.C.: May 13, 2009.

GAO Contact

Stephen M. Lord, Director, Homeland Security and Justice Issues, lords@gao.gov, (202) 512-4379.

Appendix III: Comments on the Department of Housing and Urban Development’s Priority Goals

Based on our past work, as well as that of the HUD IG, we commented on each of HUD’s six priority goals for 2012 to 2013: 1. Improve program effectiveness by awarding funds fairly and quickly.

By September 30, 2013, HUD will improve internal processes to ensure that HUD can obligate 90 percent of Notice of Funding Availability (NOFA) programs within 180 calendar days from budget passage, ensuring that America’s neediest families have the shelter and services they need, when they need them. The timely obligation and subsequent disbursement of funds will positively impact the agency’s ability to achieve all of its priority goals. 2. Increase the energy efficiency and health of the nation’s housing stock. By September 30, 2013, HUD will enable a total of 159,000 cost effective energy efficient or healthy housing units, as part of a joint HUD-Department of Energy (DOE) goal of 520,000 in 2012 to 2013 and a total goal of 1.2 million units from 2010 through 2013. 3. Preserve affordable rental housing. By September 30, 2013, preserve affordable rental housing by continuing to serve 5.4 million families and serve an additional 61,000 families through HUD’s affordable rental housing programs. 4. Prevent foreclosures. By September 30, 2013, assist 700,000 homeowners who are at risk of losing their homes due to foreclosure. 5. Reduce vacancy rates. By September 30, 2013 reduce the average residential vacancy rate in 70 percent of the neighborhoods hardest hit by the foreclosure crisis relative to comparable areas. Hardest hit neighborhoods are defined as Neighborhood Stabilization Program (NSP) 2 Neighborhood Investment Clusters (NIC). 6. Reducing homelessness. By September 30 2013, in partnership with the VA, reduce the number of homeless veterans to 35,000 by serving 35,500 additional homeless veterans. HUD is also committed to making progress towards reducing family and chronic homelessness and is working towards milestones to allow for tracking of these populations.

For each goal, we also identify our related past reports and provide an update on the status of any open recommendations and matters for congressional consideration that we previously made related to the goal. We also identify a GAO contact for our work related to each goal.

Goal Statement

Improve program effectiveness by awarding funds fairly and quickly. By September 30, 2013, HUD will improve internal processes to ensure that HUD can obligate 90 percent of NOFA programs within 180 calendar days from budget passage, ensuring that America’s neediest families have the shelter and services they need, when they need them. The timely obligation and subsequent disbursement of funds will positively impact the agency’s ability to achieve all of its priority goals.

GAO Comments

Although we have not conducted an in-depth analysis of HUD’s NOFA processes, our recent work and a recent bid protest decision highlight some of the challenges HUD has faced when trying to award funds quickly and the importance of using appropriate processes to award funds. In our bimonthly reviews of selected states’ and localities’ use of funds made available under the American Recovery and Reinvestment Act of 2009 (Recovery Act), we commented on the NOFA process HUD used to award nearly $1 billion in public housing capital funds to public housing agencies based on competition for priority investments, including investments that leveraged private sector funding or financing for renovations and energy conservation retrofit investments. In September 2009, we reported that HUD had received almost 1,800 applications for the funds and that its review process had been slower than expected. According to HUD officials, this was due to the number of applications with lengthy narratives needing review. Further, HUD officials stated that their staff were reviewing these applications while carrying out their ongoing responsibilities related to managing the public housing capital fund program. Despite these challenges, we reported in December 2009 that HUD had met the Recovery Act requirement to obligate all of the funds to public housing agencies by September 30, 2009. Specifically, HUD accepted applications from June 22 to August 18, 2009, and according to a HUD official, 746 housing agencies submitted 1,817 applications for these competitive grants. In September 2009, HUD awarded 396 competitive grants to housing agencies that successfully addressed the NOFA requirements.

In addition, a recent bid protest decision highlights the importance of using appropriate processes to award funds regardless of the time involved. On August 15, 2012, we concluded that HUD’s use of a NOFA that resulted in the issuance of a cooperative agreement to obtain services for the administration of Project-Based Section 8 Housing Assistance Payment contracts was improper because the “principal purpose” of the NOFA was to obtain contract administration services for HUD’s direct benefit and use, which should be acquired under a procurement instrument that results in the award of a contract.

Status of Key Open Recommendations and Matters for Congressional Consideration

In our August 15, 2012, bid protest decision, we recommended that HUD cancel the NOFA and solicit the contract administration services for the Project-Based Section 8 rental assistance program through a procurement instrument that would result in the award of contracts. In its response, HUD informed us of its intention to proceed with the NOFA and of its plan to make awards. However, as a result of litigation filed in the Court of Federal Claims that sought to enjoin it from proceeding with the NOFA, HUD announced its agreement not to make the awards until the court rules on the matter. As of March 2013, the court had not yet issued its decision.

Selected GAO Reports

Assisted Housing Services Corporation; North Tampa Housing Development Corporation; The Jefferson County Assisted Housing Corporation; National Housing Compliance; Southwest Housing Compliance Corporation; CMS Contract Management Services and the Housing Authority of the City of Bremerton; Massachusetts Housing Finance Agency. B-406738 et al. August 15, 2012.

Recovery Act: Status of States’ and Localities’ Use of Funds and Efforts to Ensure Accountability. GAO-10-231. Washington, D.C.: December 10, 2009.

Recovery Act: Funds Continue to Provide Fiscal Relief to States and Localities, While Accountability and Reporting Challenges Need to Be Fully Addressed. GAO-09-1016. Washington, D.C.: September 23, 2009.

GAO Contact

Mathew J. Scirè, Director, Financial Markets and Community Investment, sciremj@gao.gov, (202) 512-8678.

Goal Statement

Increase the energy efficiency and health of the nation’s housing stock. By September 30, 2013, HUD will enable a total of 159,000 cost effective energy efficient or healthy housing units, as part of a joint HUD-DOE goal of 520,000 in 2012 to 2013 and a total goal of 1.2 million units from 2010 through 2013.

GAO Comments

Energy-efficient green building practices can increase up-front costs but also may provide long-term financial, environmental, and health benefits. In prior work, we credit HUD for using accepted energy-efficient green building standards developed by others, such as the Environmental Protection Agency’s (EPA) Energy Star program and the Enterprise Green Communities, as criteria for measuring progress toward its goal. These standards are generally recognized as effective measures of increased energy efficiency. However, our prior work also found that HUD could do more to promote energy efficiency.

For example, in October 2008, we found that while HUD’s public housing office had shown leadership and initiative in partnering to develop a benchmarking tool that could be used to identify properties with high levels of utility consumption, HUD’s multifamily assisted housing had no such tool. In the absence of such a tool, HUD cannot target certain multifamily properties for green building improvements, which could result in benefits that include reduced resource consumption. In April 2013, HUD officials told us that they were collaborating with other federal agencies and industry partners to develop such a tool for its multifamily portfolio.

Our October 2008 report also identifies ways that HUD could better meet its priority goal for cost-effective energy-efficient measures, particularly for water conservation. HUD officials we interviewed identified water conservation savings as significant and among the biggest potential opportunities for financial savings, but HUD had provided few incentive points for water conservation or indoor air quality measures in its competitive grant programs. Since our report, a number of HUD programs have added water savings devices to requirements for new construction and rehabilitation projects. In addition, the Interagency Rental Policy Working Group, which includes HUD, USDA, and Treasury, has adopted requirements for water saving products and energy star appliances for rehabilitation projects.

As stated above, HUD’s priority goal is a portion of a larger HUD-DOE joint goal. DOE’s weatherization assistance program is one of the largest residential energy-efficiency programs in the nation and some DOE weatherization grantees also received HUD assistance. HUD officials told us that DOE grantees do not report which weatherization recipients received HUD assistance and HUD grantees are not required to report to HUD whether they received weatherization assistance. Consequently, double counting could occur, although HUD indicated that the likelihood of such double-counting is small. In October 2012, HUD officials told us that they were planning to use the results of a DOE evaluation of its weatherization program to identify any double counting and, if necessary, revise the overall HUD-DOE totals reported previously for 2009 and 2010. In April 2013, HUD officials told us that the data collection portion of DOE’s evaluation was complete and they were awaiting the results from DOE.

Status of Key Open Recommendations and Matters for Congressional Consideration

We recommended in October 2008 that HUD ensure the completion of the regulation that would require the use of energy-efficient products and appliances for public housing as directed by the Energy Policy Act of 2005. HUD included the statutory requirement in a proposed rule published in February 2011, but as of March 2013, HUD had not published the final rule.

We also recommended in October 2008 that HUD work with DOE to expeditiously implement energy-efficiency updates to the HUD manufactured housing code. Although manufactured housing is not part of HUD’s agency priority goal, we believe that energy-efficiency efforts in this area are directly related to the goal. Manufactured housing is an area in which HUD has significant influence because it has been responsible for establishing manufactured building code requirements since 1974. We found that HUD had not made significant energy efficiency updates to code for this program since 1994. HUD officials told us that pursuant to the requirements of the Energy Independence and Security Act of 2007 which moved responsibility for promulgating manufactured energy efficiency standards to DOE, they intended to wait to make energy- efficiency updates to the code because they were concerned about overlapping agency responsibilities between DOE and HUD. We concluded in October 2008 that waiting to take action could result in years more of some manufactured homes being built without improved energy standards. HUD has worked with DOE in developing more stringent energy standards for manufactured homes. For example, in February 2010, DOE published an advance notice of proposed rulemaking on energy efficient standards for manufactured homes pursuant to the 2007 Act and HUD officials told us that they met with DOE on the proposal. Until the rule is finalized, HUD and DOE will continue to miss an opportunity to improve the energy efficiency of manufactured housing units.

Additionally, in October 2008 we recommended that HUD work with DOE’s Oak Ridge National Laboratory and EPA to develop a utility benchmarking tool for multifamily properties. We pointed out that HUD’s public housing office had shown leadership and initiative in partnering to develop a utility benchmarking tool that could be used to identify multifamily properties with high levels of utility consumption, and that HUD’s multifamily assisted housing could benefit from a similar tool that would allow properties to be targeted for green building improvements. In October 2012, HUD officials told us that Oak Ridge’s data tool is now out of date. They added that HUD was actively working with DOE, EPA, Fannie Mae, and industry representatives on a strategy to develop common data inputs and reporting standards for multifamily properties that could lead to a multifamily benchmarking tool. In April 2013, HUD officials told us that they are working to develop a multifamily energy star benchmark that will provide information on building performance on a portfolio basis. However, it is not clear when HUD intends to complete its energy start benchmark. Until such a tool is in place and HUD is able to benchmark utility costs in its multifamily portfolio, HUD will continue to miss opportunities to target less-efficient multifamily properties for green building improvements, and reduce resource consumption and utility expenses for itself and its funding recipients.

In November 2011 we recommended that DOE, HUD, and EPA lead an effort to collaborate with other agencies to identify performance information, such as shared goals and common performance measures, for green building initiatives for the nonfederal sector. About one-third of the 94 initiatives we identified have goals and performance measures specific to green building and about two-thirds do not; therefore the results of most initiatives and their related investments in green building are unknown. DOE, HUD, and EPA generally agreed with the recommendation. In November 2012, HUD officials stated that they had met with EPA and DOE representatives to review our recommendation. The agencies generally agreed that initiatives that show potential for collaboration would be best served through existing interagency partnerships. HUD stated that the agencies might explore a higher level of centralized collaboration for the long term, but such efforts would require additional legislative or executive authority to implement.

In October 2012, we found that key standards for manufactured homes provide a lower margin of safety against a carbon monoxide exposure incident than those for site built homes. We found that HUD’s ventilation standards establish standards for airflow, not air quality, and recommended that HUD test the performance of its installed ventilation systems and reassess its ventilation standards. Measuring the actual airflow achieved by installed ventilation systems would not only permit HUD to know whether its standards are being met, but also permit HUD to better understand the potential impact ventilation systems may have on indoor air quality and the overall health of the homes. HUD generally agreed with both recommendations and stated that it would bring them before the Manufactured Housing Consensus Committee, which is responsible for recommending proposed rules to HUD, for consideration.

Selected GAO Reports

Manufactured Housing Standards: Testing and Performance Evaluation Could Better Ensure Safe Indoor Air Quality. GAO-13-52. Washington, D.C.: October 24, 2012. 2012 Annual Report: Opportunities to Reduce Duplication, Overlap and Fragmentation, Achieve Savings, and Enhance Revenue. GAO-12-342SP. Washington, D.C.: February 28, 2012.

Green Building: Federal Initiatives for the Nonfederal Sector Could Benefit from More Interagency Collaboration. GAO-12-79. Washington, D.C.: November 2, 2011.

Green Affordable Housing: HUD Has Made Progress in Promoting Green Building, but Expanding Efforts Could Help Reduce Energy Costs and Benefit Tenants. GAO-09-46. Washington D.C.: October 7, 2008.

GAO Contacts

William B. Shear, Director, Financial Markets and Community Investment, shearw@gao.gov, (202) 512-8678.

Frank Rusco, Director, Natural Resources and Environment, ruscof@gao.gov, (202) 512-3841.

Goal Statement

Preserve affordable rental housing. By September 30, 2013, preserve affordable rental housing by continuing to serve 5.4 million families and serve an additional 61,000 families through HUD’s affordable rental housing programs.

GAO Comments

HUD’s efforts to achieve this goal involve several HUD programs including the housing choice voucher, public housing, and project-based programs, which are HUD’s key programs for delivering rental assistance.factors that could impact HUD’s ability to meet this goal including the department’s ability to keep property owners in the project-based program and serve additional households.

Our past work on these programs has identified a number of HUD has implemented recommendations we made in April 2007 to help enhance HUD’s ability to keep project-based property owners in the program. Specifically, we found that although HUD offered several incentives to keep property owners in the program, some property owners, managers, and industry representatives cited concerns with certain HUD policies and practices, especially the one-to-one replacement policy (which prohibited reductions in the total number of program units in a property when a contract was renewed), and the reimbursement process for operating costs in high cost areas. In 2011, HUD modified this policy and revised the way it calculates reimbursement for operating costs. We also found that between 2001 and 2005 owners renewed 92 percent of contracts and 95 percent of units covered by these contracts. Property owners, managers, and industry representatives with whom we spoke as part of our April 2007 report indicated that market conditions were the primary factors in owners’ decisions to leave or remain in the program. Similarly, HUD has implemented a recommendation we made in November 2005 to address late housing assistance payments to landlords, which may encourage owners to participate in HUD’s project-based program. Specifically, in 2007 HUD made improvements to its data, including verifying information on contract renewal dates and project costs, which should help the department more reliably determine the timing and amount of funding needed by the landlords, thereby improving the timeliness of its payments.

Our work has also identified certain factors that could enhance HUD’s ability to meet its goal to preserve affordable rental housing. For instance, our March 2012 report on the housing choice voucher program identified potential areas that, if implemented, could help HUD reach more renter households. Specifically, excess reserves in the voucher program could be used to serve additional families, if authorized by Congress. In addition, certain rent reform options (that is, changes to the calculation of households’ payment toward rent) may allow HUD to serve more people. For example, if implemented, rent reform could reduce the federal cost burden—in some cases, quite considerably—or if Congress chose to reinvest cost savings in the program, allow the program to serve between 1,400 to 287,000 additional households, depending on which rent reform option was implemented.

We also noted that HUD could do more to ensure that certain housing agencies continue to serve households. Specifically, our April 2012 report found that HUD may not be able to systematically ensure that agencies participating in the Moving to Work (MTW) demonstration program are meeting the requirement to serve substantially the same number of households through their rental assistance programs that they would As a result, the program have been able to serve prior to participation.may affect HUD’s ability to meet its goal of preserving affordable rental housing. We found that, contrary to internal control guidance, HUD did not have a process in place to systematically review compliance with several program requirements, including the requirement to serve substantially the same number of households. We concluded that because Congress is considering expanding the program to many more housing agencies, the absence of information needed to conduct compliance reviews is significant. We further stated that without more complete knowledge of the extent to which agencies are adhering to program requirements, it is difficult for Congress to know whether an expanded MTW will benefit additional agencies and the residents they serve.

More broadly, as part of our work on overlap, fragmentation, and duplication in federal programs, we reported in August 2012, that although selected HUD, USDA, and the Department of the Treasury (Treasury) rental housing programs had overlapping purposes, the agencies’ products, areas served, and delivery methods differed. Specifically, we found that although HUD is the only agency that has a specific priority goal to preserve affordable rental housing, seven of the nine selected HUD, USDA, and Treasury programs we reviewed have the shared purpose of financing the development of new rental units or preserving existing units through refinancing or rehabilitation. However, we found that five of these programs differ in terms of tenant and geographic eligibility. Additionally, we found that HUD and USDA administer project-based rental assistance programs, which provide rental subsidies to property owners that provide housing to low-income households. We reported that although HUD serves more households in rural areas, a large share of units with USDA’s rental assistance were in rural ZIP codes, while a smaller share of units with HUD rental assistance were in these areas. In addition, we also found that all three agencies have been working to consolidate and align requirements in rental We housing programs through the Rental Policy Working Group.concluded that although its efforts have been consistent with many key collaborative practices, the group has not taken full advantage of opportunities to reinforce agency accountability for collaborative efforts through the agencies’ annual and strategic plans, or expanded its guiding principles to evaluate areas requiring statutory action to generate savings and efficiencies.

Status of Key Open Recommendations and Matters for Congressional Consideration

In August 2012, we recommended that to further improve HUD, USDA, and Treasury’s efforts through the Rental Policy Working Group to consolidate and align certain requirements in multifamily housing programs, the Rental Working Group should take steps to document collaborative efforts in strategic and annual plans to help reinforce agency accountability for these efforts. HUD and USDA agreed with the recommendation.

In April 2012, we recommended that the Secretary of the Department of Housing and Urban Development develop and implement a systematic process for assessing compliance with statutory requirements. In response to this recommendation, HUD stated that the agency had conducted an extensive effort that allowed it to monitor compliance with the requirement for agencies to continue assisting substantially the same total number of households that they would have been able to serve prior to participating in the MTW program. HUD further stated that it was testing implementation of the process and planned to formalize the process through the publication of a notice. On January 10, 2013, HUD issued a notice that describes a compliance effort that, according to HUD, will ensure that MTW agencies continue to meet the statutory obligation to serve substantially the same number of families as if they had not participated in the MTW demonstration.use a numerical indicator to make annual determinations of compliance.

According to the notice, HUD will In March 2012, we recommended that the HUD Secretary provide information to Congress on (1) housing agencies’ estimated amount of excess subsidy reserves and (2) HUD’s criteria for how it will redistribute excess reserves among housing agencies so that they can serve more households. In taking these steps, the Secretary should determine a level of subsidy reserves housing agencies should retain on an ongoing basis to effectively manage their voucher programs. HUD neither agreed nor disagreed with our recommendation. HUD noted that it currently provides quarterly reports to the Congressional Budget Office on subsidy reserve levels. However, these quarterly reports do not include information on the estimated amount of agencies’ subsidy reserves that exceed prudent levels. HUD did not comment on its efforts to provide information to Congress on the criteria for how it will redistribute excess reserves among agencies so it can serve more households. In March 2013, HUD officials told us that, upon request, they provide information to HUD’s Appropriations Committee on subsidy reserves levels, including those balances above certain minimum reserve levels. We will continue monitoring the agency’s progress in implementing our recommendations.

Selected GAO Reports

Housing Assistance: Opportunities Exist to Increase Collaboration and Consider Consolidation. GAO-12-554. Washington D.C.: August 16, 2012.

Moving to Work Demonstration: Opportunities Exist to Improve Information and Monitoring. GAO-12-490. Washington D.C.: April 19, 2012.

Housing Choice Vouchers: Options Exist to Increase Program Efficiencies. GAO-12-300. Washington D.C.: March 19, 2012.

Project-Based Rental Assistance: HUD Should Update Its Policies and Procedures to Keep Pace with the Changing Housing Market. GAO-07-290. Washington D.C.: April 11, 2007.

Project-Based Rental Assistance: HUD Should Streamline Its Processes to Ensure Timely Housing Assistance Payments. GAO-06-57. Washington D.C.: November 15, 2005.

GAO Contacts

Daniel Garcia-Diaz, Director, Financial Markets and Community Investment, garciadiazd@gao.gov, (202) 512-8678.

Goal Statement

Prevent foreclosures. By September 30, 2013, assist 700,000 homeowners who are at risk of losing their homes due to foreclosure.

GAO Comments:

HUD’s efforts to achieve this goal involve the Federal Housing Administration’s (FHA) early delinquency interventions and loss mitigation programs. However, our past work raised questions about whether FHA has collected and analyzed data to assess the effectiveness of these efforts in preventing redefaults. Further, the HUD IG raised questions about the extent to which certain efforts were conducted in accordance with program requirements.

In June 2012, we reported that millions of borrowers faced an elevated risk of foreclosure and that various indicators showed that the housing market remained weak. In particular, we noted that the serious delinquency rate for FHA loans increased in the second half of 2011, counter to trends in the broader market. We reported that FHA had been working with loan servicers to identify best practices for reaching borrowers and had reporting requirements for servicers throughout the delinquency process. However, we found that although FHA had begun to calculate redefault rates for specific home retention actions, it had not used this information to assess the effectiveness of its foreclosure mitigation efforts. Doing so is particularly important because FHA loan modifications typically do not reduce borrower’s monthly payments to the levels that our analysis indicated result in more sustainable modifications. We also found that FHA had not assessed the impact of loan and borrower characteristics on the performance of its foreclosure mitigation efforts. In some cases, FHA did not have the data needed to conduct these analyses.

In a September 2012 report, the HUD IG estimated that 11,693 preforeclosure sales completed during the 12-month period it reviewed did not meet HUD’s requirements for participation and recommended that HUD strengthen controls over the preforeclosure sale program. Preforeclosure sales are one type of FHA loss mitigation action included in HUD’s calculation of borrowers assisted. Including ineligible preforeclosure sales in the calculation of borrowers assisted could overstate foreclosure prevention efforts.

HUD has previously reported performance that exceeds its target for preventing foreclosures for fiscal years 2012 and 2013. During the period covering fiscal years 2010 and 2011, HUD reported assisting 902,431 homeowners that were in danger of losing their homes to foreclosure— 496,197 through FHA early delinquency interventions and 406,234 through FHA loss mitigation programs. For fiscal years 2012 and 2013, HUD anticipates meeting its foreclosure prevention goal by reaching 500,000 homeowners with early delinquency interventions and an additional 200,000 through loss mitigation programs. Through the end of fiscal year 2012, HUD reported that it was more than halfway to meeting its goal, having reached 290,216 homeowners with early delinquency interventions and 154,933 homeowners through loss mitigation programs.

Status of Key Open Recommendations and Matters for Congressional Consideration

In June 2012, we recommended that FHA conduct periodic analyses of the effectiveness and the long-term costs and benefits of its loss mitigation strategies and actions. These analyses should consider (1) the redefault rates associated with each type of home retention action and (2) the impact that loan and borrower characteristics have on the performance of different home retention actions. FHA should use the results from these analyses to reevaluate its loss mitigation approach and provide additional guidance to servicers to effectively target foreclosure mitigation actions. If FHA does not maintain data needed to consider this information, it should require servicers to provide the data. In an August 2012 response to our recommendations, HUD noted that it was performing a complete review of the structure of its home-retention assistance. HUD is also undertaking an analysis of borrower and loan data with the goal of proactively directing servicers as to which assistance actions should be targeted to particular borrowers. In November 2012, FHA issued Mortgagee Letter 2012-22, which contained changes to the requirements for servicers to follow when assessing borrowers for FHA loss mitigation home-retention options. We requested and plan to assess the analysis HUD completed as the basis for this change in FHA’s loss mitigation strategies to determine whether it fully responds to our recommendation.

Selected GAO Reports

Foreclosure Mitigation: Agencies Could Improve Effectiveness of Federal Efforts with Additional Data Collection and Analysis, GAO-12-296. Washington, D.C.: June 28, 2012.

GAO Contact

Mathew J. Scirè, Director, Financial Markets and Community Investment, sciremj@gao.gov, (202) 512-8678.

Goal Statement

Reduce vacancy rates. By September 30, 2013 reduce the average residential vacancy rate in 70 percent of the neighborhoods hardest hit by the foreclosure crisis relative to comparable areas. Hardest hit neighborhoods are defined as Neighborhood Stabilization Program (NSP) 2 Neighborhood Investment Clusters (NIC).

GAO Comments

HUD will apply the results of the second phase of its Neighborhood Stabilization Program (NSP 2), funded under the American Recovery and Reinvestment Act, towards achieving this agency priority goal. The agency considers NSP, which provides grants to government and other entities to try to reduce the number of foreclosed and abandoned properties, its primary tool for mitigating the effects of foreclosures on As we reported in November 2011, high foreclosure neighborhoods.rates have contributed to increased vacancies, which can impose additional costs and challenges on communities, including increased public safety costs and lower tax revenues. While we have not directly assessed HUD’s capacity to achieve this agency priority goal, our prior work evaluating NSP indicates that through this program, HUD has the potential to reduce vacancies in areas receiving NSP funding. In December 2010, we examined HUD’s implementation of the first phase of NSP and grantees’ compliance with program requirements in using their funds to mitigate the impacts of foreclosures, which can include increased vacancies. We found that HUD and grantees had taken actions to try to ensure program compliance. While three phases of NSP were authorized by different pieces of legislation, all three rounds of NSP generally follow the same requirements. Therefore our previous findings are applicable to NSP 2, the contributing program to this agency priority goal.

Through its NSP technical assistance program, HUD has hired The Reinvestment Fund (TRF) to conduct analyses of NSP investments across the United States. HUD is using aspects of this analysis to measure progress toward meeting this agency priority goal. As part of its quarterly studies, TRF is conducting analysis of trends in vacancy rates within NICs versus comparable areas (or neighborhoods). NICs are geographic areas with a concentration of properties to which NSP funds have been applied. On the Performance.gov website, HUD uses “NIC” to refer to those areas to which NSP 2 funds have been applied. For both the first and second quarter of 2012, HUD reported that about 78 percent of NICs had lower vacancy rates than at least one comparable area— outperforming the agency priority goal’s target of 70 percent.

We have not assessed the reliability of TRF’s studies. However, in its analysis, TRF uses data from HUD’s Disaster Recovery Grant Reporting (DRGR) system, the information system used by all NSP grantees to report on their activities and results. In December 2010, we reported that inconsistencies in the manner in which grantees entered data into DRGR could complicate the analysis of program outputs and result in over counting and under counting of program outputs. We recommended that HUD take several actions to improve the consistency of the data collected from NSP grantees. In October 2012, HUD addressed these recommendations by issuing detailed guidance. For the purposes of measuring progress towards this goal, TRF uses property address information from DRGR for units that received NSP 2 funding. Quarterly, HUD standardizes and removes duplicate address information from DRGR to try to prevent any double counting of properties in TRF’s analyses.

Status of Key Open Recommendations and Matters for Congressional Consideration

We currently have no open recommendations or matters for congressional consideration related to this priority goal.

Selected GAO Reports

Vacant Properties: Growing Number Increases Communities’ Costs and Challenges. GAO-12-34. Washington, D.C.: November 4, 2011.

Neighborhood Stabilization Program: HUD and Grantees Are Taking Actions to Ensure Program Compliance but Data on Program Outputs Could be Improved. GAO-11-48. Washington, D.C.: December 17, 2010.

GAO Contact

Mathew J. Scirè, Director, Financial Markets and Community Investment, sciremj@gao.gov, (202) 512-8678.

Goal Statement

Reducing homelessness. By September 30 2013, in partnership with the VA, reduce the number of homeless veterans to 35,000 by serving 35,500 additional homeless veterans. HUD is also committed to making progress towards reducing family and chronic homelessness and is working towards milestones to allow for tracking of these populations.

GAO Comments

HUD notes that several programs are expected to contribute to the achievement of its priority goal to reduce homelessness, including the HUD Veterans Affairs Supportive Housing (HUD-VASH) program, HUD homeless assistance programs, and the Homelessness Prevention and Our past work has identified a number of Rapid Re-Housing Program.issues related to these programs. For example, our June 2012 report on the HUD-VASH program—a collaborative initiative between HUD and VA that targets the most vulnerable, most needy, and chronically homeless veterans—states that the program has moved veterans out of homelessness. Specifically, according to VA, as of March 2012, nearly 31,200 veterans lived in HUD-VASH supported housing, which represents about 83 percent of the vouchers authorized under the program. In addition, our December 2011 report on homeless women veterans notes that although HUD collects data on homeless women and on homeless veterans, the department does not collect detailed information on homeless women veterans and neither HUD nor VA captures data on the overall population of homeless women veterans. Further, our report states that HUD and VA lack data on the characteristics and needs of these women on a national, state, and local level. Finally, our report notes that absent more complete data on homeless women veterans, VA does not have the information needed to plan services effectively, allocate grants to providers, and track progress toward its overall goal of ending veteran homelessness.

Our May 2012 report on the fragmentation, overlap, and duplication among federal homelessness programs also identified issues related to the programs that are expected to contribute to the achievement of this priority goal. For example, our report noted HUD was one of eight federal agencies that administered 26 targeted homelessness programs in fiscal year 2011, suggesting fragmentation and some overlap among these programs.fund housing assistance, but also provides funding for mental health care, substance abuse treatment, and employment services. Similarly, HHS and VA administer programs that provide housing and employment assistance. Fragmentation and overlap can lead to inefficient use of resources. Some local service providers told us that managing multiple applications and reporting requirements was burdensome, difficult, and costly. Moreover, according to providers, persons experiencing More specifically, HUD not only administers programs that homelessness have difficulties navigating services that are fragmented across agencies. Further, our report states that limited information exists about the efficiency or effectiveness of targeted homelessness programs because evaluations have not been conducted recently—including for the six programs HUD administers. Finally, our report states that the U.S. Interagency Council on Homelessness (Interagency Council) strategic plan to prevent and end homelessness has served as a useful and necessary first step in increasing agency coordination and focusing attention on ending homelessness; however, the plan lacks key characteristics desirable in a national strategy. For example, the plan does not list priorities or milestones and does not discuss resource needs or assign clear roles and responsibilities to federal partners.

Status of Key Open Recommendations and Matters for Congressional Consideration

In May 2012, we recommended that the Interagency Council and the Office of Management and Budget, in conjunction with the Secretaries of HHS, HUD, Labor, and VA, should consider examining inefficiencies that may result from overlap and fragmentation in their programs for persons experiencing homelessness. VA agreed with this recommendation. HHS, HUD, Labor, and the Interagency Council did not explicitly agree or disagree. We also recommended that to help prioritize, clarify, and refine efforts to improve coordination across agencies, and improve the efficiency and effectiveness of federal homelessness programs, the Interagency Council, in consultation with its member agencies, should incorporate additional elements into updates to the national strategic plan or other planning and implementation documents to help set priorities, measure results, and ensure accountability. According to the Interagency Council, its fiscal year 2013 report will focus on updates and progress made on the national strategic plan’s objectives. The Interagency Council’s national strategic plan broadly describes the federal approach to preventing and ending homelessness; however, until the key member agencies fully implement their plans, including setting priorities, measuring progress and results, and holding federal and nonfederal partners accountable, they are at risk of not reaching their goal of ending veteran and chronic homelessness by 2015, and ending homelessness among children, youth, and families by 2020.

In December 2011, we recommended that in order to help achieve the goal of ending homelessness among veterans, the Secretaries of HUD and VA should collaborate to ensure appropriate data are collected on homeless women veterans, including those with children and those with disabilities, and use these data to strategically plan for services. In concurring with this recommendation, VA stated it had several initiatives already planned or under way to gather information on those homeless women veterans who are in contact with VA, including the development of a more streamlined and comprehensive data collection system. In April 2013, VA stated that it had taken additional actions to inform policy and operational decisions about homeless and at-risk women veterans. For example, VA stated that in 2013 it worked with HUD to ensure that gender specific data were collected during the 2013 Point in Time count of homeless persons. VA added that the results of the 2013 Point in Time count will be included in the Annual Homeless Assessment Report to Congress which will be published later in 2013 and will be used by the department to strategically plan and implement services for all homeless and at-risk veterans, including women veterans. In addition, VA stated that in 2012 it revised the Community Homelessness Assessment, Local Education and Networking Groups survey to capture gender specific data for homeless veterans to better identify the needs of women veterans and influence service provision.

Selected GAO Reports

Veteran Homelessness: VA and HUD Are Working to Improve Data on Supportive Housing Program. GAO-12-726. Washington, D.C.: June 26, 2012.

Homelessness: Fragmentation and Overlap in Programs Highlight the Need to Identify, Assess, and Reduce Inefficiencies. GAO-12-491. Washington, D.C.: May 10, 2012.

Homeless Women Veterans: Actions Needed to Ensure Safe and Appropriate Housing. GAO-12-182. Washington, D.C.: December 23, 2011.

Homelessness: A Common Vocabulary Could Help Agencies Collaborate and Collect More Consistent Data. GAO-10-702. Washington, D.C.: June 30, 2010.

GAO Contact

Alicia Puente Cackley, Director, Financial Markets and Community Investment, cackleya@gao.gov, (202)512-8678.

Appendix IV: Comments on the Department of Transportation’s Priority Goals

Based on our past work, as well as that of the DOT IG, we commented on each of DOT’s four priority goals for 2012 to 2013: 1. Air traffic control systems can improve the efficiency of airspace. By September 30, 2013, replace a 40-year-old computer system serving 20 air traffic control centers with a modern, automated system that tracks and displays information on high altitude planes. 2. Advance the development of passenger rail in the United States. By September 30, 2013, initiate construction on all 7 high speed rail corridors and 36 individual high speed rail projects. 3. Reduce risk of aviation accidents. By September 30, 2013, reduce aviation fatalities by addressing risk factors both on the ground and in the air. Commercial aviation (i.e. airlines): Reduce fatalities to no more than 7.4 per 100 million people on board. General aviation (i.e. private planes): Reduce fatal accident rate per 100,000 flight hours to no more than 1.06. 4. Reduce the rate of roadway fatalities. Reduce the rate of roadway fatalities from 1.26 in 2008 to 1.03 per 100 million vehicle miles traveled by December 31, 2013.

For each goal, we also identify our related past reports and provide an update on the status of any open recommendations and matters for congressional consideration that we previously made related to the goal. We also identify a GAO contact for our work related to each goal.

Goal Statement

Air traffic control systems can improve the efficiency of airspace. By September 30, 2013, replace a 40-year-old computer system serving 20 air traffic control centers with a modern, automated system that tracks and displays information on high altitude planes.

GAO Comments

The priority goal refers to the Federal Aviation Administration’s (FAA) replacement of the existing en route air traffic control automation system used in its en route air traffic control centers (centers) with a new system architecture, the En Route Automation Modernization system (ERAM). While our previous work has shown that FAA has experienced delays in deploying ERAM, FAA has since made progress toward achieving this goal. As we reported in September 2012, FAA has experienced delays in deploying ERAM, which affected overall acquisition and maintenance costs as well as time frames for other programs. Specifically, the ERAM program is almost 4 years behind its original schedule and about $330 million, or about 15 percent, over its original budget because of the following factors: unanticipated risks associated with operational complexities at the field sites, insufficient testing to identify software issues before deployment at the field sites, insufficient communication between the program office and field sites, and insufficient stakeholder (e.g., air traffic controller) involvement during system development and deployment.

The delays added an estimated $18 million per year to the costs of maintaining the system that ERAM was meant to replace.

Since new budget and schedule baselines for the ERAM program were established in June 2011, according to FAA reports, the program has made progress toward its goal of initial operating capability of ERAM by September 30, 2013. As of March 2013, FAA had achieved initial operating capability at 16 out of 20 centers and expects to achieve this goal. In September 2012, the DOT IG reported that FAA’s use of initial operating capability for tracking progress with ERAM gave FAA decision makers a false sense of confidence in the maturity of the system when in reality, much work and time still remained in implementing the system. For example, after FAA declared its first two sites as achieving initial operating capability, these sites experienced multiple failures after the milestone was achieved and went through a measured transition from limited operations to eventual continuous operations. In response to the DOT IG’s recommendation to better define key milestones to reflect progress, FAA is planning to establish criteria for entrance and exit at the various key milestones, including initial operating capability. FAA also plans to have all 20 centers operationally ready by August 2014.

As we reported in September 2012, looking more broadly, ERAM is considered to be the backbone that will support the Next Generation Air Transportation System (NextGen)—a new air traffic management system that will replace the current radar-based system and is expected to enhance the safety and capacity of the air transport system—and delays with ERAM’s deployment illustrate challenges FAA faces in implementing NextGen. For example, delays in ERAM affected the implementation of two other key NextGen acquisitions—Data Communications and System Wide Information Management. In part because of ERAM’s delay, FAA pushed the Data Communications program’s start date from September 2011 to May 2012, revised the original plan for the first System Wide Information Management segment, and delayed the start date for another segment from 2010 to July 2012. The implementation of NextGen—both in the midterm (through 2020) and in the long term (beyond 2020)—will be affected by how well FAA manages these and other program interdependencies.

Status of Key Open Recommendations and Matters for Congressional Consideration

To address past issues with cost estimate and schedule accuracy, such as those with ERAM, in February 2012, we recommended that when appropriate for major acquisition programs based on a program’s cost, schedule, complexity, and risk, FAA conduct an assessment of major acquisition programs to ensure they meet all of the established best practices for cost estimates and schedules contained in our guidance; require a fully integrated master schedule for each major acquisition program, including those that are NextGen components; and conduct independent cost estimates and schedule risk analysis.

As of March 2013, FAA has taken steps to implement these recommendations. For example, according to FAA officials, FAA’s Acquisition Executive Board now considers whether an independent cost estimate or schedules risk analysis is advisable as parts of its program review. FAA is also developing an Integrated Master Schedule for the entire NextGen initiative that is, in part, intended to show how changes in program schedules affect other programs and the timelines for the NextGen initiative as a whole. To further strengthen schedule integration, FAA plans to continue populating the integrated master schedule and then begin integrating this tool with other FAA planning tools, including the National Airspace System Enterprise Architecture and NextGen Implementation Plan, in December 2013.

Selected GAO Reports

Next Generation Air Transportation System: FAA Faces Implementation Challenges. GAO-12-1011T. September 12, 2012.

Air Traffic Control Modernization: Management Challenges Associated with Program Costs and Schedules Could Hinder NextGen Implementation. GAO-12-223. February 16, 2012.

NextGen Air Transportation System: FAA’s Metrics Can Be Used to Report on Status of Individual Programs, but Not of Overall NextGen Implementation or Outcomes. GAO-10-629. July 27, 2010.

GAO Contact

Gerald L. Dillingham, Ph.D., Director, Physical Infrastructure Issues, dillinghamg@gao.gov, (202) 512-2834.

Goal Statement

Advance the development of passenger rail in the United States. By September 30, 2013, initiate construction on 7 high speed rail corridors and 36 individual high speed rail projects.

GAO Comments

While our past work does not cover activities related to construction of high speed rail projects, we and the DOT IG have reported on planning and other weaknesses with the Federal Railroad Administration’s (FRA) High Speed Intercity Passenger Rail (HSIPR) program related to FRA’s capacity to achieve this APG. The federal government had not historically had a strong leadership role in intercity passenger rail but this changed in 2009 when the HSIPR program was authorized. The program provides grants to states and other entities to develop high speed intercity passenger rail corridors and projects. As of January 2013, FRA had awarded grants for 150 high speed intercity passenger rail projects. Of these projects, FRA had obligated about $9.2 billion for 9 corridor programs and 57 individual high speed rail projects as of December 2012. According to FRA, by this same date, states and other project sponsors had begun construction on 5 of the high speed rail corridors and 33 of the individual high speed rail projects. By the end of fiscal year 2013, FRA’s goal is to begin construction on a total of 7 corridor and 36 individual high speed rail projects.

The fundamental weaknesses of the HSIPR program include not having well-defined goals, a clear strategic vision, and performance measures to track program progress. In March 2009, before the HSIPR program was established, we reported that several principles could help guide the potential federal role in high speed rail. These principles included, among other things, creating well-defined goals based on identified areas of national interest, incorporating performance and accountability for results into funding decisions, and employing the best analytical tools and approaches to emphasize return on investment. Similarly, in June 2010, we reported that FRA’s strategic vision for high speed rail, as outlined in the agency’s April 2009 Vision for High-Speed Rail in America, did not define the goals, stakeholder roles, or objectives for federal involvement in high speed intercity passenger rail and that the agency’s preliminary national rail plan did not include recommendations for future action. Further, although states would be among primary HSIPR grant recipients, many did not have rail plans that would establish strategies and priorities for rail investments or identify the public benefits of such investments. Many of these weaknesses continue. For example, in September 2012 the DOT IG reported that FRA’s HSIPR goals lacked the thoroughness needed to ensure that grant managers and decision makers, including Congress, could understand them, and that FRA generally lacked performance measures needed to assess the program’s progress in achieving its goals as well as complete monitoring mechanisms. According to DOT, FRA has taken action to address some of the issues reported by the DOT IG, including developing a standardized mechanism for collecting and tracking HSIPR grantee performance and compliance metrics, and developing a comprehensive grants management training curriculum.

There are also weaknesses in how the HSIPR program is administered starting with how HSIPR grants are awarded. In March 2011 we reported that, although FRA had applied its established criteria during the eligibility and technical reviews of the HSIPR grant applications, we could not verify whether it applied its final selection criteria because the documented rationales for selecting projects were typically vague. We concluded that without a record that provided insight into why decisions were made, FRA invited skepticism about the overall fairness of its decision making. Other program weaknesses include the lack of guidance issued to HSIPR applicants and FRA’s grants administration framework. In March 2009 we recommended the Secretary of Transportation develop guidance and methods for ensuring reliability of ridership and other forecasts used to determine the viability of high speed rail projects. In March 2012, the DOT IG also reported that FRA had established only minimal requirements and guidance on the information HSIPR grant applicants must provide to FRA on project viability which did not provide enough detail to minimize bias and ensure accuracy in project viability assessments. In addition, in September 2012, the DOT IG reported that FRA had issued the policies and procedures for HSIPR grants management several years after the program had been established and that insufficient staffing and training undermined FRA’s efforts to effectively administer and ensure the accountability of HSIPR grant funds once they are awarded. FRA’s Grants Management Manual was not issued until April 2012, almost 3 years after the HSIPR program was authorized. FRA’s monitoring plan, which will, among other things, guide performance and compliance monitoring for the HSIPR program, was not finalized until March 2012.

Aside from program weaknesses, we have found that implementing high speed rail projects is difficult. This difficulty could affect achievement of program goals. Our March 2009 report identified some of the challenges in developing and financing high speed rail projects, including securing the up-front investments for such projects and sustaining public and political support and stakeholder consensus. We concluded that whether any high speed rail proposals are eventually built hinges on addressing the funding, public support, and other challenges facing these projects.

Status of Key Open Recommendations and Matters for Congressional Consideration

In March 2011, we recommended that FRA create additional records to document the substantive reasons behind award decisions in future HSIPR funding rounds to better ensure accountability for its use of federal funds. As of November 2012, FRA had enhanced its grant management manual with more explicit requirements for documenting the rationale behind its funding selections.

In March 2009, we recommended, among other things, that the Secretary of Transportation develop guidance and methods to improve the reliability and accuracy of ridership, cost, and other forecasts for these systems. As of November 2012, FRA said it is implementing this recommendation in conjunction with stakeholders, partners, and researchers, through an iterative process of developing methods and guidance, using them, and then refining them. FRA is also working with a research panel of the Transportation Research Board to develop a handbook that will provide tools to decision makers in such areas as ridership forecasting and service characteristics (e.g., frequency of service). FRA said it expects implementing this recommendation will require from 5 to 10 years.

In September 2012, DOT’s IG recommended that before awarding, obligating, and disbursing additional grant funds, FRA should take several actions to establish a comprehensive grants management program with clear program goals and mechanisms to track grantee performance toward those goals. In response to the IG’s recommendations, FRA officials concurred with each recommendation and said they would implement reports, tools and training programs to meet the IG’s recommendations starting in late 2012.

Selected GAO Reports

Intercity Passenger Rail: Recording Clearer Reasons for Awards Decisions Would Improve Otherwise Good Grantmaking Practices. GAO-11-283. Washington, D.C.: March 10, 2011.

High Speed Rail: Learning from Service Start-Ups, Prospects for Increased Industry Investment, and Federal Oversight. GAO-10-625. Washington, D.C.: June 17, 2010.

High Speed Passenger Rail: Future Development Will Depend on Addressing Financial and Other Challenges and Establishing a Clear Federal Role. GAO-09-317. Washington, D.C.: March 19, 2009.

GAO Contact

Susan Fleming, Director, Physical Infrastructure Issues, 202-512-2834, flemings@gao.gov.

Goal Statement

Reduce risk of aviation accidents. By September 30, 2013, reduce aviation fatalities by addressing risk factors both on the ground and in the air. Commercial aviation (i.e. airlines): Reduce fatalities to no more than 7.4 per 100 million people on board. General aviation (i.e. private planes): Reduce fatal accident rate per 100,000 flight hours to no more than 1.06.

GAO Comments

DOT’s FAA has worked toward these goals by partnering with the airline industry and other stakeholders through the Commercial Aviation Safety Team (CAST), improving runway safety, shifting toward a risk-based analysis of airborne aviation system information, establishing safety management systems (SMS), renewing the General Aviation Joint Steering Committee (GAJSC), and developing a 5-year strategy for reducing general aviation fatalities. DOT reported that from 2009 through 2011 FAA exceeded its targets for reducing commercial air carrier fatalities, well below its 2013 goal. For general aviation fatality rates, however, FAA has not yet achieved its goal, in part due to challenges we have recently discussed.

As we reported in April 2012, CAST has contributed to reducing commercial aviation accidents by analyzing past accidents and incidents to identify precursors and contributing factors, and ensuring that efforts to improve safety focus on the most prevalent accident categories. CAST has reduced commercial aviation risks by focusing on areas including controlled flight into terrain, loss of control, and runway incursions. CAST analyzes accident and incident data to identify precipitating conditions and causes, and then formulates an intervention strategy designed to reduce the likelihood of a recurrence. According to CAST, its work—along with new aircraft, regulations, and other activities—reduced the commercial aviation fatal accident rate by 83 percent from 1998 to 2008 and is an important aspect of FAA’s efforts to improve aviation safety by sharing and analyzing data. However, as we reported in October 2011, for safety at and around airports, including runways, the overall rate of runway incursions (the unauthorized presence of an airplane, vehicle, or person on the runway) at towered airports has trended steadily upward, as has the rate and number of airborne operational errors (errors made by air traffic controllers), though it is not clear whether these recent increases in operational errors can be attributed to several changes in reporting policies and procedures at FAA, or increases in actual incidents.

We reported in September 2012 that FAA is seeking to further enhance commercial aviation safety by shifting to a data-driven, risk-based safety oversight approach—referred to as SMS. SMS represents a proactive approach to safety and is intended to continually monitor all aspects of aviation operations and collect appropriate data to identify emerging safety problems before they result in death, injury, or significant property damage. SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. Several challenges remain that may affect FAA’s ability to effectively implement SMS. FAA is taking steps to address some of these, but challenges related to data concerns, its capacity to conduct analysis and oversight, and standardization of policies and procedures could negatively affect FAA’s efforts to implement SMS in a timely and efficient manner and require some skills that agency employees do not have. Addressing these challenges is ever more important with air travel projected to increase over the next 20 years.

FAA has embarked on several initiatives to meet its goal of reducing the fatal general aviation accident rate; however it reported not meeting its target fatality rates in any year from 2009 through 2011. As we reported in October 2012, FAA reported the general aviation fatality rate exceeded the target rate by 7.4 percent for 2011. FAA initiatives to improve aviation safety include renewing the GAJSC and implementing the Flight Standards Service’s 5-year strategy for reducing general aviation fatalities. The GAJSC, a government-industry partnership similar to the CAST approach for commercial aviation, focuses on analyzing general aviation accident data to develop effective intervention strategies. We believe that the GAJSC has the potential to contribute to a reduction in general aviation accidents and fatalities over the long term. However, the 5-year strategy has shortcomings that jeopardize its potential for success because, among other things, the strategy lacks performance measures for significant activities. Without a strong performance management structure, FAA will not be able to determine the success or failure of the significant activities that underlie the strategy. Furthermore, there are some limitations in flight activity data and other data that preclude a confident assessment of general aviation safety. For example, FAA’s survey of general aviation operators, on which the agency bases its annual flight-hour estimates, continues to suffer from methodological and conceptual limitations, even with FAA’s efforts to improve it.

Status of Key Open Recommendations and Matters for Congressional Consideration

In October 2012, we recommended that FAA (1) improve measures of general aviation activity by requiring the collection of the number of hours that general aviation aircraft fly, (2) set specific general aviation safety improvement goals—such as targets for fatal accident reductions—for individual industry segments (e.g. personal or corporate operations) using a data-driven, risk management approach and (3) determine whether the programs and activities underlying the 5-year strategy are successful and if additional actions are needed, develop performance measures for each significant program and activity underlying the 5-year strategy. In its comments to our report, FAA reported that it is working toward implementing these recommendations.

In September 2012, we recommended that FAA develop a system to assess whether SMS meets its goals and objectives by identifying and collecting related data on performance measures. In comments to our report, FAA stated that it is currently involved in activities directed towards the development of safety performance measurement capabilities, including a process and measures for measuring safety performance. This activity is expected to be completed by April 2015.

In October 2011, we recommended that FAA develop separate risk-based assessment processes, measures, and performance goals for runway safety incidents involving commercial and general aviation aircraft, and to expand the existing risk-based process for assessing airborne losses of separation. In comments to our report, FAA reported that it is working toward implementing these recommendations.

Selected GAO Reports

General Aviation Safety: Additional FAA Efforts Could Help Identify and Mitigate Safety Risks. GAO-13-36. Washington D.C.: October 4, 2012.

Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk Management. GAO-12-898. Washington D.C.: September 12, 2012.

Aviation Safety: FAA Is Taking Steps to Improve Data, but Challenges for Managing Safety Risks Remain. GAO-12-660T. Washington D.C.: April 25, 2012.

Aviation Safety: Enhanced Oversight and Improved Availability of Risk- Based Data Could Further Improve Safety. GAO-12-24. Washington D.C.: October 5, 2011.

Aviation Safety: Improved Data Quality and Analysis Capabilities Are Needed as FAA Plans a Risk-Based Approach to Safety Oversight. GAO-10-414. Washington D.C.: May 6, 2010.

GAO Contact

Gerald L. Dillingham, Ph.D., Director, Physical Infrastructure Issues, dillinghamg@gao.gov, (202) 512-2834.

Goal Statement

Reduce the rate of roadway fatalities. Reduce the rate of roadway fatalities from 1.26 in 2008 to 1.03 per 100 million vehicle miles traveled by December 31, 2013.

GAO Comments

We have issued a number of reports related to DOT’s efforts to reduce highway fatalities that highlight the need for improved performance accountability and data. The number of traffic fatalities decreased from 41,000 in 2000 to fewer than 33,000 in 2010; the fatality rate per 100 million miles traveled also decreased from 1.53 in 2000 to 1.11 in 2010.To help states reduce traffic fatalities, the National Highway Traffic Safety Administration (NHTSA) within DOT provides traffic safety grants to states to, among other things, promote and enforce safety belt use and impaired driving laws and improve traffic safety data systems. While NHTSA has made progress in developing performance measures to help NHTSA and states evaluate the effectiveness of traffic safety programs, we reported in March 2008 that state performance has generally not been tied to receipt of the grants and improvements in state traffic safety data are needed to support a more performance-based approach to improving traffic safety programs.

On July 6, 2012, President Obama signed into law the Moving Ahead for Progress in the 21st Century Act.transportation program framework more performance-based by: (1) establishing national performance goals for the federal-aid highway program in several areas, including goals for the safety of the nation’s highways; and (2) requiring the Secretary of Transportation, in consultation with state departments of transportation and others, to establish performance measures linked to national goals, including measures for serious injuries and fatalities on public roads. Finally, the act required states to establish performance targets for those measures and report their progress in achieving planned outcomes through the statewide transportation plans. These provisions are consistent with a performance-based planning framework we recommended to Congress in December 2010 and, as they are implemented over the next several years, should help NHTSA and states focus their efforts on key actions needed to improve traffic safety and reduce highway fatalities.

The act made the surface While states have implemented projects to improve traffic safety data systems, such as switching to electronic data reporting and adopting data collection forms consistent with national guidelines, enhancements in these systems are still needed to support a performance-based approach to improving traffic safety. In April 2010, we reported that our analysis of traffic records assessments—conducted for states by NHTSA technical teams or contractors at least every 5 years—indicated that the quality of state traffic safety data systems varied across the six data systems maintained by states. Assessments include an evaluation of system quality based on six performance measures. Across all states, we found that vehicle and driver data systems met performance measures 71 percent and 60 percent of the time, respectively, while roadway, crash, citation and adjudication, and injury surveillance data systems met performance measures less than 50 percent of the time. States face resource and coordination challenges in improving traffic safety data systems. For example, custodians of data systems are often located in different state agencies, which may make coordination difficult. In addition, rural and urban areas may face different challenges in improving data systems, such as limited technology options for rural areas or timely processing of large volumes of data in urban areas. States we visited have used strategies to overcome these challenges, including establishing an executive-level traffic records coordinating committee (TRCC), in addition to the technical-level committee that states are required to establish to qualify for federal traffic safety grant funding. An executive-level committee could help states address challenges by targeting limited resources and facilitating data sharing.

Status of Key Open Recommendations and Matters for Congressional Consideration

In April 2010, we recommended that the Secretary of Transportation should direct the NHTSA Administrator to (1) ensure that traffic records assessments provide an in-depth evaluation that is complete and consistent in addressing all performance measures across all state traffic safety data systems and (2) study and communicate to Congress the value of requiring states to establish an executive-level TRCC in order to qualify for traffic safety data system grant funding.

In response to the first recommendation, NHTSA developed a comprehensive approach for assessing the systems and processes that govern the collection, management, and analysis of traffic records data. Core to this approach is the set of questions for conducting assessments published in September 2012 in the Traffic Records Program Assessment Advisory. The Advisory includes standards of evidence to guide state officials in providing the information necessary to answer each assessment question. The assessment now asks a comprehensive, uniform set of questions about data quality performance measures across all state traffic safety data systems. NHTSA kicked off a pilot program to test the new process in Indiana in November 2012. This pilot was successfully completed in February 2013. As part of the new assessment process, NHTSA will create a database to house data from the new traffic records systems, conduct research and identify national trends. NHTSA can then inform states about how the ratings for each of their assessment questions compares with a national average.

In response to the second recommendation, NHTSA’s study examining how executive level and technical level TRCCs coordinate traffic records systems management was initiated with the 2012 pilot test of the new traffic records assessment process in Indiana. Data collection will continue through the fiscal year 2015 assessment cycle at which point 10 states will have been assessed and the data set will be large enough to enable a quality analysis. The data for this study will include the information submitted by the states as well as the ratings received on the comprehensive, uniform set of assessment questions. Specifically, NHTSA plans to examine states’ responses to questions about their TRCC management, strategic planning, data integration, and the capabilities of the six core traffic records components. Notable practices demonstrated by effective TRCC organizations—particularly at the executive level—will be highlighted.

Selected GAO Reports

Statewide Transportation Planning: Opportunities Exist to Transition to Performance-Based Planning and Federal Oversight. GAO-11-77. Washington, D.C.: December 15, 2010.

Traffic Safety Data: State Data System Quality Varies and Limited Resources and Coordination Can Inhibit Further Progress. GAO-10-454. Washington, D.C.: April 15, 2010.

Traffic Safety Programs: Progress, States’ Challenges, and Issues for Reauthorization. GAO-08-990T. Washington, D.C.: July 16, 2008.

Traffic Safety: NHTSA’s Improved Oversight Could Identify Opportunities to Strengthen Management and Safety in Some States. GAO-08-788. Washington, D.C.: July 14, 2008.

Traffic Safety: Improved Reporting and Performance Measures Would Enhance Evaluation of High-Visibility Campaigns. GAO-08-477. Washington, D.C.: April 25, 2008.

Traffic Safety: Grants Generally Address Key Safety Issues, Despite State Eligibility and Management Difficulties. GAO-08-398. Washington, D.C.: March 14, 2008.

Surface Transportation: Restructured Federal Approach Needed for More Focused, Performance-Based, and Sustainable Programs. GAO-08-400. Washington, D.C.: March 6, 2008.

GAO Contacts

Phillip R. Herr, Managing Director, Physical Infrastructure Issues, herrp@gao.gov, (202) 512-2834.

Susan Fleming, Director, Physical Infrastructure Issues, flemings@gao.gov, (202) 512-2834.

Appendix V: Comments on the Department of Veterans Affairs’ Priority Goals

Based on our past work, as well as that of the VA IG, we commented on each of VA’s three priority goals for 2012 to 2013: 1. Assist in housing 24,400 additional homeless veterans (12,200 per year) and reduce the number of homeless veterans to 35,000 in 2013, to be measured in the January 2014 Point-In-Time homelessness count. By September 2013, working in conjunction with the U.S. Interagency Council on Homelessness (Interagency Council) and HUD, VA will also assist homeless Veterans in obtaining employment, accessing VA services, and securing permanent supportive housing, with a long-range goal of eliminating homelessness among Veterans by 2015. 2. Improve accuracy and reduce the amount of time it takes to process veterans’ disability benefit claims. By September 30, 2013, reduce the veterans’ disability compensation and pension claims backlog to 40 percent from 60.2 percent while achieving 90 percent rating accuracy, up from 83.8 percent, in pursuit of eliminating the Veterans’ claims backlog (defined as claims pending more than 125 days) by 2015. 3. Improve awareness of VA services and benefits by increasing the timeliness and relevance of on-line information available to veterans, service members and eligible beneficiaries. By September 30, 2013, increase the number of registered eBenefits users from 1.0 million to 2.5 million.

For each goal, we also identify our related past reports and provide an update on the status of any open recommendations and matters for congressional consideration that we previously made related to the goal. We also identify a GAO contact for our work related to each goal.

Goal Statement

Assist in housing 24,400 additional homeless veterans (12,200 per year) and reduce the number of homeless veterans to 35,000 in 2013, to be measured in the January 2014 Point-In-Time homelessness count. By September 2013, working in conjunction with the U.S. Interagency Council on Homelessness (Interagency Council) and HUD, VA will also assist homeless Veterans in obtaining employment, accessing VA services, and securing permanent supportive housing, with a long-range goal of eliminating homelessness among veterans by 2015.

GAO Comments

VA notes that several programs are expected to contribute to the achievement of its priority goal to reduce homelessness among veterans, including the HUD Veterans Affairs Supportive Housing (HUD-VASH) program, Grant and Per Diem program, Domiciliary Care for Homeless Veterans program, and Health Care for Homeless Veterans program. Our past work has identified a number of issues related to these programs. For example, our June 2012 report on the HUD-VASH program—a collaborative initiative between HUD and VA that targets the most vulnerable, most needy, and chronically homeless veterans—states that the program has moved veterans out of homelessness. Specifically, according to VA, as of March 2012, nearly 31,200 veterans lived in HUD- VASH supported housing, which represents about 83 percent of the rental assistance vouchers authorized under the program. In addition, our December 2011 report on homeless women veterans notes that although HUD collects data on homeless women and on homeless veterans, the department does not collect detailed data information on homeless women veterans and neither HUD nor VA captures data on the overall population of homeless women veterans. Further, our report states that HUD and VA lack data on the characteristics and needs of these women on a national, state, and local level. Finally, our report notes that absent more complete data on homeless women veterans, VA does not have the information needed to plan services effectively, allocate grants to providers, and track progress toward its overall goal of ending veteran homelessness.

Our May 2012 report on the fragmentation, overlap, and duplication among federal homelessness programs also identified issues related to the programs that are expected to contribute to the achievement of this priority goal. For example, our report noted VA was one of eight federal agencies that administered 26 targeted homelessness programs in fiscal year 2011, suggesting fragmentation and some overlap among these programs.funding for supportive services such as health care, substance abuse treatment, and employment assistance, but also administers programs that provide housing and employment assistance. Similarly, HUD not only administers housing assistance, but also provides funding for mental More specifically, VA typically operates programs or provides health care, substance abuse treatment, and employment services. Fragmentation and overlap can lead to inefficient use of resources. Some local service providers told us that managing multiple applications and reporting requirements was burdensome, difficult, and costly. Moreover, according to providers, persons experiencing homelessness have difficulties navigating services that are fragmented across agencies. Further, our report states that limited information exists about the efficiency or effectiveness of targeted homelessness programs because evaluations have not been conducted recently—including for the 13 programs VA administers or co-administers. Finally, our report states that the Interagency Council strategic plan to prevent and end homelessness has served as a useful and necessary first step in increasing agency coordination and focusing attention on ending homelessness; however, the plan lacks key characteristics desirable in a national strategy. For example, the plan does not list priorities or milestones and does not discuss resource needs or assign clear roles and responsibilities to federal partners.

Status of Key Open Recommendations and Matters for Congressional Consideration

In May 2012, we recommended that the Interagency Council and the Office of Management and Budget––in conjunction with the Secretaries of HHS, HUD, Labor, and VA––should consider examining inefficiencies that may result from overlap and fragmentation in their programs for persons experiencing homelessness. VA agreed with this recommendation. HHS, HUD, Labor, and the Interagency Council did not explicitly agree or disagree. We also recommended that to help prioritize, clarify, and refine efforts to improve coordination across agencies, and improve the efficiency and effectiveness of federal homelessness programs, the Interagency Council, in consultation with its member agencies, should incorporate additional elements into updates to the national strategic plan or other planning and implementation documents to help set priorities, measure results, and ensure accountability. According to the Interagency Council, its fiscal year 2013 report will focus on updates and progress made on the national strategic plan’s objectives. The Interagency Council’s national strategic plan broadly describes the federal approach to preventing and ending homelessness; however, until the key member agencies fully implement their plans, including setting priorities, measuring progress and results, and holding federal and nonfederal partners accountable, they are at risk of not reaching their goal of ending veteran and chronic homelessness by 2015, and ending homelessness among children, youth, and families by 2020.

In December 2011, we recommended that in order to help achieve the goal of ending homelessness among veterans, the Secretaries of HUD and VA should collaborate to ensure appropriate data are collected on homeless women veterans, including those with children and those with disabilities, and use these data to strategically plan for services. In concurring with this recommendation, VA stated it had several initiatives already planned or under way to gather information on those homeless women veterans who are in contact with VA, including the development of a more streamlined and comprehensive data collection system. In April 2013, VA stated that it had taken additional actions to inform policy and operational decisions about homeless and at-risk women veterans. For example, VA stated that in 2013 it worked with HUD to ensure that gender specific data were collected during the 2013 Point in Time count of homeless persons. VA added that the results of the 2013 Point in Time count will be included in the Annual Homeless Assessment Report to Congress which will be published later in 2013 and will be used by the department to strategically plan and implement services for all homeless and at-risk veterans, including women veterans. In addition, VA stated that in 2012 it revised the Community Homelessness Assessment, Local Education and Networking Groups survey to capture gender specific data for homeless veterans to better identify the needs of women veterans and influence service provision.

Selected GAO Reports

Veteran Homelessness: VA and HUD Are Working to Improve Data on Supportive Housing Program. GAO-12-726. Washington, D.C.: June 26, 2012.

Homelessness: Fragmentation and Overlap in Programs Highlight the Need to Identify, Assess, and Reduce Inefficiencies. GAO-12-491. Washington, D.C.: May 10, 2012.

Homeless Women Veterans: Actions Needed to Ensure Safe and Appropriate Housing. GAO-12-182. Washington, D.C.: December 23, 2011.

Homelessness: A Common Vocabulary Could Help Agencies Collaborate and Collect More Consistent Data. GAO-10-702. Washington, D.C.: June 30, 2010.

GAO Contact

Alicia Puente Cackley, Director, Financial Markets and Community Investment, cackleya@gao.gov, (202) 512-8678.

Goal Statement

Improve accuracy and reduce the amount of time it takes to process veterans’ disability benefit claims. By September 30, 2013, reduce the veterans’ disability compensation and pension claims backlog to 40 percent from 60.2 percent while achieving 90 percent rating accuracy, up from 83.8 percent, in pursuit of eliminating the veterans’ claims backlog (defined as claims pending more than 125 days) by 2015.

GAO Comments

As we and other organizations have reported over the last decade, VA has faced challenges improving the accuracy and timeliness of its disability claims process. VA’s disability compensation benefits program has been included in our High Risk List, under “Improving and Modernizing Federal Disability Programs,” since 2003. Our December 2012 report on VA claims processing found that VA’s disability claims backlog—defined as claims awaiting a decision over 125 days—had more than tripled since September 2009. In fact, two-thirds of all disability claims awaiting a decision in August 2012 met VA’s backlog definition. Moreover, the timeliness of disability claims processing over the last several years has worsened: the average length of time to complete a claim increased from 161 days in 2009 to 260 days in 2012. The number of disability claims received is likely to remain high as VA projects that 1 million service members will become veterans over the next 5 years, portending ongoing challenges for VA to meet its goal of processing claims within 125 days by 2015.

While our December 2012 report did not look at claims processing accuracy, the VA IG established a benefits inspection program in March 2009 which examines claims processing accuracy at VA regional offices. Based on a review of the VA IG’s benefits inspection findings across 21 VA regional offices in fiscal year 2012, accuracy rates ranged from 40 to 87 percent per office for a sample of selected types of claims.

In December 2012, we reported that the Veterans Benefits Administration (VBA) has a number of ongoing initiatives designed to improve claims processing and help VA meet its timeliness goals, but the impact of some initiatives is uncertain:

The Fully Developed Claims program, implemented nationally in June 2010, provides priority processing to veterans who submit claims with all relevant private medical evidence. The average processing time for claims involved in the program is 98 days, but veteran participation in the program has been low—only 4 percent of all compensation claims submitted in 2012—minimizing the impact on VA’s claims backlog.

The Claims Organizational Model, which reorganizes claim staff into cross-functional teams, processes claims by complexity, and redesigns mailroom functions, was piloted in 3 regional offices in March 2012 and implemented in all regional offices as of March 2013.

VBA developed standard medical forms—called Disability Benefits Questionnaires—designed to speed up the claims process by more accurately capturing medical evidence needed from medical providers. Although VBA tracks the number and completeness of questionnaires submitted, VBA is not measuring their impact on processing time. In 2010, VBA began to develop the Veterans Benefit Management System (VBMS), an initiative to help streamline the claims process and reduce processing times. According to VA officials, VBMS is intended to convert existing paper-based claims folders into electronic claims folders that will allow VBA employees electronic access to claims and their support evidence. Once completed, VBMS will allow veterans, physicians, and other external parties to submit claims and supporting evidence electronically. In August 2012, VBA officials told us that VBMS was not ready for national deployment, citing delays in scanning claims folders into VBMS as well as other software performance issues. A recent VA IG report also concluded that VBMS has experienced some performance issues and the scanning and digitization of claims lacked a detailed plan. However, according to VA, as of December 2012, 18 regional offices had implemented VBMS and all regional offices will implement VBMS by the end of calendar year 2013.

In our December 2012 report, we stated that without a comprehensive plan to strategically manage resources and evaluate the effectiveness of each initiative, VBA risks spending limited resources on initiatives that may not speed up disability claims processes. In response to our December 2012 report, on January 25, 2013, VA published the Strategic Plan to Eliminate the Compensation Claims Backlog.

Status of Key Open Recommendations and Matters for Congressional Consideration

In a December 2012 report, we reviewed the timeliness of VA claims processing and recommended that the Secretary of Veterans Affairs direct the Veterans Benefits Administration to: 1. Develop improvements for partnering with relevant federal and state military officials to reduce the time it takes to gather military service records for National Guard and Reserve sources. 2. Develop improvements for partnering with Social Security Administration (SSA) officials to reduce the time it takes to gather SSA medical records. 3. Ensure the development of a robust backlog reduction plan for VBA’s initiatives that, among other best practice elements, identifies implementation risks and strategies to address them and performance goals that incorporate the impact of individual initiatives on processing timeliness.

VA generally concurred with our recommendations, and has taken steps to address the recommendations. For example, VA stated it has recently initiated several interagency efforts to improve receipt of military service records. According to VA, on December 3, 2012, the joint VBA and DOD Disability Claims Reduction Task Force met to begin to evaluate the process to request records, among other issues, with the aim of improving the timeliness of record exchanges between the two agencies. In addition, the National Guard Bureau and the VA recently agreed to create a collaboration group that will examine ways to improve the timeliness and completeness of the records submitted in support of VA benefit claims. Furthermore, VA officials stated that VBA staff are currently meeting with SSA on a weekly basis to develop strategies to improve the records acquisition process and piloting a tool with four VA regional offices to provide VA staff with direct electronic access to SSA medical records. We believe these initiatives are heading in the right direction in order to improve the timeliness of meeting VA requests for SSA medical records and National Guard and Reservists records.

VA agreed with our recommendation to develop a robust backlog plan for VBA’s initiatives, and subsequent to our report, published the Strategic Plan to Eliminate the Compensation Claims Backlog, which identifies implementation risks as well as tracks overall performance based on a number of metrics, including processing timeliness. However, this plan does not provide individual performance goals and metrics for all initiatives, which are needed to ensure VA is spending its limited resources on initiatives that are proven to speed up disability claims and appeals processes.

Selected GAO Reports

High-Risk Series: An Update. GAO-13-283. Washington, D.C.: February 14, 2013.

Veterans’ Disability Benefits: Timely Processing Remains a Daunting Challenge. GAO-13-89. Washington, D.C.: December 21, 2012.

VA Disability Compensation: Actions Needed to Address Hurdles Facing Program Modernization. GAO-12-846. Washington, D.C.: September 10, 2012.

Veterans Disability Benefits: Clearer Information for Veterans and Additional Performance Measures Could Improve Appeal Process. GAO-11-812. Washington, D.C.: September 29, 2011.

Veterans’ Disability Benefits: Further Evaluation of Ongoing Initiatives Could Help Identify Effective Approaches for Improving Claims Processing. GAO-10-213. Washington, D.C.: January 29, 2010.

GAO Contact

Daniel Bertoni, Director, Education, Workforce, and Income Security Issues, bertonid@gao.gov, 202-512-7215.

Goal Statement

Improve awareness of VA services and benefits by increasing the timeliness and relevance of on-line information available to veterans, service members and eligible beneficiaries. By September 30, 2013, increase the number of registered eBenefits users from 1.0 million to 2.5 million.

GAO Comments

While we have not specifically looked at VA’s eBenefits efforts, our past work has identified challenges VA has faced in its efforts to increase awareness of its services and benefits which could be applicable to its ability to achieve this goal.

In February 2011, we reported that VA has a variety of activities to reach out to and support veterans and service members who may be eligible for VA education benefits, including the posting of information on its Web site to support those individuals in the process of applying for education benefits. We found that veterans service organizations, school officials, and students receiving VA education benefits had positive feedback for a recent redesign of the GI Bill web site that highlighted the three main steps in applying for Post-9/11 GI Bill benefits. However, we also found that VA did not provide links on the GI Bill Web site to consumer-focused information generated by other entities. In contrast, the Department of Education’s College Navigator site aggregated, for example, information on graduation rates, loan default rates, costs of attendance and available scholarships. Moreover, we found that little was known about the effectiveness of VA’s education outreach and support because VA did not have outcome-oriented performance measures for these activities. For example, while VA’s education program estimates the number of people who view or listen to a particular Post-9/11 GI Bill online, radio, or print advertisement, VA had not determined the extent to which its outreach campaign has been effective in informing or changing the behavior of target audiences.

In December 2011, we reported that VA and the Department of Defense had recently developed the eBenefits web portal to provide veterans with customized information on VA benefits and assistance, and how to apply for them, but the portal did not include a direct link to information on enhanced monthly benefits (which increase recipients’ monthly disability compensation or pension payments). A few of our focus group participants—conducted with veterans and their family representatives— commented that they had difficulty finding information about enhanced monthly benefits on VA’s website. Federal website guidelines recommend that navigation procedures to access online information should be simple and that links should be properly labeled to help users obtain desired results.

In December 2012, we reported on the status of VBA’s recent efforts to improve disability claims and appeals processing timeliness. In that report, we noted that veterans can learn about the status of their claims in several ways, including the use of eBenefits. However, we did not review veterans’ actual use of eBenefits.

Status of Key Open Recommendations and Matters for Congressional Consideration

In our February 2011 report, to improve VA’s outreach and support for eligible service members and veterans, communication with school officials, and oversight of its education benefit programs, we recommended that the Secretary of Veterans Affairs, among other actions, (1) develop outcome-oriented performance measures for outreach to service members and veterans who are seeking VA education benefits and (2) establish performance measures for the quality of information provided by VA’s toll-free hotline and for the timeliness and quality of its Right Now Web service. VA concurred with these recommendations in commenting upon a draft of the report. With respect to the first recommendation, VA reported that it had deployed an early communication tool to inform service members and veterans about their eligibility for education benefits and, as of April 2013, reported that it was in the first phase of capturing data on the frequency of visits to dedicated website Uniform Resource Locators. VA anticipated it will be able to establish baseline performance measures by the end of July 2013. With respect to measuring the quality of VA’s customer service on its toll-free hotline and its online Right Now Web service, VA has established applicable national performance standards and, in April 2013, VA reported that the standards have been issued to its field. We have requested to review the performance standards prior to considering this recommendation fully implemented, and are awaiting further status updates with regard to the implementation of the first recommendation.

Selected GAO Reports

Veterans’ Disability Benefits: Timely Processing Remains a Daunting Challenge. GAO-13-89. Washington, D.C.: December 21, 2012.

VA Enhanced Monthly Benefits: Recipient Population Is Changing, and Awareness Could Be Improved. GAO-12-153. Washington, D.C.: December 14, 2011.

VA Education Benefits: Actions Taken, but Outreach and Oversight Could Be Improved. GAO-11-256. Washington, D.C.: February 28, 2011.

GAO Contact

Daniel Bertoni, Director, Education, Workforce, and Income Security Issues, bertonid@gao.gov, 202-512-7215.

Appendix VI: Comments on the Office of Personnel Management’s Priority Goals

Based on our past work, we commented on each of OPM’s five priority goals for 2012 to 2013: 1. Ensure high quality federal employees. By September 30, 2013, increase federal manager satisfaction with applicant quality (as an indicator of hiring quality) from 7.7 to 8.3 on a scale of 1 to 10, while continually improving timeliness, applicant satisfaction, and other hiring process efficiency and quality measures. 2. Improve performance culture in the Goals-Engagement- Accountability-Results (GEAR) pilot agencies to inform the development of government-wide policies. By September 30, 2013, employee responses to the annual Employee Viewpoint Survey in each of the five agencies participating in a performance culture pilot project will increase by 5 percent or greater on the results-oriented culture index and the conditions for employee engagement index, using 2011 survey results as the baseline. 3. Increase health insurance choices for Americans. By October 1, 2013, expand competition within health insurance markets by ensuring participation of at least two multi-state health plans in the State Affordable Insurance Exchanges. 4. Maintain speed of national security background investigations.

Through September 30, 2013, maintain a 40 day or less average completion time for the fastest 90 percent of initial national security investigations. 5. Reduce federal retirement processing time. By July 31, 2013, Retirement Services will have reduced its case inventory so that 90 percent of all claims will be adjudicated within 60 days.

For each goal, we also identify our related past reports and provide an update on the status of any open recommendations and matters for congressional consideration that we previously made related to the goal. We also identify a GAO contact for our work related to each goal.

Goal Statement

Ensure high quality federal employees. By September 30, 2013, increase federal manager satisfaction with applicant quality (as an indicator of hiring quality) from 7.7 to 8.3 on a scale of 1 to 10, while continually improving timeliness, applicant satisfaction, and other hiring process efficiency and quality measures.

GAO Comments

We have not specifically reported on OPM’s ability to ensure high-quality federal employees by increasing satisfaction with applicant quality. However, our past work has highlighted efforts OPM has undertaken to recruit and maintain a high-quality workforce.

Our past work on OPM’s efforts to improve the federal government’s competitiveness in recruiting and maintaining a high-quality workforce has shown that in 2005, and again in 2008, OPM issued guidance on the use of hiring authorities and flexibilities. As we reported in September 2012, in 2006 OPM developed the Hiring Toolkit to assist agency officials in determining the appropriate hiring flexibilities to use given their specific situations, and in 2008 OPM launched an 80-day hiring model to help speed up the hiring process. Also in 2008, OPM established standardized vacancy announcement templates for common occupations, such as contract specialist and accounting technician positions, in which agencies can insert summary information concerning their specific jobs prior to posting for public announcement. In 2012, OPM issued regulations launching the Pathways program in order to make it easier to recruit and hire students and recent graduates and allow for noncompetitive conversion to permanent positions after meeting certain requirements. If successfully implemented, initiatives such as Pathways could help agencies further close critical skills gaps. We narrowed the scope of the human capital high-risk area in February 2011 to focus on this challenge of closing mission critical skills gaps and although progress has been made, the area remains on our recently issued High Risk List in February 2013.

In January 2010, we reported on the use of recruitment, relocation, and retention (3R) incentives at the Food and Drug Administration and the oversight provided by Health and Human Services, and how OPM provides oversight to agency 3R programs. We found that these flexibilities were widely used by agencies, and that retention incentives accounted for the majority of these incentive costs. Federal 3R incentives are among the human capital flexibilities intended to help federal agencies address human capital challenges and to build and maintain a high-performing workforce with essential skills and competencies. According to OPM, the 3R incentives are intended to provide agencies with discretionary authority to use compensation other than base pay to help recruit, relocate, and retain employees in difficult staffing situations. Our review of the steps OPM has taken to help ensure that agencies have effective oversight of their incentive programs found that while OPM provided oversight of such incentives through various mechanisms, including guidance and periodic evaluations and accountability reviews, there are opportunities for improvement.

Status of Key Open Recommendations and Matters for Congressional Consideration

In January 2010, we recommended that the Director of OPM require agencies to incorporate succession planning efforts into the decision process for awarding retention incentives and document this requirement for succession planning in their 3R incentive plans. In January 2011, OPM issued proposed regulations to add succession planning to the list of factors an agency may consider before approving a retention incentive for an employee who would be likely to leave the federal service in the absence of the incentive. OPM stated that specifically listing this factor in the regulations will strengthen the relationship between succession planning and retention incentives. OPM anticipates publishing the final regulations this year.

Selected GAO Reports

High Risk Series: An Update. GAO-13-283. Washington, D.C.: February, 14, 2013.

Human Capital Management: Effectively Implementing Reforms and Closing Critical Skills Gaps are Key to Addressing Federal Workforce Challenges. GAO-12-1023T. Washington, D.C.: September 19, 2012.

Human Capital: Continued Opportunities Exist for FDA and OPM to Improve Oversight of Recruitment, Relocation, and Retention Incentives. GAO-10-226. Washington, D.C.: January 22, 2010.

GAO Contacts

Robert Goldenkoff, Director, Strategic Issues, goldenkoffr@gao.gov, 202- 512-2757.

Yvonne Jones, Director, Strategic Issues, jonesy@gao.gov, 202-512- 2717.

Goal Statement

Improve performance culture in the GEAR pilot agencies to inform the development of government-wide policies. By September 30, 2013, employee responses to the annual Employee Viewpoint Survey in each of the five agencies participating in a performance culture pilot project will increase by 5 percent or greater on the results-oriented culture index and the conditions for employee engagement index, using 2011 survey results as the baseline.

GAO Comments

We have not specifically reported on improving the performance culture of the five GEAR pilot agencies—OPM, the Departments of Energy, Housing and Urban Development, and Veterans Affairs, and the U.S. GEAR is an effort to create high-performing organizations Coast Guard.that are aligned, accountable, and focused on results. Our past work has highlighted steps that OPM and agencies should take to improve their performance cultures.

In March 2003, we reported that effective performance management systems are not merely used for once or twice-yearly individual expectation setting and rating processes, but are tools to help the organization manage on a day-to-day basis. We identified key practices that create a clear linkage—”line of sight”—between individual performance and organizational success and, thus, transform agency cultures to be more results-oriented, customer-focused, and collaborative in nature. These key practices are: (1) align individual performance expectations with organizational goals; (2) connect performance expectations to crosscutting goals; (3) provide and routinely use performance information to track organizational priorities; (4) require follow-up actions to address organizational priorities; (5) use competencies to provide a fuller assessment of performance; (6) link pay to individual and organizational performance; (7) make meaningful distinctions in performance; (8) involve employees and stakeholders to gain ownership of performance management systems; and (9) maintain continuity during transitions.

As the federal government’s human capital leader, OPM must have the capacity to effectively assist agencies and to successfully lead and implement these important human capital management transformations. In January 2007, we reported that to enhance its capacity to do so, OPM is working to transform its own organization from less of a rulemaker, enforcer, and independent agent to more of a consultant, toolmaker, and strategic partner. We recommended that OPM reexamine its agency-wide skills and competencies in light of its updated strategic management document. OPM implemented this recommendation in 2008 by completing an agencywide competency assessment of all mission critical occupations. As reform initiatives move forward, it is increasingly important for OPM to complete this transformation and clearly demonstrate its capacity to lead and implement such reforms.

Status of Key Open Recommendations and Matters for Congressional Consideration

We currently have no open recommendations or matters for congressional consideration related to this priority goal.

Selected GAO Reports

Results-Oriented Management: Opportunities Exist for Refining the Oversight and Implementation of the Senior Executive Performance- Based Pay System. GAO-09-82. Washington, D.C.: November 21, 2008.

Office of Personnel Management: Key Lessons Learned to Date for Strengthening Capacity to Lead and Implement Human Capital Reforms. GAO-07-90. Washington, D.C.: January 19, 2007.

Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success. GAO-03-488. Washington, D.C: March 14, 2003.

GAO Contact

Robert Goldenkoff, Director, Strategic Issues, goldenkoffr@gao.gov, (202) 512-2757.

Yvonne Jones, Director, Strategic Issues, jonesy@gao.gov, (202) 512- 2717.

Goal Statement

Increase health insurance choices for Americans. By October 1, 2013, expand competition within health insurance markets by ensuring participation of at least two multi-state health plans in the State Affordable Insurance Exchanges.

GAO Comments

Our past work has not specifically focused on the priority goal of ensuring the participation of at least two multi-state health plans in each insurance exchange. However, we know from prior work that OPM had awarded a contract by early 2011 to provide policy and analytical support for this effort, and in March of 2012 it had issued a notice of proposed rulemaking.

The achievement of this goal will require OPM to contract with multiple private health insurance issuers and to coordinate closely with HHS, which is partnering with states to assure an operating Affordable Insurance Exchange in each state by January 1, 2014. Our work on OPM’s role overseeing the Federal Employees Health Benefits Program and its role in implementing the high risk pool program under the Patient Protection and Affordable Care Act are illustrative of OPM’s experience in two activities central to the achievement of this goal.

Negotiate and contract with health insurance issuers. Through its oversight of the Federal Employees Health Benefits Program, OPM has long been responsible for selecting, contracting with, and regulating hundreds of health insurance issuers that offer health plans to millions of federal employees, dependents and retirees, as well as negotiating benefits and premium rates, as we reported in December 2002. OPM will likely leverage this experience and these relationships to contract with issuers to offer plans through state exchanges.

Coordinate and collaborate with other agencies. In July 2011, we reported on OPM’s recent collaboration with HHS in implementing the new Pre-Existing Condition Insurance Plan (high risk pool) program required under the Patient Protection and Affordable Care Act. Under an interagency agreement, OPM assists with the administration of the program, including reviewing the performance of the health insurance issuer chosen to offer health plans within the federal program, and overseeing its operations on an ongoing basis.

Status of Key Open Recommendations and Matters for Congressional Consideration

We currently have no open recommendations or matters for congressional consideration related to this priority goal.

Selected GAO Reports

Patient Protection and Affordable Care Act: Contracts Awarded and Consultants Retained by Federal Departments and Agencies to Assist in Implementing the Act. GAO-11-797R. Washington, D.C.: July 14, 2011.

Pre-existing Condition Insurance Plans: Program Features, Early Enrollment and Spending Trends, and Federal Oversight Activities. GAO-11-662. Washington, D.C.: July 27, 2011.

Federal Employees’ Health Plans: Premium Growth and OPM’s Role in Negotiating Benefits. GAO-03-236. Washington, D.C.: December 31, 2002.

GAO Contacts

John E. Dicken, Director, Health Care, dickenj@gao.gov, 202-512-7043.

Stanley J. Czerwinski, Director, Strategic Issues, czerwinskis@gao.gov, 202-512-6520.

Goal Statement

Maintain speed of national security background investigations. Through September 30, 2013, maintain a 40 day or less average completion time for the fastest 90 percent of initial national security investigations.

GAO Comments

OPM’s priority goal of maintaining a 40 day or less average completion time for the fastest 90 percent of initial national security investigations is related to an area we previously designated as high risk. Specifically, in January 2005 we first placed DOD’s security clearance program—which comprises the vast majority of government-wide clearances—on our High Risk List because we identified significant delays in completing security clearances, which sometimes took up to a year to complete. OPM is currently the investigative service provider for the majority of the executive branch, including DOD. We removed the high-risk designation from DOD’s program in February 2011 due to both high-level attention from various executive branch agencies, including DOD, OMB, and the Director of National Intelligence and improvement in the timeliness of DOD clearances, among other things. For example, we found in January 2011 that DOD met the congressionally directed Intelligence Reform and Terrorism Prevention Act of 2004 goal of 40 days for initial investigations throughout fiscal year 2010. This timeliness measure does not include data on periodic reinvestigations. Timeliness data for investigations is OPM’s responsibility as the investigative service provider for DOD. For example, the fastest 90 percent of DOD initial clearance investigations were processed by OPM in an average of 35 days in fiscal year 2010. In addition, in 2010 the Performance Accountability Council (PAC) reported that OPM was meeting investigation timeliness goals for many of the agencies for which it conducts national security background investigations. However, we have not comprehensively reported on the timeliness statistics for all of the national security background investigations conducted by OPM. Instead, our previous work has focused on the timeliness of DOD clearances because DOD’s program was on our High Risk List.

While OPM conducts background investigations for most of the federal government, executive branch agencies conduct other phases in the federal government’s personnel security clearance process. For example, the requesting agency determines which positions—military, civilian, or private-industry contracts—require access to classified information and, therefore, which people must apply for and undergo a security clearance investigation. OPM, in turn, conducts these investigations using federal investigative standards and OPM internal guidance as criteria for collecting background information on applicants. Adjudicators from the requesting agencies use the information contained in the resulting OPM investigative reports and consider federal guidelines to determine whether an applicant is eligible for a personnel security clearance.

During the time while DOD’s security clearance program was on our High Risk List, the executive branch initiated actions to reform the government- wide security clearance process, in which OPM had a role as the investigative service provider. As part of this government-wide reform effort, Executive Order 13467 established a leadership structure by creating the PAC. The order appointed the Deputy Director for Management at OMB as the chair of the council and designated the Director of National Intelligence as the Security Executive Agent and the Director of OPM as the Suitability Executive Agent. The PAC is responsible for holding agencies accountable for the implementation of suitability, security, and, as appropriate, contractor employee fitness processes and procedures. In turn, the PAC issued a Strategic Framework in February 2010, which set forth a mission and strategic goals, performance measures, a communications strategy, roles and responsibilities, and metrics to measure the quality of security clearance investigations and adjudications. Some of the goals and performance measures developed by the PAC were aimed at addressing the timeliness of initial security clearance investigations. In addition to the timeliness of initial investigations and as the result of our work, members of Congress and federal agencies have expressed concerns about the quality of the background investigations. The leaders of the PAC also committed to measuring the quality of investigations by further developing quality metrics in a memorandum to Congress on May 31, 2010. While OPM’s agency priority goal to measure timeliness is important; it does not capture the competing priority of measuring the quality of the investigations. Finally, the Intelligence Reform and Terrorism Prevention Act required the executive branch to annually report on timeliness of background investigations; however, this requirement expired in 2011, so there is no mechanism to report the timeliness of the end-to-end clearance process—including timeliness of initiation and adjudication phases of the process, the timeliness of investigations that took OPM longer than 40 days to complete, or other security clearance reform- related goals—to Congressional oversight committees.

Status of Key Open Recommendations and Matters for Congressional Consideration

In our May 2009 report, we recommended that the Director of OPM direct the Associate Director of OPM’s Federal Investigative Services Division to measure the frequency with which its investigative reports meet federal investigative standards in order to improve the completeness of future investigation documentation. As of March 2013, OPM has not implemented the recommendation to measure how frequently investigative reports meet federal investigative standards. Instead, OPM assesses the quality of investigations based on voluntary reporting from customer agencies. Specifically, OPM tracks investigations that are (1) returned for rework from the requesting agency; (2) identified as deficient using a web-based survey; and (3) identified as deficient through adjudicator calls to OPM’s quality hotline. In our past work, we noted that the number of investigations returned for rework is not by itself a valid indicator of the quality of investigative work because adjudication officials said they were reluctant to return incomplete investigations in anticipation of delays that would impact timeliness. Further, relying on agencies to voluntarily provide information on investigation quality may not reflect the quality of OPM’s total investigation workload. One of OPM’s customer agencies, DOD developed and implemented a tool known as RAISE to monitor the quality of investigations completed by OPM, However, OPM does not use DOD’s tool. According to an OPM official, OPM is working through the PAC to decide how the executive branch will measure quality government-wide. While the PAC considered using DOD’s RAISE tool, among others, according to the OPM official, they opted to develop another tool that better captures quality. Further, the OPM official stated OPM’s intent to implement that tool once it is developed by the PAC, but did not provide an estimated timeframe for development and implementation. Our prior work noted that in May 2010, leaders of the reform effort provided congressional members with metrics assessing According to officials quality and other aspects of the clearance process.from one of the PAC’s working groups, these metrics were communicated to executive branch agencies in June 2010. RAISE was one tool the reform team members planned to use for measuring quality.

Selected GAO Reports

Personnel Security Clearances: Continuing Leadership and Attention Can Enhance Momentum Gained from Reform Effort. GAO-12-815T. Washington, D.C.: June 21, 2012.

Background Investigations: Office of Personnel Management Needs to Improve Transparency of Its Pricing and Seek Cost Savings. GAO-12-197. Washington, D.C.: February 28, 2012.

High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February 2011.

Personnel Security Clearances: Progress Has Been Made to Improve Timeliness but Continued Oversight Is Needed to Sustain Momentum. GAO-11-65. Washington, D.C.: November 19, 2010.

Personnel Security Clearances: An Outcome-Focused Strategy Is Needed to Guide Implementation of the Reformed Clearance Process. GAO-09-488. Washington, D.C.: May 19, 2009.

DOD Personnel Clearances: Comprehensive Timeliness Reporting, Complete Clearance Documentation, and Quality Measures Are Needed to Further Improve the Clearance Process. GAO-09-400. Washington, D.C.: May 19, 2009.

GAO Contact

Brenda S. Farrell, Director, Defense Capabilities and Management, farrellb@gao.gov, 202-512-3604.

Goal Statement

Reduce federal retirement processing time. By July 31, 2013, Retirement Services will have reduced its case inventory so that 90 percent of all claims will be adjudicated within 60 days.

GAO Comments

OPM’s efforts to reduce federal retirement processing time have included attempts over 2 decades to modernize its retirement processing system by automating paper-based processes and replacing antiquated information systems. However, these efforts have been unsuccessful due to weaknesses in key management practices. Our previous reviews have identified weaknesses in project management, risk management, organizational change management, system testing, cost estimating, and progress reporting. Specifically, in February 2005, we made recommendations to address weaknesses in the following areas:

Project management: OPM had defined major components of its retirement modernization effort, such as data conversion of paper files and development of electronic processes for capture and storage of data. However, it had not identified the dependencies among these efforts, increasing the risk that delays in one activity could have unforeseen impacts on the progress of others.

Risk management: OPM did not have a process for identifying and tracking project risks and mitigation strategies on a regular basis. Thus, OPM lacked a mechanism to address potential problems that could adversely impact the cost, schedule, and quality of the modernization effort.

Organizational change management: OPM had not adequately prepared its staff for changes to job responsibilities resulting from the modernization by developing a detailed transition plan. The absence of such a plan could lead to confusion about roles and responsibilities and hinder effective system implementation.

In January 2008, as OPM was on the verge of deploying an automated retirement processing system, we reported deficiencies in management capabilities, and made recommendations to address them:

Testing: The results of tests 1 month prior to the deployment of a major system component revealed that it had not performed as intended. These defects, along with a compressed testing schedule, increased the risk that the retirement processing system would not work as intended upon deployment.

Cost estimating: The cost estimate OPM developed for the modernization effort was not fully reliable. This meant that the agency did not have a sound basis for formulating budgets or developing a program baseline.

Progress reporting: The baseline against which OPM was measuring the progress of the program did not reflect the full scope of the program; this increased the risk that variances from planned performance would not be detected.

In April 2009, we reported that OPM continued to have deficiencies in its cost estimating, progress reporting, and testing practices and we made recommendations to address these deficiencies, as well as additional weaknesses in the planning and oversight of the modernization effort. OPM agreed with these recommendations and began to address them, but the agency cancelled its most recent large-scale retirement modernization effort in February 2011.

In November 2011, agency officials, including the Chief Information Officer, Chief Operating Officer, and Associate Director for Retirement Services, told us that OPM does not plan to initiate another large-scale effort to modernize the retirement process. Rather, the officials said the agency intends to take targeted steps to improve retirement processing. More recently, in January 2012, OPM released a new plan to improve retirement processing that aims at targeted, incremental improvements rather than a large-scale modernization. Under this plan, the agency expects to eliminate its retirement processing backlog by July 2013 and accurately process 90 percent of its cases within 60 days. To meet this goal, OPM reported that it plans to hire and train 76 new staff to address retirement claims; establish higher production standards and identify potential retirement process improvements; and work with other agencies to improve the accuracy and completeness of the data they provide to OPM for use in retirement processing. However, as we have previously noted in February 2012, the plan does not address improving or eliminating the legacy information systems that currently support retirement processing. Although we have not assessed OPM’s actions or progress toward fulfilling its January 2012 plan, Performance.gov was updated to include information about the agency’s progress in December 2012. For example, the agency reported that it had met its targets for hiring new staff, as well as improving the accuracy and completeness of retirement data other agencies provide to OPM.

Status of Key Open Recommendations and Matters for Congressional Consideration

We currently have no open recommendations or matters for congressional consideration related to this priority goal.

Selected GAO Reports

OPM Retirement Modernization: Progress Has Been Hindered by Longstanding Information Technology Weaknesses. GAO-12-430T. Washington, D.C.: February 1, 2012.

OPM Retirement Modernization: Longstanding Information Technology Management Weaknesses Need to Be Addressed. GAO-12-226T. Washington, D.C.: November 15, 2011.

Office of Personnel Management: Retirement Modernization Planning and Management Shortcomings Need to Be Addressed. GAO-09-529. Washington, D.C.: April 21, 2009.

Office of Personnel Management: Improvements Needed to Ensure Successful Retirement Systems Modernization. GAO-08-345. Washington, D.C.: January 31, 2008.

Office of Personnel Management: Retirement Systems Modernization Program Faces Numerous Challenges. GAO-05-237. Washington, D.C.: February 28, 2005.

GAO Contact

Valerie C. Melvin, Director, Information Management and Technology Resources Issues, melvinv@gao.gov, (202) 512-6304.

Appendix VII: Comments from the Department of Veterans Affairs

GAO Comments

1. Although our report included information about VA’s Strategic Plan to Eliminate the Compensation Claims Backlog in a subsequent paragraph, we revised the cited paragraph to note that VA, in response to our December 2012 report, published the plan on January 25, 2013. 2. We revised the cited paragraph to focus on VA’s Strategic Plan to Eliminate the Compensation Claims Backlog. However, as VA acknowledges in its comments, the plan does not provide individual performance goals and metrics for all initiatives. We continue to believe that without performance goals and measures clearly aligned with each of its initiatives, VA lacks assurance that it is spending its limited resources on proven methods to speed up disability claims and appeals processes.

Appendix VIII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the above contact, Elizabeth Curda (Assistant Director) and Benjamin T. Licht supervised this review and the development of the resulting report. Virginia Chanley, Karin Fangman, Patricia Norris, Daniel Ramsey, and Dan Webb made significant contributions to this report. Robert Gebhart, Donna Miller, Jessica Nierenberg, and Ulyana Panchishin also made key contributions.