prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: Concerned that the federal government was more focused on program activities and processes than the results to be achieved, Congress passed the Government Performance and Results Act of 1993 (GPRA)  GPRA sought to focus federal agencies on performance by requiring agencies to develop long-term and annual goals, and measure and report on progress towards those goals annually  Based on our analyses of the act’s implementation, we concluded in March 2004 that GPRA’s requirements had laid a solid foundation for results-oriented management  At that time, we found that performance planning and measurement had slowly yet increasingly become a part of agencies’ cultures  For example, managers reported having significantly more performance measures in 2003 than in 1997, when GPRA took effect government-wide  However, the benefit of collecting performance information is fully realized only when that information is actually used by managers to make decisions aimed at improving results  Although our 2003 survey found greater reported availability of performance information than in 1997, it also showed managers’ use of that information for various management activities generally had remained unchanged  Based on those results, and in response to a request from Congress, in September 2005, we developed a framework intended to help agencies better incorporate performance information into their decision making  As shown in figure 1, we identified five leading practices that can promote the use of performance information for policy and program decisions; and four ways agency managers can use performance information to make program decisions aimed at improving results  Our September 2005 report also highlighted examples of how agencies had used performance information to improve results  For example, we described how the Department of Transportation’s National Highway Traffic Safety Administration used performance information to identify, develop, and share effective strategies that increased national safety belt usage—which can decrease injuries and fatalities from traffic accidents— from 11 percent in 1985 to 80 percent in 2004  Subsequently, the GPRA Modernization Act of 2010 (GPRAMA) was enacted, which significantly expanded and enhanced the statutory framework for federal performance management  The Senate Committee on Homeland Security and Governmental Affairs report accompanying the bill that would become GPRAMA stated that agencies were not consistently using performance information to improve their management and results  The report cited the results of our 2007 survey of federal managers  That survey continued to show little change in managers’ use of performance information  The report further stated that provisions in GPRAMA are intended to address those findings and increase the use of performance information to improve performance and results  For example, GPRAMA requires certain agencies to designate a subset of their respective goals as their highest priorities—known as agency priority goals—and to measure and assess progress toward those goals at least quarterly through data-driven reviews  Our recent work and surveys suggest that data-driven reviews are having their intended effect  For example, in July 2015, we found that agencies reported that their reviews had positive effects on progress toward agency goals and efforts to improve the efficiency of operations, among other things  In addition, for those managers who were familiar with their agencies’ data-driven reviews, our 2013 and 2017 surveys showed that the more managers viewed their programs as being subject to a review, the more likely they were to report their agencies’ reviews were driving results and conducted in line with our leading practices  Recognizing the important role these reviews were playing in improving data-driven decision making, our management agenda for the presidential and congressional transition in 2017 included a key action to expand the use of data-driven reviews beyond agency priority goals to other agency goals  More broadly, our recent surveys of federal managers have continued to show that reported government-wide uses of performance information generally have not changed or in some cases have declined  As we found in September 2017, and as illustrated in figure 2, the 2017 update to our index suggests that government-wide use of performance information did not improve between 2013 and 2017  In addition, it is statistically significantly lower relative to our 2007 survey, when we created the index  Moreover, in looking at the government-wide results on the 11 individual survey questions that comprise the index, we found few statistically significant changes in 2017 when compared to (1) our 2013 survey or (2) the year each question was first introduced  For example, in comparing 2013 and 2017 results, two questions had results that were statistically significantly different: The percentage of managers who reported that employees who report to them pay attention to their agency’s use of performance information was statistically significantly higher (from 40 to 46 percent)  The percentage of managers who reported using performance information to adopt new program approaches or change work processes was statistically significantly lower (from 54 to 47 percent)  As we stated in our September 2017 report, the decline on the latter question was of particular concern as agencies were developing plans to improve their efficiency, effectiveness, and accountability, as called for by an April 2017 memorandum from OMB  In early 2017, the administration announced several efforts intended to improve government performance  OMB issued several memorandums detailing the administration’s plans to improve government performance by reorganizing the government, reducing the federal workforce, and reducing federal agency burden  As part of the reorganization efforts, OMB and agencies were to develop government-wide and agency reform plans, respectively, designed to leverage various GPRAMA provisions  For instance, the April 2017 memorandum mentioned above stated that OMB intends to monitor implementation of the reforms using, among other things, agency priority goals  While many agency-specific organizational improvements were included in the President’s fiscal year 2019 budget, released in February 2018, OMB published additional government-wide and agency reform proposals in June 2018  The President’s Management Agenda (PMA), released in March 2018, outlines a long-term vision for modernizing federal operations and improving the ability of agencies to achieve outcomes  To address the issues outlined in the PMA, the administration established a number of cross-agency priority (CAP) goals  CAP goals, required by GPRAMA, are to address issues in a limited number of policy areas requiring action across multiple agencies, or management improvements that are needed across the government  The PMA highlights several root causes for the challenges the federal government faces  Among them is that agencies do not consistently apply data-driven decision-making practices  The PMA states that smarter use of data and evidence is needed to orient decisions and accountability around service and results  To that end, in March 2018, the administration established the Leveraging Data as a Strategic Asset CAP goal to improve the use of data in decision making to increase the federal government’s effectiveness  Over the past 25 years, various organizations, roles, and responsibilities have been created by executive action or in law to provide leadership in federal performance management  At individual agencies and across the federal government, these organizations and officials have key responsibilities for improving performance, as outlined below  OMB: At least every four years, OMB is to coordinate with other agencies to develop CAP goals—such as the one described earlier on leveraging data as an asset—to improve the performance and management of the federal government  OMB is also required to coordinate with agencies to develop annual federal government performance plans to define, among other things, the level of performance to be achieved toward the CAP goals  Following GPRAMA’s enactment, OMB issued guidance for initial implementation, as required by the act, and continues to provide updated guidance in its annual Circular No  A-11, additional memorandums, and other means  Chief Operating Officer (COO): The deputy agency head, or equivalent, is designated as the COO, with overall responsibility for improving agency management and performance through, among other things, the use of performance information  President’s Management Council (PMC): The PMC is comprised of OMB’s Deputy Director for Management and the COOs of major departments and agencies, among other individuals  Its responsibilities include improving overall executive branch management and implementing the PMA  Performance Improvement Officer (PIO): Agency heads designate a senior executive as the PIO, who reports directly to the COO  The PIO is responsible for assisting the head of the agency and COO to ensure that agency goals are achieved through, among other things, the use of performance information  Performance Improvement Council (PIC): The PIC is charged with assisting OMB to improve the performance of the federal government  It is chaired by the Deputy Director for Management at OMB and includes PIOs from each of the 24 Chief Financial Officers Act agencies, as well as other PIOs and individuals designated by the chair  Among its responsibilities, the PIC is to work to resolve government-wide or cross-cutting performance issues, and facilitate the exchange among agencies of practices that have led to performance improvements  Previously, the General Service Administration’s (GSA) Office of Executive Councils provided analytical, management, and administrative support for the PIC, the PMC, and other government-wide management councils  In January 2018, the office was abolished and its functions, staff, and authorities, along with those of the Unified Shared Services Management Office, were reallocated to GSA’s newly created Shared Solutions and Performance Improvement Office  As at the government-wide level—where, as described earlier, the use of performance information did not change from 2013 to 2017—managers’ reported use of performance information at most agencies also did not improve since 2013 (illustrated in figure 3)  At the agency level, 3 of the 24 agencies had statistically significant changes in their index scores—1 increase (National Science Foundation) and 2 decreases (Social Security Administration and the Office of Personnel Management)  Also, in 2017, 6 agencies had results that were statistically significantly different—4 higher and 2 lower—than the government-wide average (see sidebar)  Throughout the report, we highlight two different types of statistically significant results—changes from our last survey in 2013 and differences from the 2017 government-wide average  The former indicates when an agency’s reported use of performance information or leading practices has measurably improved or declined  The latter indicates when it is statistically significantly higher or lower than the rest of government  These results suggest agencies have taken actions that led to improvements in their use of performance information  For example, when a result is a statistically significant increase since 2013, as with the National Science Foundation index score in 2017, this suggests that the agency has adopted practices that led to a measurable increase in the use of performance information by managers  When a result is statistically significantly higher than the government-wide average, like GSA’s 2017 index score, this suggests that the agency’s use of performance information is among the highest results when compared to the rest of government  These agencies could also have insights into practices that led to relatively high levels of performance information use  Finally, when a result is a statistically significant decrease since 2013, as with the Social Security Administration’s index score in 2017, or statistically significantly lower than the government-wide average, like the Department of Homeland Security’s 2017 index score, this suggests the agencies face challenges that are hampering their ability to use performance information  Appendix III provides each agency’s index scores from 2007, 2013, and 2017 to show changes between survey years  When we disaggregated the index and analyzed responses from the 11 questions that comprise the index—which could help pinpoint particular actions that improved the use of performance information—we similarly found relatively few changes in agencies’ recent results  Specifically, we identified 16 instances where agency responses on individual questions were statistically significantly different from 2013 to 2017—10 increases and 6 decreases  This represents about 6 percent of the total possible responses to the 11 survey questions from each of the agencies  In addition, we found 12 instances where an agency’s result on a question was statistically significantly higher (11) or lower (1) than the government-wide average in 2017  For example, the percentage of Social Security Administration (SSA) managers reporting that their peers use performance information to share effective approaches was statistically significantly higher than the government-wide average  Although SSA’s index score had a statistically significant decline in 2017 compared to 2013, the agency’s index score remains relatively high, as it has in prior years  The scope of our work has not allowed us to determine definitively what factors caused the decline in SSA’s index score and whether the decline is likely to continue, although its result on this particular question may indicate a continued strength  Each agency’s results on the 11 questions that comprise the index are presented in appendix I  The agencies’ respective statistically significant results are identified in figure 4  While some agencies had statistically significant improvements on individual questions, and could point to actions that led to improvements in their use of performance information, these improvements should be considered in relation to the range of agency results and the government- wide average  In figure 4, there are five agencies with statistically significant increases on responses to individual questions, where those results were not statistically significantly higher than the government-wide average (see arrows without plus signs for the Departments of Agriculture, Defense, and Justice; the Environmental Protection Agency; and the National Science Foundation)  While these represent improvements, they should be considered in relation to the range of agency results and the government-wide average (provided in detail in the agency summaries in appendix I)  For example, in 2017, the percentage of managers at the Department of Agriculture who reported that upper management use performance information to inform decisions about program changes was statistically significantly higher than in 2013  However, the department’s 2017 result (37 percent) was relatively lower when compared to the maximum agency result on that question (60 percent)  Appendix I presents the results on the index and the 11 questions that comprise it for each of the 24 agencies  When we compared government-wide and agency-level results on selected survey questions that reflect practices that promote the use of performance information, we found that results between 2013 and 2017 generally remained unchanged  As described earlier, there are 10 survey questions that both reflect the five leading practices identified in our past work and had statistically significant associations with higher index scores  As shown in figure 5, government-wide results on 2 of the 10 questions were statistically significantly different, both increases, from 2013 to 2017  Despite these two increases, the overall results suggest these practices are not widely followed government-wide  On most of the 10 questions, only about half (or fewer) of the managers reported their agencies were following them to a “great” or “very great” extent  When we analyzed agency-level responses to these 10 questions, we also found relatively few changes in recent results  Specifically, our analysis found 20 instances—16 increases and 4 decreases—where agencies’ responses on individual questions were statistically significantly different from 2013 to 2017  This represents about 8 percent of the total possible responses to the 10 survey questions from each of the agencies  In addition, we found 10 instances where an agency’s result on a question was statistically significantly higher (8) or lower (2) than the government-wide average in 2017  Each agency’s results on these 10 questions are presented in appendix I, and the statistically significant results are identified in figure 6  Those agencies with results on individual questions that are either statistically significantly higher than 2013, higher than the 2017 government-wide average, or both may have taken actions in line with our leading practices for promoting the use of performance information  For example, the National Science Foundation had both types of statistically significant results on a question about having sufficient information on the validity of their performance data  Here, the agency’s result increased 27 percentage points from 2013 to 2017  While the scope of our review does not allow us to definitively determine the reasons for the National Science Foundation’s higher results, they suggest the agency has taken recent actions that greatly improved the availability and accessibility of information on the validity of performance data  In both 2013 and 2017, our analyses found this particular question to be the strongest predictor of higher performance information use when we tested for associations between the questions that reflect leading practices and our index  Our 2017 survey results show that managers who reported their programs were subject to data-driven reviews also were more likely to report using performance information in decision making to a greater extent (see figure 7)  For the 35 percent of managers who reported being familiar with data-driven reviews, those who reported their programs had been subject to data-driven reviews to a “great” or “very great” extent had index scores that were statistically significantly higher than those whose programs were subject to these reviews to a lesser extent  Similarly, we found that being subject to data-driven reviews to a greater extent was also related to greater reporting of agencies following practices that can promote the use of performance information  As figure 8 shows, managers who reported their programs were subject to these reviews to a “great” or “very great” extent more frequently reported that their agencies followed the five leading practices that promote the use of performance information, as measured by the 10 related survey questions associated with higher scores on the index  For example, of the estimated 48 percent of managers who reported their programs were subject to data-driven reviews to a “great” or “very great” extent, 72 percent also reported that managers at their level (peers) effectively communicate performance information on a routine basis to a “great” or “very great” extent  Conversely, for the 24 percent of managers who reported their programs were subject to data-driven reviews to a “small” or “no” extent, only 30 percent reported that managers at their level do this to a “great” or “very great” extent  Our past work has found that the Executive Branch has taken steps to improve the use of performance information in decision making by senior leaders at federal agencies  However, our survey results indicate those steps have not led to similar improvements in use by managers at lower levels  Through its guidance to implement GPRAMA, OMB developed a framework for performance management in the federal government that involves agencies setting goals and priorities, measuring performance, and regularly reviewing and reporting on progress  This includes expectations for how agency senior leaders should use performance information to assess progress towards achieving agency priority goals through data-driven reviews, and strategic objectives through strategic reviews  For example, GPRAMA requires, and OMB’s guidance reinforces, that data-driven reviews should involve the agency head, Chief Operating Officer, Performance Improvement Officer, and other senior officials responsible for leading efforts to achieve each goal  OMB’s guidance also identifies ways in which agency leaders should use the results of those reviews to inform various decision-making activities, such as revising strategies, formulating budgets, and managing risks  Our past work also found that agencies made progress in implementing these reviews and using performance information  In July 2015, we found that agencies generally were conducting their data-driven reviews in line with GPRAMA requirements and our related leading practices, including that agency leaders used the reviews to drive performance improvement  In addition, in September 2017, we reported on selected agencies’ experiences in implementing strategic reviews and found that the reviews helped direct leadership attention to progress on strategic objectives  Despite those findings, our survey results continue to show that the reported use of performance information by federal managers has generally not improved, and actually declined at some agencies  This could be because of the two different groups of agency officials covered by our work  GPRAMA’s requirements, and the federal performance management framework established by OMB’s guidance, apply at the agency-wide level and generally involve senior leaders  Our past work reviewing implementation of the act therefore focused on improvements in the use of performance information by senior leaders at the agency- wide level  In contrast, our surveys covered random samples of mid- and upper-level managers within those agencies, including at lower organizational levels such as component agencies  Their responses indicate that the use of performance information more broadly within agencies—at lower organizational levels—generally has not improved over time  The exception to this was managers whose programs were subject to the data-driven reviews required by GPRAMA  As described above, those managers were more likely to report greater use of performance information in their agencies  This reinforces the value of the processes and practices put in place by GPRAMA  Our survey results suggest that limited actions have been taken to diffuse processes and practices related to the use of performance information to lower levels within federal agencies, where mid-level and senior managers make decisions about managing programs and operations  Although OMB staff agreed that diffusing processes and practices to lower levels could lead to improved use of performance information, they told us they have not directed agencies to do so for a few reasons  First, OMB staff expressed concerns about potentially imposing a “one-size-fits- all” approach on agencies  They stated that agencies are best positioned to improve their managers’ use of performance information, given their individual and unique missions and cultures, and the environments in which they operate  We agree that it makes sense for agencies to be able to tailor their approaches for those reasons  OMB’s existing guidance provides an overarching framework that recognizes the need for flexibility and for agencies to tailor their approaches  Moreover, given the long- standing and cross-cutting nature of this challenge, a government-wide approach also would provide a consistent focus on improving the use of performance information more extensively within agencies  OMB staff also told us that they believed it would go beyond their mandate to direct agencies to extend GPRAMA requirements to lower levels  GPRAMA requires OMB to provide guidance to agencies to implement its requirements, which only apply at the agency-wide level  As noted earlier, however, GPRAMA also requires OMB to develop cross- agency priority (CAP) goals to improve the performance and management of the federal government  The President’s Management Agenda established a CAP goal to leverage data as a strategic asset, in part, to improve the use of data for decision making and accountability throughout the federal government  This new CAP goal presents an opportunity for OMB and agencies to identify actions to expand the use of performance information in decision making throughout agencies  As of June 2018, the action plan for implementing the Leveraging Data as a Strategic Asset CAP goal is limited  According to the President’s Management Agenda and initial CAP goal action plan, the goal primarily focuses on developing and implementing a long-term, enterprise-wide federal data strategy to better govern and leverage the federal government’s data  It is through this strategy that, among other things, the administration intends to improve the use of data for decision making and accountability  However, the strategy is under development and not expected to be released until January 2019, with a related plan to implement it expected in April 2019  The existing action plan, released in March 2018 and updated in June 2018, does not yet include specific steps needed to improve the use of data—including performance information—more extensively within agencies  According to the action plan for the goal, potential actions currently under consideration focus on establishing agency “learning agendas” that prioritize the development and use of data and other evidence for decision-making; building agency capacity to use data and other evidence; and improving the timeliness of performance information and other data, and making that information available to decision makers and the public  Although developing learning agendas and building capacity could help improve the use of performance information in agencies, improving availability of data may be less effective  For example, as our past survey results have shown, increasing the availability of performance information has not resulted in corresponding increases in its use in decision making  We recognize that the CAP goal was created in March 2018  Nonetheless, it is important that OMB and its fellow goal leaders develop the action plan and related federal data strategy consistent with all key requirements to better ensure successful implementation  The action plan does not yet include complete information related to the following GPRAMA requirements: performance goals that define the level of performance to be achieved each year for the CAP goal; the various federal agencies, organizations, programs, and other activities that contribute to the CAP goal; performance measures to assess overall progress towards the goal as well as the progress of each agency, program, and other activity contributing to the goal; and clearly defined quarterly targets  Consistent with GPRAMA, Standards for Internal Control in the Federal Government identifies information that agencies are required to include in their plans to help ensure they achieve their goals  The standards state that objectives—such as improving the use of data in decision making— should be clearly defined to enable the identification of risks  Objectives are to be defined in specific terms so they can be understood at all levels of the entity—in this case, government-wide as well as within individual agencies  This involves defining what is to be achieved, who is to achieve it, how it will be achieved, and the time frames for achievement  Ensuring that future updates to the new CAP goal’s action plan includes all required elements is particularly important, as our previous work has found that some past CAP goal teams did not meet all planning and reporting requirements  For example, in May 2016 we found that most of the CAP goal teams we reviewed had not established targets for all performance measures they were tracking  This limited the transparency of their efforts and the ability to track progress toward established goals  We recommended that OMB, working with the Performance Information Council (PIC), report on actions that CAP goal teams are taking, or plan to take, to develop such targets and performance measures  OMB staff generally agreed and, in July 2017, told us they were working, where possible, to assist the development of measures for CAP goals  However, the recommendation has not been addressed and OMB staff said the next opportunity to address it would be when the administration established new CAP goals (which took place in March 2018)  Following the initial release of the new CAP goals, CAP goal teams are to more fully develop the related action plans through quarterly updates  Given the ongoing importance of meeting these planning and reporting requirements, we will continue to monitor the status of actions to address this recommendation as implementation of the new CAP goals proceeds  While the PIC, which is chaired by OMB, has contributed to efforts to enhance the use of performance information, our survey results identify additional opportunities to further those efforts  The PIC’s past efforts have included hosting various working groups and learning events for agency officials to provide performance management guidance, and developing resources with relevant practices  For example, the PIC created a working group focused on agency performance reviews, which was used to share recommendations for how agencies can implement reviews, along with a guide with practices for effectively implementing strategic reviews  In January 2018, staff supporting the PIC joined with staff from another GSA office to create a new group called Fed2Fed Solutions  This group consults with agencies and provides tailored support, such as data analysis and performance management training for agency officials, to help them address specific challenges related to organizational transformation, data-driven decision making, and other management improvement efforts  Our survey results identify useful information related to potential promising practices and challenges that OMB and the PIC could use to inform efforts to enhance the use of performance information more extensively within agencies (e g , at lower levels)  As was previously described, the PIC has responsibilities to (1) facilitate the exchange among agencies of proven practices, and (2) work to resolve government- wide or cross-cutting performance issues, such as challenges  Our analyses of 2017 survey results identified instances where agencies may have found effective ways to enhance the use of performance information by agency leaders and managers in decision making, as well as instances where agencies (and their managers) face challenges in doing so  Specifically, based on analyses of our survey responses, we identified 14 agencies that may have insights into specific practices that led to recent improvements in managers’ use of performance information, or ways that they maintain relatively high levels of use by their managers when compared to the rest of the government  Figure 9 summarizes the agencies identified earlier in the report that had statistically significant increases, or results higher than the government-wide average, on our index or individual survey questions  As the figure shows, several agencies had statistically significant results across all three sets of analyses and therefore may have greater insights to offer: the General Services Administration, National Aeronautics and Space Administration, and the National Science Foundation  In addition, our analyses identified nine agencies where results suggest managers face challenges that have hampered their ability to use performance information  Figure 10 summarizes the agencies identified earlier in the report that had statistically significant decreases, or results lower than the government-wide average, on our index or individual survey questions  As the figure shows, the Office of Personnel Management had statistically significant decreases in all three sets of analyses  Four agencies—the Departments of the Treasury and Veterans Affairs, the Nuclear Regulatory Commission, and the Social Security Administration—were common to both of the figures above  That is, they had results that indicate they may have insights on some aspects of using performance information and face challenges in other aspects  As was mentioned earlier, to provide proper context, these results should be considered in relation to the range of agency results and the government- wide average (provided in detail in the agency summaries in appendix I)  Given the prioritization of other activities, such as the recent creation of the Fed2Fed Solutions program, the PIC has not yet undertaken a systematic approach that could improve the use of performance information by managers at lower levels within agencies  Such an approach would involve identifying and sharing practices that have led to improved use, as well as identifying common or cross-cutting challenges that have hampered such use  The results of our analyses could help the PIC do so, and in a more targeted manner  By identifying and sharing proven practices, the PIC could further ensure that agency leaders and managers are aware of effective or proven ways they can use performance information to inform their decisions across the spectrum of activities they manage within their agencies  Those proven practices also may help agency leaders and managers resolve any identified challenges  Furthermore, in September 2017, we found that, for the estimated 35 percent of managers who reported familiarity with data-driven reviews, the more they viewed their programs being subject to a review, the more likely they were to report the reviews were driving results and were conducted in line with our leading practices for using performance information  Despite the reported benefits of and results achieved through data-driven reviews, they were not necessarily widespread  As noted above, GPRAMA requires agencies to conduct such reviews for agency priority goals, which represent a small subset of goals, and they are required at the departmental level  These reasons may explain why most managers reported they were not familiar with the reviews  As a result, we recommended that OMB should work with the PIC to identify and share among agencies practices for expanding the use of data-driven reviews  OMB staff agreed with our recommendation but have yet to address it  In June 2018, OMB updated its annual guidance to agencies to explicitly encourage them to expand data-driven reviews to include other goals, priorities, and management areas as applicable to improve organizational performance  However, as of June 2018, OMB and the PIC have yet to take any steps to identify and share practices for expanding the use of these reviews in line with our recommendation  Given the additional analyses we conducted for this report—which show that being subject to data-driven reviews is related to greater reported use of performance information and leading practices that promote such use—we continue to believe these further actions would help agencies implement these reviews more extensively  We reiterate the importance of the September 2017 recommendation and will continue to monitor OMB’s progress to address it  For more than 20 years, our work has highlighted weaknesses in the use of performance information in federal decision making  While the Executive Branch has taken some actions in recent years, such as establishing a framework for performance management across the federal government, our survey results underscore that more needs to be done to improve the use of performance information more extensively within agencies and government-wide  The President’s Management Agenda and its related CAP goal to leverage data as a strategic asset present an opportunity to do so, as it aims to improve data-driven decision making  As OMB and its fellow goal leaders more fully develop the action plan for achieving this goal, providing additional details for its plans to improve data-driven decision making would help provide assurance that it can be achieved  As part of those initiatives, our survey results could provide a useful guide for targeting efforts  Officials at each agency could use these results to identify areas for additional analysis and potential actions that could help improve the use of performance information across the agency and at lower levels  Similarly, OMB and the PIC could use the results to identify broader issues in need of government-wide attention  It will also be important, however, for OMB and the PIC to go beyond this analysis and work with agencies to identify and share proven practices for increasing the use of performance information at lower levels within agencies, as well as challenges that may be hampering agencies’ ability to do so  We are making the following two recommendations to OMB: The Director of OMB should direct the leaders of the Leveraging Data as a Strategic Asset CAP Goal to ensure future updates to the action plan, and the resulting federal data strategy, provide additional details on improving the use of data, including performance information, more extensively within federal agencies  The action plan should identify performance goals; contributing agencies, organizations, programs, and other activities; those responsible for leading implementation within these contributors; planned actions; time frames; and means to assess progress  (Recommendation 1) The Director of OMB, in coordination with the PIC, should prioritize efforts to identify and share among agencies proven practices for increasing, and challenges that hamper, the use of performance information in decision making more extensively within agencies  At a minimum, this effort should involve the agencies that our survey suggests may offer such insights  (Recommendation 2) We provided a draft of this report to the Director of the Office of Management and Budget for review and comment  We also provided a draft of the report to the heads of each of the 24 federal agencies covered by our survey  OMB had no comments, and informed us that it would assess our recommendations and consider how best to respond  We are sending copies of this report to congressional requesters, the Director of the Office of Management and Budget, the heads of each of the 24 agencies, and other interested parties  This report will also be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-6806 or mcneilt@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report  Key contributors to this report are listed in appendix IV  (USDA) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Commerce) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Goverment-wide) (DOD) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Education) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (Energy) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (HHS) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DHS) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (HUD) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Interior) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOJ) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOL) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (State) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOT) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Treasury) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (VA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Goverment-wide) (USAID) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (EPA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (GSA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (NASA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (NSF) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (NRC) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (OPM) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (SBA) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (SSA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” This report responds to a request that we analyze agency-level results from our 2017 survey of federal managers at the 24 agencies covered by the Chief Financial Officers (CFO) Act of 1990, as amended, to determine the extent agencies are using performance information  This report assesses the extent to which: 1  the reported use of performance information and related leading practices at 24 agencies has changed compared to our prior survey in 2013; 2  being subject to data-driven reviews related to managers’ reported use of performance information and leading practices; and 3  the Executive Branch has taken actions to enhance agencies’ use of performance information in various decision-making activities  From November 2016 through March 2017, we administered our online survey to a stratified random sample of 4,395 individuals from a population of 153,779 mid- and upper-level civilian managers and supervisors at the 24 CFO Act agencies  The management levels covered general schedule (GS) or equivalent schedules at levels comparable to GS-13 through GS-15, and career Senior Executive Service (SES) or equivalent  We obtained the sample from the Office of Personnel Management’s Enterprise Human Resources Integration database as of September 30, 2015—the most recent fiscal year data available at the time  The sample was stratified by agency and whether the manager or supervisor was a member of the SES  To help determine the reliability and accuracy of the database elements used to draw our sample of federal managers for the 2017 survey, we checked the data for reasonableness and the presence of any obvious or potential errors in accuracy and completeness and reviewed our past analyses of the reliability of this database  We concluded in our September 2017 report that the data used to draw our sample were sufficiently reliable for the purpose of the survey  For the 2017 survey, we received usable questionnaires from about 67 percent of the eligible sample  The weighted response rate at each agency generally ranged from 57 percent to 82 percent, except the Department of Justice, which had a weighted response rate of 36 percent  The overall survey results are generalizable to the population of managers government-wide and at each individual agency  To assess the potential bias from agencies with lower response rates, we conducted a nonresponse bias analysis using information from the survey and sampling frame as available  The analysis confirmed discrepancies in the tendency to respond to the survey related to agency and SES status  The analysis also revealed some differences in response propensity by age and GS level; however, the direction and magnitude of the differences on these factors were not consistent across agencies or strata  Our data may be subject to bias from unmeasured sources for which we cannot control  Results, and in particular estimates from agencies with low response rates such as the Department of Justice, should be interpreted with caution  However, the survey’s results are comparable to five previous surveys we conducted in 1997, 2000, 2003, 2007, and 2013  To address the first objective, we used data from our 2017 survey to update agency scores on our use of performance information index  This index, which was last updated using data from our 2013 survey, averages managers’ responses on 11 questions related to the use of performance information for various management activities and decision making  Using 2017 survey data, we conducted statistical analyses to ensure these 11 questions were still positively correlated  That analysis confirmed that no negative correlations existed and therefore no changes to the index were needed  Figure 11 shows the questions that comprise the index  After calculating agency index scores for 2017, we compared them to previous results from 2007 and 2013, and to the government-wide average for 2017, to identify any statistically significant differences  We focus on statistically significant results because these indicate that observed relationships between variables and differences between groups are likely to be valid, after accounting for the effects of sampling and other sources of survey error  For each of the 11 questions that comprise the index, we identified individual agency results, excluding missing and no basis to judge responses, and determined when they were statistically significantly different from (1) the agency’s results on the same question in 2013, or (2) the government-wide average results on the question in 2017  In this report, we analyzed and summarized the results of our 2017 survey of federal managers  Due to the limited scope of the engagement, we did not conduct additional audit work to determine what may have caused statistically significant changes between our 2017 and past survey results  To further address this objective we completed several statistical analyses that allowed us to assess the association between the index and 22 survey questions that we determined relate to leading practices we previously found promote the use of performance information  See figure 12 for the 22 specific questions related to these five practices that we included in the analysis  When we individually tested these 22 survey questions (bivariate regression), we found that each was statistically significantly and positively related to the index in 2017  This means that each question, when tested in isolation from other factors, was associated with higher scores on the index  However, when all 22 questions were tested together (multivariate regression), we found that 5 questions continued to be positively and significantly associated with the index in 2017, after controlling for other factors  To conduct this multivariate analysis, we began with a base model that treated differences in managers’ views of agency performance management use as a function of the agency where they worked  We found, however, that a model based on agency alone had little predictive power (R-squared of 0 04)  We next examined whether managers’ responses to these questions reflecting practices that promote the use of performance information related to their perceptions of agency use of performance information, independent of agency  The results of this analysis are presented in table 1 below  Each coefficient reflects the increase in our index associated with a one-unit increase in the value of a particular survey question  Our final multivariate regression model had an R-squared of 0 67, suggesting that the variables in this model explain approximately 67 percent of the variation in the use index  We also tested this model controlling for whether a respondent was a member of the SES and found similar results  As shown above in table 1, five questions related to three of the leading practices that promote agencies’ use of performance information were statistically significant in 2017  These results suggest that, when controlling for other factors, certain specific efforts to increase agency use of performance information—such as providing information on the validity of performance data—may have a higher return and lead to higher index scores  With respect to aligning agency-wide goals, objectives, and measures, we found that each increase in terms of the extent to which individuals felt that managers aligned performance measures with agencywide goals and objectives was associated with a 0 08 increase in their score on the use index  In terms of improving the usefulness of performance information, we found that having information on the validity of performance data for decision making was the strongest predictor in our model (0 18)  As measured here, taking steps to ensure the performance information is useful and appropriate was associated with almost as large a change in a managers’ index score (0 16)  In terms of developing agency capacity to use performance information, we found that having sufficient analytical tools to collect, analyze, and use performance information (0 07), and providing or paying for training that would help link their programs to achievement of agency strategic goals (0 10), were also statistically significantly related to a manager’s reported use of performance information  When we combined these results with what we previously found through a similar analysis of 2013 survey results in September 2014, we identified 10 questions that have had a statistically significant association with higher index scores  This reinforces the importance of the five leading practices to promote the use of performance information  For each of these questions, which are outlined in figure 13 below, we determined when agency results were statistically significantly different from 2013 results or the 2017 government-wide average  For the second objective, we examined, based on the extent they responded their programs had been subject to agency data-driven reviews, differences in managers’ use index scores and responses on questions related to practices that promote the use of performance information  We grouped managers based on the extent they reported their programs had been subject to these reviews, from “no extent” through “very great extent ” We then calculated the average index scores for the managers in each of those five categories  We also examined differences in how managers responded to the 10 questions reflecting practices that can promote the use of performance information based on the extent they reported their programs had been subject to data-driven reviews  We grouped managers into three categories based on the extent they reported their programs had been subject to these reviews (no-small extent, moderate extent, great-very great extent)  We then compared how these groups responded to the ten questions  For the third objective, we reviewed our past work that assessed Executive Branch activities to enhance the use of performance information; various resources (i e , guidance, guides, and playbooks) developed by the Office of Management and Budget (OMB) and the Performance Improvement Council (PIC) that could support agencies’ use of performance information; and the President’s Management Agenda, and related materials with information on cross-agency efforts to improve the use of data in federal decision making  Lastly, for the third objective we also interviewed OMB and PIC staff about any actions they have taken, or planned to take, to further support the use of performance information across the federal government  We conducted this performance audit from October 2017 to September 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the above contact, Benjamin T  Licht (Assistant Director) and Adam Miles (Analyst-in-Charge) supervised this review and the development of the resulting report  Arpita Chattopadhyay, Caitlin Cusati, Meredith Moles, Dae Park, Amanda Prichard, Steven Putansu, Alan Rozzi, Shane Spencer, and Khristi Wilkins also made key contributions  Robert Robinson developed the graphics for this report  Alexandra Edwards, Jeff DeMarco, Mark Kehoe, Ulyana Panchishin, and Daniel Webb verified the information presented in this report  Results of the Periodic Surveys on Organizational Performance and Management Issues Managing for Results: Further Progress Made in Implementing the GPRA Modernization Act, but Additional Actions Needed to Address Pressing Governance Challenges  GAO-17-775  Washington, D C : September 29, 2017  Supplemental Material for GAO-17-775: 2017 Survey of Federal Managers on Organizational Performance and Management Issues  GAO-17-776SP  Washington, D C : September 29, 2017  Program Evaluation: Annual Agency-wide Plans Could Enhance Leadership Support for Program Evaluations  GAO-17-743  Washington, D C : September 29, 2017  Managing for Results: Agencies’ Trends in the Use of Performance Information to Make Decisions  GAO-14-747  Washington, D C : September 26, 2014  Managing for Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges  GAO-13-518  Washington, D C : June 26, 2013  Managing for Results: 2013 Federal Managers Survey on Organizational Performance and Management Issues, an E-supplement to GAO-13-518  GAO-13-519SP  Washington, D C : June 26, 2013  Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation in Program Management and Policy Making  GAO-13-570  Washington, D C : June 26, 2013  Government Performance: Lessons Learned for the Next Administration on Using Performance Information to Improve Results  GAO-08-1026T  Washington, D C : July 24, 2008  Government Performance: 2007 Federal Managers Survey on Performance and Management Issues, an E-supplement to GAO-08-1026T  GAO-08-1036SP  Washington, D C : July 24, 2008  Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results  GAO-04-38  Washington, D C : March 10, 2004  Managing for Results: Federal Managers’ Views on Key Management Issues Vary Widely Across Agencies  GAO-01-592  Washington, D C : May 25, 2001  Managing for Results: Federal Managers’ Views Show Need for Ensuring Top Leadership Skills GAO-01-127  Washington, D C : October 20, 2000  The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven  GAO/GGD-97-109  Washington, D C : June 2, 1997 


Report 2:  Developing international military capacity  U S  officials told us that UN peacekeeping operations provide an indirect benefit of helping to professionalize the military units from many developing countries that contribute troops to the UN  We have previously reported that building military capacity of foreign partners to address security-related threats is an important goal of U S  national security strategy and foreign policy  According to U S  and UN officials, the relative strengths of U S  peacekeeping operations would be faster deployment and superior command and control, logistics, intelligence, and counterterrorism capabilities  Deployment speed  State, DOD, and UN officials highlighted the United States’ ability to deploy troops and police to peacekeeping operations more quickly than the UN  Unlike the U S  military, which can draw from a ready pool of military personnel, the UN must seek troops from UN member states, which takes time  UN officials told us that the UN faces a shortage of both troops and UN police, which slows deployment  Further, a 2015 report by the UN High-level Panel on Peacekeeping stated that the UN “has struggled to get sufficient forces on the ground quickly enough and relies on under-resourced uniformed capabilities ” The report also stated that aviation, medical, and engineering specialists, among others, are difficult to mobilize in advance of infantry units  Command and control  State, DOD, and UN officials told us that U S  operations would enable the U S  military to have direct command and control, whereas UN operations, which are inherently multinational, face challenges with command and control over troops from several different countries  The UN High-level Panel report noted that UN peacekeeping operations’ weak command and control is a well-known constraint that limits the type of operations the UN can undertake  Logistics support  U S  and UN officials told us that U S  operations have superior logistics systems  U S  procurement likely would be faster than UN procurement, which lacks a standing supply chain and, therefore, relies on third-party vendors, according to UN officials  In addition, the UN High Level Panel report stated that UN peacekeeping operations’ logistics systems and structures in the field are under severe strain, which can limit the mobility of these operations  Intelligence capability  U S  and UN officials agreed that U S  operations would involve superior intelligence capability  The UN only recently established an intelligence policy—in May 2017—having recognized that some peacekeeping operations had been deployed in fragile political and security environments with asymmetrical and complex threats  However, UN officials acknowledged that the scope of UN intelligence capability remains limited  Counterterrorism capability  DOD officials told us that a U S  peacekeeping operation would have the capability to include a counterterrorism component and would not be constrained in the use of force, if needed, in response to terrorist threats  UN peacekeeping operations, on the other hand, lack the capabilities and specialized military preparation to engage in counterterrorism operations, according to the UN High-level Panel report  The UN report stated that counterterrorism should be undertaken by the host government, a capable regional force, or an ad hoc coalition authorized by the UN Security Council  According to the UN report, UN peacekeeping operations may engage in proactive and preemptive use of force to protect civilians and UN personnel from threats; however, offensive force to degrade, neutralize or defeat an opponent is a fundamentally different type of posture that should be authorized by the Security Council only under limited and exceptional circumstances  We provided a draft of this report to DOD, State, and the UN for review and comment  DOD provided a letter, reproduced in appendix II, which stated that it had no comments  State did not provide comments  The UN provided technical comments, which we incorporated into our report as appropriate  We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and State, and the Secretary- General of the United Nations  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-9601 or melitot@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made significant contributions to this report are listed in appendix III  The objectives of this report were to (1) compare the reported costs of a specific United Nations (UN) peacekeeping operation to the estimated costs of a hypothetical, comparable operation implemented by the United States; (2) identify factors that affect cost differences; and (3) identify stakeholder views on the relative strengths of UN and U S  peacekeeping operations  To compare the reported costs of a specific UN peacekeeping operation to the estimated costs of a hypothetical, comparable operation implemented by the United States, we selected the UN Multidimensional Integrated Stabilization Mission in the Central African Republic (MINUSCA) as a case study  We compared the reported UN expenditures for MINUSCA, which included both military and civilian components, with estimated costs for a hypothetical U S  operation with a similar level of military and civilian personnel  Our comparison covers a total of 3 years and 3 months—from MINUSCA’s establishment in April 2014 through June 30, 2017, the end of UN fiscal year 2017  We selected MINUSCA because it is in sub-Saharan Africa, where most UN peacekeeping operations established since 2003 have taken place, and has a typical scope and budget compared to other UN peacekeeping operations in sub-Saharan Africa, according to U S  and UN officials  In addition, MINUSCA is one of the most recent UN peacekeeping operations; thus, initial expenditures for the operation are relatively current  Because the results of our cost comparison are based on a single case study, they cannot be generalized to all UN peacekeeping operations  To determine the UN’s costs for MINUSCA, we analyzed UN budget and expenditure data covering the initial start-up period (April 2014 to June 2014) and the first 3 UN fiscal years (July 1, 2014 to June 30, 2017)  We spoke with officials of the UN Departments of Peacekeeping Operations, Field Support, and Management at UN Headquarters in New York, New York, to better understand the characteristics of MINUSCA and the different costs affecting MINUSCA’s budget and expenditures  We assessed UN expenditure data through discussions with cognizant UN officials and a review of external audits of UN budgetary information and found them sufficiently reliable for our purposes  We also analyzed data on U S  contributions to UN peacekeeping operations for fiscal years 2014 through 2017 from the Department of State’s (State) Bureau of International Organization Affairs to determine total U S  contributions to MINUSCA, and UN peacekeeping operations overall  To estimate the costs of a hypothetical, comparable operation implemented by the United States, we developed a hypothetical scenario for a U S  operation based on the MINUSCA budget and supporting documents, assuming deployment of the same number of military, civilian, and police personnel in the Central African Republic (CAR) over the same time period (April 2014 through June 2017)  To estimate the military portion of the operation, we interviewed Department of Defense (DOD) officials and staff at the Institute for Defense Analyses (IDA), a DOD-sponsored non-profit corporation involved in developing cost estimates for U S  contingency operations  The Office of the Undersecretary of Defense-Comptroller and IDA generated a cost estimate for the military components included in the hypothetical operation using the Contingency Operations Support Tool (cost estimating tool)  DOD uses this tool to develop cost estimates for all military contingency operations  The cost estimate included only the incremental costs of the operation—those directly attributable to the operation that would not be incurred if the operation did not take place  For example, the estimate produced by the cost estimating tool did not include the direct salaries of active duty personnel as those costs would be incurred by the United States regardless of a possible decision to undertake the hypothetical operation  We assessed the cost estimating tool’s applicability to developing a hypothetical cost estimate for the purposes of this report through discussions with DOD and IDA officials, and compared the tool to the accurate and comprehensive characteristics of a high-quality cost estimate, as described in the GAO Cost Estimating and Assessment Guide  While we found the DOD cost estimating tool generated a sufficiently reliable cost estimate for a hypothetical U S  peacekeeping operation, we did not assess the overall reliability of the tool or its capability to generate accurate or comprehensive estimates for future U S  operations  To generate our estimate of U S  military costs using the DOD’s estimating tool, we used UN military deployment numbers as a baseline for the scale of a hypothetical, comparable U S  peacekeeping operation, while using unit sizes and rotations in deployment that were considered appropriate for the U S  military, according to DOD and IDA officials  We based the hypothetical U S  operation, and hence the cost estimate, on the following assumptions, which correspond approximately with MINUSCA’s actual UN personnel deployments: Theater of operation: Central African Republic (CAR) Type of operation: military contingency Operation time frame: April 10, 2014 through June 30, 2017 Military contingents: as of June 30, 2017, 11,495 total personnel Infantry: 10 units of 630-785 personnel per unit, approximately 90 percent active duty / 10 percent reserves Communication / signals: 1 unit, 124 personnel per unit Engineering: 4 units, 200 personnel per unit Military police: 1 unit, 120 personnel per unit Formed police units (military police): 12 units, 140 personnel per Hospital / medical: 1 level III hospital, 248 beds, 495 personnel Helicopter units: 2 UH-60 C3 units, 1 MH-60M Assault attack helicopter unit, 100 personnel per unit Quick reaction force: 1 unit, 160 personnel per unit Special forces: 1 tactical civilian affairs unit, 1 Marine special operations intelligence unit, 160 personnel per unit Unmanned aerial vehicle: 1 unit, 84 personnel Transportation: 1 heavy transport unit, 120 personnel per unit Operational tempo: 1 0 for all phases of operation and units, except aviation units (set at 1 5) Deployment schedule and phasing: phased deployment, including 14 days for predeployment (e g , training), 5 days for deployment, 180 days for active duty unit sustainment and 270 days for reserve unit sustainment, 5 days for redeployment, and 0 days for reconstitution Housing: contractor-provided semi-permanent housing Transportation: personnel and equipment transported by airlift from the United States (primarily Fort Hood, Texas), material (such as water, food, and other consumables) transported by airlift from Italy We obtained input on the operational design for the military portion of the cost estimate from DOD officials in the Joint Chiefs of Staff, the Office of the Undersecretary of Defense-Policy, and the Office of the Undersecretary of Defense-Comptroller, and IDA officials  However, the military portions of the scenario and their corresponding cost estimate have some limitations  As a result of rounding for some units, U S  military personnel numbers do not exactly match the MINUSCA deployment levels  In addition, based on input from DOD officials, we attempted to select military units that would provide an essential function per U S  common practices while keeping the overall personnel deployment level as close as possible to MINUSCA’s deployment level  An actual U S  military plan may differ significantly from the UN plan as a result of differences between U S  and UN military operations, structure, doctrine, and circumstances at the time of the operation  To estimate U S  civilian costs, we matched the number of U S  civilian police and personnel to the number serving in MINUSCA  We then estimated the costs of deploying these U S  civilian personnel in CAR for the same time period as MINUSCA  We did not attempt to determine how the U S  government would actually implement civilian components of a peacekeeping operation in CAR  To estimate U S  civilian police costs, we met with State’s Bureau of International Narcotics and Law Enforcement Affairs (INL) to identify State’s costs for civilian police contractors providing police training and technical assistance in sub-Saharan Africa  Based on INL’s input we assumed that the base salary of civilian police would be grade 13, step 5 on the Office of Personnel Management’s general schedule salary tables for federal employees  In addition to the average base salary, we identified other costs—with input from INL—including, among others, personal equipment, travel from the United States, and State’s published allowances specific to CAR for local cost of living, post hardship differential, danger pay, and living quarters  We applied the average cost per officer to the average number of UN civilian police officers deployed in MINUSCA  To estimate U S  civilian personnel costs, we met with State’s Bureau of Budget and Planning to identify the costs of State Foreign Service officers and locally employed staff, based on the number of UN international and national civilian staff deployed to MINUSCA, respectively  We matched the number of State Foreign Service officers for the U S  cost estimate to the number of UN international staff in MINUSCA, with input from State to align the grade levels  The estimated costs for Foreign Service officers include average salaries based on State’s Foreign Service salary tables and State’s allowances specific to CAR, including local cost of living, post hardship differential, and danger pay  We also met with State Bureau of Budget and Planning officials to estimate other costs for Foreign Service officers, which we included in our cost estimate, including post assignment travel, administrative support costs, residential furnishings, and residential guards, among others, but we did not assess the reliability of these additional costs provided by State  In addition, State’s Bureau of Overseas Buildings Operations provided the actual costs of residential leases for Foreign Service officers in CAR in fiscal year 2017, which we used to estimate the cost of housing Foreign Service officers in CAR  We also matched the number of State locally employed staff to the number of UN national staff deployed to MINUSCA and added their average salaries and other costs in CAR based on data provided by State’s Bureau of the Comptroller and Global Financial Services  While MINUSCA’s expenditures also included costs for sending an annual average of up to about 200 UN volunteers to CAR, State officials told us that the United States generally would not send volunteers through its assistance efforts to a high-risk post, such as CAR  Therefore, we did not include any costs related to volunteers in the cost estimate  We also did not include costs related to host-government-provided personnel serving in MINUSCA  In addition, UN expenditures included about $7 million for “quick-impact projects” to support local government infrastructure and civil society initiatives  We did not include comparable costs for quick-impact projects in our U S  cost estimate because we did not have a basis for matching these costs  To identify factors that affect cost differences between MINUSCA and a hypothetical, comparable operation implemented by the United States, we reviewed the U S  cost estimate generated in conjunction with DOD, IDA, and State officials, and identified significant areas of cost for the United States and the assumptions incorporated in the estimate or factors specified by U S  officials that drive those costs  We compared the U S  cost estimate, including these significant areas of cost, to UN costs to identify differences  We interviewed U S  and UN officials regarding U S  and UN standards and policies that explain differences between MINUSCA costs and the estimated costs of a U S  operation  To identify stakeholder views on the relative strengths of UN and U S  peacekeeping operations, we reviewed UN reports on peacekeeping operations and interviewed UN, DOD, and State officials  In addition, we reviewed GAO’s 2006 report comparing the costs as well as the strengths of a UN peacekeeping operation in Haiti with those of a hypothetical U S  operation  We conducted our review from February 2017 through February 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the individual named above, Drew Lindsey (Assistant Director), Howard Cott, Juan Pablo Avila-Tournut, Debbie Chung, Martin de Alteriis, Neil Doherty, Jennifer Leotta, Caitlin Mitchell, Elizabeth Repko, and Alex Welsh made significant contributions to this report 

ground_truth: To reform the federal government and make it more efficient and effective, agencies need to use data about program performance. The benefit of collecting performance information is only fully realized when it is used by managers to make decisions aimed at improving results. GAO was asked to review agencies' use of performance information. This report assesses, among other things, the extent to which: (1) 24 agencies' reported use of performance information and related leading practices has changed since 2013 and (2) the Executive Branch has taken actions to enhance the use of performance information. To address the first objective, GAO analyzed results from its 2017 survey of federal managers, and compared them to 2013 results. The survey covered a stratified random sample of 4,395 managers from the 24 Chief Financial Officers Act agencies. The survey had a 67 percent response rate and results can be generalized to the population of managers government-wide and at each agency. For the second objective, GAO reviewed agency documents and interviewed staff from OMB and the PIC. Agencies' reported use of performance information to make decisions, and leading practices that can promote such use, generally has not improved since GAO's last survey of federal managers in 2013. However, GAO's survey results continue to point to certain practices that could help agencies improve managers' use of performance information. For example, as shown in the table below, GAO's survey found that managers whose programs were subject to data-driven reviews (regular reviews used to assess progress on select agency goals) to a greater extent reported statistically significantly greater use of performance information to make decisions. The Executive Branch has begun taking steps to improve the use of performance information within agencies and across the government. For example, In the President's Management Agenda and government-wide reform plan, released in March and June 2018 respectively, the administration acknowledged the need to do more, and announced a goal, among other actions, to improve the use of data in federal decision making. However, the Office of Management and Budget (OMB) and others responsible for this goal have yet to fully develop action plans to hold agencies accountable for achieving it. The Performance Improvement Council (PIC), which is chaired by OMB, has undertaken efforts to improve the use of performance information by, for example, creating a working group on agency performance reviews. But it has not yet taken a systematic approach to identify and share proven practices that led to, or challenges that may be hampering, increased use of performance information by managers. GAO's survey results identified agencies that may have insights into such practices and challenges. More fully developing action plans for the new goal, and identifying and sharing proven practices and challenges, could help ensure the Executive Branch takes further steps to improve the use of performance information by managers within agencies and across the federal government. To improve the use of performance information within agencies and across the federal government, GAO recommends that OMB work with (1) fellow goal leaders to more fully develop action plans for the new goal to improve the use of data and (2) the PIC to prioritize efforts to identify and share proven practices and challenges. OMB had no comments on this report.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.



Report 2: DOJ and its components, as well as the judiciary, play important roles in requesting and collecting restitution  DOJ and select components: Prosecutors in DOJ’s Criminal Division and the Criminal Divisions of the 94 USAOs are responsible for overseeing criminal matters, including identifying and notifying victims, determining their losses as part of a case investigation, prosecuting cases and negotiating the terms of plea agreements, of which restitution may be a part  Within DOJ’s Criminal Division, the Money Laundering and Asset Recovery Section manages DOJ’s Asset Forfeiture Program  As previously stated, FLUs within each USAO undertake activities to collect restitution from offenders in their district  Additionally, all USAOs have asset forfeiture staff responsible for forfeiting property seized by law enforcement agencies because the property was used in criminal activities or purchased with the proceeds of criminal activities  According to EOUSA guidance, coordination between the FLU and Asset Forfeiture units is highly encouraged to use forfeited assets as a means to collect on unpaid restitution debts  DOJ requires each USAO to have its own policies and procedures related to debt collection efforts but allows them discretion in developing these policies and procedures to ensure that they are appropriate for local conditions  DOJ also requires USAOs to have policies and procedures to make early, effective, and coordinated asset investigations and recovery a routine part of every case involving victims but allows USAOs to specify these policies and procedures  DOJ’s EOUSA provides USAOs with management assistance, guidance, training, and administrative support  Among other activities, EOUSA provides management assistance to USAOs by administering internal evaluations for each USAO, which are intended to provide on-site management support for that office  Further, EOUSA provides guidance to enhance offices’ efforts to request and collect restitution  Judiciary: Within the judiciary, the 94 federal district courts order restitution, receipt restitution payments, and disburse restitution to victims  Within the federal district where the offender was convicted, a probation officer prepares the presentence investigation report (PSR) for the court, which includes information on the victim’s losses and an offender’s financial information  Probation officers may obtain this information from DOJ, which has the statutory responsibility for the enforcement and collection of criminal debt  The court uses the PSR, among other things, to determine whether to order restitution  If an offender is released to the community by the court and placed on supervision, probation officers are responsible for ensuring the offender abides by the terms of release, including paying any restitution owed to victims  The Clerk of each District Court is responsible for the receipt of restitution from offenders and for disbursing payments to victims  The Judicial Conference is the national policy-making body for the federal courts  The Conference operates through a network of committees created to address and advise courts on a wide variety of subjects such as information technology, personnel, probation and pretrial services, space and facilities, security, judicial salaries and benefits, budget, defender services, court administration, and rules of practice and procedure  The Judicial Conference has taken policy positions on restitution-related issues and has supported legislative proposals to improve the restitution process  AOUSC is the agency within the judiciary that provides a broad range of legislative, legal, financial, technology, management, administrative, and program support services to federal courts  AOUSC is responsible for carrying out Judicial Conference policies and a primary responsibility of AOUSC is to provide staff support and counsel to the Judicial Conference and its committees  USSC is an independent agency within the judiciary which, among other activities, establishes and promulgates detailed sentencing guidelines that judges are to consider in sentencing offenders convicted of federal crimes, including guidelines on when and how to order restitution  Additionally, each district court is required to submit to USSC a report of each offender’s sentence that includes, among other information, details on the offenses for which the offender was convicted; the sentence imposed on the offender; and if the judge departed from the sentencing guidelines, information on reasons why  USSC maintains a database containing sentencing data on federal offenders convicted of felonies or serious misdemeanors, analyzes it and publishes these data on an annual basis  USSC is also statutorily required to annually report to Congress its analysis of sentencing-related documents, including an accounting of districts USSC believes have not submitted appropriate information to the commission, among other things  During the course of a federal criminal investigation, federal prosecutors identify and notify victims, as well as determine their losses in conjunction with the federal agents investigating the case  If a defendant pleads guilty or is found guilty at trial, the prosecutor has the burden of proving the victims’ losses in court  To facilitate this, a Victim-Witness coordinator within the USAO responsible for the case provides victims the opportunity to explain their losses in detail, usually through a Victim Impact Statement  This information is then to be provided to a federal probation officer who uses it to begin a PSR  To develop the PSR, probation officers use information provided by the USAO and may contact victims and verify the loss amounts  Additionally, probation officers will investigate an offender’s economic circumstances— such as if the offender has a job, any assets or any dependents  If a judge determines that restitution is to be ordered, the judge must order restitution for the full amount of a victim’s losses for offenses without consideration of the economic circumstances of the defendant  Judges may decline to order restitution in certain instances, for example, where restitution is discretionary, or in certain cases where the number of identifiable victims makes restitution impracticable or the complexity of calculating restitution would unduly prolong the sentencing process  If the court does not order restitution, or orders only partial restitution, the judge must provide the reason, and judges usually do so in a written Statement of Reasons document  Figure 1 provides an overview of the federal restitution process  Upon imposition of a restitution debt by the court, FLU staff use two mechanisms to determine the collectability of the debt and what collection actions to take  First, FLU staff classify the debt into one of four categories to determine the extent to which the FLU will pursue enforcement actions to collect upon the debt  FLUs classify debts from a Priority Code 1 debt (indicating that FLUs will make collection of this debt the highest priority) to a Priority Code 4 debt (indicating that FLUs will make collection of this debt the lowest priority)  Second, FLUs may suspend collection action on criminal debts, regardless of their categorization, under certain circumstances if they determine the debts are uncollectible  FLU staff may also determine that debts are permanently uncollectible and categorize them as Priority Code 4 debts  If a debtor does not provide payment, FLU staff then use various enforcement actions to collect the restitution debt  These can include, among other actions, filing liens against an offender’s property, coordinating with asset forfeiture staff to use forfeited assets to pay the restitution debt, and garnishing wages an offender may earn  Victims can be compensated for losses with the proceeds of forfeited assets through DOJ’s Asset Forfeiture Program and in accordance with law and regulation  Federal regulations provide that the proceeds from forfeited assets are first used to cover program costs associated with forfeiture-related activities and next to pay valid owners, lien-holders, and federal financial regulatory agencies  Forfeited assets can then be distributed to other victims of crime as compensation for their losses if their loss is a direct result of the commission of the offense underlying forfeiture or a related offense  Any remaining funds from the forfeited asset may be placed into official use, distributed to foreign governments, state or local law enforcement agencies as part of the equitable sharing program to enhance cooperation with federal investigations  When victims are eligible for compensation using forfeited assets, DOJ employs two processes: restoration and remission  The restoration process involves the USAO staff requesting funds on behalf of a victim when there is both an order of forfeiture and an order of restitution  Under the restoration process, USAO staff request DOJ’s Money Laundering and Asset Recovery Section to use the forfeited asset to pay a restitution debt  If DOJ approves the request for restoration, the funds from the forfeited property are then transferred to the Clerk of the Court who disburses this money to the victim  The remission process requires a victim of a crime to directly petition DOJ to receive funds from the forfeited property  According to officials in DOJ’s Criminal Division, the courts may not order restitution on behalf of victims who suffered a specific actual loss as a direct result of a crime for a variety of reasons, and therefore the remission process serves as a complement to the restoration process to ensure victims are made whole  For example, these officials stated that, among other reasons, the courts may not order restitution if a defendant dies prior to sentencing or if the case is one in which a court is not required to, and does not, order restitution, but the victim has suffered eligible losses  EOUSA and USAO officials in all six of the offices with whom we spoke told us that prosecutors document requests for restitution in their case files and that their offices employ other internal controls, such as the use of templates and forms, throughout the prosecution process to ensure that prosecutors request restitution as appropriate  EOUSA officials told us that although the agency does not track this information, they believed all USAOs generally document requests for restitution in their offices’ case files  Further, USAO officials in all six offices told us that prosecutors document requests for the court to order restitution in their case files by including this information in a written memorandum  To support prosecutors in documenting this information, all six offices we selected provide prosecutors with a prosecution memorandum template  Of the six templates we reviewed, four explicitly include a section for prosecutors to indicate whether victims have been identified and the extent of any victim losses  In addition to these templates, four of six USAOs we selected had forms that prosecutors could use to identify whether cases have victims and their need for restitution when drafting criminal charging documents  Moreover, officials from two of the six USAOs told us their offices use this form as an internal control to ensure prosecutors have identified all victims and considered their need for restitution, if applicable  All six offices we selected also provided prosecutors templates for drafting plea agreements, and templates we reviewed from all six USAOs included language requesting the offender pay restitution, if applicable  However, prosecutors are not required to use plea agreement templates, nor are they required to request restitution as part of a plea agreement  USAO officials from one office stated that including this language in the plea agreement template served to remind prosecutors of their requirement to consider requesting restitution as stated in the U S  Attorney’s Manual  Select USAO officials also described various forms of management oversight to ensure prosecutors request restitution as appropriate  Specifically, four USAOs we selected require supervisory review of the form that prosecutors fill out when drafting criminal charging documents, which includes information on victims  Additionally, officials in all six USAOs told us that they require supervisory review of plea agreements for every case  For example, officials from two USAOs told us their office requires the Criminal Chief, the supervisor of all criminal cases, to approve documents in the plea agreement, which may include requests for restitution  Federal courts sent information on sentencing decisions to USSC and USSC had information on restitution decisions for 95 percent of all offenders from fiscal years 2014 through 2016  According to our analysis of USSC data, 214,578 federal offenders were sentenced from fiscal years 2014 through 2016 and restitution was ordered for 33,158 of those offenders, or 15 percent  Collectively, courts ordered these offenders to pay $33 9 billion in restitution during this period  Courts did not order restitution for the remaining 181,420 offenders, or 85 percent  Table 1 shows the number of federal offenders sentenced and ordered to pay restitution for fiscal years 2014 through 2016, as well as the total amount of restitution ordered by the courts  The majority of federal offenders were sentenced for immigration or drug- related offenses, and USAO officials in all six offices we selected told us that these types of offenses do not typically have victims with actual losses  For example, from fiscal years 2014 through 2016, USSC data showed that 131,088 offenders, 61 percent of offenders sentenced, were sentenced for immigration or drug-related offenses and courts ordered 999 (or less than 1 percent) of these offenders to pay restitution  USSC data show that courts ordered restitution more often for offenders sentenced for other offenses, such as fraud  For example, courts sentenced 21,551 offenders for fraud offenses from fiscal years 2014 through 2016, and courts ordered restitution for 15,902 of these offenders, or 74 percent  Table 2 shows the number of offenders sentenced and the number ordered to pay restitution by offenses for which restitution was most often and least often ordered by courts from fiscal years 2014 through 2016  The percentage of federal offenders ordered to pay restitution varied across federal court districts; from 2 percent of offenders in one district to 42 percent in another district  USAO officials we interviewed stated that some of this variation may be due to the types of offenses prosecuted within different districts  For example, officials from one USAO stated that their office, which had a high volume of immigration–related offenders, had few cases in which restitution was applicable  Our analysis of USSC data showed that from fiscal year 2014 through fiscal year 2016 and across all districts, districts with a higher than average rate of immigration-related offenders had lower than average rates of restitution ordered  Conversely, districts with above-average rates of offenders convicted of financial offenses such as fraud, embezzlement, money laundering, tax offenses, counterfeiting or bribery had higher than average rates of restitution ordered, as shown in table 3  Judges indicated on documents sent to USSC that restitution was not applicable and thus did not order it for most offenders sentenced from fiscal years 2014 through 2016—167,230 offenders—or 78 percent of all offenders sentenced during this time period  Our analysis of sentencing information for the remaining offenders found that courts ordered restitution at a higher rate as compared to all offenders  Specifically, after excluding offenders for whom restitution was not applicable and were not ordered to pay it, we found that courts ordered restitution for 70 percent of the remaining 47,348 offenders  EOUSA and USAO officials told us that in cases where there are identifiable victims, restitution may not be ordered for other reasons  EOUSA officials told us that restitution may not be ordered for several reasons, such as when victims provide no proof of their losses or when victims recover compensation through other means, such as through civil proceedings  Further, officials from one USAO told us that victims must provide documentation of their losses for restitution and, if victims are not able to provide this documentation, courts may decline to order restitution  Also, in certain cases, courts are not required to order restitution—such as when there is no identifiable victim or, on the other hand, when the number of identifiable victims is so large as to make restitution impractical, among other reasons  Additionally, the court might not order, or order only partial restitution for other reasons, such as when the value of property the defendant returned to the victim was deducted from the restitution award or because the victim received compensation from insurance  If a court does not order restitution, or orders partial restitution, it is required to provide the reason for its decision and to provide that reason to USSC, but our analysis showed USSC did not always have these data  Specifically, from fiscal years 2014 through 2016, we found that restitution was not ordered—and no reason was documented in USSC data for that decision—for 9,848 offenders (5 percent of the 214,578 offenders sentenced during this time period)  Information on offenders’ sentences, including restitution, assists USSC in its continuous reexamination of its guidelines and policy statements and ensures that various sentencing practices are achieving their stated purposes  Further, Standards for Internal Control in the Federal Government state that management should evaluate issues identified through monitoring activities or reported by personnel to determine whether any of the issues rise to the level of an internal control deficiency  In response to our questions about the missing information on reasons why restitution was not ordered, AOUSC and USSC officials stated that they were unaware of the missing information or why it was missing  Judiciary officials stated that because various entities within the judiciary participate in the process of collecting and recording information on reasons restitution was not ordered, they did not know which entities could take action to improve USSC data  However, as previously discussed, if the court does not order restitution, or orders only partial restitution, the judge must provide the reason, and judges usually do so in a written Statement of Reasons form  The Judicial Conference, along with USSC, has developed guidance to help judges fill out the Statement of Reasons form and AOUSC supports the Judicial Conference in carrying out its policies  Further, courts must provide USSC the written Statement of Reasons form for sentences imposed  USSC is also responsible for collecting, analyzing, and distributing information on federal sentences provided by each district court, including information related to orders for restitution  However, judicial officials, including from the entities listed above, agreed that further studying the missing data may inform the judiciary of the cause of the missing data, as well as any efforts needed to improve USSC information  Courts are required to provide reasons for not ordering restitution and to provide this information to USSC so that the agency can analyze and report on sentencing data  Determining why USSC data are incomplete could help inform the judiciary whether the issue rises to the level of an internal control deficiency and whether additional action can be taken to improve the transparency of sentencing decisions  Doing so could help the judiciary ensure reasons for not ordering restitution are provided consistently in all cases and potentially improve data provided to USSC, in turn supporting its mission to promote transparency in sentencing decisions  Our analysis of DOJ data showed that DOJ collected $2 95 billion in restitution debt from fiscal years 2014 through 2016, half of which was collected on debts imposed during this period  The extent of collections across the 94 USAOs ranged from a high of $848 million in one USAO to a low of $1 2 million in another USAO  The median amount collected for USAOs was $10 7 million  DOJ was more successful at collecting restitution on newer debts—debts imposed from fiscal years 2014 through 2016  Of the $2 95 billion in restitution debt collected, about half was collected from new debts imposed by courts during this time period  Specifically, DOJ collected $1 5 billion (4 percent), of the $34 billion ordered from fiscal years 2014 through 2016  The remaining half of the debt collected during this time frame was collected from debts imposed between fiscal year 1988 and fiscal year 2014  New debts—imposed in fiscal years 2014 through 2016—were also more likely to be fully paid during this time period compared to all debts  Specifically, from fiscal years 2014 through 2016, DOJ collected the full amount of restitution on 4,003 of the 24,950 debts imposed during this time, 16 percent  However, across all debts, including debts imposed prior to fiscal year 2014, DOJ collected the full amount of restitution ordered on only 5 percent of debts  Across all restitution debts, DOJ collected at least some of the debt for one-third of debts and did not collect any restitution on the remaining two-thirds  More than 60 percent of the restitution DOJ collected in fiscal years 2014 through 2016 was owed to non-federal victims ($1 8 billion), including individuals, corporations and state and local governments  An additional 37 percent of restitution was collected on behalf of federal agencies that were victims of crimes  One percent of restitution collected was community restitution, which is restitution collected for drug offenses that otherwise have no victims and which is disbursed to state victim assistance agencies and state agencies dedicated to the reduction of substance abuse, as shown in table 4  AOUSC officials noted that some collected restitution is not disbursed to non-federal victims due to a lack of accurate contact information for these victims  Specifically, according to AOUSC, as of June 2017, courts had more than $132 million in restitution due to 113,260 victims that could not be disbursed because of a lack of accurate contact information for these victims  DOJ is required to provide courts with victim contact information, and victims are to notify DOJ if their contact information changes  However, AOUSC and USAO officials told us that this notification by victims may not always occur  For example, officials in one USAO told us that due to the length of court proceedings, victims may move without notifying the court prior to the disbursement of restitution and, as a result, the court is unable to disburse restitution to those victims  According to our analysis of DOJ data, at the end of fiscal year 2016, $110 billion in restitution was outstanding and USAOs had identified $100 billion of that debt as uncollectible, as shown in figure 2  USAOs may identify debts as uncollectible and suspend collection actions on a debt for a variety of reasons, including that the offender has no, or only a nominal, ability to pay the debt  Probation officials, EOUSA officials, and officials from five of six USAOs we interviewed stated that most outstanding restitution debt is identified as uncollectible and collection action is suspended because many offenders have little ability to pay the debt—a conclusion supported by USSC data  For example, according to USSC data, 95 percent of offenders ordered to pay restitution from fiscal years 2014 through 2016 received a waiver from paying a court-ordered fine, indicating their inability to pay  While courts are allowed to take an offender’s economic circumstances into consideration when issuing fines, they generally may not do so when ordering restitution  As a result, EOUSA and federal probation officials with whom we spoke stated that offenders ordered to pay restitution often do not have an ability to do so and therefore a large amount of restitution orders is uncollectible  Through various guidance documents, DOJ has identified and recommended numerous practices for DOJ prosecutors and FLU staff to use throughout the restitution process to help ensure full and timely restitution for victims  USAO officials in all six offices with whom we spoke stated that, based on their experience, these practices were generally effective  Specifically, DOJ and EOUSA officials identified practices for prosecutors and FLU staff to use when requesting restitution, facilitating court orders for restitution, and collecting restitution and documented these practices in several guidance manuals  Officials we interviewed from all six USAOs stated they were generally satisfied with the guidance from EOUSA and that they thought most of DOJ’s recommended practices were effective when requesting restitution, facilitating court orders for restitution, and collecting restitution  Requesting restitution  Officials we interviewed from three USAOs identified coordination between prosecutors and case investigators prior to sentencing to identify victims and their losses as an important practice for requesting restitution  USAO officials from three of the six offices stated that gathering detailed information on an offender’s financial resources, which include assets that could be forfeited and used to pay a restitution debt, was a very effective practice related to requesting restitution  Facilitating court orders of restitution  Although the courts, and not prosecutors, are responsible for ordering restitution, DOJ guidance identifies several practices that prosecutors can use to facilitate orders of restitution that may increase the likelihood of full and timely restitution for victims  Officials from three of six USAOs stated that the most effective practice related to ordering restitution was ensuring courts ordered restitution as due and payable immediately  Specifically, when offenders cannot pay restitution in an immediate lump-sum payment, the courts must specify a payment schedule through which the offender will pay restitution based on the offender’s ability to pay  In these cases, USAO officials stated that it is effective for prosecutors to ensure the restitution order specifies that restitution is due and payable immediately  According to an EOUSA official, this permits the agency to immediately pursue all collection remedies allowed by law whenever the debtor has or subsequently obtains the ability to pay  Collecting restitution  Officials from all six USAOs stated that using the Treasury Offset Program (TOP), a program that allows for the reduction or withholding of a debtor’s federal benefits, such as a tax refund, was one of the most effective practices for collecting restitution  Specifically, officials in one USAO told us that TOP requires minimal effort for FLU staff and can result in a high amount of collections  As an example, officials from two USAOs told us their respective offices each recovered more than $500,000 dollars in restitution debt in fiscal year 2016 through TOP  Officials from three of the six offices also identified using wage garnishment as an effective practice for collecting restitution  Across all parts of the restitution process, USAO officials we spoke with also consistently identified DOJ recommended practices related to internal and external communication and collaboration as effective for improving the restitution process  Specifically, the officials identified collaboration between various units in the USAO as an effective practice to ensuring restitution for victims  For example, USAO officials in two of the six offices highlighted coordination between Victim-Witness coordinators and prosecutors to help identify victims and quantify their losses as effective to assisting in the request for restitution  Additionally, USAO officials in all six offices stated that strong coordination between FLU personnel and criminal prosecutors to identify an offender’s financial resources and available assets was an effective practice to help ensure FLU staff could collect restitution using those resources or assets  USAO officials from five of six offices identified external communication between FLU and the federal probation office as an effective practice  Specifically, officials from these USAOs stated that FLUs coordinating with probation officers during the offender’s supervision period to enforce restitution terms was an effective practice for collecting restitution  Additionally, according to EOUSA guidance, FLU staff can use outreach and training with other partners such as the probation office to facilitate information sharing on restitution collection issues and officials from five of six USAOs told us that FLUs conducting training and outreach is a very effective practice  In addition, probation officials we interviewed in each of the six federal judicial districts we selected stated that ongoing communication between USAO staff and probation officers is effective to ensuring victims are identified and receive full and timely restitution  Probation officials from one court district emphasized the importance of a good working relationship with the USAO, stating that the probation office and USAO are better able to ensure victims and their losses are accurately identified and defendants’ ability to pay is adequately addressed when working collaboratively  A probation official from another office said that probation officers regularly coordinated with the USAO’s FLU, and this coordination was particularly important on cases involving complex financial crimes, where the offender has a complicated financial portfolio  Further, probation officials from five of six probation offices also stated that attending training conducted by the FLU is a very effective practice  EOUSA and selected USAO officials told us that while these practices may be useful in some circumstances, they may not be effective or applicable in all cases or in all districts  Specifically, practices DOJ recommends may be effective when offenders have the ability to pay restitution but are simply unwilling to do so; however, USAO officials in five of six offices stated that these practices cannot mitigate the fact that many offenders lack the ability to pay restitution because they lack assets and income  Additionally, while EOUSA guidance recommends that FLU staff contact co-defendants or victims for information on the whereabouts or assets of offenders who owe restitution, officials from three USAOs told us this was not effective  According to one official, although co- defendants are sometimes eager to share information, the information is usually unreliable  USAO officials also identified some recommended practices as not applicable to their district  For example, EOUSA recommends that FLU units request Asset Investigation assistance from EOUSA for complex cases involving large amounts of valuable assets  However, USAO officials in a small, rural district with whom we spoke stated that the types of cases their office prosecutes tend not to be the type of financial cases that warranted use of this resource  DOJ has identified improving debt collection—including court-ordered restitution—as a major management initiative in its 2014-2018 Strategic Plan  However, it does not have any measures or goals in place to assess its performance in meeting this initiative or meet requirements that it evaluate its performance in seeking and recovering restitution as required by statute  In 2001, we recommended that DOJ adequately measure its criminal debt collection performance against established goals to help improve collections and stem the growth in uncollected criminal debt  DOJ concurred with this recommendation, and as of fiscal year 2003, annually assessed each district based on established collection goals for that district  However, as of September 2017, DOJ no longer evaluates each district based on established goals  EOUSA officials stated that DOJ no longer uses these performance goals and that the agency did not maintain records for when or why it stopped  EOUSA officials stated that while the agency does not have any measures or goals to assess USAOs’ performance in improving debt collection, including the collection of federal restitution, they are working with DOJ’s Justice Management Division to develop a suite of analytical tools to monitor the collection of debt across all offices  According to DOJ, some of these analytical tools have been implemented and additional tools will be implemented by March 2018  EOUSA officials stated that these tools will help the agency determine which cases are most likely to result in significant collections and the types and timing of enforcement actions that generate maximum debt recovery results  EOUSA officials further stated the analytical tools will allow the agency to compare districts’ efforts based on a variety of factors (e g , caseload, staff size, and enforcement actions)  These analytical tools may provide EOUSA with valuable insight into the present condition of the collection of restitution across USAOs, but they will not provide DOJ with a baseline performance standard that could be used to indicate if USAOs’ efforts to collect restitution debts are having a measurable impact in meeting DOJ’s objective of improving debt collection  Additionally, EOUSA conducts evaluations of each USAO every 4 years, which include a review of FLU operations, but EOUSA officials stated that these reviews do not include oversight of the collection of restitution  Among other aspects of USAO operations, these internal evaluations review the extent to which each FLU is complying with statutory and DOJ requirements related to debt collection, has sufficient program resources, and adequately manages its caseload  However, DOJ and EOUSA officials told us that it did not plan to use these internal evaluations to meet the Justice for All Reauthorization Act of 2016 requirement to evaluate each USAO on its performance in seeking and recovering restitution for victims  Specifically, the officials stated that these internal evaluations are not an appropriate mechanism to meet the law’s requirements because the internal evaluations do not specifically review the seeking and recovery of restitution for victims  According to DOJ officials responsible for the internal evaluation program, these evaluations are largely intended to provide onsite management assistance and analysis of how the USAO allocates its administrative and legal personnel resources rather than the office’s efficacy in collecting restitution  Consistent with requirements outlined in the Government Performance and Results Act Modernization Act of 2010 (GPRAMA), performance measurement is the ongoing monitoring and reporting of program accomplishments—particularly towards pre-established, objective and quantifiable goals—and agencies are to establish performance measures to assess progress towards those goals  While GPRAMA is applicable to the department or agency level, performance measures and goals are important management tools at all levels of an agency, including the program, project, or activity level  Agencies can use performance measurement to make various types of management decisions to improve programs and results, such as developing strategies and allocating resources, including identifying problems and taking corrective action when appropriate  Further, the Justice for All Reauthorization Act of 2016 requires DOJ to evaluate each USAO in its performance in recovering restitution for victims  DOJ and EOUSA officials told us that DOJ does not require USAOs to establish performance measures or goals to assess their progress in improving the collection of restitution  DOJ and EOUSA officials also told us that each USAO could develop performance goals but that they were unaware of the extent to which USAOs did so, and further, they do not track the extent to which USAOs met performance goals  Additionally, these officials stated that because each USAO faces different constraints in its ability to collect restitution, establishing a uniform and consistent performance measure and goal would be challenging  EOUSA officials noted that some USAOs may have more resources, such as more FLU staff or specialized asset investigators, available to pursue collections as compared to other offices and therefore offices with fewer resources could have difficulty meeting a performance goal  Further, EOUSA and USAO officials stated that the extent to which DOJ can collect on a debt is heavily influenced by factors outside of the agency’s control, such as an offender’s ability to pay  USAOs could use information provided by performance measures and goals—such as an office’s ability to meet a performance goal—to make managerial decisions to help address these constraints, such as by increasing the allocation of staff resources  Further, to avoid comparing USAOs to a nationally set performance goal that does not account for specific constraints faced by each office, DOJ could—as it did in fiscal year 2003—require each USAO to establish its own objective, quantitative collection goals based on historical, district-specific collection statistics  Finally, as previously discussed, each USAO already accounts for external factors that affect the collectability of a debt, such as an offender’s ability to pay, by suspending collection action on debts it identifies as uncollectible  Therefore, any performance measures and goals developed could be based solely on debts that the USAO already has determined to be collectible  Stakeholders we interviewed—including officials from one USAO, probation officials in two districts, and officials with DOJ’s Office of Crime Victims—noted that receiving restitution is both emotionally and financially important to victims  Specifically, officials from one USAO and one probation office noted that while many victims may never receive the full amount of restitution ordered, receiving even a minimal amount of restitution is a symbolic victory and that it is important for victims to know the government is making efforts to collect restitution on their behalf  The legislative history of the MVRA echoes these sentiments, providing that even nominal restitution payments have benefits for the victim of crime, and that orders of restitution are largely worthless without enforcement  Yet, according to our analysis, $10 billion of restitution debt DOJ identified as collectible remained outstanding at the end of fiscal year 2016  Further, the extent to which USAOs collected restitution varied widely— from a high of one USAO district collecting nearly 350 percent of all collectible debt in fiscal years 2014 through 2016 to a low of one district collecting less than one percent of collectible debt in the same period  Without performance measures, including the establishment of goals, DOJ cannot assess if this variation is due to factors outside the control of USAOs or due to management deficiencies that require corrective action  Developing performance measures and goals for each USAO related to the collection of restitution would allow DOJ to assess its progress in achieving its major management initiative in improving debt collection— including debts owed to victims as court-ordered restitution  Doing so would also better position DOJ to meet the requirements of the Justice for All Reauthorization Act of 2016 to evaluate offices in their performance in recovering restitution on behalf of victims and to use performance information to improve the practices of offices as needed  Although asset forfeiture and restitution are separate parts of a criminal sentence, DOJ guidance states that using forfeited assets to benefit victims is a way that DOJ can help ensure eligible victims of crime are compensated for their losses  Further, DOJ regulations and policy require that eligible victims receive compensation from forfeited assets before certain other uses, such as official use or equitable sharing  However, while DOJ tracks the amount of compensation provided to victims through forfeited assets, it does not have assurances that forfeited assets are being used to compensate victims to the greatest extent possible  According to DOJ information, the agency made payments of about $595 million to eligible victims other than owners of the property from the Assets Forfeiture Fund from fiscal years 2014 through 2016, or 15 percent of $3 9 billion in paid expenditures during this period, as shown in table 5  As table 5 shows, DOJ can account for cases in which forfeited assets were used to compensate eligible victims who were not owners or lienholders  However, DOJ does not have information on the overall universe of victims who could have been eligible to receive compensation from forfeited assets  Further, it does not have insight into any reasons why funds from forfeited assets were not used for these victims  Specifically, DOJ officials stated that the department collects information on whether victims have been identified in cases associated with forfeited assets, and if restitution is anticipated in these cases, but it does not track the extent to which these victims were ultimately compensated using forfeited assets  Further, DOJ also does not collect information on reasons why victims were not compensated using funds from forfeited assets  While DOJ is required to use forfeited assets to compensate victims before using those assets for certain other purposes, the agency is unable to provide assurances that it is always doing so because it does not have information on the overall universe of victims or reasons why victims were not compensated using forfeited assets  As a result, DOJ does not have a basis to know whether the $595 million provided to victims from fiscal years 2014 through 2016 is the maximum amount of compensation the agency could have provided to victims using forfeited assets  Full use of forfeited assets for victim compensation has long been, and continues to be, a goal of DOJ  In 2005, an interagency task force—led by DOJ and including the Department of Treasury, Office of Management and Budget and AOUSC—developed a strategic plan to improve the collection of criminal debt  Among other goals included in its strategic plan, the task force stated a goal of examining how asset seizure and forfeiture procedures can be used to maximize recoveries for victims  More recently, DOJ reported in its 2014-2018 Strategic Plan that it would make every effort to recover full and fair restitution for victims using the federal forfeiture statutes to preserve and recover criminal proceeds  Specifically, DOJ stated that using federal forfeiture statutes to recover full and fair restitution for victims is one part of its strategy to protect the rights of the American people and enforce the rule of law  Finally, DOJ officials told us they considered providing compensation to victims as one goal of the Asset Forfeiture Program and EOUSA stated in guidance that asset forfeiture is the most widely available and effective tool to seize assets for restitution purposes  Standards for Internal Control in the Federal Government call on federal managers to design control activities to achieve the agency’s objectives  These controls can include using quality information to make informed decisions, evaluate the entity’s performance in achieving key objectives, and address risks  DOJ officials told us that they do not track the extent to which victims were not compensated using forfeited assets because USAO staff are not required to request that these assets be used for victim compensation  DOJ officials explained that staff are required to indicate in the agency’s forfeited asset database, the Consolidated Asset Tracking System, if victims exist in cases associated with forfeited assets and if restitution is anticipated in these cases  However, these officials stated that staff are not required to then compensate these victims using the forfeited assets or to indicate why these assets were not used for this purpose  DOJ officials told us that decisions to compensate victims using forfeited assets are best left to the judgment of the USAO staff familiar with the case, such as the prosecuting attorney or asset forfeiture staff  DOJ officials pointed to informal communication and coordination among prosecutors, the FLU, and the Asset Forfeiture unit in each USAO as a means to provide compensation to victims as appropriate  However, communication and coordination among these groups has been a challenge for USAOs, as the DOJ Inspector General found in a June 2015 review of DOJ’s debt collection program  Similarly, during our current review, EOUSA and USAO officials we spoke with identified communication and coordination as an area for improvement  EOUSA officials told us that while they thought that FLU staff and Asset Forfeiture unit staff were collaborating more frequently to use forfeited assets to collect restitution debts since the issuance of the DOJ Inspector General’s report, the extent of collaboration between these two units still varied across USAOs  Further, officials we talked to in two USAOs and one probation office noted that USAO staff could improve their use of forfeited assets for restitution payments  For example, officials in one probation office noted that it was their practice to identify forfeited assets that could be used for compensation in the PSR because they had observed that USAO staff were frequently not applying such assets to victim compensation  While DOJ may allow USAO staff to use discretion when requesting restoration or alerting victims to assets available for compensation, increasing the agency’s understanding of the extent to which assets could have been—but were not—used for victim compensation, and the reasons for those decisions, does not affect that discretion  There are legitimate reasons why victims might not be compensated using forfeited assets; for example, the assets may have other owners or lienholders that must be compensated prior to victims, or offenders may have other means by which to pay victims restitution  However, there are also instances where victims may have not received compensation through forfeited assets as a result of unintentional circumstances  For example, according to DOJ’s Asset Forfeiture Manual, forfeiture actions can proceed faster than the parallel criminal case  Consequently, assets might be equitably shared, placed into official use, or remitted to victims who file petitions long before restitution is ordered, and therefore would not be available for other victims who wait for restitution to be ordered after an offender is sentenced  To avoid this outcome, DOJ recommends that USAOs coordinate to ensure the retention of property for victim compensation  However, although DOJ officials responsible for leading DOJ’s asset forfeiture efforts highlighted the need for expedient coordination when USAO staff are considering using forfeited assets to compensate victims, they stated this may not always occur  As a result, otherwise eligible victims may not always be compensated through forfeited assets  By gathering information about the extent to which assets were used for victim compensation—including when they were not used and reasons why not—DOJ could have a better understanding of potential instances where victims could be, but are not, receiving compensation through forfeited funds and could take steps to address them accordingly  Options for gathering such information could include doing a one-time retrospective study of forfeited assets with victims or anticipated restitution to determine the extent that assets were used for victim compensation, or creating a tracking mechanism through its forfeited assets database, or another system  Gathering information on the extent to which forfeited assets were used for victim compensation, including when not used and reasons why not, could position DOJ to take action to increase the use of these assets for victim compensation if warranted  These actions could include providing funds for increased asset forfeiture staff in USAOs, providing additional training or changing policies or procedures for using forfeited assets to compensate victims  Fully and systematically understanding the extent to which issues, such as a lack of coordination within USAOs, result in victims not being compensated using forfeited assets would give DOJ a basis upon which to develop improvements to the Asset Forfeiture Program  Such information would also provide DOJ and staff at all USAOs with information to evaluate its performance in achieving one of the goals of the Asset Forfeiture Program and taking action to meet the agency goal of protecting the rights of the American people—including the right to full and fair restitution for victims  Restitution serves the criminal justice goal of holding offenders accountable and, to the extent possible, restoring victims of federal crimes to their prior position had the crime not occurred  Many victims are unlikely to receive any meaningful portion of court-ordered restitution owed to them because of offenders’ inability to pay these debts  However, the fact that restitution is difficult to collect does not negate the important responsibilities of the judiciary and DOJ to properly manage and oversee all aspects of the restitution process  By law, courts are to state why they did not order restitution and provide that information to USSC  While this information was collected and recorded in USSC data for most offenders, we found that this information was missing for thousands of offenders  It is important for the judiciary to ensure that this information is consistently collected and recorded to assist USSC in its continuous re-examination of its guidelines and policy statements and ensure that various sentencing practices are achieving their stated purposes  The judiciary could support USSC in this endeavor by determining why this information is missing  Results from this study could help inform the judiciary whether this issue rises to the level of an internal control deficiency and whether additional action can be taken to improve the transparency of sentencing decisions  While DOJ has delegated collection activities for restitution to USAOs, it could provide better oversight to ensure it is making reasonable efforts to collect restitution and meeting its responsibility to victims  USAOs have identified a significant portion of outstanding restitution debt as uncollectible, but they have also identified $10 billion of outstanding restitution debt that could be collected  Developing and implementing performance measures and goals for each USAO would allow DOJ to gauge USAOs’ success in collecting this restitution and, by extension, the department’s success in achieving its major management initiative to increase the collection of debt  Further, DOJ could use performance information to improve the practices of offices in seeking and recovering restitution, consistent with a requirement in the Justice for All Reauthorization Act of 2016  Finally, DOJ could gain greater visibility into the use of forfeited assets to compensate victims by gathering information on cases in which victims have been identified and restitution is anticipated but forfeited assets are not used, and any reasons why  Doing so would better position DOJ to take action to increase the use of forfeited assets to compensate eligible victims if warranted and to provide assurance that it is maximizing the use of asset forfeiture in satisfying restitution debts, one of the agency’s most effective mechanisms for satisfying restitution  We are making three recommendations, including one to the judiciary and two to DOJ  Specifically: Judiciary officials, including AOUSC, USSC, and the Judicial Conference, should determine why USSC data on the reasons restitution was not ordered are incomplete  Additionally, if warranted based on this information, judiciary officials should take action to ensure USSC data records include all required information for orders of restitution  (Recommendation 1) To improve oversight of the collection of restitution we recommend that the Attorney General: Develop and implement performance measures and goals for each USAO related to the collection of restitution, and measure progress towards meeting those goals  (Recommendation 2) In cases where forfeited assets were not used to compensate victims, gather information on reasons why forfeited assets were not used for victims  If warranted based on this information, take action to increase the use of forfeited assets to compensate eligible victims  (Recommendation 3) We provided a draft of this report for review and comment to DOJ, the Judicial Conference of the United States, AOUSC, USSC, and the Federal Judicial Center  DOJ concurred with our recommendations and provided technical comments, which we incorporated as appropriate  AOUSC provided written comments, which are reproduced in appendix III  In its written comments, AOUSC noted that it would work with the USSC to address our recommendation  We are sending copies of this report to the appropriate congressional committees and the Attorney General, the Judicial Conference of the United States, the Directors of AOUSC, the Staff Director of USSC, the Federal Judicial Center and other interested parties  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you and your staff have any questions about this report, please contact me at (202) 512-8777 or goodwing@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions are listed in appendix IV  According to our analysis of data from the U S  Sentencing Commission (USSC), 214,578 federal offenders were sentenced from fiscal years 2014 through 2016  Table 6 shows the number of offenders sentenced and the number and percentage of offenders ordered to pay restitution for each primary offense of conviction in fiscal years 2014 through 2016  The Department of Justice (DOJ) has identified and recommended numerous practices for federal prosecutors and Financial Litigation Unit (FLU) staff to use throughout the restitution process through various guidance documents  We conducted semi-structured interviews with officials from six U S  Attorneys’ Offices (USAO) to obtain their views on the restitution process and the extent to which they believed DOJ- recommended restitution practices related to the restitution process were effective  In particular, we spoke with USAO officials from the District of Connecticut; the Southern District of California; the District of New Jersey; the Southern District of Ohio; the District of South Dakota; and the District of Wyoming  Tables 7 through 9 show the results of our semi-structured interviews  In particular, table 7 shows practices related to requesting restitution and the extent to which USAO officials found these practices effective  Table 8 shows practices related to facilitating orders of restitution and the extent to which USAO officials found these practices effective  Table 9 shows practices related to collecting restitution and the extent to which USAO officials found these practices effective  Each table also indicates practices that officials we interviewed considered as most important or effective for helping ensure victims receive full and timely restitution  In addition to the contact named above, Chris Ferencik (Assistant Director); Kathleen Donovan (Analyst-in-Charge); Enyinnaya David Aja; David Alexander; Lacinda Ayers; Carla Brown; Emily Hutz; Janet Temko- Blinder; and Adam Vogt, made key contributions to this report 
Report 2:  Developing international military capacity  U S  officials told us that UN peacekeeping operations provide an indirect benefit of helping to professionalize the military units from many developing countries that contribute troops to the UN  We have previously reported that building military capacity of foreign partners to address security-related threats is an important goal of U S  national security strategy and foreign policy  According to U S  and UN officials, the relative strengths of U S  peacekeeping operations would be faster deployment and superior command and control, logistics, intelligence, and counterterrorism capabilities  Deployment speed  State, DOD, and UN officials highlighted the United States’ ability to deploy troops and police to peacekeeping operations more quickly than the UN  Unlike the U S  military, which can draw from a ready pool of military personnel, the UN must seek troops from UN member states, which takes time  UN officials told us that the UN faces a shortage of both troops and UN police, which slows deployment  Further, a 2015 report by the UN High-level Panel on Peacekeeping stated that the UN “has struggled to get sufficient forces on the ground quickly enough and relies on under-resourced uniformed capabilities ” The report also stated that aviation, medical, and engineering specialists, among others, are difficult to mobilize in advance of infantry units  Command and control  State, DOD, and UN officials told us that U S  operations would enable the U S  military to have direct command and control, whereas UN operations, which are inherently multinational, face challenges with command and control over troops from several different countries  The UN High-level Panel report noted that UN peacekeeping operations’ weak command and control is a well-known constraint that limits the type of operations the UN can undertake  Logistics support  U S  and UN officials told us that U S  operations have superior logistics systems  U S  procurement likely would be faster than UN procurement, which lacks a standing supply chain and, therefore, relies on third-party vendors, according to UN officials  In addition, the UN High Level Panel report stated that UN peacekeeping operations’ logistics systems and structures in the field are under severe strain, which can limit the mobility of these operations  Intelligence capability  U S  and UN officials agreed that U S  operations would involve superior intelligence capability  The UN only recently established an intelligence policy—in May 2017—having recognized that some peacekeeping operations had been deployed in fragile political and security environments with asymmetrical and complex threats  However, UN officials acknowledged that the scope of UN intelligence capability remains limited  Counterterrorism capability  DOD officials told us that a U S  peacekeeping operation would have the capability to include a counterterrorism component and would not be constrained in the use of force, if needed, in response to terrorist threats  UN peacekeeping operations, on the other hand, lack the capabilities and specialized military preparation to engage in counterterrorism operations, according to the UN High-level Panel report  The UN report stated that counterterrorism should be undertaken by the host government, a capable regional force, or an ad hoc coalition authorized by the UN Security Council  According to the UN report, UN peacekeeping operations may engage in proactive and preemptive use of force to protect civilians and UN personnel from threats; however, offensive force to degrade, neutralize or defeat an opponent is a fundamentally different type of posture that should be authorized by the Security Council only under limited and exceptional circumstances  We provided a draft of this report to DOD, State, and the UN for review and comment  DOD provided a letter, reproduced in appendix II, which stated that it had no comments  State did not provide comments  The UN provided technical comments, which we incorporated into our report as appropriate  We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and State, and the Secretary- General of the United Nations  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-9601 or melitot@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made significant contributions to this report are listed in appendix III  The objectives of this report were to (1) compare the reported costs of a specific United Nations (UN) peacekeeping operation to the estimated costs of a hypothetical, comparable operation implemented by the United States; (2) identify factors that affect cost differences; and (3) identify stakeholder views on the relative strengths of UN and U S  peacekeeping operations  To compare the reported costs of a specific UN peacekeeping operation to the estimated costs of a hypothetical, comparable operation implemented by the United States, we selected the UN Multidimensional Integrated Stabilization Mission in the Central African Republic (MINUSCA) as a case study  We compared the reported UN expenditures for MINUSCA, which included both military and civilian components, with estimated costs for a hypothetical U S  operation with a similar level of military and civilian personnel  Our comparison covers a total of 3 years and 3 months—from MINUSCA’s establishment in April 2014 through June 30, 2017, the end of UN fiscal year 2017  We selected MINUSCA because it is in sub-Saharan Africa, where most UN peacekeeping operations established since 2003 have taken place, and has a typical scope and budget compared to other UN peacekeeping operations in sub-Saharan Africa, according to U S  and UN officials  In addition, MINUSCA is one of the most recent UN peacekeeping operations; thus, initial expenditures for the operation are relatively current  Because the results of our cost comparison are based on a single case study, they cannot be generalized to all UN peacekeeping operations  To determine the UN’s costs for MINUSCA, we analyzed UN budget and expenditure data covering the initial start-up period (April 2014 to June 2014) and the first 3 UN fiscal years (July 1, 2014 to June 30, 2017)  We spoke with officials of the UN Departments of Peacekeeping Operations, Field Support, and Management at UN Headquarters in New York, New York, to better understand the characteristics of MINUSCA and the different costs affecting MINUSCA’s budget and expenditures  We assessed UN expenditure data through discussions with cognizant UN officials and a review of external audits of UN budgetary information and found them sufficiently reliable for our purposes  We also analyzed data on U S  contributions to UN peacekeeping operations for fiscal years 2014 through 2017 from the Department of State’s (State) Bureau of International Organization Affairs to determine total U S  contributions to MINUSCA, and UN peacekeeping operations overall  To estimate the costs of a hypothetical, comparable operation implemented by the United States, we developed a hypothetical scenario for a U S  operation based on the MINUSCA budget and supporting documents, assuming deployment of the same number of military, civilian, and police personnel in the Central African Republic (CAR) over the same time period (April 2014 through June 2017)  To estimate the military portion of the operation, we interviewed Department of Defense (DOD) officials and staff at the Institute for Defense Analyses (IDA), a DOD-sponsored non-profit corporation involved in developing cost estimates for U S  contingency operations  The Office of the Undersecretary of Defense-Comptroller and IDA generated a cost estimate for the military components included in the hypothetical operation using the Contingency Operations Support Tool (cost estimating tool)  DOD uses this tool to develop cost estimates for all military contingency operations  The cost estimate included only the incremental costs of the operation—those directly attributable to the operation that would not be incurred if the operation did not take place  For example, the estimate produced by the cost estimating tool did not include the direct salaries of active duty personnel as those costs would be incurred by the United States regardless of a possible decision to undertake the hypothetical operation  We assessed the cost estimating tool’s applicability to developing a hypothetical cost estimate for the purposes of this report through discussions with DOD and IDA officials, and compared the tool to the accurate and comprehensive characteristics of a high-quality cost estimate, as described in the GAO Cost Estimating and Assessment Guide  While we found the DOD cost estimating tool generated a sufficiently reliable cost estimate for a hypothetical U S  peacekeeping operation, we did not assess the overall reliability of the tool or its capability to generate accurate or comprehensive estimates for future U S  operations  To generate our estimate of U S  military costs using the DOD’s estimating tool, we used UN military deployment numbers as a baseline for the scale of a hypothetical, comparable U S  peacekeeping operation, while using unit sizes and rotations in deployment that were considered appropriate for the U S  military, according to DOD and IDA officials  We based the hypothetical U S  operation, and hence the cost estimate, on the following assumptions, which correspond approximately with MINUSCA’s actual UN personnel deployments: Theater of operation: Central African Republic (CAR) Type of operation: military contingency Operation time frame: April 10, 2014 through June 30, 2017 Military contingents: as of June 30, 2017, 11,495 total personnel Infantry: 10 units of 630-785 personnel per unit, approximately 90 percent active duty / 10 percent reserves Communication / signals: 1 unit, 124 personnel per unit Engineering: 4 units, 200 personnel per unit Military police: 1 unit, 120 personnel per unit Formed police units (military police): 12 units, 140 personnel per Hospital / medical: 1 level III hospital, 248 beds, 495 personnel Helicopter units: 2 UH-60 C3 units, 1 MH-60M Assault attack helicopter unit, 100 personnel per unit Quick reaction force: 1 unit, 160 personnel per unit Special forces: 1 tactical civilian affairs unit, 1 Marine special operations intelligence unit, 160 personnel per unit Unmanned aerial vehicle: 1 unit, 84 personnel Transportation: 1 heavy transport unit, 120 personnel per unit Operational tempo: 1 0 for all phases of operation and units, except aviation units (set at 1 5) Deployment schedule and phasing: phased deployment, including 14 days for predeployment (e g , training), 5 days for deployment, 180 days for active duty unit sustainment and 270 days for reserve unit sustainment, 5 days for redeployment, and 0 days for reconstitution Housing: contractor-provided semi-permanent housing Transportation: personnel and equipment transported by airlift from the United States (primarily Fort Hood, Texas), material (such as water, food, and other consumables) transported by airlift from Italy We obtained input on the operational design for the military portion of the cost estimate from DOD officials in the Joint Chiefs of Staff, the Office of the Undersecretary of Defense-Policy, and the Office of the Undersecretary of Defense-Comptroller, and IDA officials  However, the military portions of the scenario and their corresponding cost estimate have some limitations  As a result of rounding for some units, U S  military personnel numbers do not exactly match the MINUSCA deployment levels  In addition, based on input from DOD officials, we attempted to select military units that would provide an essential function per U S  common practices while keeping the overall personnel deployment level as close as possible to MINUSCA’s deployment level  An actual U S  military plan may differ significantly from the UN plan as a result of differences between U S  and UN military operations, structure, doctrine, and circumstances at the time of the operation  To estimate U S  civilian costs, we matched the number of U S  civilian police and personnel to the number serving in MINUSCA  We then estimated the costs of deploying these U S  civilian personnel in CAR for the same time period as MINUSCA  We did not attempt to determine how the U S  government would actually implement civilian components of a peacekeeping operation in CAR  To estimate U S  civilian police costs, we met with State’s Bureau of International Narcotics and Law Enforcement Affairs (INL) to identify State’s costs for civilian police contractors providing police training and technical assistance in sub-Saharan Africa  Based on INL’s input we assumed that the base salary of civilian police would be grade 13, step 5 on the Office of Personnel Management’s general schedule salary tables for federal employees  In addition to the average base salary, we identified other costs—with input from INL—including, among others, personal equipment, travel from the United States, and State’s published allowances specific to CAR for local cost of living, post hardship differential, danger pay, and living quarters  We applied the average cost per officer to the average number of UN civilian police officers deployed in MINUSCA  To estimate U S  civilian personnel costs, we met with State’s Bureau of Budget and Planning to identify the costs of State Foreign Service officers and locally employed staff, based on the number of UN international and national civilian staff deployed to MINUSCA, respectively  We matched the number of State Foreign Service officers for the U S  cost estimate to the number of UN international staff in MINUSCA, with input from State to align the grade levels  The estimated costs for Foreign Service officers include average salaries based on State’s Foreign Service salary tables and State’s allowances specific to CAR, including local cost of living, post hardship differential, and danger pay  We also met with State Bureau of Budget and Planning officials to estimate other costs for Foreign Service officers, which we included in our cost estimate, including post assignment travel, administrative support costs, residential furnishings, and residential guards, among others, but we did not assess the reliability of these additional costs provided by State  In addition, State’s Bureau of Overseas Buildings Operations provided the actual costs of residential leases for Foreign Service officers in CAR in fiscal year 2017, which we used to estimate the cost of housing Foreign Service officers in CAR  We also matched the number of State locally employed staff to the number of UN national staff deployed to MINUSCA and added their average salaries and other costs in CAR based on data provided by State’s Bureau of the Comptroller and Global Financial Services  While MINUSCA’s expenditures also included costs for sending an annual average of up to about 200 UN volunteers to CAR, State officials told us that the United States generally would not send volunteers through its assistance efforts to a high-risk post, such as CAR  Therefore, we did not include any costs related to volunteers in the cost estimate  We also did not include costs related to host-government-provided personnel serving in MINUSCA  In addition, UN expenditures included about $7 million for “quick-impact projects” to support local government infrastructure and civil society initiatives  We did not include comparable costs for quick-impact projects in our U S  cost estimate because we did not have a basis for matching these costs  To identify factors that affect cost differences between MINUSCA and a hypothetical, comparable operation implemented by the United States, we reviewed the U S  cost estimate generated in conjunction with DOD, IDA, and State officials, and identified significant areas of cost for the United States and the assumptions incorporated in the estimate or factors specified by U S  officials that drive those costs  We compared the U S  cost estimate, including these significant areas of cost, to UN costs to identify differences  We interviewed U S  and UN officials regarding U S  and UN standards and policies that explain differences between MINUSCA costs and the estimated costs of a U S  operation  To identify stakeholder views on the relative strengths of UN and U S  peacekeeping operations, we reviewed UN reports on peacekeeping operations and interviewed UN, DOD, and State officials  In addition, we reviewed GAO’s 2006 report comparing the costs as well as the strengths of a UN peacekeeping operation in Haiti with those of a hypothetical U S  operation  We conducted our review from February 2017 through February 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the individual named above, Drew Lindsey (Assistant Director), Howard Cott, Juan Pablo Avila-Tournut, Debbie Chung, Martin de Alteriis, Neil Doherty, Jennifer Leotta, Caitlin Mitchell, Elizabeth Repko, and Alex Welsh made significant contributions to this report 

ground_truth: One of the goals of federal criminal restitution is to restore victims of federal crimes to the position they occupied before the crime was committed by providing compensation. Various entities within the federal government are involved in the process of requesting, ordering, and collecting restitution for crime victims, including DOJ and the judiciary. The Justice for All Reauthorization Act of 2016 includes a provision for GAO to review the federal criminal restitution process for fiscal years 2014 through 2016. This report addresses, among other things: (1) the extent to which information is available on restitution requested by DOJ and ordered by courts; (2) the amount of restitution debt DOJ collected and the amount that remains outstanding; and, (3) the extent to which DOJ has conducted oversight on the collection of restitution. GAO analyzed laws, policies and procedures as well as USSC data on restitution orders and DOJ data on restitution collected from fiscal years 2014 through 2016. GAO also selected a non-generalizable sample of six federal judicial districts based on restitution collections and spoke with USAO officials and federal probation officers. Officials from selected U.S. Attorney's Offices (USAO) stated that they document requests for restitution in case files and employ other internal controls, such as the use of templates and forms, throughout the prosecution process to ensure that prosecutors request restitution as appropriate. GAO's analysis of U.S. Sentencing Commission (USSC) data—an agency within the judiciary—showed that information on restitution orders was available for 95 percent of all offenders sentenced from fiscal years 2014 through 2016. Specifically, 214,578 federal offenders were sentenced during this time period and restitution was ordered for 33,158, or 15 percent, of those offenders. Collectively, courts ordered these offenders to pay $33.9 billion in restitution. Most federal offenders sentenced during these years were sentenced for immigration or drug-related offenses. In interviews, USAO officials stated that these offenses do not typically have victims requiring restitution. GAO found that data on reasons why restitution was not ordered were incomplete for 5 percent of all offenders sentenced from fiscal years 2014 through 2016. Determining why data on restitution orders are incomplete may inform the judiciary of the cause of the incomplete data and any efforts needed to improve USSC data. GAO's analysis of Department of Justice (DOJ) data showed that USAOs collected $2.95 billion in restitution debt in fiscal years 2014 through 2016, see figure below. However, at the end of fiscal year 2016, $110 billion in previously ordered restitution remained outstanding, and USAOs identified $100 billion of that outstanding debt as uncollectible due to offenders' inability to pay. DOJ identified improving debt collection—including restitution—as a major management initiative in its 2014-2018 Strategic Plan. While DOJ is developing analytical tools to monitor the collection of restitution, it has not established performance measures or goals. Performance measures and goals would allow DOJ to gauge USAOs' success in collecting restitution and, by extension, the department's success in achieving a major management initiative. GAO is making three recommendations. GAO is making one to the judiciary to determine why data on restitution orders are incomplete. GAO is making two recommendations to DOJ, including one to implement performance measures and goals for the collection of restitution. The judiciary and DOJ concurred with the recommendations.