prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.



Report 2: Concerned that the federal government was more focused on program activities and processes than the results to be achieved, Congress passed the Government Performance and Results Act of 1993 (GPRA)  GPRA sought to focus federal agencies on performance by requiring agencies to develop long-term and annual goals, and measure and report on progress towards those goals annually  Based on our analyses of the act’s implementation, we concluded in March 2004 that GPRA’s requirements had laid a solid foundation for results-oriented management  At that time, we found that performance planning and measurement had slowly yet increasingly become a part of agencies’ cultures  For example, managers reported having significantly more performance measures in 2003 than in 1997, when GPRA took effect government-wide  However, the benefit of collecting performance information is fully realized only when that information is actually used by managers to make decisions aimed at improving results  Although our 2003 survey found greater reported availability of performance information than in 1997, it also showed managers’ use of that information for various management activities generally had remained unchanged  Based on those results, and in response to a request from Congress, in September 2005, we developed a framework intended to help agencies better incorporate performance information into their decision making  As shown in figure 1, we identified five leading practices that can promote the use of performance information for policy and program decisions; and four ways agency managers can use performance information to make program decisions aimed at improving results  Our September 2005 report also highlighted examples of how agencies had used performance information to improve results  For example, we described how the Department of Transportation’s National Highway Traffic Safety Administration used performance information to identify, develop, and share effective strategies that increased national safety belt usage—which can decrease injuries and fatalities from traffic accidents— from 11 percent in 1985 to 80 percent in 2004  Subsequently, the GPRA Modernization Act of 2010 (GPRAMA) was enacted, which significantly expanded and enhanced the statutory framework for federal performance management  The Senate Committee on Homeland Security and Governmental Affairs report accompanying the bill that would become GPRAMA stated that agencies were not consistently using performance information to improve their management and results  The report cited the results of our 2007 survey of federal managers  That survey continued to show little change in managers’ use of performance information  The report further stated that provisions in GPRAMA are intended to address those findings and increase the use of performance information to improve performance and results  For example, GPRAMA requires certain agencies to designate a subset of their respective goals as their highest priorities—known as agency priority goals—and to measure and assess progress toward those goals at least quarterly through data-driven reviews  Our recent work and surveys suggest that data-driven reviews are having their intended effect  For example, in July 2015, we found that agencies reported that their reviews had positive effects on progress toward agency goals and efforts to improve the efficiency of operations, among other things  In addition, for those managers who were familiar with their agencies’ data-driven reviews, our 2013 and 2017 surveys showed that the more managers viewed their programs as being subject to a review, the more likely they were to report their agencies’ reviews were driving results and conducted in line with our leading practices  Recognizing the important role these reviews were playing in improving data-driven decision making, our management agenda for the presidential and congressional transition in 2017 included a key action to expand the use of data-driven reviews beyond agency priority goals to other agency goals  More broadly, our recent surveys of federal managers have continued to show that reported government-wide uses of performance information generally have not changed or in some cases have declined  As we found in September 2017, and as illustrated in figure 2, the 2017 update to our index suggests that government-wide use of performance information did not improve between 2013 and 2017  In addition, it is statistically significantly lower relative to our 2007 survey, when we created the index  Moreover, in looking at the government-wide results on the 11 individual survey questions that comprise the index, we found few statistically significant changes in 2017 when compared to (1) our 2013 survey or (2) the year each question was first introduced  For example, in comparing 2013 and 2017 results, two questions had results that were statistically significantly different: The percentage of managers who reported that employees who report to them pay attention to their agency’s use of performance information was statistically significantly higher (from 40 to 46 percent)  The percentage of managers who reported using performance information to adopt new program approaches or change work processes was statistically significantly lower (from 54 to 47 percent)  As we stated in our September 2017 report, the decline on the latter question was of particular concern as agencies were developing plans to improve their efficiency, effectiveness, and accountability, as called for by an April 2017 memorandum from OMB  In early 2017, the administration announced several efforts intended to improve government performance  OMB issued several memorandums detailing the administration’s plans to improve government performance by reorganizing the government, reducing the federal workforce, and reducing federal agency burden  As part of the reorganization efforts, OMB and agencies were to develop government-wide and agency reform plans, respectively, designed to leverage various GPRAMA provisions  For instance, the April 2017 memorandum mentioned above stated that OMB intends to monitor implementation of the reforms using, among other things, agency priority goals  While many agency-specific organizational improvements were included in the President’s fiscal year 2019 budget, released in February 2018, OMB published additional government-wide and agency reform proposals in June 2018  The President’s Management Agenda (PMA), released in March 2018, outlines a long-term vision for modernizing federal operations and improving the ability of agencies to achieve outcomes  To address the issues outlined in the PMA, the administration established a number of cross-agency priority (CAP) goals  CAP goals, required by GPRAMA, are to address issues in a limited number of policy areas requiring action across multiple agencies, or management improvements that are needed across the government  The PMA highlights several root causes for the challenges the federal government faces  Among them is that agencies do not consistently apply data-driven decision-making practices  The PMA states that smarter use of data and evidence is needed to orient decisions and accountability around service and results  To that end, in March 2018, the administration established the Leveraging Data as a Strategic Asset CAP goal to improve the use of data in decision making to increase the federal government’s effectiveness  Over the past 25 years, various organizations, roles, and responsibilities have been created by executive action or in law to provide leadership in federal performance management  At individual agencies and across the federal government, these organizations and officials have key responsibilities for improving performance, as outlined below  OMB: At least every four years, OMB is to coordinate with other agencies to develop CAP goals—such as the one described earlier on leveraging data as an asset—to improve the performance and management of the federal government  OMB is also required to coordinate with agencies to develop annual federal government performance plans to define, among other things, the level of performance to be achieved toward the CAP goals  Following GPRAMA’s enactment, OMB issued guidance for initial implementation, as required by the act, and continues to provide updated guidance in its annual Circular No  A-11, additional memorandums, and other means  Chief Operating Officer (COO): The deputy agency head, or equivalent, is designated as the COO, with overall responsibility for improving agency management and performance through, among other things, the use of performance information  President’s Management Council (PMC): The PMC is comprised of OMB’s Deputy Director for Management and the COOs of major departments and agencies, among other individuals  Its responsibilities include improving overall executive branch management and implementing the PMA  Performance Improvement Officer (PIO): Agency heads designate a senior executive as the PIO, who reports directly to the COO  The PIO is responsible for assisting the head of the agency and COO to ensure that agency goals are achieved through, among other things, the use of performance information  Performance Improvement Council (PIC): The PIC is charged with assisting OMB to improve the performance of the federal government  It is chaired by the Deputy Director for Management at OMB and includes PIOs from each of the 24 Chief Financial Officers Act agencies, as well as other PIOs and individuals designated by the chair  Among its responsibilities, the PIC is to work to resolve government-wide or cross-cutting performance issues, and facilitate the exchange among agencies of practices that have led to performance improvements  Previously, the General Service Administration’s (GSA) Office of Executive Councils provided analytical, management, and administrative support for the PIC, the PMC, and other government-wide management councils  In January 2018, the office was abolished and its functions, staff, and authorities, along with those of the Unified Shared Services Management Office, were reallocated to GSA’s newly created Shared Solutions and Performance Improvement Office  As at the government-wide level—where, as described earlier, the use of performance information did not change from 2013 to 2017—managers’ reported use of performance information at most agencies also did not improve since 2013 (illustrated in figure 3)  At the agency level, 3 of the 24 agencies had statistically significant changes in their index scores—1 increase (National Science Foundation) and 2 decreases (Social Security Administration and the Office of Personnel Management)  Also, in 2017, 6 agencies had results that were statistically significantly different—4 higher and 2 lower—than the government-wide average (see sidebar)  Throughout the report, we highlight two different types of statistically significant results—changes from our last survey in 2013 and differences from the 2017 government-wide average  The former indicates when an agency’s reported use of performance information or leading practices has measurably improved or declined  The latter indicates when it is statistically significantly higher or lower than the rest of government  These results suggest agencies have taken actions that led to improvements in their use of performance information  For example, when a result is a statistically significant increase since 2013, as with the National Science Foundation index score in 2017, this suggests that the agency has adopted practices that led to a measurable increase in the use of performance information by managers  When a result is statistically significantly higher than the government-wide average, like GSA’s 2017 index score, this suggests that the agency’s use of performance information is among the highest results when compared to the rest of government  These agencies could also have insights into practices that led to relatively high levels of performance information use  Finally, when a result is a statistically significant decrease since 2013, as with the Social Security Administration’s index score in 2017, or statistically significantly lower than the government-wide average, like the Department of Homeland Security’s 2017 index score, this suggests the agencies face challenges that are hampering their ability to use performance information  Appendix III provides each agency’s index scores from 2007, 2013, and 2017 to show changes between survey years  When we disaggregated the index and analyzed responses from the 11 questions that comprise the index—which could help pinpoint particular actions that improved the use of performance information—we similarly found relatively few changes in agencies’ recent results  Specifically, we identified 16 instances where agency responses on individual questions were statistically significantly different from 2013 to 2017—10 increases and 6 decreases  This represents about 6 percent of the total possible responses to the 11 survey questions from each of the agencies  In addition, we found 12 instances where an agency’s result on a question was statistically significantly higher (11) or lower (1) than the government-wide average in 2017  For example, the percentage of Social Security Administration (SSA) managers reporting that their peers use performance information to share effective approaches was statistically significantly higher than the government-wide average  Although SSA’s index score had a statistically significant decline in 2017 compared to 2013, the agency’s index score remains relatively high, as it has in prior years  The scope of our work has not allowed us to determine definitively what factors caused the decline in SSA’s index score and whether the decline is likely to continue, although its result on this particular question may indicate a continued strength  Each agency’s results on the 11 questions that comprise the index are presented in appendix I  The agencies’ respective statistically significant results are identified in figure 4  While some agencies had statistically significant improvements on individual questions, and could point to actions that led to improvements in their use of performance information, these improvements should be considered in relation to the range of agency results and the government- wide average  In figure 4, there are five agencies with statistically significant increases on responses to individual questions, where those results were not statistically significantly higher than the government-wide average (see arrows without plus signs for the Departments of Agriculture, Defense, and Justice; the Environmental Protection Agency; and the National Science Foundation)  While these represent improvements, they should be considered in relation to the range of agency results and the government-wide average (provided in detail in the agency summaries in appendix I)  For example, in 2017, the percentage of managers at the Department of Agriculture who reported that upper management use performance information to inform decisions about program changes was statistically significantly higher than in 2013  However, the department’s 2017 result (37 percent) was relatively lower when compared to the maximum agency result on that question (60 percent)  Appendix I presents the results on the index and the 11 questions that comprise it for each of the 24 agencies  When we compared government-wide and agency-level results on selected survey questions that reflect practices that promote the use of performance information, we found that results between 2013 and 2017 generally remained unchanged  As described earlier, there are 10 survey questions that both reflect the five leading practices identified in our past work and had statistically significant associations with higher index scores  As shown in figure 5, government-wide results on 2 of the 10 questions were statistically significantly different, both increases, from 2013 to 2017  Despite these two increases, the overall results suggest these practices are not widely followed government-wide  On most of the 10 questions, only about half (or fewer) of the managers reported their agencies were following them to a “great” or “very great” extent  When we analyzed agency-level responses to these 10 questions, we also found relatively few changes in recent results  Specifically, our analysis found 20 instances—16 increases and 4 decreases—where agencies’ responses on individual questions were statistically significantly different from 2013 to 2017  This represents about 8 percent of the total possible responses to the 10 survey questions from each of the agencies  In addition, we found 10 instances where an agency’s result on a question was statistically significantly higher (8) or lower (2) than the government-wide average in 2017  Each agency’s results on these 10 questions are presented in appendix I, and the statistically significant results are identified in figure 6  Those agencies with results on individual questions that are either statistically significantly higher than 2013, higher than the 2017 government-wide average, or both may have taken actions in line with our leading practices for promoting the use of performance information  For example, the National Science Foundation had both types of statistically significant results on a question about having sufficient information on the validity of their performance data  Here, the agency’s result increased 27 percentage points from 2013 to 2017  While the scope of our review does not allow us to definitively determine the reasons for the National Science Foundation’s higher results, they suggest the agency has taken recent actions that greatly improved the availability and accessibility of information on the validity of performance data  In both 2013 and 2017, our analyses found this particular question to be the strongest predictor of higher performance information use when we tested for associations between the questions that reflect leading practices and our index  Our 2017 survey results show that managers who reported their programs were subject to data-driven reviews also were more likely to report using performance information in decision making to a greater extent (see figure 7)  For the 35 percent of managers who reported being familiar with data-driven reviews, those who reported their programs had been subject to data-driven reviews to a “great” or “very great” extent had index scores that were statistically significantly higher than those whose programs were subject to these reviews to a lesser extent  Similarly, we found that being subject to data-driven reviews to a greater extent was also related to greater reporting of agencies following practices that can promote the use of performance information  As figure 8 shows, managers who reported their programs were subject to these reviews to a “great” or “very great” extent more frequently reported that their agencies followed the five leading practices that promote the use of performance information, as measured by the 10 related survey questions associated with higher scores on the index  For example, of the estimated 48 percent of managers who reported their programs were subject to data-driven reviews to a “great” or “very great” extent, 72 percent also reported that managers at their level (peers) effectively communicate performance information on a routine basis to a “great” or “very great” extent  Conversely, for the 24 percent of managers who reported their programs were subject to data-driven reviews to a “small” or “no” extent, only 30 percent reported that managers at their level do this to a “great” or “very great” extent  Our past work has found that the Executive Branch has taken steps to improve the use of performance information in decision making by senior leaders at federal agencies  However, our survey results indicate those steps have not led to similar improvements in use by managers at lower levels  Through its guidance to implement GPRAMA, OMB developed a framework for performance management in the federal government that involves agencies setting goals and priorities, measuring performance, and regularly reviewing and reporting on progress  This includes expectations for how agency senior leaders should use performance information to assess progress towards achieving agency priority goals through data-driven reviews, and strategic objectives through strategic reviews  For example, GPRAMA requires, and OMB’s guidance reinforces, that data-driven reviews should involve the agency head, Chief Operating Officer, Performance Improvement Officer, and other senior officials responsible for leading efforts to achieve each goal  OMB’s guidance also identifies ways in which agency leaders should use the results of those reviews to inform various decision-making activities, such as revising strategies, formulating budgets, and managing risks  Our past work also found that agencies made progress in implementing these reviews and using performance information  In July 2015, we found that agencies generally were conducting their data-driven reviews in line with GPRAMA requirements and our related leading practices, including that agency leaders used the reviews to drive performance improvement  In addition, in September 2017, we reported on selected agencies’ experiences in implementing strategic reviews and found that the reviews helped direct leadership attention to progress on strategic objectives  Despite those findings, our survey results continue to show that the reported use of performance information by federal managers has generally not improved, and actually declined at some agencies  This could be because of the two different groups of agency officials covered by our work  GPRAMA’s requirements, and the federal performance management framework established by OMB’s guidance, apply at the agency-wide level and generally involve senior leaders  Our past work reviewing implementation of the act therefore focused on improvements in the use of performance information by senior leaders at the agency- wide level  In contrast, our surveys covered random samples of mid- and upper-level managers within those agencies, including at lower organizational levels such as component agencies  Their responses indicate that the use of performance information more broadly within agencies—at lower organizational levels—generally has not improved over time  The exception to this was managers whose programs were subject to the data-driven reviews required by GPRAMA  As described above, those managers were more likely to report greater use of performance information in their agencies  This reinforces the value of the processes and practices put in place by GPRAMA  Our survey results suggest that limited actions have been taken to diffuse processes and practices related to the use of performance information to lower levels within federal agencies, where mid-level and senior managers make decisions about managing programs and operations  Although OMB staff agreed that diffusing processes and practices to lower levels could lead to improved use of performance information, they told us they have not directed agencies to do so for a few reasons  First, OMB staff expressed concerns about potentially imposing a “one-size-fits- all” approach on agencies  They stated that agencies are best positioned to improve their managers’ use of performance information, given their individual and unique missions and cultures, and the environments in which they operate  We agree that it makes sense for agencies to be able to tailor their approaches for those reasons  OMB’s existing guidance provides an overarching framework that recognizes the need for flexibility and for agencies to tailor their approaches  Moreover, given the long- standing and cross-cutting nature of this challenge, a government-wide approach also would provide a consistent focus on improving the use of performance information more extensively within agencies  OMB staff also told us that they believed it would go beyond their mandate to direct agencies to extend GPRAMA requirements to lower levels  GPRAMA requires OMB to provide guidance to agencies to implement its requirements, which only apply at the agency-wide level  As noted earlier, however, GPRAMA also requires OMB to develop cross- agency priority (CAP) goals to improve the performance and management of the federal government  The President’s Management Agenda established a CAP goal to leverage data as a strategic asset, in part, to improve the use of data for decision making and accountability throughout the federal government  This new CAP goal presents an opportunity for OMB and agencies to identify actions to expand the use of performance information in decision making throughout agencies  As of June 2018, the action plan for implementing the Leveraging Data as a Strategic Asset CAP goal is limited  According to the President’s Management Agenda and initial CAP goal action plan, the goal primarily focuses on developing and implementing a long-term, enterprise-wide federal data strategy to better govern and leverage the federal government’s data  It is through this strategy that, among other things, the administration intends to improve the use of data for decision making and accountability  However, the strategy is under development and not expected to be released until January 2019, with a related plan to implement it expected in April 2019  The existing action plan, released in March 2018 and updated in June 2018, does not yet include specific steps needed to improve the use of data—including performance information—more extensively within agencies  According to the action plan for the goal, potential actions currently under consideration focus on establishing agency “learning agendas” that prioritize the development and use of data and other evidence for decision-making; building agency capacity to use data and other evidence; and improving the timeliness of performance information and other data, and making that information available to decision makers and the public  Although developing learning agendas and building capacity could help improve the use of performance information in agencies, improving availability of data may be less effective  For example, as our past survey results have shown, increasing the availability of performance information has not resulted in corresponding increases in its use in decision making  We recognize that the CAP goal was created in March 2018  Nonetheless, it is important that OMB and its fellow goal leaders develop the action plan and related federal data strategy consistent with all key requirements to better ensure successful implementation  The action plan does not yet include complete information related to the following GPRAMA requirements: performance goals that define the level of performance to be achieved each year for the CAP goal; the various federal agencies, organizations, programs, and other activities that contribute to the CAP goal; performance measures to assess overall progress towards the goal as well as the progress of each agency, program, and other activity contributing to the goal; and clearly defined quarterly targets  Consistent with GPRAMA, Standards for Internal Control in the Federal Government identifies information that agencies are required to include in their plans to help ensure they achieve their goals  The standards state that objectives—such as improving the use of data in decision making— should be clearly defined to enable the identification of risks  Objectives are to be defined in specific terms so they can be understood at all levels of the entity—in this case, government-wide as well as within individual agencies  This involves defining what is to be achieved, who is to achieve it, how it will be achieved, and the time frames for achievement  Ensuring that future updates to the new CAP goal’s action plan includes all required elements is particularly important, as our previous work has found that some past CAP goal teams did not meet all planning and reporting requirements  For example, in May 2016 we found that most of the CAP goal teams we reviewed had not established targets for all performance measures they were tracking  This limited the transparency of their efforts and the ability to track progress toward established goals  We recommended that OMB, working with the Performance Information Council (PIC), report on actions that CAP goal teams are taking, or plan to take, to develop such targets and performance measures  OMB staff generally agreed and, in July 2017, told us they were working, where possible, to assist the development of measures for CAP goals  However, the recommendation has not been addressed and OMB staff said the next opportunity to address it would be when the administration established new CAP goals (which took place in March 2018)  Following the initial release of the new CAP goals, CAP goal teams are to more fully develop the related action plans through quarterly updates  Given the ongoing importance of meeting these planning and reporting requirements, we will continue to monitor the status of actions to address this recommendation as implementation of the new CAP goals proceeds  While the PIC, which is chaired by OMB, has contributed to efforts to enhance the use of performance information, our survey results identify additional opportunities to further those efforts  The PIC’s past efforts have included hosting various working groups and learning events for agency officials to provide performance management guidance, and developing resources with relevant practices  For example, the PIC created a working group focused on agency performance reviews, which was used to share recommendations for how agencies can implement reviews, along with a guide with practices for effectively implementing strategic reviews  In January 2018, staff supporting the PIC joined with staff from another GSA office to create a new group called Fed2Fed Solutions  This group consults with agencies and provides tailored support, such as data analysis and performance management training for agency officials, to help them address specific challenges related to organizational transformation, data-driven decision making, and other management improvement efforts  Our survey results identify useful information related to potential promising practices and challenges that OMB and the PIC could use to inform efforts to enhance the use of performance information more extensively within agencies (e g , at lower levels)  As was previously described, the PIC has responsibilities to (1) facilitate the exchange among agencies of proven practices, and (2) work to resolve government- wide or cross-cutting performance issues, such as challenges  Our analyses of 2017 survey results identified instances where agencies may have found effective ways to enhance the use of performance information by agency leaders and managers in decision making, as well as instances where agencies (and their managers) face challenges in doing so  Specifically, based on analyses of our survey responses, we identified 14 agencies that may have insights into specific practices that led to recent improvements in managers’ use of performance information, or ways that they maintain relatively high levels of use by their managers when compared to the rest of the government  Figure 9 summarizes the agencies identified earlier in the report that had statistically significant increases, or results higher than the government-wide average, on our index or individual survey questions  As the figure shows, several agencies had statistically significant results across all three sets of analyses and therefore may have greater insights to offer: the General Services Administration, National Aeronautics and Space Administration, and the National Science Foundation  In addition, our analyses identified nine agencies where results suggest managers face challenges that have hampered their ability to use performance information  Figure 10 summarizes the agencies identified earlier in the report that had statistically significant decreases, or results lower than the government-wide average, on our index or individual survey questions  As the figure shows, the Office of Personnel Management had statistically significant decreases in all three sets of analyses  Four agencies—the Departments of the Treasury and Veterans Affairs, the Nuclear Regulatory Commission, and the Social Security Administration—were common to both of the figures above  That is, they had results that indicate they may have insights on some aspects of using performance information and face challenges in other aspects  As was mentioned earlier, to provide proper context, these results should be considered in relation to the range of agency results and the government- wide average (provided in detail in the agency summaries in appendix I)  Given the prioritization of other activities, such as the recent creation of the Fed2Fed Solutions program, the PIC has not yet undertaken a systematic approach that could improve the use of performance information by managers at lower levels within agencies  Such an approach would involve identifying and sharing practices that have led to improved use, as well as identifying common or cross-cutting challenges that have hampered such use  The results of our analyses could help the PIC do so, and in a more targeted manner  By identifying and sharing proven practices, the PIC could further ensure that agency leaders and managers are aware of effective or proven ways they can use performance information to inform their decisions across the spectrum of activities they manage within their agencies  Those proven practices also may help agency leaders and managers resolve any identified challenges  Furthermore, in September 2017, we found that, for the estimated 35 percent of managers who reported familiarity with data-driven reviews, the more they viewed their programs being subject to a review, the more likely they were to report the reviews were driving results and were conducted in line with our leading practices for using performance information  Despite the reported benefits of and results achieved through data-driven reviews, they were not necessarily widespread  As noted above, GPRAMA requires agencies to conduct such reviews for agency priority goals, which represent a small subset of goals, and they are required at the departmental level  These reasons may explain why most managers reported they were not familiar with the reviews  As a result, we recommended that OMB should work with the PIC to identify and share among agencies practices for expanding the use of data-driven reviews  OMB staff agreed with our recommendation but have yet to address it  In June 2018, OMB updated its annual guidance to agencies to explicitly encourage them to expand data-driven reviews to include other goals, priorities, and management areas as applicable to improve organizational performance  However, as of June 2018, OMB and the PIC have yet to take any steps to identify and share practices for expanding the use of these reviews in line with our recommendation  Given the additional analyses we conducted for this report—which show that being subject to data-driven reviews is related to greater reported use of performance information and leading practices that promote such use—we continue to believe these further actions would help agencies implement these reviews more extensively  We reiterate the importance of the September 2017 recommendation and will continue to monitor OMB’s progress to address it  For more than 20 years, our work has highlighted weaknesses in the use of performance information in federal decision making  While the Executive Branch has taken some actions in recent years, such as establishing a framework for performance management across the federal government, our survey results underscore that more needs to be done to improve the use of performance information more extensively within agencies and government-wide  The President’s Management Agenda and its related CAP goal to leverage data as a strategic asset present an opportunity to do so, as it aims to improve data-driven decision making  As OMB and its fellow goal leaders more fully develop the action plan for achieving this goal, providing additional details for its plans to improve data-driven decision making would help provide assurance that it can be achieved  As part of those initiatives, our survey results could provide a useful guide for targeting efforts  Officials at each agency could use these results to identify areas for additional analysis and potential actions that could help improve the use of performance information across the agency and at lower levels  Similarly, OMB and the PIC could use the results to identify broader issues in need of government-wide attention  It will also be important, however, for OMB and the PIC to go beyond this analysis and work with agencies to identify and share proven practices for increasing the use of performance information at lower levels within agencies, as well as challenges that may be hampering agencies’ ability to do so  We are making the following two recommendations to OMB: The Director of OMB should direct the leaders of the Leveraging Data as a Strategic Asset CAP Goal to ensure future updates to the action plan, and the resulting federal data strategy, provide additional details on improving the use of data, including performance information, more extensively within federal agencies  The action plan should identify performance goals; contributing agencies, organizations, programs, and other activities; those responsible for leading implementation within these contributors; planned actions; time frames; and means to assess progress  (Recommendation 1) The Director of OMB, in coordination with the PIC, should prioritize efforts to identify and share among agencies proven practices for increasing, and challenges that hamper, the use of performance information in decision making more extensively within agencies  At a minimum, this effort should involve the agencies that our survey suggests may offer such insights  (Recommendation 2) We provided a draft of this report to the Director of the Office of Management and Budget for review and comment  We also provided a draft of the report to the heads of each of the 24 federal agencies covered by our survey  OMB had no comments, and informed us that it would assess our recommendations and consider how best to respond  We are sending copies of this report to congressional requesters, the Director of the Office of Management and Budget, the heads of each of the 24 agencies, and other interested parties  This report will also be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-6806 or mcneilt@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report  Key contributors to this report are listed in appendix IV  (USDA) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Commerce) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Goverment-wide) (DOD) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Education) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (Energy) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (HHS) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DHS) (Goverment-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (HUD) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Interior) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOJ) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOL) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (State) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOT) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Treasury) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (VA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Goverment-wide) (USAID) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (EPA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (GSA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (NASA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (NSF) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (NRC) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (OPM) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (SBA) (Government-wide) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (SSA) 10  The individual I report to 11  Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” This report responds to a request that we analyze agency-level results from our 2017 survey of federal managers at the 24 agencies covered by the Chief Financial Officers (CFO) Act of 1990, as amended, to determine the extent agencies are using performance information  This report assesses the extent to which: 1  the reported use of performance information and related leading practices at 24 agencies has changed compared to our prior survey in 2013; 2  being subject to data-driven reviews related to managers’ reported use of performance information and leading practices; and 3  the Executive Branch has taken actions to enhance agencies’ use of performance information in various decision-making activities  From November 2016 through March 2017, we administered our online survey to a stratified random sample of 4,395 individuals from a population of 153,779 mid- and upper-level civilian managers and supervisors at the 24 CFO Act agencies  The management levels covered general schedule (GS) or equivalent schedules at levels comparable to GS-13 through GS-15, and career Senior Executive Service (SES) or equivalent  We obtained the sample from the Office of Personnel Management’s Enterprise Human Resources Integration database as of September 30, 2015—the most recent fiscal year data available at the time  The sample was stratified by agency and whether the manager or supervisor was a member of the SES  To help determine the reliability and accuracy of the database elements used to draw our sample of federal managers for the 2017 survey, we checked the data for reasonableness and the presence of any obvious or potential errors in accuracy and completeness and reviewed our past analyses of the reliability of this database  We concluded in our September 2017 report that the data used to draw our sample were sufficiently reliable for the purpose of the survey  For the 2017 survey, we received usable questionnaires from about 67 percent of the eligible sample  The weighted response rate at each agency generally ranged from 57 percent to 82 percent, except the Department of Justice, which had a weighted response rate of 36 percent  The overall survey results are generalizable to the population of managers government-wide and at each individual agency  To assess the potential bias from agencies with lower response rates, we conducted a nonresponse bias analysis using information from the survey and sampling frame as available  The analysis confirmed discrepancies in the tendency to respond to the survey related to agency and SES status  The analysis also revealed some differences in response propensity by age and GS level; however, the direction and magnitude of the differences on these factors were not consistent across agencies or strata  Our data may be subject to bias from unmeasured sources for which we cannot control  Results, and in particular estimates from agencies with low response rates such as the Department of Justice, should be interpreted with caution  However, the survey’s results are comparable to five previous surveys we conducted in 1997, 2000, 2003, 2007, and 2013  To address the first objective, we used data from our 2017 survey to update agency scores on our use of performance information index  This index, which was last updated using data from our 2013 survey, averages managers’ responses on 11 questions related to the use of performance information for various management activities and decision making  Using 2017 survey data, we conducted statistical analyses to ensure these 11 questions were still positively correlated  That analysis confirmed that no negative correlations existed and therefore no changes to the index were needed  Figure 11 shows the questions that comprise the index  After calculating agency index scores for 2017, we compared them to previous results from 2007 and 2013, and to the government-wide average for 2017, to identify any statistically significant differences  We focus on statistically significant results because these indicate that observed relationships between variables and differences between groups are likely to be valid, after accounting for the effects of sampling and other sources of survey error  For each of the 11 questions that comprise the index, we identified individual agency results, excluding missing and no basis to judge responses, and determined when they were statistically significantly different from (1) the agency’s results on the same question in 2013, or (2) the government-wide average results on the question in 2017  In this report, we analyzed and summarized the results of our 2017 survey of federal managers  Due to the limited scope of the engagement, we did not conduct additional audit work to determine what may have caused statistically significant changes between our 2017 and past survey results  To further address this objective we completed several statistical analyses that allowed us to assess the association between the index and 22 survey questions that we determined relate to leading practices we previously found promote the use of performance information  See figure 12 for the 22 specific questions related to these five practices that we included in the analysis  When we individually tested these 22 survey questions (bivariate regression), we found that each was statistically significantly and positively related to the index in 2017  This means that each question, when tested in isolation from other factors, was associated with higher scores on the index  However, when all 22 questions were tested together (multivariate regression), we found that 5 questions continued to be positively and significantly associated with the index in 2017, after controlling for other factors  To conduct this multivariate analysis, we began with a base model that treated differences in managers’ views of agency performance management use as a function of the agency where they worked  We found, however, that a model based on agency alone had little predictive power (R-squared of 0 04)  We next examined whether managers’ responses to these questions reflecting practices that promote the use of performance information related to their perceptions of agency use of performance information, independent of agency  The results of this analysis are presented in table 1 below  Each coefficient reflects the increase in our index associated with a one-unit increase in the value of a particular survey question  Our final multivariate regression model had an R-squared of 0 67, suggesting that the variables in this model explain approximately 67 percent of the variation in the use index  We also tested this model controlling for whether a respondent was a member of the SES and found similar results  As shown above in table 1, five questions related to three of the leading practices that promote agencies’ use of performance information were statistically significant in 2017  These results suggest that, when controlling for other factors, certain specific efforts to increase agency use of performance information—such as providing information on the validity of performance data—may have a higher return and lead to higher index scores  With respect to aligning agency-wide goals, objectives, and measures, we found that each increase in terms of the extent to which individuals felt that managers aligned performance measures with agencywide goals and objectives was associated with a 0 08 increase in their score on the use index  In terms of improving the usefulness of performance information, we found that having information on the validity of performance data for decision making was the strongest predictor in our model (0 18)  As measured here, taking steps to ensure the performance information is useful and appropriate was associated with almost as large a change in a managers’ index score (0 16)  In terms of developing agency capacity to use performance information, we found that having sufficient analytical tools to collect, analyze, and use performance information (0 07), and providing or paying for training that would help link their programs to achievement of agency strategic goals (0 10), were also statistically significantly related to a manager’s reported use of performance information  When we combined these results with what we previously found through a similar analysis of 2013 survey results in September 2014, we identified 10 questions that have had a statistically significant association with higher index scores  This reinforces the importance of the five leading practices to promote the use of performance information  For each of these questions, which are outlined in figure 13 below, we determined when agency results were statistically significantly different from 2013 results or the 2017 government-wide average  For the second objective, we examined, based on the extent they responded their programs had been subject to agency data-driven reviews, differences in managers’ use index scores and responses on questions related to practices that promote the use of performance information  We grouped managers based on the extent they reported their programs had been subject to these reviews, from “no extent” through “very great extent ” We then calculated the average index scores for the managers in each of those five categories  We also examined differences in how managers responded to the 10 questions reflecting practices that can promote the use of performance information based on the extent they reported their programs had been subject to data-driven reviews  We grouped managers into three categories based on the extent they reported their programs had been subject to these reviews (no-small extent, moderate extent, great-very great extent)  We then compared how these groups responded to the ten questions  For the third objective, we reviewed our past work that assessed Executive Branch activities to enhance the use of performance information; various resources (i e , guidance, guides, and playbooks) developed by the Office of Management and Budget (OMB) and the Performance Improvement Council (PIC) that could support agencies’ use of performance information; and the President’s Management Agenda, and related materials with information on cross-agency efforts to improve the use of data in federal decision making  Lastly, for the third objective we also interviewed OMB and PIC staff about any actions they have taken, or planned to take, to further support the use of performance information across the federal government  We conducted this performance audit from October 2017 to September 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the above contact, Benjamin T  Licht (Assistant Director) and Adam Miles (Analyst-in-Charge) supervised this review and the development of the resulting report  Arpita Chattopadhyay, Caitlin Cusati, Meredith Moles, Dae Park, Amanda Prichard, Steven Putansu, Alan Rozzi, Shane Spencer, and Khristi Wilkins also made key contributions  Robert Robinson developed the graphics for this report  Alexandra Edwards, Jeff DeMarco, Mark Kehoe, Ulyana Panchishin, and Daniel Webb verified the information presented in this report  Results of the Periodic Surveys on Organizational Performance and Management Issues Managing for Results: Further Progress Made in Implementing the GPRA Modernization Act, but Additional Actions Needed to Address Pressing Governance Challenges  GAO-17-775  Washington, D C : September 29, 2017  Supplemental Material for GAO-17-775: 2017 Survey of Federal Managers on Organizational Performance and Management Issues  GAO-17-776SP  Washington, D C : September 29, 2017  Program Evaluation: Annual Agency-wide Plans Could Enhance Leadership Support for Program Evaluations  GAO-17-743  Washington, D C : September 29, 2017  Managing for Results: Agencies’ Trends in the Use of Performance Information to Make Decisions  GAO-14-747  Washington, D C : September 26, 2014  Managing for Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges  GAO-13-518  Washington, D C : June 26, 2013  Managing for Results: 2013 Federal Managers Survey on Organizational Performance and Management Issues, an E-supplement to GAO-13-518  GAO-13-519SP  Washington, D C : June 26, 2013  Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation in Program Management and Policy Making  GAO-13-570  Washington, D C : June 26, 2013  Government Performance: Lessons Learned for the Next Administration on Using Performance Information to Improve Results  GAO-08-1026T  Washington, D C : July 24, 2008  Government Performance: 2007 Federal Managers Survey on Performance and Management Issues, an E-supplement to GAO-08-1026T  GAO-08-1036SP  Washington, D C : July 24, 2008  Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results  GAO-04-38  Washington, D C : March 10, 2004  Managing for Results: Federal Managers’ Views on Key Management Issues Vary Widely Across Agencies  GAO-01-592  Washington, D C : May 25, 2001  Managing for Results: Federal Managers’ Views Show Need for Ensuring Top Leadership Skills GAO-01-127  Washington, D C : October 20, 2000  The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven  GAO/GGD-97-109  Washington, D C : June 2, 1997 
Report 2:  OMB issues guidance for agencies to implement various requirements of the improper payment laws  Specifically, OMB is required by IPERIA to issue guidance to set standards for agencies to follow in determining the underlying validity of sampled payments to ensure that amounts being billed, paid, or obligated for payment are proper  Although existing OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, like eligibility, are addressed through testing processes  Without such guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs—for example, the cases of DOD Military Pay and OPM Retirement described above—calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions  According to OMB guidance, when an agency’s review is unable to determine whether a payment was proper because of insufficient or lack of documentation, the payment must be considered an improper payment  Among the six agencies and 10 programs we reviewed, treatment of insufficient documentation varied by program, as did the classification of these issues for root cause reporting in the AFRs  HHS’s programs were the only ones we reviewed that reported improper payments in the insufficient documentation root cause category for fiscal years 2016 or 2017, as shown in table 6  Some agencies stated that they report insufficient documentation in other root cause categories that they consider more appropriate  For example, Education officials stated that for the Direct Loan program, payments that lack sufficient supporting documentation may be placed in the “Administrative or Process Error Made by Other Party” root cause category  In these cases, a third party—such as a loan servicer—is unable to provide sufficient documentation supporting that the sampled payment was proper  OMB guidance states that in cases where the agency believes that more than one root cause category might be suitable, the agency should determine which category it believes to be the most appropriate  Additionally, some agencies stated that the “insufficient documentation” category was not always relevant when they recreated sampled cases to estimate a program’s improper payments  For example, according to officials, to complete an OASDI stewardship review of a sampled case, a Social Security Administration (SSA) quality reviewer reviews the documentation related to the original determination and then independently re-develops all factors of the payment and interviews the associated beneficiary  According to agency officials, insufficient documentation would not apply as all improper payments identified in the stewardship sample are supported by documentation and payment has been verified in all reviewed cases  As noted previously, the processes for estimating DOD Military Pay and OPM Retirement improper payments were limited, and these limitations may have an impact on the agencies’ ability to identify improper payments related to insufficient documentation  Treatment of cases of nonresponse  Some agencies contact outside entities—such as payees or beneficiaries—as part of their improper payment testing processes  Among the six agencies we reviewed, treatment of cases of nonresponse differed  For example: SSA officials stated that in cases where quality reviewers do not receive responses from OASDI beneficiaries they contact, they exclude the cases from review (unless the reviewer identifies an improper payment in the initial review that is completed prior to reaching out to the beneficiary)  For EITC improper payment estimation purposes, the Internal Revenue Service (IRS) stated that the agency does not consider the sampled payment associated with a nonresponse case to be proper or improper  It sets the sampling weight of nonresponse cases to zero and adjusts the sampling weights of respondents upward to account for the nonresponse cases  IRS’s methodology assumes nonresponse and response cases have an equal likelihood of improper payment  For Medicare Fee-for-Service and Medicaid, HHS’s policies and procedures consider payments associated with nonresponse cases to be improper  OMB guidance states that when an agency’s review is unable to discern whether a payment was proper as a result of insufficient or lack of documentation, this payment must be considered an improper payment  However, it does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes  As a result, without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable  When agencies identify improper payments, they must determine the amount of the payment that was improperly made  The six agencies we reviewed generally reported using the definition of improper payment in relevant laws and OMB guidance to determine the amount of improper payments identified  OMB guidance provides agencies with instructions on how to calculate the amount of improper payments  However, when developing its improper payment estimate for EITC, IRS subtracted overpayments that were paid out and later recovered  By subtracting recovered overpayments, IRS excluded them from the EITC improper payment estimate  For 2013—the tax year used to produce the fiscal year 2017 improper payment estimate—IRS estimated that $1 2 billion in EITC overpayments would be recovered  IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered  Although the OMB guidance was revised in October 2014 to implement this requirement, IRS has not updated its estimation methodology for EITC  By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur  Improper payments are a long-standing, significant problem in the federal government  Estimation of improper payments is key to understanding the extent of the problem and to developing effective corrective actions to address it  Among the six agencies we reviewed, processes to estimate improper payments in their programs varied, and certain differences in these processes may affect the quality of the resulting estimates and consequently these agencies’ efforts to reduce improper payments  Specifically, policies and procedures for DOD’s Military Pay and OPM’s Retirement programs’ improper payment estimation methodologies do not address certain key risks, like eligibility, in part because these agencies have not fully assessed their processes  Further, although OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments  Without such assessments and guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs, calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions  Additionally, for agencies we reviewed that contact outside entities as part of their improper payment estimation processes, the treatment of cases of nonresponse varied  OMB guidance does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes  Without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable  Finally, although IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered, IRS has not updated its processes to reflect the change  By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur  We are making two recommendations to the Director of OMB that have government-wide implications and specific recommendations to DOD, OPM, and IRS regarding their programs included in this review  The Director of OMB should develop guidance on how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, such as eligibility, are addressed through testing processes  (Recommendation 1) The Director of OMB should develop guidance clarifying the appropriate treatment of nonresponse cases during improper payment testing  (Recommendation 2) The Under Secretary of Defense (Comptroller) should assess the processes for estimating Military Pay improper payments to determine whether they effectively address key risks of improper payments— including eligibility for different types of pay and allowances—and take steps to update the processes to incorporate key risks that are not currently addressed  (Recommendation 3) The Director of OPM should assess the processes to estimate Retirement improper payments to determine whether they effectively address key risks of improper payments—including eligibility and whether older claims face different risks of improper payments than new claims—and take steps to update the processes to incorporate key risks that are not currently addressed  (Recommendation 4) The Commissioner of IRS should update IRS’s improper payment estimation methodology to not exclude recovered overpayments from its EITC improper payment estimate  (Recommendation 5) We provided a draft of this report for comment to OMB, DOD, Education, HHS, Treasury, OPM, SSA, and USDA  OMB provided oral comments, which are summarized below  OPM, DOD, and IRS provided written comments, which are reproduced in appendixes II through IV, respectively  Education, HHS, SSA, and USDA did not provide written comments on the draft report  In addition, HHS, IRS, OMB, OPM, and SSA provided technical comments, which we have incorporated, as appropriate  In oral comments provided on April 30, 2018, a Senior Policy Advisor in OMB’s Office of Federal Financial Management stated that OMB partially agreed with our first recommendation and agreed with our second recommendation  Regarding the first recommendation, the Senior Policy Advisor stated that OMB should not have to develop more specific guidance as each program and activity has its own risks  Instead, inspectors general are better equipped and positioned to review the sampling and estimation plans as part of their annual IPERA compliance audits and that agencies, their statisticians, and inspectors general should work out the best testing procedures for their agencies  OMB could provide suggestions during OMB’s annual town hall meeting related to improper payments for areas that inspectors general may consider  Although we agree that programs and activities may face different risks of improper payment, we continue to believe that guidance from OMB on how agencies test to identify improper payments—such as directing agencies to take a risk-based approach in developing their testing procedures—could help ensure that agencies address the specific risks they identify when developing improper payment estimates  Further, such guidance could also help ensure that testing processes are designed to address an agency’s identified risks before the estimate is developed, whereas an inspector general’s review—as well as related recommendations for improvement—would generally occur after the agency’s improper payment estimate had been developed and reported  Regarding the second recommendation, the Senior Policy Advisor noted that OMB plans to update its guidance to direct agencies to treat nonresponse cases as improper payments and to include a new category for tracking such cases  In its written comments, OPM partially concurred with our recommendation to assess the processes to estimate Retirement improper payments to determine whether they effectively address the key risks of improper payments  OPM agreed to conduct an audit of older claims to determine if they face different risks than new claims  However, OPM did not agree with the part of the recommendation to assess the risk of improper payments related to eligibility in the estimation process  OPM stated that eligibility is determined before annuity or survivor benefits are fully adjudicated  However, the objective of an improper payment estimate is to determine whether payments were made properly  To do so, an agency should determine whether the payee was eligible for the payment that was made, among other things  As such, we continue to believe that the recommendation—including the assessment of the risk of improper payments related to eligibility—is warranted  In their written comments, DOD and IRS both agreed with our recommendations directed to them and described the steps they plan to take to implement them  We are sending copies of this report to the appropriate congressional committees; the Secretaries of Agriculture, Defense, Education, Health and Human Services, and the Treasury; the Director of the Office of Personnel Management; the Administrator of the Social Security Administration; the Director of the Office of Management and Budget; and other interested parties  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2623 or davisbh@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  Table 7 lists the fiscal year 2017 improper payment estimates by agency and program, as reported by agencies in their fiscal year 2017 agency financial reports and compiled on the Office of Management and Budget’s payment integrity website, paymentaccuracy gov  In addition to the contact named above, Phillip McIntyre (Assistant Director), James M  Healy (Auditor in Charge), Daniel Flavin, and Fabiola Torres made key contributions to this report 

ground_truth: The National Popular Vote (NPV) initiative proposes an agreement among the states, an interstate compact that would effectively achieve direct popular election of the President and Vice President without a constitutional amendment. It relies on the Constitution's grant of authority to the states in Article II, Section 1 to appoint presidential electors "in such Manner as the Legislature thereof may direct.... " Any state that joins the NPV compact pledges that if the compact comes into effect, its legislature will award all the state's electoral votes to the presidential ticket that wins the most popular votes nationwide, regardless of who wins in that particular state. The compact would, however, come into effect only if its success has been assured; that is, only if states controlling a majority of electoral votes (270 or more) join the compact. By early May 2019, 14 states and the District of Columbia had joined the compact. After early momentum—eight states and the District of Columbia joined the NPV Compact between 2007 and 2011—the pace of state accessions slowed through 2018. Since then, four additional states joined, bringing the total number of electoral votes controlled by NPV member states to 189. During the same period, legislation to join the compact had been introduced during the current session in at least one chamber of the legislature in 14 additional states that control an additional 150 electors. The NPV initiative emerged following the presidential election of 2000, in which one ticket gained an electoral vote majority, winning the presidency, but received fewer popular votes than its opponents. NPV grew out of subsequent discussions among scholars and activists about how to avoid similar outcomes in the future and to achieve direct popular election. Proponents of NPV assert that it would guarantee the presidential candidates who win the most popular votes nationwide will always win the presidency; that it would end the inequities of the general ticket/winner-take-all system of awarding electoral votes; and that candidates would extend their focus beyond winning the "battleground states," campaigning more widely and devoting greater attention to issues of concern to other parts of the country. They further assert that NPV would accomplish this while avoiding the exacting standards set for the proposal and ratification of constitutional amendments. Opponents argue that NPV would undermine the authority of states under the Constitution and the Founders' intention that presidential elections should be both national and federal contests; that it is an admitted "end run" around the Constitution that would circumvent the amendment process; and that it might actually lead to more disputed presidential elections characterized by politically contentious state recounts. The NPV has also been debated on legal grounds. Some observers maintain that it must be approved by Congress, because it is an interstate compact that would affect key provisions of constitutional presidential election procedures. NPV Inc., the organization managing the initiative's advocacy campaign, responds that congressional approval is not necessary because NPV concerns the appointment of electors, a subject that falls within state constitutional authority, and that the Supreme Court has previously rejected arguments that similar compacts would impair the rights of nonmember states. Other critics claim that NPV might violate the Voting Rights Act by diluting minority voter influence and avoiding the recently invalidated preclearance requirement for election procedure changes in covered jurisdictions. In response, NPV Inc. has asserted that the compact is "entirely consistent with the goal of the Voting Rights Act." This report monitors the NPV's progress in the states and will identify and analyze further developments as warranted.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: Since its founding in 2006, the National Popular Vote (NPV) initiative has promoted an agreement among the states, an interstate compact that would effectively establish direct popular election of the President and Vice President without a constitutional amendment, while retaining the structure of the electoral college system  The United States is unique among "presidential" republics by providing an indirect election to choose its chief executive  The President and Vice President of the United States are selected not by registered voters, but by the electoral college, electors appointed in the states "in such Manner as the Legislature thereof may direct     " Alexander Hamilton, who was "present at the creation" of the Constitution in 1787, commented favorably on the electoral college system in The Federalist : The mode of appointment of the Chief Magistrate of the United States is almost the only part of the system, of any consequence, which has escaped without severe censure, or which has received the slightest mark of approbation from its opponents     I venture somewhat further, and hesitate not to affirm that if the manner of it be not perfect, it is at least excellent  It unites in an eminent degree all the advantages the union of which was to be wished for  Notwithstanding Hamilton's endorsement, the first proposal to change the electoral college system by constitutional amendment was introduced as early as 1800, and since that time more than 700 proposals to reform or eliminate the college have been introduced in Congress  Reform advocates have long focused on the fact that it does not provide for direct democratic election, that less-populous states are afforded an arithmetical advantage due to the assignment of two electors to each state, regardless of population, and that the winner-take-all system makes it possible for candidates to win an electoral college majority and the presidency, while gaining fewer votes than their principal opponents in the popular election  Between 1949 and 1979, Congress considered amendments to reform the electoral college, or replace it with direct popular election, in committee and on the floor of both chambers  Proposed amendments must, however, meet the requirements of the Constitution's Article V, which calls for two-thirds approval by both houses of Congress, and ratification by three-fourths of the states; to date, no electoral college reform proposal has met these requirements  Proponents of the National Popular Vote initiative contend that their plan will achieve direct popular election while circumventing the requirements of Article V, and will guarantee that the popular vote winners will always be elected President and Vice President  The fundamentals of the electoral college system were established by Article II, Section 1 of the U S  Constitution, and subsequently revised by the Twelfth Amendment  The Constitution's minimal provisions have been complemented over the past two centuries by a range of federal and state laws, political party procedures, and enduring political traditions, leading to the system as it exists today  The salient features of the contemporary system are detailed below  The electors are collectively known as the electoral college; although this phrase does not appear in the Constitution, it gained currency in the early days of the republic, and was recognized in federal law in 1845  The electoral college has no continuing existence; its sole purpose is to elect the President and Vice President  Each state is allocated a number of electors equal to the combined total of its U S  Senate and House of Representatives delegations  The District of Columbia is also allocated three electors  At present, the total is 538, reflecting the combined size of the Senate (100 Members), the House (435 Members), and the District of Columbia electors  Any person may serve as an elector, except Senators and Representatives, or any other person holding an office of "trust or profit" under the United States  Article II, Section 1 of the Constitution empowers the states to "appoint [electors], in such Manner as the Legislature thereof may direct     " This grant of authority provides the constitutional basis claimed for the NPV initiative  In practice, all states currently provide for popular election of their electoral college delegations  Candidates for the office of elector are nominated by political parties and other groups on the presidential ballot in each state  In most cases, the candidates for the office of elector are nominated by the state party committee or the party's statewide convention  The winning presidential nominees must gain a national majority of 270 or more electoral votes, out of the 538 total, in order to be elected  If no ticket of candidates attains a majority, then the House of Representatives elects the President, and the Senate the Vice President, in a procedure known as contingent election  Candidates for the office of elector are selected by their respective political parties  They are expected to vote for the presidential and vice presidential candidates to whom they are pledged  Some states seek to require them to so vote by law or other means, but most constitutional scholars hold that the electors remain free agents under the Constitution, and that they may vote for any person they choose  On rare occasions, an elector will vote for a different candidate, or abstain from casting a vote for any candidate; he or she is known as a "faithless elector " The goal of presidential campaigns under the existing system is to win by carrying states that collectively cast a majority of electoral votes  Political parties and presidential campaigns tend to focus on states that are closely contested (widely referred to as "battleground" states), or that have large delegations of electoral votes, or both  Winning a majority of the more populous and/or battleground states is considered crucial to obtaining the necessary electoral vote majority  In 48 states and the District of Columbia, the presidential/vice presidential ticket winning the most popular votes (a plurality or more) in that state is awarded all its electoral votes  That is to say, the winning party's entire ticket of electors is elected  This is referred to as the "winner-take-all" or "general ticket" system  Presidential Election Day is set by law for Tuesday after the first Monday in November every fourth year succeeding the election of President and Vice President  On Presidential Election Day, voters cast one vote for the candidates they support  They are, however, actually voting for the state political party "ticket" of electors supporting those presidential and vice presidential candidates  Presidential electors assemble on the first Monday after the second Wednesday in December following the general election  They meet in their respective states, not collectively, and cast separate votes by ballot for the President and Vice President  After the electors vote, the results are sent by the states to Congress and various other federal authorities  On January 6 of the year following a presidential election, Congress meets in a joint session to count the electoral votes and make a formal declaration of which candidates have been elected President and Vice President  A range of factors contributed to the emergence of the National Popular Vote initiative in the first decade of the 21 st century  A major source was frustration by reform advocates after three decades of failed attempts to secure congressional approval for a direct popular election amendment  A more immediate spur was the contentious and disputed presidential election of 2000, which is regarded as having been a major factor contributing to the development of the NPV proposal  One of the factors cited for the emergence of the NPV initiative has been the exacting requirements set by the Constitution for amendments, in this case, a direct popular election constitutional amendment  As noted previously, approval by two-thirds of Members present and voting is required in both houses when Congress proposes an amendment, followed by ratification by three-fourths of the states, 38 at present, usually within a seven-year period specified by Congress  Between 1948 and 1979, Congress debated electoral college reform at length; throughout this time, hundreds of reform proposals were introduced in both chambers  They generally centered on one of two courses: "end it" by eliminating the entire electoral college system and establishing direct popular election, or "mend it" by reforming its more controversial provisions  Between 1948 and 1979, proposed amendments were the subject of hearings in the Senate and House Judiciary Committees on 17 different occasions, while electoral college reform was debated in the Senate on five occasions and twice in the House during this period  Proposals were approved by the necessary two-thirds majority twice in the Senate and once in the House, but never in the same Congress  Following the 1979 defeat of a direct popular election amendment on the Senate floor, and the retirement or defeat of prominent congressional advocates, the question of electoral college reform largely disappeared from public attention and Congress's legislative agenda  Although Senators and Representatives continued to introduce reform proposals, few received action beyond routine committee referral, and in time, the number of measures introduced dropped to zero  Even after the presidential elections of 2000 and 2016, in which the winner of the electoral vote won fewer popular votes than his opponent (a so-called "misfire") , there was little evidence that Congress was prepared to consider an electoral college reform amendment  Proposals to replace the electoral college system with direct popular election continued to be introduced, but in dwindling numbers as the years passed  No proposal for direct popular election was introduced in the 113 th Congress  By comparison, 41 direct popular election or electoral college reform amendments were proposed in the 95 th Congress (1977-1978)  Following the 2016 election, however, four constitutional amendments introduced late in the 114 th Congress proposed eliminating the electoral college and replacing it with direct election  To date in the 116 th Congress, three amendments to establish direct popular election by constitutional amendment have been introduced, but no action beyond committee referral has been taken on them  Until recently, survey research findings showed public support for presidential election reform through direct popular election by sizable margins  As early as 1967, the Gallup Poll reported that 58% of respondents supported direct election, compared with 22% who favored retaining the electoral college  More recently, Gallup's 2013 survey recorded that 63% of respondents favored an amendment providing for direct election, while 29% favored retention of the electoral college  Following the 2016 election, however, overall support for direct election was measured at 49% in favor to 47% opposed  It is arguable that the change in public attitudes was influenced by the 2016 election results, in which the Republican nominees won the election with a majority of electoral votes, but fewer popular votes than their Democratic opponents  For instance, Gallup reported a shift to greater support for the electoral college system by respondents who identified themselves as "Republican" or "Lean Republican " Conversely, already high levels of support for direct popular election among respondents who identified themselves as "Democratic" or "Lean Democratic" rose still further in the post-2016 election Gallup Poll  To date, CRS has identified one survey that was specifically designed to measure public commitment to the NPV initiative  A March 27, 2019, Politico/Morning Consult poll posed relevant questions on the presidential election process and the NPV compact  The first question, which presented a general outline of the existing electoral college system and the generic alternative of direct popular election, reported that respondents preferred direct election by 50% to 34% for retaining the electoral college, and 16% reporting "Don't know/No opinion " The next question explained the proposed NPV compact and asked respondents' preference for NPV or the electoral college method  Although the level of support for NPV was lower than that measured for generic direct popular election, a plurality of respondents to this question favored the "National Popular Vote Interstate Compact" by a plurality of 43% in favor, to 33% opposed and 23% who reported "Don't know/No opinion " The disputed presidential election of 2000 was arguably a catalyst for new thinking on electoral college reform   Following a closely contested campaign, Republican candidates George W  Bush and Richard Cheney were elected over Democratic nominees Al Gore Jr  and Joseph Lieberman following a bitter dispute over election results in Florida tha t was ultimately decided by the Supreme Court   The high court's decision left Bush and Cheney with a narrow plurality in Florida of 537 popular votes and a similarly narrow electoral college majority of 30 states with 271 electoral votes, while their Democratic opponents took 20 states and the District of Columbia with 266 electoral votes (one District of Columbia elector cast a blank ballot in protest against the outcome)   It was the first election since 1888 in which the candidates elected uncontestably won fewer popular votes than their principal opponents : the Gore / Lieberman Democratic ticket gained 50,992,335 popular votes to 50,455,156 for Bush / Cheney  The se election results generated considerable discontent with the system  Some critics argued for a constitutional amendment, but the 107 th Congress faced a heavy legislative workload throughout this period, which initially included enactment of President George W  Bush's legislative program and was later expanded to urgent responses to the terrorist attacks of September 1 1, 2001   Rather than focus on the lengthy process associated with consideration of a constitutional amendment, Congress focused on legislati ve remedies  The Help America Vote Act of 2002, passed in response to the numerous irregularities in voting systems and procedures revealed by the 2000 election, mandated election administration reforms and v oting system technology enhancements (funded in part by federal grants to the states) intended to ensure accurate and timely voting and vote tabulation in future elections  In 2016, the presidential election was again won by nominees who gained a majority of electoral votes but fewer popular votes than their major party opponents  Although proposals to amend the Constitution to provide direct popular election were introduced in response to this occurrence late in the 114 th Congress , and again in the 115 th Congress, the 2016 results did not result in the degree of c ontroversy a nd activism that followed the electoral college "misfire " of 2000  T he following factor may have contributed to this situation : i n 2000, a shift in Florida's electoral votes from Bush/Cheney to Gore/Lieberman would have changed the election result; by comparison, in 2016, a shift in the state with the closest vote margin, Michigan , would not have altered the election  While the 2000 election's "misfire" did not result in consideration of a constitutional amendment, it did prompt considerable study and investigation into new approaches to electoral reform among scholars of the presidential election process and political activists  Law professors Robert W  Bennett of Northwestern University, Vikram Amar of the University of California-Davis, and Akhil Amar of Yale University School of Law are generally credited as the intellectual godparents of the concept that ultimately evolved into the National Popular Vote Interstate Compact, which relies on the Constitution's broad grant of power to each state to "appoint, in such Manner as the Legislature thereof may direct [emphasis added], a Number of Electors, equal to the whole Number of Senators and Representatives to which the State may be entitled in the Congress " Project FairVote, an issue advocacy group self-described as a nonprofit, nonpartisan "501(c)(4)" organization, appears to have been an incubator of the NPV concept  FairVote has supported NPV for "over a decade," and was an early supporter of National Popular Vote Inc , the plan's official advocacy group; moreover, longtime FairVote board members Robert Richie and the late Representative John B  Anderson were early supporters of the National Popular Vote initiative and contributors to its manifesto, Every Vote Equal   As noted previously, the NPV initiative was the ultimate result of the various studies and proposals offered following the presidential election of 2000  The NPV initiative seeks to establish direct popular election of the President and Vice President through an interstate compact, rather than by constitutional amendment  Ideally, under the compact's provisions, legislatures of the 50 states and the District of Columbia would pass legislation binding the signatories to appoint presidential electors committed to the presidential/vice presidential ticket that gained the most votes nationwide   If all 50 states and the District of Columbia were compact members, this would deliver a unanimous electoral college decision for the candidates winning a plurality of the popular vote  Specifically, the plan calls for an agreement among the states, an interstate compact effected through state legislation, in which the legislature in each of the participating states agrees to appoint electors pledged to the candidates who won the nationwide popular vote   State election authorities would count and certify the popular vote in each state, which would be aggregated and certified as the "nationwide popular vote " The participating state legislatures would then choose the slate of electors pledged to the "nationwide popular vote winner," notwithstanding the results within their particular state s   To ensure success, the initiative would come into effect only if states whose total electoral votes equal or exceed the constitutional majority of 270 were to approve the plan  If the nationwide popular vote were effectively tied, the states would be released from their commitment under the compact, and could choose electors who represented the presidential ticket that gained the most votes in each particular state  One novel NPV provision would enable the presidential candidate who won the national popular vote to fill any vacancies in the electoral college with electors of his or her own choice  States would retain the right to withdraw from the compact, but if a state chose to withdraw within six months of the end of a presidential term, the withdrawal would not be effective until after the succeeding President and Vice President had been elected  The NPV advocacy effort is managed by National Popular Vote Inc , a "501(c)(4)" nonprofit corporation established in California in 2006 by Barry Fadem, an attorney specializing in initiative and referendum law, and John R  Koza, Ph D , an automated systems scientist and entrepreneur  As a 501(c)(4) entity, it is permitted to engage in political activity in furtherance of its goal, without forfeiting its tax-exempt status, so long as this is not its primary activity  NPV states on its website that its "specific purpose is to study, analyze and educate the public regarding its proposal to implement a nationwide popular election of the President of the United States " Dr  Koza serves as chairman of NPV Inc , and Mr  Fadem serves as president  NPV's advisory board includes former Senators and Representatives of both major political parties  In 2006, National Popular Vote Inc  published a detailed handbook, Every Vote Equal: A State-Based Plan for Electing the President by National Popular Vote   This publication, in its fourth edition at the time of this writing, provides a detailed account of various issues related to the NPV initiative, including the electoral college, earlier reform efforts, interstate compacts, the text of the proposed compact, a strategy for advancing the initiative, and a 340-page section addressing "myths about the National Popular Vote Compact " National Popular Vote Inc  maintains an office in Mountain View, CA  Supporters in various state legislatures began to introduce measures to adopt the interstate compact shortly after NPV's inaugural press conference on February 23, 2006  The NPV Compact has been introduced at various sessions in the legislatures of all 50 states and the Council of the District of Columbia, which performs the functions of a state legislature in the nation's capital, and has received some form of active consideration in 38 states and the D C  Council  Among other activities, NPV maintains a regular communications program of email newsletters announcing activities and soliciting readers to petition governors and state legislators to support the compact  At the time of this writing, NPV claims that 3,112 state legislators have either sponsored or cast a recorded vote in their respective legislatures for the compact  NPV also claims endorsements from legislators and endorsements by the New York Times , Los Angeles Times , Chicago Sun-Times , Minneapolis Star Tribune , Boston Globe , Miami Herald , and other newspapers  NPV also advocates use of the citizen initiative process where available to enact state adherence to the compact; it asserts that when Article II, Section 1, clause 2 grants authority to the states to appoint "in such Manner as the Legislature thereof may direct," the authority extends to the states' entire lawmaking process, which in some states includes the proposal and passage of legislation and constitutional amendments through citizen initiative  The citizen initiative approach to the interstate compact, however, has yet to be used at the time of this writing  At the time of this writing, in May 2019, the following 14 states and the District of Columbia have adopted the National Popular Vote Compact  Collectively, they are assigned a total of 189 electoral votes  The National Popular Vote Interstate Compact has been introduced since its inception in all 50 states and the District of Columbia  States that have adopted NPV at the time of this writing are listed in chronological order, by year of adoption, as follows: Hawaii (4 electoral votes), 2008; Illinois (20 electoral votes), 2008; Maryland (10 electoral votes), 2008; New Jersey (14 electoral votes), 2008; Washington (12 electoral votes), 2009; Massachusetts (11 electoral votes), 2010; District of Columbia (3 electoral votes), 2010; Vermont (3 electoral votes), 2011; California (55 electoral votes), 2011; Rhode Island (4 electoral votes), 2013; New York (29 electoral votes), 2014; Connecticut (7 electoral votes), 2018; Colorado (9 electoral votes), 2019; Delaware (3 electoral votes), 2019; and New Mexico (5 electoral votes), 2019  After initial momentum in 2008, when four states joined the compact in one year, NPV made slower progress toward its goal of approval by states accounting for 270 electoral votes  Highlights were California's approval in 2011, which added 55 electoral votes to the tally, and New York's accession to the compact in 2014  Beginning with Connecticut's approval in 2018, followed in 2019 by Colorado, Delaware, and New Mexico, 24 additional electoral votes were added to the NVP count, bringing the total to 189, 70% of the 270 votes needed for NPV to go into effect  By early May 2019, legislation to join the NPV had been introduced in the current session of at least one chamber of the legislature in 14 states that controlled a combined total of 150 electoral votes  As of April 17, the compact had been approved in Nevada by the Assembly (lower chamber of the legislature) and in Oregon by the Senate  Accession by these two states would raise the NPV member total to 202 electoral votes  Conversely, proposals to rescind approval of the NPV Interstate Compact have been introduced in the legislatures of Connecticut, Hawaii, Maryland, Massachusetts, New Jersey, and Washington, to date; none has been approved  Some observers note that, despite NPV's assertion of bipartisan support, all the jurisdictions that have joined the compact to date could be identified as "leaning" Democratic or "solid" Democratic in their support of the Democratic Party, as classified by a recent Gallup survey  For instance, 11 of the 14, including California, Delaware, the District of Columbia, Hawaii, Illinois, Maryland, Massachusetts, New Jersey, New Mexico, Rhode Island, and Vermont, were found by Gallup to be among the "most solidly Democratic states in 2017 " Alluding to this fact, one commentator observed that [a]ll the states to have joined so far are very blue  Until some purple states and red states sign on, the compact has little in the way of territory to conquer     The seven states where President Obama won [in 2012] by the widest margins, along with D C , have joined  So have three others—New Jersey, Illinois and Washington—where Obama won by at least 15 percentage points  But none below that threshold have done so  The 2016 presidential election results in Colorado, Connecticut, Delaware, and New Mexico, the most recent adherents to the NPV compact, arguably confirm this observation—the Democratic candidates won the popular vote in all four states, by margins of between 4 9% in Colorado to 13 7% in Connecticut  On the other hand, several states where the NPV remained under active consideration in 2019—Arizona, Florida, Georgia, Idaho, Indiana, Kansas, North Carolina, Ohio, and South Carolina—were carried by the Republican presidential ticket in 2016  As the NPV campaign developed momentum in the states, particularly between 2008 and 2011, defenders of the existing arrangements and the electoral college announced measures to promote retention of the electoral college system  In October 2011, the Heritage Foundation, a conservative public policy institute, released a report opposing the NPV compact  That same month, Roll Call reported that the State Government Leadership Foundation, a project of the Republican State Leadership Committee, would begin a campaign to defend the electoral college and counter recent NPV gains  Further activity, however, does not appear to have been undertaken by these groups by the time of this writing  Arguments in support of and opposed to the National Popular Vote proposal resemble those advanced in favor of and against direct popular election of the President  The central issue turns on the question of the asserted simplicity and democratic attractiveness of the direct election idea as compared to a more complex array of factors cited by supporters of the electoral college system  Proponents of the NPV initiative arguably share the philosophical criticism voiced by proponents of direct popular election, who maintain that the electoral college system is intrinsically undemocratic—it provides for "indirect" election of the President and Vice President  This, they assert, is an 18 th century anachronism, dating from a time when communications were poor, the literacy rate was much lower, and the nation had yet to develop the durable, sophisticated, and inclusive democratic political system it now enjoys  They maintain that only direct popular election of the President and Vice President is consistent with modern democratic values and practice  Beyond this fundamental challenge, critics cite what they identify as a wide range of technical failings of the electoral college arrangement  Perhaps the most prominent of these is that the electoral college system can result in the election of a President and Vice President who have won the electoral vote, but gained fewer popular votes than their major opponent  This condition results at least in part from the nearly universal reliance on the "winner-take-all" or general ticket system of awarding electoral votes in the states, which is also criticized by NPV advocates  Under the general ticket system, the candidates winning the most popular votes in a state (a plurality is sufficient) are awarded all that state's electors and electoral votes; under these circumstances, a presidential ticket can gain all of a state's electoral votes on even a slim margin of popular votes  Presidents were elected in 1876, 1888, 2000, and 2016 who received fewer popular votes than their major party opponents, while the runner-up in both popular and electoral votes was elected by the House of Representatives when four candidates split the vote in the presidential election of 1824  NPV supporters advocate the compact on the grounds of fairness and respect for the voters' choice  At the core of their arguments, they assert that the process would be simple, national, and democratic; the NPV interstate compact would provide de facto for a single, democratic choice, allowing all the nation's voters to choose the President and Vice President directly, with no intermediaries  The "people's choice," they assert, would win in every election, and every vote would carry the same weight in the election, no matter where in the nation it was cast  No state would be advantaged, nor would any be disadvantaged  According to NPV, the central argument in favor is that the compact "would guarantee the Presidency to the candidate who receives the most popular votes [or at least a plurality] in all 50 states (and the District of Columbia) " According to NPV, there would never again be a presidential election "misfire" or another "wrong winner " Other elements of the electoral college system criticized by NPV advocates (and other electoral college reformers) would arguably disappear or be rendered irrelevant  These include the faithless elector phenomenon, the general ticket system's asserted "disfranchisement" of voters who backed the losing candidates, and various asserted "voting power" advantages attributed to large (populous) states, small states, states with large populations of noncitizens, states with low rates of voter participation, and populous states with concentrations of minority-group voters  In addition, the NPV compact would almost certainly eliminate the need for contingent election of the President and Vice President under the Twelfth Amendment  NPV advocates also assert the compact would provide a practical benefit to states that tend to be noncompetitive in presidential elections and which therefore receive fewer campaign visits by major party candidates  With "every vote equal," NPV maintains that presidential and vice presidential nominees and their organizations would need to spread their presence and resources more evenly as they campaigned for every vote nationwide, rather than concentrate on winning key "battleground" states  They assert that, under the present system candidates have no reason to poll, visit, organize, campaign, or worry about the concerns of voters of states that they cannot possibly win or lose  This means that voters in two thirds of the states are effectively disenfranchised in presidential elections because candidates concentrate their attention on a small handful of "battleground" states  In 2004, candidates concentrated over two-thirds of their money and campaign visits in just five states; over 80% in nine states, and over 99% of their money in just 16 states  For instance, NPV notes that California voters seldom see the presidential or vice presidential nominees or benefit from campaign spending because the Golden State, having voted Democratic since 1988, is considered to be reliably "blue," and Democratic Party candidates are said to take its 55 electoral votes for granted  They also note that Republican candidates make few California appearances, but, NPV asserts, for the opposite reason: why spend time and resources in support of an apparently hopeless cause? Similar arguments made by NPV on the Republican side apply to Texas, a state that has voted for Republican presidential nominees since 1980  In 2016 for instance, NPV claims that no Democratic nominee participated in a general election campaign event in California or Texas, while a Republican nominee appeared in only one campaign event in each of those states  By comparison, according to their calculations, the hotly contested battleground states of Florida, North Carolina, and Pennsylvania received, respectively, 71, 55, and 54 candidate appearances  According to NPV's analysis of campaign appearances, the 2016 major party candidates for President and Vice President appeared at a total of 375 campaign events during the general election campaign, but they visited only 12 states; by NPV's calculation, 38 states and the District of Columbia were bypassed during the campaign  NPV advocates also maintain that the concentration of campaign resources, advertising, and candidate appearances in battleground states depresses turnout in "flyover" states, where candidates make few campaign appearances  The U S  Elections Project report, America Goes to the Polls, 2016 , appears to offer statistics consistent with this assertion, finding that the participation rate of the population eligible to vote in 14 battleground states was 65% in the 2016 presidential election, as opposed to comparable nationwide turnout of 60%  It also reports findings similar to those advanced by NPV: 95% of campaign visits during the 2016 campaign were made in battleground states, as was 99% of "ad spending " The NPV manifesto also cites a Brookings Institution study of the 2004 presidential election in support of its argument, stating, "Because the electoral college has effectively narrowed elections like the last one to a quadrennial contest for the votes of a relatively small number of states, people elsewhere are likely to feel that their votes don't matter " It should be noted, however, that a range of other political, social, cultural, and economic factors may also contribute to the disparity in turnout between battleground and non-battleground states  NPV further suggests that the disparity in participation may ultimately damage the ability to govern on the state and local levels and could have a negative impact on the legitimacy of public institutions: Diminished voter turnout in presidential races in non-battleground states weakens down-ballot candidates, thereby making the state even less competitive in the future  Governance—not just electioneering—is affected by the winner-take-all rule  National Popular Vote opponents oppose the compact on various grounds  Some argue that it is unconstitutional or "anticonstitutional," that is, contrary to the Founders' intentions and the spirit of the nation's fundamental charter  It is also asserted that NPV would solve few of the electoral college system's alleged problem issues and would create some of its own  Finally, some observers note that the NPV compact is an interstate compact as defined in Article I, Section 10, clause 3 of the Constitution, and as such would be subject to congressional approval  This issue is examined in greater detail in a separate section of this report  On the most fundamental philosophical basis, opponents might argue that the NPV compact violates one of the basic principles of majoritarian democracy: it does not require that candidates win a majority of the popular vote in order to gain the presidency  Rather, it would anoint as winner the ticket that gains more popular votes than any other  A majoritarian democracy, it may be argued, should require a majority in order to elect; it may be further noted that the existing system, by comparison, requires a majority in the electoral college  As one commentary noted only the strictest of majoritarians desire a purely majoritarian presidential election system, and those individuals should be deeply troubled by the prospect of plurality presidencies, which the NPVC [sic] expressly countenances  Indeed, the NPVC promises to create more difficulties and "misfires" in its own way than the Electoral College system its proponents so earnestly seek to replace  Further, opponents might ask how the NPV compact would function in the event of a multicandidate election, a phenomenon that recurs from time to time in U S  presidential elections  One commentator posited the following problematic scenario under such circumstances: Under the compact, one can easily imagine a multi-candidate race in which a candidate would win, say, a thirty-four percent plurality of the popular vote nationwide while losing in every state and D C  If all of the states and D C  were signatories to the compact, all the electoral votes in such a hypothetical race would be awarded contrary to the will of voters choosing electors (still not voting directly for President under this plan)  Would the United States accept a President who wasn't the choice of sixty-six percent of those voting, nor even the choice of a single state? The existing electoral college system, NPV skeptics might also assert, is a fundamental element in the federal constitutional arrangements established by the Constitution  Fearing "the tyranny of the majority," the Founders established a system of government that provides checks and balances designed to restrain the majority and secure minority rights  These principles are also embedded in the structure of federal elections: the Senate, the House of Representatives, and the presidency were deliberately provided with different terms of office and different electorates, and the states were given an important role in the federal election process  In particular, through the electoral college the United States elects its national Presidents and Vice Presidents in a state-based federal election  Successful nominees are compelled under this system to present a broad political vision that commands nation-spanning "concurrent majorities" and appeals to the great variety of Americans  As in the case of the Senate, less populous states are accorded a small numerical advantage by assignment of two at-large electors reflecting the Constitution's equal apportionment of Senators to each state regardless of its population  The NPV initiative, they could claim, would discard the Founders' intentions in favor of what they consider to be a flawed "majoritarian" presidency that would ill-serve a continent-spanning and profoundly diverse republic  Another criticism centers on the use of the NPV compact to effect a fundamental change in the presidential election process and a de facto amendment to the Constitution, but without following the procedures set out in Article V  Critics may note that NPV's founders admit their plan is an "end run" around the Constitution  Proponents might counter with the argument that Article V presents too high a hurdle for what they consider a necessary reform of the system  Opponents, however, could respond that the Founders intended the various supermajority requirements in Congress and the states to ensure that successful constitutional amendments enjoy broad national support  The bare majority of electoral votes required to implement NPV, they might note, meets none of these supermajority requirements  As one study critical of the NPV initiative concluded, because the use of an interstate compact "does not conform to the constitutional means of changing the original decisions of the Framers, NPV could not [therefore] be a legitimate innovation " A final argument on this line might be that one "end run" around the amendment process might lead to others, setting a dangerous precedent for similar efforts in the future  Opponents might note that the NPV would eliminate the electoral college system's multiplier effect generated by the winner-take-all or general ticket system used in 48 states and the District of Columbia, which tends to magnify the winning ticket's margin of victory, and is said to confer greater legitimacy to the victors  For instance, in 2016, Republican nominee Donald Trump's clear electoral vote majority of 304 votes (56 5% of the total), compared with Democratic nominee Hillary Clinton's 227 (42 2% of the total) could be said to reinforce and confirm his victory, notwithstanding Clinton's plurality of the popular vote (48 2%), compared with Trump's 46 1%  From a practical standpoint, NPV opponents might argue that the NPV would actually lead to an increase in contested election results and legal challenges in the states, as the political parties maneuver to claim every possible vote  They assert that the existing tabulation of popular votes within each state reduces contested results and recounts  Under NPV, the incentive to gain every vote would arguably lead to far broader disputes and widespread recounts at every level of election administration  As a Heritage Foundation study concluded Under the NPV … any suspicions necessitating a recount in even a single district would be an incentive for a national recount     The prospect of a candidate challenging "every precinct, in every county, in every state of the Union" should be abhorrent to anyone who witnessed the drama, cost, delay, and undue litigation sparked by the Florida recount of 2000  Opponents might also assert that the increased incidence of recounts would be further complicated by wide-ranging disparities in state procedures, potentially leading to prolonged periods of uncertainty following close presidential elections  Critics may also note that the NPV plan contains no "statute of limitations," unlike proposed constitutional amendments, for which Congress typically sets a seven-year ratification period  Where, critics may ask, is a similar time limit that would "sunset" the NPV compact, after which it would expire or return to "square one"? According to its website, NPV was launched on February 23, 2006; if it were a constitutional amendment proposed by Congress, it would have expired on February 23, 2013, since by the end of the customary seven-year deadline it was "ratified" by only eight states and the District of Columbia  By what reasoning, they might ask, should the NPV be exempt from the standards of timeliness and contemporaneity Congress customarily sets for constitutional amendments? Opponents might reject claims that, under NPV, campaign spending and candidate appearances would be spread and scheduled more widely, beyond the current concentration of time and resources in battleground states  They might argue that spreading campaign resources and candidate events in non-battleground states is a questionable argument to justify a fundamental change in the presidential election process  Campaign appearances and spending, they could assert, should not be considered to be a local economic stimulus package, nor are the amounts in question sufficient to make much of a difference in the economic condition of most states  As one critical analysis notes, "    the nation does not hold presidential elections to foster local economic development " Moreover, they might continue, it is equally dubious to assert that nominees will slight the concerns of citizens of the non-battleground states from which they draw their greatest support, or that concentrated campaigning in the battleground states somehow "disenfranchises" voters in others  In the modern era, a small percentage of voters actually attends an in-person presidential or vice presidential candidate appearance  Television (especially broadcast and cable TV news networks), social media, the internet, and newspapers—not the traditional rallies, torchlight parades, and handbills—dominate presidential election campaigns in the 21 st century  In addition to policy issues discussed previously, some observers have also raised questions related to the NPV initiative based on the fact that it is an interstate compact as defined in the Constitution  Others have questioned whether NPV might conflict with some provisions of the Voting Rights Act  The NPV initiative has been described by its supporters variously as a bill, a state-level statute, and an interstate compact  The latter reference necessitates an analysis of whether the initiative complies with the Compact Clause of the Constitution  An interstate compact—under the broadest understanding—is a contract between two or more consenting states  The Supreme Court has further suggested that an interstate compact often requires reciprocal commitments between the governments of two or more states, such that one state's commitment is conditioned on the action of another state and no state can unilaterally repeal its commitment  The use of interstate compacts predates the Constitution, as the Articles of Confederation contained a similar Compact Clause that provided a qualified prohibition on states entering into any agreements between them without the consent of Congress  The chaos resulting from the disunity created by the Articles of Confederation prompted the Framers of the Constitution generally to "impose more uniformity" among the states, resulting in a Constitution that wholly prohibits states from entering into any treaties, alliances, and federations  Nonetheless, the Constitution maintained the Articles of Confederation's qualified prohibition on interstate compacts and agreements, allowing states to enter into an interstate compact so long as the participating states seek the consent of Congress  Specifically, the Compact Clause provides that "No State shall, without the Consent of Congress     enter into any Agreement or Compact with another State     " While the historical rationale for Article I's qualified prohibition on interstate compacts is unclear, the Compact Clause generally reflects the view of the Framers that states should be able to work cooperatively together, as well as the concern that unchecked interstate alliances might threaten the harmony of the Union or the authority vested by the Constitution in the federal government  As the Supreme Court noted in Cuyler v  Adams , "By vesting in Congress the power to grant or withhold consent, or to condition consent on the states' compliance with specified conditions, the Framers sought to ensure that Congress would maintain ultimate supervisory power over cooperative state action that might otherwise interfere with the full and free exercise of federal authority " The Compact Clause places no limits on what might be done through an interstate compact other than the requirement of congressional consent  In the early years of government under the Constitution, compacts were used almost exclusively to settle boundary disputes  Beginning with the establishment of the Port of New York Authority in 1921, however, compacts began to be used to address more complex, regional issues requiring intergovernmental cooperation  Some compacts are merely advisory in form, but others may be regulatory, with significant powers granted to multistate commissions  More recently, compacts have addressed such wide-ranging concerns as mental health treatment, law enforcement and crime control, education, driver licensing and enforcement, environmental conservation, energy, nuclear waste control, facilities operations, transportation, economic development, insurance regulation, placement of children and juveniles, disaster assistance, and pollution control  Approximately 200 interstate compacts are in effect today  Accordingly, the central legal issue with respect to the Compact Clause is whether a given interstate compact requires the consent of Congress  While a "literal" reading of the Compact Clause "would require the States to obtain congressional approval before entering into any agreement among themselves, irrespective of form, subject, duration, or interest to the United States [emphasis added]," the Supreme Court has repeatedly rejected such a reading  In 1893, in Virginia v  Tenness e e , Justice Stephen Field, writing for the Court, contended that a broad reading of the Compact Clause would "embrace all forms of stipulation, written or verbal, and relating to all kinds of subjects[,]" requiring congressional consent to agreements "which the United States can have no possible objection or have any interest in interfering with," as well as those that "may tend to increase     the political influence of the contracting states, so as to encroach upon or impair the supremacy of the United States     " Surmising that the Compact Clause could not have been intended to have such a broad reach, Justice Field concluded that the Clause prohibits states from entering into compacts without congressional consent only when the underlying compact is "directed to the formation of any combination tending to the increase of political power in the States, which may encroach upon or interfere with the just supremacy of the United States " The Supreme Court has subsequently reaffirmed Justice Field's "functional view of the Compact Clause," and, accordingly, generally where an agreement does not fall within the scope of the Compact Clause as envisioned by the Court in Virginia v  Tennessee , the agreement "will not be invalidated for lack of congressional consent " Whether the NPV initiative requires congressional consent under the Compact Clause first requires a determination as to whether NPV even constitutes an interstate compact  At times, its supporters have resisted framing the initiative as an interstate compact, arguably out of concern for running afoul of the Compact Clause's provisions  For example, Professor Akhil Amar has argued that because the initiative does not create a "new interstate governmental apparatus," the NPV should not be considered an interstate compact, as NPV compact signatory states are merely exercising power collectively that each state could exercise on its own  It is unclear, however, whether the creation of a new interstate governmental entity formed out of an agreement between two or more states is necessary, as opposed to sufficient, in order to deem an agreement as being an interstate compact subject to the Compact Clause  While the Supreme Court, in Northeast Bancorp , suggested that a "joint organization or body" formed out of an interstate agreement is a "classic indic[ium] of a compact," the Court has never adopted a definition of an interstate compact that solely rests on the existence of an interstate governmental body  Instead, the Court appears to have adopted a broader definition of what an interstate compact can entail  For example, in Virginia v  Tennessee , the Court noted that the words "compacts" and "agreements" are synonymous and "cover all stipulations affecting the conduct or claims of the parties " In other words, when two or more states enter into a stipulated agreement whereby one state agrees to perform an act in consideration for a reciprocal act by the other state(s), that agreement can be considered an interstate compact  This broad definition of a compact appears to encompass the NPV compact, as the initiative requires signatory states to agree mutually to appoint their electors to the winner of the national popular vote  Moreover, NPV binds each assenting state, as no member state can withdraw from it within six months or less of the end of a President's term  Because NPV prohibits states from freely "modify[ing] or repeal[ing] [the agreement] unilaterally" and requires "reciprocation" of mutual obligations, it appears that the initiative can be described as an interstate compact  Assuming the NPV initiative is an interstate compact, the question remains whether it is one that implicates the Compact Clause  The answer to that question primarily depends on whether NPV is "directed to the formation of any combination tending to the increase of political power in the States, which may encroach upon or interfere with the just supremacy of the United States  " In other words, the "test" for whether a particular interstate compact requires congressional consent is centrally concerned with vertical balances of power between the federal government and the states; namely, "whether the Compact enhances state power quoad the National Government " While the NPV arguably increases the political power of the states that have consented to it by ensuring that those states' desired outcome for the presidential election—the awarding of the majority of electoral votes to the presidential candidate supported by the majority of the voting populace—it is unclear how that increase in political power would be at the expense of the power of the federal government  After all, the Constitution provides the federal government with no role in determining the members of the electoral college  One scholar has suggested that the NPV initiative would lead to a vertical alteration of power by eliminating the possibility that the House of Representatives would resolve a presidential election in the absence of an electoral majority for a single candidate because it is premised on a majority of electoral votes going to a single candidate  The House of Representatives, however, has decided only two presidential elections in American history, and whether such an arguably hypothetical and de minimis diminishment of federal power through the NPV would be sufficient to require congressional consent under the Compact Clause is simply unresolved by the relevant case law  While the Supreme Court's case law interpreting the Compact Clause is centrally concerned with vertical federalism concerns (i e , the balance of power between the state and federal governments), the Court has recognized a potential secondary rationale suggested for the Compact Clause: to preserve the horizontal balance of powers among the various states  And horizontal federalism concerns could very well be implicated by the Compact Clause, as the provision appears to have been included in the Constitution out of concern both for the supremacy of the federal government and unity among the various states  Whether the NPV compact threatens the powers of nonconsenting states has been the subject of much debate among academics  Those in support of the initiative have contended that the nonconsenting states do not lose any power as a result of the NPV  According to this line of argumentation, even under the NPV, all states would retain their right to select the electors of their choosing, as nonmember state electors would still be counted in the electoral vote  Others, however, have pointed to the underlying premise of the NPV—to enhance the political power of more populous states in presidential elections—as evidence that the initiative diminishes the power of nonconsenting states  In other words, while non-compacting states would still retain the power to appoint electors, the influence that comes with that power would arguably be diminished because a state's role in the national election would be defined by its percentage of the popular vote and not by its percentage of electors, warranting congressional interest in approving a compact that effectuated such a change in national elections  Ultimately, however, whether the NPV actually threatens the power of nonconsenting states is a debate that remains active within academia but would likely be the source of considerable litigation if the initiative ever became effective  If congressional consent is needed for the NPV, that consent can take various forms  Usually congressional consent to an interstate compact takes the form of a joint resolution or act of Congress specifying its approval of the text of the compact and adding any conditions or provisions it deems necessary, often embodying the compact document  As with most congressional actions, consent to an interstate compact must occur with the approval of both houses and must be signed by the President before it becomes law  Rarely has the President vetoed or threatened to veto consent legislation by Congress  While congressional consent to an interstate compact is most often explicit, consent by Congress may also be implied by subsequent acts of Congress as "[a]n inference clear and satisfactory that Congress     intended to consent" to a compact may be sufficient  Congress may also delegate its power to approve a compact to a federal official so long as an "intelligible principle" against which approval can be measured is apparent  Ultimately, if congressional consent is truly needed for NPV to be effective, the initiative might have difficulty ever being enacted because the approval of both houses of Congress and the President would likely necessitate additional hurdles beyond the already challenging task of amassing support at the state level for the NPV  Beyond the legal issues raised with respect to the Compact Clause, the NPV initiative also potentially raises other broader constitutional concerns, including whether the states can functionally obviate the role of the electoral college through the NPV  Article II of the Constitution establishes that the election of the President should occur indirectly through the election by the electoral college  The choice of an indirect election for the President was a deliberate one by the Framers of the Constitution, because, while noting the importance that the "sense of the people" should influence the choice for President, they found it "equally desirable" for the "immediate election" of the President to be made by a body representative of distinct state interests and removed from the threat of unchecked majoritarianism  The result was that the Constitution established a presidential election process that was "manifestly nonmajoritarian," with the electoral college, a body established to represent the distinct views of each state, as the centerpiece of the election process  The central constitutional issue presented by the NPV, therefore, is whether the states, through an interstate compact, can functionally transform the presidential election system enshrined in the Constitution into a more majoritarian process  Supporters of the NPV argue that the Constitution provides the legal means for states to transform the presidential election system into one where the President is elected based solely on the result of the national popular vote  Specifically, clause 2 of Article II, Section 1 of the Constitution provides the states with the power to "appoint, in such Manner as the Legislature thereof may direct," the electors who represent the state in the electoral college  Facially, the Constitution's primary limitation on the power of a state to select its electors is the final number of electors awarded to each state  While perhaps an argument can be made that the structure, logic, and history of the Constitution place limits on the manner or method in which a state chooses its electors, the text of the Constitution simply does not impose any such limits  Supreme Court case law also supports reading Article II of the Constitution to broadly provide states with wide discretion as to the manner in which its electors are selected  Specifically, in 1892 in McPherson v  Blacker, a unanimous Supreme Court upheld a Michigan law providing for the election by individual congressional district of presidential electors against a challenge that the law violated Article II of the Constitution  In so holding, the Court placed great emphasis on a number of state laws that existed shortly after the ratification that provided a variety of "modes of choosing the electors," including selection by the legislature itself, by a "vote of the people for a general ticket," "by vote of the people in districts," or by some permutation of those methods  Viewing this evidence together with the text of Article II and the historical evidence from the Constitutional Convention led the Court to broadly conclude state legislatures have "conceded plenary power     in the matter of the appointment of electors," allowing the Michigan law to stand  Applying McPherson to the case of the NPV, the argument can and has been made that if the states have plenary power with respect to the manner of how electors are appointed, the power necessarily allows states to select electors in line with the results of the national popular vote  More recently, supporters of the NPV have relied on the Supreme Court's 2015 ruling in Arizona Legislature v  Arizona Independent Redistricting Commission (AIRC) —which held that the State of Arizona had wide discretion under the Elections Clause of the Constitution to select the method by which the state provided for redistricting —to argue that the states retain broad discretion in selecting electors under Article II, which uses similar language to the provision interpreted in AIRC  Others have argued that the structure of the Constitution and historical evidence suggest that the states do not have such vast discretion in appointing electors as to functionally transform the election for President into a national popular referendum  As noted elsewhere in this report, the electoral college was created by the Framers to ensure that states with the least population retained power in the selection of the President, providing a check against domination by the most populous states  The electoral college, being a product of the choices of individual state legislatures, was envisioned by the Framers as a body that would represent the specific interests of a given state, as opposed to the undifferentiated nation at large  Accordingly, it may be argued that allowing the most populous states to collude to ensure that the national popular vote, as opposed to the wishes of an individual state, dictates the results of a state's slate of electors, could arguably be irreconcilable with the Framers' intentions with respect to the electoral college  As such, for those who find the NPV compact constitutionally suspect under Article II, McPherson 's broad pronouncements about the nature of a state's power to appoint electors should be viewed in the context of that particular case, where the state of Michigan was attempting to appoint its electors based on the votes of an individual district in the state, as opposed to the state as a whole  In contrast to the law at issue in McPherson , with NPV, there appears to be no evidence contemporaneous with the ratification of the Constitution of a state selecting its electors in accordance with the results of the national popular vote  Unlike the State of Michigan in McPherson , an NPV state's electors might not be a product of the views of the state at the time of the election, but instead would reflect national popular sentiment about who should be the President  Moreover, the Supreme Court, in interpreting arguably analogous language from Article I of the Constitution allowing states to regulate the manner of the selection of the Members of the House of Representatives and Senate, concluded that the states cannot exercise their delegated authority in a way that would "effect a fundamental change in the constitutional structure " The question that remains is whether the Court in a future case challenging the NPV compact would interpret the states' authority under Article II to appoint electors to be broad enough to allow the President to be selected as a result of the national popular vote, a question that, given the lack of any precise precedent respecting the constitutionality of the NPV compact under Article II, will likely remain unresolved until such time  Other critics claim the NPV compact might violate Sections 2 and 5 of the Voting Rights Act (VRA)  Writing in Columbia Law Review , David Gringer invokes the voting power theory  He argues that the plan conflicts with Section 2 of the VRA because moving from "a state-based [vote] to a national popular vote dilutes the voting strength of a given state's minority population by reducing its ability [voting power] to influence the outcome of presidential elections " Gringer also asserts that the NPV compact may violate Section 5 of the act  In 2013, however, the U S  Supreme Court invalidated Section 4(b) of the VRA, which contained a formula prescribing which states and jurisdictions with a history of discrimination were required to obtain prior approval or "preclearance" under Section 5 before changing any voting standard, practice, or procedure  Although the Court invalidated only the coverage formula in Section 4, by extension, Section 5 has been rendered currently inoperable  Prior to the Supreme Court ruling, Gringer argued that the NPV compact would qualify as a covered practice under Section 5, and that the legislatures of all the "covered" states would have been required to obtain preclearance before implementing the compact  Responding to this point, National Popular Vote Inc  noted the following: The National Popular Vote bill manifestly would make every person's vote for President equal throughout the United States in an election to fill a single office (the Presidency)  It is entirely consistent with the goal of the Voting Rights Act  There have been court cases under the Voting Rights Act concerning contemplated changes in voting methods for various representative legislative bodies     However, these cases do not bear on elections to fill a single office (i e , the Presidency)  In 2012, the Justice Department's Civil Rights Division specifically declined to challenge California's accession to the NPV compact on VRA grounds  The states' authority to appoint electors by any method their legislatures choose is not absolute  Federal court decisions have struck down state laws concerning appointment of electors that were found to be in violation of the Fourteenth Amendment's guarantee of equal protection: Although Clause 2 (of Article II, Section 1 of the Constitution) seemingly vests complete discretion in the states, certain older cases had recognized a federal interest in protecting the integrity of the process  Thus, the Court upheld the power of Congress to protect the right of all citizens who are entitled to vote to lend aid and support in any legal manner to the election of any legally qualified person as a presidential elector     [I]n Oregon v  Mitchell (42 U S  112 (1970)), the Court upheld the power of Congress to reduce the voting age in presidential elections and to set a thirty-day durational residency period as a qualification for voting in presidential elections  Although the Justices were divided on the reasons, the rationale emerging from this case, considered with Williams v  Rhodes , (393 U S  20 1968)) is that the Fourteenth Amendment limits state discretion in prescribing the manner of selecting electors and that Congress in enforcing the Fourteenth Amendment may override state practices that violate that Amendment and may substitute standards of its own  Critics of the electoral college system have sought direct election of the President and Vice President without success for more than two centuries  The NPV initiative represents a novel effort to achieve this goal by use of an interstate compact that would circumvent the stringent requirements necessary for the proposal and ratification of constitutional amendments  Since its inception in 2006, NPV has achieved a degree of success: 14 states and the District of Columbia, controlling a total of 189 electoral votes, have joined the compact since 2008  Progress has arguably been sporadic, however, notwithstanding active campaigning by National Popular Vote Inc  Over the course of more than a decade, NPV has heretofore failed to develop a sustained momentum toward its stated goal of states controlling 270 electoral votes  The action of three states in joining NPV to date in 2019 marks the most activity in a single year since 2008; it remains to be seen whether this trend will continue  To date, certain Democratic-leaning states have joined the compact  The arguable lack of support in Republican-controlled state legislatures raises questions about further accessions to the compact in the immediate future, particularly given the fact that the GOP controlled both legislative chambers in 30 states following the 2018 elections  To date, while the NPV initiative has generated interest among supporters of direct popular election of the President, it does not appear to have gained widespread awareness among the public at large  The findings of the March 27, 2019, Politico/Morning Consult survey cited earlier in this report arguably suggest that greater public knowledge of NPV might spur popular support for the compact  This might then contribute to further momentum if additional states were to join, particularly populous ones like Florida (29 electoral votes), Georgia (16 electoral votes), and Ohio (18 electoral votes) where the compact was under active consideration in 2019  Under these circumstances, proponents might be energized and encouraged by a sense of progress for the initiative  At the same time, NPV opponents could be expected to coalesce around the issues identified earlier in this report, and renew and increase their efforts in defense of the electoral college system  The activities of both might ultimately bring the NPV initiative to the more immediate attention of Congress 


Report 2:  OMB issues guidance for agencies to implement various requirements of the improper payment laws  Specifically, OMB is required by IPERIA to issue guidance to set standards for agencies to follow in determining the underlying validity of sampled payments to ensure that amounts being billed, paid, or obligated for payment are proper  Although existing OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, like eligibility, are addressed through testing processes  Without such guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs—for example, the cases of DOD Military Pay and OPM Retirement described above—calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions  According to OMB guidance, when an agency’s review is unable to determine whether a payment was proper because of insufficient or lack of documentation, the payment must be considered an improper payment  Among the six agencies and 10 programs we reviewed, treatment of insufficient documentation varied by program, as did the classification of these issues for root cause reporting in the AFRs  HHS’s programs were the only ones we reviewed that reported improper payments in the insufficient documentation root cause category for fiscal years 2016 or 2017, as shown in table 6  Some agencies stated that they report insufficient documentation in other root cause categories that they consider more appropriate  For example, Education officials stated that for the Direct Loan program, payments that lack sufficient supporting documentation may be placed in the “Administrative or Process Error Made by Other Party” root cause category  In these cases, a third party—such as a loan servicer—is unable to provide sufficient documentation supporting that the sampled payment was proper  OMB guidance states that in cases where the agency believes that more than one root cause category might be suitable, the agency should determine which category it believes to be the most appropriate  Additionally, some agencies stated that the “insufficient documentation” category was not always relevant when they recreated sampled cases to estimate a program’s improper payments  For example, according to officials, to complete an OASDI stewardship review of a sampled case, a Social Security Administration (SSA) quality reviewer reviews the documentation related to the original determination and then independently re-develops all factors of the payment and interviews the associated beneficiary  According to agency officials, insufficient documentation would not apply as all improper payments identified in the stewardship sample are supported by documentation and payment has been verified in all reviewed cases  As noted previously, the processes for estimating DOD Military Pay and OPM Retirement improper payments were limited, and these limitations may have an impact on the agencies’ ability to identify improper payments related to insufficient documentation  Treatment of cases of nonresponse  Some agencies contact outside entities—such as payees or beneficiaries—as part of their improper payment testing processes  Among the six agencies we reviewed, treatment of cases of nonresponse differed  For example: SSA officials stated that in cases where quality reviewers do not receive responses from OASDI beneficiaries they contact, they exclude the cases from review (unless the reviewer identifies an improper payment in the initial review that is completed prior to reaching out to the beneficiary)  For EITC improper payment estimation purposes, the Internal Revenue Service (IRS) stated that the agency does not consider the sampled payment associated with a nonresponse case to be proper or improper  It sets the sampling weight of nonresponse cases to zero and adjusts the sampling weights of respondents upward to account for the nonresponse cases  IRS’s methodology assumes nonresponse and response cases have an equal likelihood of improper payment  For Medicare Fee-for-Service and Medicaid, HHS’s policies and procedures consider payments associated with nonresponse cases to be improper  OMB guidance states that when an agency’s review is unable to discern whether a payment was proper as a result of insufficient or lack of documentation, this payment must be considered an improper payment  However, it does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes  As a result, without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable  When agencies identify improper payments, they must determine the amount of the payment that was improperly made  The six agencies we reviewed generally reported using the definition of improper payment in relevant laws and OMB guidance to determine the amount of improper payments identified  OMB guidance provides agencies with instructions on how to calculate the amount of improper payments  However, when developing its improper payment estimate for EITC, IRS subtracted overpayments that were paid out and later recovered  By subtracting recovered overpayments, IRS excluded them from the EITC improper payment estimate  For 2013—the tax year used to produce the fiscal year 2017 improper payment estimate—IRS estimated that $1 2 billion in EITC overpayments would be recovered  IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered  Although the OMB guidance was revised in October 2014 to implement this requirement, IRS has not updated its estimation methodology for EITC  By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur  Improper payments are a long-standing, significant problem in the federal government  Estimation of improper payments is key to understanding the extent of the problem and to developing effective corrective actions to address it  Among the six agencies we reviewed, processes to estimate improper payments in their programs varied, and certain differences in these processes may affect the quality of the resulting estimates and consequently these agencies’ efforts to reduce improper payments  Specifically, policies and procedures for DOD’s Military Pay and OPM’s Retirement programs’ improper payment estimation methodologies do not address certain key risks, like eligibility, in part because these agencies have not fully assessed their processes  Further, although OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments  Without such assessments and guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs, calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions  Additionally, for agencies we reviewed that contact outside entities as part of their improper payment estimation processes, the treatment of cases of nonresponse varied  OMB guidance does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes  Without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable  Finally, although IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered, IRS has not updated its processes to reflect the change  By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur  We are making two recommendations to the Director of OMB that have government-wide implications and specific recommendations to DOD, OPM, and IRS regarding their programs included in this review  The Director of OMB should develop guidance on how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, such as eligibility, are addressed through testing processes  (Recommendation 1) The Director of OMB should develop guidance clarifying the appropriate treatment of nonresponse cases during improper payment testing  (Recommendation 2) The Under Secretary of Defense (Comptroller) should assess the processes for estimating Military Pay improper payments to determine whether they effectively address key risks of improper payments— including eligibility for different types of pay and allowances—and take steps to update the processes to incorporate key risks that are not currently addressed  (Recommendation 3) The Director of OPM should assess the processes to estimate Retirement improper payments to determine whether they effectively address key risks of improper payments—including eligibility and whether older claims face different risks of improper payments than new claims—and take steps to update the processes to incorporate key risks that are not currently addressed  (Recommendation 4) The Commissioner of IRS should update IRS’s improper payment estimation methodology to not exclude recovered overpayments from its EITC improper payment estimate  (Recommendation 5) We provided a draft of this report for comment to OMB, DOD, Education, HHS, Treasury, OPM, SSA, and USDA  OMB provided oral comments, which are summarized below  OPM, DOD, and IRS provided written comments, which are reproduced in appendixes II through IV, respectively  Education, HHS, SSA, and USDA did not provide written comments on the draft report  In addition, HHS, IRS, OMB, OPM, and SSA provided technical comments, which we have incorporated, as appropriate  In oral comments provided on April 30, 2018, a Senior Policy Advisor in OMB’s Office of Federal Financial Management stated that OMB partially agreed with our first recommendation and agreed with our second recommendation  Regarding the first recommendation, the Senior Policy Advisor stated that OMB should not have to develop more specific guidance as each program and activity has its own risks  Instead, inspectors general are better equipped and positioned to review the sampling and estimation plans as part of their annual IPERA compliance audits and that agencies, their statisticians, and inspectors general should work out the best testing procedures for their agencies  OMB could provide suggestions during OMB’s annual town hall meeting related to improper payments for areas that inspectors general may consider  Although we agree that programs and activities may face different risks of improper payment, we continue to believe that guidance from OMB on how agencies test to identify improper payments—such as directing agencies to take a risk-based approach in developing their testing procedures—could help ensure that agencies address the specific risks they identify when developing improper payment estimates  Further, such guidance could also help ensure that testing processes are designed to address an agency’s identified risks before the estimate is developed, whereas an inspector general’s review—as well as related recommendations for improvement—would generally occur after the agency’s improper payment estimate had been developed and reported  Regarding the second recommendation, the Senior Policy Advisor noted that OMB plans to update its guidance to direct agencies to treat nonresponse cases as improper payments and to include a new category for tracking such cases  In its written comments, OPM partially concurred with our recommendation to assess the processes to estimate Retirement improper payments to determine whether they effectively address the key risks of improper payments  OPM agreed to conduct an audit of older claims to determine if they face different risks than new claims  However, OPM did not agree with the part of the recommendation to assess the risk of improper payments related to eligibility in the estimation process  OPM stated that eligibility is determined before annuity or survivor benefits are fully adjudicated  However, the objective of an improper payment estimate is to determine whether payments were made properly  To do so, an agency should determine whether the payee was eligible for the payment that was made, among other things  As such, we continue to believe that the recommendation—including the assessment of the risk of improper payments related to eligibility—is warranted  In their written comments, DOD and IRS both agreed with our recommendations directed to them and described the steps they plan to take to implement them  We are sending copies of this report to the appropriate congressional committees; the Secretaries of Agriculture, Defense, Education, Health and Human Services, and the Treasury; the Director of the Office of Personnel Management; the Administrator of the Social Security Administration; the Director of the Office of Management and Budget; and other interested parties  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2623 or davisbh@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  Table 7 lists the fiscal year 2017 improper payment estimates by agency and program, as reported by agencies in their fiscal year 2017 agency financial reports and compiled on the Office of Management and Budget’s payment integrity website, paymentaccuracy gov  In addition to the contact named above, Phillip McIntyre (Assistant Director), James M  Healy (Auditor in Charge), Daniel Flavin, and Fabiola Torres made key contributions to this report 

ground_truth: To reform the federal government and make it more efficient and effective, agencies need to use data about program performance. The benefit of collecting performance information is only fully realized when it is used by managers to make decisions aimed at improving results. GAO was asked to review agencies' use of performance information. This report assesses, among other things, the extent to which: (1) 24 agencies' reported use of performance information and related leading practices has changed since 2013 and (2) the Executive Branch has taken actions to enhance the use of performance information. To address the first objective, GAO analyzed results from its 2017 survey of federal managers, and compared them to 2013 results. The survey covered a stratified random sample of 4,395 managers from the 24 Chief Financial Officers Act agencies. The survey had a 67 percent response rate and results can be generalized to the population of managers government-wide and at each agency. For the second objective, GAO reviewed agency documents and interviewed staff from OMB and the PIC. Agencies' reported use of performance information to make decisions, and leading practices that can promote such use, generally has not improved since GAO's last survey of federal managers in 2013. However, GAO's survey results continue to point to certain practices that could help agencies improve managers' use of performance information. For example, as shown in the table below, GAO's survey found that managers whose programs were subject to data-driven reviews (regular reviews used to assess progress on select agency goals) to a greater extent reported statistically significantly greater use of performance information to make decisions. The Executive Branch has begun taking steps to improve the use of performance information within agencies and across the government. For example, In the President's Management Agenda and government-wide reform plan, released in March and June 2018 respectively, the administration acknowledged the need to do more, and announced a goal, among other actions, to improve the use of data in federal decision making. However, the Office of Management and Budget (OMB) and others responsible for this goal have yet to fully develop action plans to hold agencies accountable for achieving it. The Performance Improvement Council (PIC), which is chaired by OMB, has undertaken efforts to improve the use of performance information by, for example, creating a working group on agency performance reviews. But it has not yet taken a systematic approach to identify and share proven practices that led to, or challenges that may be hampering, increased use of performance information by managers. GAO's survey results identified agencies that may have insights into such practices and challenges. More fully developing action plans for the new goal, and identifying and sharing proven practices and challenges, could help ensure the Executive Branch takes further steps to improve the use of performance information by managers within agencies and across the federal government. To improve the use of performance information within agencies and across the federal government, GAO recommends that OMB work with (1) fellow goal leaders to more fully develop action plans for the new goal to improve the use of data and (2) the PIC to prioritize efforts to identify and share proven practices and challenges. OMB had no comments on this report.