prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.



Report 2: Our simulations suggest that the sector will likely continue to face a difference between revenue and spending during the next 50 years  This long-term outlook is measured by the operating balance—a measure of the sector’s ability to cover its current expenditures out of current receipts  While both expenditures and revenues are projected to increase as a percentage of gross domestic product (GDP) during the simulation period, a difference between the two is projected to persist because expenditures are generally expected to grow at a faster rate than revenues  (see figure 1)  Absent any policy changes by state and local governments, revenues are likely to be insufficient to maintain the sector’s capacity to provide services at levels consistent with current policies during the next 50 years  Our simulations suggest that state and local governments will need to make policy changes to avoid fiscal imbalances before then and assure that revenues are at least equal to expenditures  We simulated the state and local government sector’s operating balance (the difference between the sector’s operating revenues and operating expenditures) in order to understand the sector’s long-term fiscal outlook  The sector’s operating expenditures were 15 1 percent of GDP in 2017  As shown in figure 2, these state and local government sector operating expenditures are comprised of employee compensation, social benefit payments, interest payments, capital outlays, and other expenditures  The sector’s operating revenues were 13 8 percent of GDP in 2017  As shown in figure 3, these state and local government sector operating revenues are comprised of taxes, transfer receipts, and other types of revenues  One way of measuring the long-term fiscal challenges faced by the state and local government sector is through an indicator known as the “fiscal gap ” The fiscal gap is an estimate of actions—such as revenue increases or expenditure reductions—that must be taken today and maintained for each year going forward to achieve fiscal balance during the simulation period  While we measured the gap as the amount of reductions in expenditures needed to prevent negative operating balances, increases in revenues, reductions in expenditures, or a combination of the two of sufficient magnitude would allow the sector to close the fiscal gap  Our simulations suggest that the fiscal gap is about 14 7 percent of total expenditures or about 2 4 percent of GDP  That is, assuming no change in projected total revenues, eliminating the difference between the sector’s expenditures and revenues during the 50-year simulation period would likely require action to be taken today, and maintained for each year equivalent to a 14 7 percent reduction in the sector’s total expenditures (see figure 4)  Alternatively, assuming no change in projected total expenditures, closing the fiscal gap by increasing revenue would also likely require actions of similar magnitude  More likely, eliminating the difference between expenditures and revenues would involve some combination of spending reductions and revenue increases  Our simulations suggest that growth in the sector’s overall spending is largely driven by health care expenditures  As shown in figure 5, these expenditures are projected to increase from about 4 1 percent of GDP in 2018 to 6 3 percent of GDP in 2067  Two types of health care expenditures—Medicaid spending and spending on health benefits for state and local government employees and retirees—will likely constitute a growing expenditure for state and local governments during the simulation period  Medicaid expenditures are expected to rise, on average, by 1 percentage point more than GDP each year  According to CBO, growth in Medicaid spending reflects growth in both the number of people receiving Medicaid benefits and the cost of Medicaid benefits each person receives  Specifically, CBO reported that between 2019 and 2028, Medicaid spending is projected to grow at an average rate of 5 5 percent per year—nearly 5 percentage points of this growth is due to an increase in per capita costs and about 1 percentage point of this growth is due to an increase in enrollment  Data from CBO and the Centers for Medicare & Medicaid Services (CMS) also suggest that growth in Medicaid spending per capita is generally expected to outpace GDP growth in the future—referred to as excess cost growth  Our estimates of Medicaid excess cost growth using CMS data suggest that Medicaid spending per capita will grow, on average, about 0 5 percent faster than GDP per capita for the period from 2018 through 2067  Our simulations also suggest that health benefits for state and local government employees and retirees—a type of employee compensation spending—are likely to rise, on average, by 0 9 percentage points more than GDP each year  Growth in these health benefits also reflects growth in the projected number of employees and retirees and growth in the projected amount of health benefits for each employee and retiree  Growth in spending by states and local governments on health care per capita, which includes spending on employee and retiree health benefits, is generally expected to outpace GDP per capita  Data from CMS suggest that national health expenditures per capita are likely to grow on average about 0 8 percent faster than GDP per capita each year during the simulation period from 2018 through 2067  If employee and retiree health benefits follow trends in overall national health spending, they will likely make up an increasingly large share of total employee compensation going forward (see figure 6)  While state and local government contributions to employee pension plans—another type of employee compensation spending—will likely decline as a percentage of GDP, as shown in figure 6, our simulations nonetheless suggest that state and local governments may need to take steps to manage their pension obligations in the future  From 1998 through 2007, state and local governments’ pension contributions amounted to about 8 percent of wages and salaries on average  In addition, for the period from 2008 through 2017, pension contributions amounted to about 12 3 percent of wages and salaries on average  Our simulations suggest that those pension contributions will need to be about 12 9 percent of wages and salaries for state and local governments to meet their long-term pension obligations  This is the case even though pension asset values have increased in recent years, from about $2 4 trillion in 2008 to about $4 2 trillion in 2017 (adjusted for inflation and measured in 2012 dollars)  This suggests that state and local governments may need to take additional steps to manage their pension obligations by reducing benefits or increasing employees’ contributions  Along with pension contributions, other types of state and local government expenditures are projected to grow more slowly than GDP  For example, in 2017, wages and salaries of state and local government employees constituted a large expenditure for the sector  However, these expenditures are projected to decline as a percentage of GDP during the simulation period  Our simulations also suggest that state and local governments’ capital outlays—which include spending on infrastructure, such as buildings, highways and streets, sewer systems, and water systems, as well as equipment and land— will grow more slowly than GDP if state and local governments continue to provide current levels of capital per resident  Our simulations suggest that federal grants overall will increase as a share of GDP, while Medicaid grants will likely grow more quickly than other types of federal grants (see figure 7)  Thus, Medicaid grants will likely make up an increasing share of revenues in the future  Since Medicaid is a matching formula grant program, the projected increase in federal Medicaid grants, therefore, reflects expected increased Medicaid expenditures that will be shared by state governments  Our simulations also suggest that federal investment grants (i e , grants intended to finance capital infrastructure investments) and other federal grants unrelated to Medicaid (i e , grants intended to finance education, social services, housing, and community investment) are likely to decline as a share of GDP  Further, our simulations suggest that if historical relationships between state and local governments’ tax revenues and tax bases persist, total tax revenues for the state and local government sector will increase from 8 8 percent of GDP in 2018 to 9 4 percent of GDP by the end of the simulation period  This increase is driven largely by the growth in personal income taxes, as shown in figure 8  Specifically, our simulations suggest that personal income tax revenues will increase as a share of GDP by about 1 percentage point during the simulation period  Sales taxes and property taxes, on the other hand, are projected to remain relatively constant as a share of GDP during the simulation period through 2067  While our long-term simulations do not account for pending or future federal policy changes that will result in changes to expenditures and revenues, an understanding of several recent federal policy changes related to taxes and health care are important to note because they present sources of uncertainty for the state and local government sector’s long-term fiscal outlook  In addition, as is the case in any model that is reliant on historical data to simulate a long-term outlook, other considerations, such as economic growth and rates of return on pension assets, could shift future fiscal outcomes  These policy changes and uncertainties are discussed below and may help federal policy makers and state and local governments consider how these changes could affect the long-term outlook  Recently enacted legislation, such as Public Law 115-97, commonly referred to by the President and administrative documents as the Tax Cuts and Jobs Act (TCJA), could affect the sector’s revenues over the long-term  Enacted in December 2017, TCJA included significant changes to corporate and individual tax law, with implications for state and local government tax collections  In particular, for individual taxpayers, for tax years 2018 through 2025, tax rates were lowered for nearly all income levels, some deductions from taxable income were changed (personal exemptions were eliminated, while the standard deduction was increased), and certain credits, such as the child tax credit, were expanded  The effect of TCJA on the long-term state and local fiscal outlook is still evolving, and will likely depend on how states incorporate the law’s changes into their state income tax rules  That is, because some states link their state income taxes to federal income tax rules, states must decide whether to let the changes from TCJA flow through to their state income tax systems, or establish new state income tax rules  For example, some states have adopted the federal definition of taxable income as a starting point for state tax calculations, while other states use the federal definition of adjusted gross income as a starting point  The choices states make to continue to link to these definitions could have long-term implications for their state tax revenues  In addition, under TCJA, the amount of the federal itemized deductions allowed for all state and local income, sales, and property taxes (commonly referred to as the state and local tax (SALT) deduction) is now capped at $10,000 for tax years 2018 to 2025  The magnitude or net effect of these changes is uncertain in that states are still working to understand the impact of the tax laws on their revenues  It remains to be seen whether and how states will see changes in their revenues in the future  Moreover, a recent U S  Supreme Court decision involving state sales taxes could have implications for states’ ability to collect revenue  Specifically, the court’s ruling in June 2018 in South Dakota v  Wayfair, Inc  held that states could require out-of-state sellers to collect and remit sales taxes on purchases made from those out-of-state sellers, even if the seller does not have a substantial physical presence in the taxing state  Prior to this ruling, a seller that did not have a substantial physical presence in a state could not be required to collect and remit a sales tax on goods sold into the state  Instead, a purchaser may have been required to pay a use tax (i e , a tax levied on the consumer for the privilege of use, ownership, or possession of taxable goods and services) in the same amount to his or her state government  In 2017, we reported that states could realize between an estimated $8 5 billion and $13 4 billion in additional state sales tax revenue across all states if all sellers were required to collect taxes on all remote sales at current rates  The extent to which states realize changes in sales tax revenue will likely depend on how they revise their state laws and enforcement efforts in response to this June 2018 ruling  Enacted health care legislation could also affect the long-term fiscal position of state and local governments  As we have reported in prior work, the effect of the Patient Protection and Affordable Care Act (PPACA) on the long-term state and local fiscal outlook could depend on how states implement PPACA, and on future rates of health care cost growth  For example, consider the states that have opted, under PPACA, to expand Medicaid program coverage to millions of lower income adults  While the federal government is expected to cover a large share of the costs of the Medicaid expansion, these states are ultimately expected to bear some of the costs  Specifically, the federal government reimbursed 100 percent of the costs of the expanded population beginning in 2014  This reimbursement rate will decline from the 2018 reimbursement rate of 94 percent to 90 percent by 2020  As such, the reduced federal reimbursement rate may affect those states that expanded their Medicaid populations in recent years  As discussed earlier in this report, our simulations suggest that Medicaid spending will make up an increasing share of the state and local government sector’s operating expenditures in the future  A weakening of the economy could add to the fiscal pressures states face in funding these Medicaid obligations  As our prior work has shown, past recessions in 2001 and 2007 hampered states’ ability to fund increased Medicaid enrollment and maintain their existing services  Specifically, Medicaid enrollment increased during these recessions, in part due to increased unemployment, which led more individuals to become eligible for the program  We have also reported on the use of Medicaid demonstrations, which allow states to test new approaches to coverage to improve quality and access, or generate savings or efficiencies  Specifically, CMS may waive certain Medicaid requirements and approve new types of expenditures that would not otherwise be eligible for federal Medicaid matching funds  For example, under demonstrations, states have extended coverage to certain populations, provided services not otherwise eligible for Medicaid, and made payments to providers to incentivize delivery system improvements  We previously reported that, as of November 2016, nearly three-quarters of states have CMS- approved demonstrations  In fiscal year 2015, federal spending under demonstrations represented a third of all Medicaid spending nationwide  We also reported that in 10 states, federal spending on demonstrations represented 75 percent or more of all federal spending on Medicaid  Joint financing of Medicaid is a fixture of this federal-state partnership  Demonstration waivers hold the potential for changing state Medicaid spending  However, as we have reported, these demonstrations are required, under HHS policy, to achieve budget neutrality and not raise costs for the federal government  In addition to federal tax- and health-related policy changes, a number of other factors could affect the state and local government sector’s long- term fiscal outlook  Specifically, we developed simulations using alternative assumptions of the growth of key model variables—which include economic growth, health care excess cost growth, and the rate of return on pension assets  We determined that changes in the growth projections of these key variables could affect the operating balance of state and local governments, thereby shifting future fiscal outcomes for the sector  Future trends in GDP growth could affect the state and local government sector’s fiscal outlook  Data from CBO and the Board of Trustees of the Federal Old-Age and Survivors Insurance and Federal Disability Insurance Trust Funds (OASDI Trustees) project real GDP to grow by 1 9 percent per year on average from 2018 through 2028, and by 2 1 percent per year on average after 2028, respectively  Using these projections, our simulations suggest that maintaining current policies would cause the sector’s expenditures to exceed its revenues and that the difference between revenues and expenditures would become increasingly negative during the next several decades  However, simulations we developed using the OASDI Trustees’ alternative projections of real GDP growth suggest that the difference between revenues and expenditures would expand before narrowing toward the end of the simulation period if real GDP were to grow at a faster rate—2 8 percent per year on average—as shown in figure 9  Our simulations also show that if GDP were to grow at a slower rate—1 5 percent per year on average—the difference between revenues and expenditures would expand  This would result in an increasingly negative operating balance during the simulation period  As discussed earlier in this report, excess cost growth in health care is another key determinant of the sector’s fiscal balance  Data from CBO project Medicaid spending per capita to grow about 1 5 percent faster than GDP per capita on average for the period from 2019 through 2028  Data from CMS project Medicaid spending per capita to grow about 0 6 percent faster on average for the period from 2029 through 2067  Data from CMS also project national health expenditures per capita to grow about 0 8 percent faster than GDP per capita for the period from 2018 through 2067  Using these projections, our simulations suggest that maintaining current policies will cause the sector’s expenditures to exceed its revenues, and that the difference between revenues and expenditures will become increasingly negative during the next several decades  However, simulations developed using alternative projections of excess cost growth in Medicaid and national health expenditures suggest that the difference between revenues and expenditures may be reduced but not eliminated within the simulation period if excess cost growth in health care is zero  In the scenario where excess cost growth rises faster—0 7 percent on average for Medicaid for the period from 2029 through 2067 and 1 percent for national health expenditures for the period from 2018 through 2067—our simulations show that the difference between revenues and expenditures will persist for the remainder of the simulation period (see figure 10)  The rate of return on pension assets could also affect the state and local government sector’s fiscal outlook  Based on an inflation-adjusted rate of return on pension assets of 5 percent, our simulations suggest that state and local governments will need to make pension contributions equivalent to about 12 9 percent of their wages and salaries to meet their long-term pension obligations  However, this estimate is sensitive to the rate of return on state and local governments’ pension assets  Simulations we developed using a higher rate of return—7 5 percent—suggest that pension contributions needed to meet pension obligations would be about 3 percent of state and local government employees’ wages and salaries  In addition, under this scenario, our simulations suggest that the difference between revenues and expenditures will be reduced, but not eliminated within the simulation period  Alternatively, we estimated that if the rate of return on pension assets is relatively low—at 2 5 percent— required pension contributions would need to be about 23 percent of state and local government employees’ wages and salaries during the simulation period  Under this scenario, our simulations show that the sector’s negative operating balance will continue to grow larger throughout the simulation period  It follows therefore, that high rates of return on pension assets are associated with an improved outlook for state and local governments, and vice versa (see figure 11)  This report was prepared under the direction of Michelle A  Sager, Director, Strategic Issues, who can be reached at (202) 512-6806 or sagerm@gao gov, and Oliver M  Richard, Director, Center for Economics, who can be reached at (202) 512-8424 or richardo@gao gov if there are any questions  GAO staff who made key contributions to this report are listed in appendix IV  To simulate measures of fiscal balance for the state and local government sector for the long term, we used aggregate data on the state and local government sector and national data on other variables from the following sources: Agency for Healthcare Research and Quality; Board of Governors of the Federal Reserve System; Board of Trustees of the Federal Old-Age and Survivors Insurance and Federal Disability Insurance Trust Funds (OASDI Trustees); Bureau of Economic Analysis (BEA); Bureau of Labor Statistics; Centers for Medicare & Medicaid Services (CMS); Congressional Budget Office (CBO); and Federal Reserve Bank of St  Louis  Our approach generally follows the approach used in GAO-08-317 and in subsequent updates of that report  Specifically, we developed a model that projects the levels of receipts and expenditures of the state and local government sector (henceforth, the sector) in future years based on current and historical spending and revenue patterns  We use table 3 3 of the National Income and Product Accounts (NIPA)—State and Local Government Current Receipts and Expenditures—prepared by BEA at the U S  Department of Commerce as an organizing framework for developing our model of the sector’s revenues and expenditures (see table 1)  In this table, current revenues are grouped in five main categories  Current tax receipts  These receipts are tax payments made by persons or businesses to state and local governments  They include income taxes, general sales taxes, property taxes, and excise taxes  Current taxes also include fees for motor vehicle licenses, drivers’ licenses, and business licenses  Social insurance contributions  These contributions finance the provision of certain social benefits to qualified persons, and include contributions from employers and employees for temporary disability insurance, worker’s compensation insurance, and other programs  Income receipts from government assets  These receipts include interest, dividends, and rental income, such as royalties paid on drilling on the outer continental shelf  Also, state and local governments earn interest and dividend income on financial assets  Current transfer receipts  Transfer receipts are receipts for which state and local governments provide nothing of value in return  Current transfer receipts include federal grants, fines, fees, donations, and tobacco settlements  Also included are net insurance settlements, certain penalty taxes, court fees, and other miscellaneous transfers  Current surplus of government enterprises  This surplus is a profit- type measure for state and local government enterprises, such as water, sewer, gas, and electricity providers; toll providers; liquor stores; air and water terminals; public transit; and state lotteries  Some types of enterprises, such as state lotteries, consistently earn surpluses which are used to fund general government activities  In contrast, many enterprises run deficits, which, in turn, reduce receipts  State and local governments also receive income from the sale of goods and services, such as school tuition  In the NIPAs, this income is treated as an offset against expenditures, not revenue  This income comes from voluntary purchases that might have been made from a private sector provider of such services  In addition to current receipts, state and local governments receive capital transfer receipts  These receipts include estate and gift taxes, and federal government investment grants for capital such as highways, transit, air transportation, and water treatment plants  State and local government current expenditures are grouped into four main categories  Consumption expenditures  Generally, spending for which some value is provided in return  State and local government consumption spending is the sum of inputs used to provide goods and services, including compensation of general government employees, consumption of general government fixed capital (depreciation), and intermediate goods and services purchased, less sales to other sectors and own-account investment  Current transfer payments  Payments for which nothing of value is provided in return  For state and local governments, current transfer payments consist primarily of social benefits, which are payments to persons to provide for needs that arise from circumstances such as sickness, unemployment, retirement, and poverty  There are two kinds of social benefits—benefits from social insurance funds, such as temporary disability insurance and workers’ compensation, and other social benefits, such as medical benefits from Medicaid and the state Children’s Health Insurance Program (CHIP), family assistance from Temporary Assistance to Needy Families, education assistance, and other public assistance programs  While NIPA table 3 3 also includes other current transfer payments to the rest of the world as part of current transfer payments, these amounts are generally equal to zero  Interest payments  These include actual and imputed interest and represent the cost of borrowing by state and local governments to finance their capital and operational costs  Subsidies  State and local government subsidies are largely payments to railroads  State and local government spending also includes gross investment, capital transfer payments, and net purchases of nonproduced assets  Gross investment is spending on capital goods like structures, equipment, and intellectual property—items that are called fixed assets or capital because of their repeated or continuous use in providing government services for more than 1 year  Structures include residential and commercial buildings, highways and streets, sewer systems, and water systems  State and local government capital transfer payments include disaster-related insurance benefits paid to the U S  territories and the Commonwealths of Puerto Rico and Northern Mariana Islands  Net purchases of nonproduced assets are composed of net purchases of land less oil bonuses (payments to states for the long-term rights to extract oil)  Our main indicator of the sector’s fiscal balance is its operating balance net of funds for capital expenditures (henceforth, operating balance), which is a measure of the sector’s ability to cover its current expenditures out of current revenues  The operating balance is defined as total receipts minus (1) capital outlays not financed by medium- and long-term debt issuance, (2) total current expenditures less depreciation, (3) current surplus of state and local government enterprises, and (4) net social insurance fund balance  Alternative indicators of fiscal balance include net saving and net lending or borrowing  Net saving is the difference between current receipts and current expenditures  Since current expenditures exclude capital investment but include a depreciation measure, net saving can be thought of as a measure of the extent to which governments are covering their current operations from current receipts  Net lending or borrowing is the difference between total receipts and total expenditures, and is analogous to the federal unified surplus or deficit  Total receipts differ from current receipts because they include capital transfer receipts  Total expenditures differ from current expenditures because they include capital investment, capital transfer payments, and net purchases of nonproduced assets  However, they exclude fixed capital consumption  The former three categories are cash expenditures, while the latter is a noncash charge  Net lending or net borrowing represents the governments’ cash surplus or borrowing requirement  This measure is normally negative because governments borrow to finance their capital investment (and sometimes to finance current operations as well)  The following equations describe how we simulated state and local government receipts and expenditures, as well as the intermediate variables used in those simulations  For this update, we started with historical data for 2017, or the most recent year available, and then simulated each variable for each year from 2018 through 2092 (the simulation period)  To simulate state and local government receipts and expenditures, we use simulations of various national-level demographic, macroeconomic, and health care variables derived from projections produced by CBO, CMS, and the OASDI Trustees, and otherwise derived using our own assumptions (see table 2)  This approach is similar to the approach we have used in prior model updates  To simulate state and local government spending on defined benefit pensions, we first estimate the contribution rate (as a fraction of state and local government general government wages and salaries) that state and local governments would need to make each year going forward to ensure that their pension systems are fully funded on an ongoing basis  Our goal is to estimate the financial commitments to employees that have been and are likely to continue to be made by the state and local sector to better understand the full fiscal outlook for the sector  As such, our analysis projects the liabilities that the sector is likely to continue to incur in the future based on simulations of future numbers of retirees receiving pension benefits and their benefit amounts; future numbers of employees, their wages and salaries, and their pension contributions; and assets in state and local government defined benefit pension funds  Although we are only interested in applying contribution rates over the simulation time frame, we actually have to derive the contribution rate for a longer time frame in order to find the steady-state level of necessary contributions
Report 2:  NHTSA began implementing a mandated 2-year pilot grant program intended to evaluate the feasibility and effectiveness of informing consumers about open auto recalls during state vehicle registration  In September 2016, NHTSA solicited applications to participate in the program, wherein selected states would inform consumers—at no charge—about open recalls using all means that permit consumers to register vehicles within the state (e g , in person, Internet, and mail)  According to NHTSA, only one state applied for the grant  In September 2017, NHTSA awarded the sole applicant $223,000  Under the program, the grantee needs to collect and report program performance data, including the extent to which open recalls have been identified and repaired  In addition, the grantee must report whether certain notification means were more effective than others and what could be done to improve the program  Upon completion of the pilot program, NHTSA is required to evaluate the extent to which open recalls identified have been remedied  Auto manufacturers we met with were generally supportive of the program  Specifically, representatives from 9 of the 10 manufacturers told us that notifying consumers about open recalls during vehicle registration can raise consumer awareness or improve recall completion rates  In September 2016, NHTSA issued a Notice of Proposed Rulemaking (NPRM), which proposes to require auto manufacturers to notify consumers about open recalls by electronic means—such as e-mails, phone calls, and text messages, in addition to First-Class Mail  As we described earlier, auto manufacturers are currently required to notify consumers about safety recalls affecting their vehicles via First-Class Mail  According to NHTSA, the NPRM aims to aid in efficiently and effectively improving recall completion rates, by proposing that manufacturers provide notification using electronic means in addition to First-Class Mail  Consumers in our focus groups as well as auto manufacturers and consumer associations we interviewed generally supported additional notification using electronic means  Consumers in our focus groups: As we discussed earlier, 69 of the consumers in our focus group discussion sessions reported they would prefer to receive additional notification by at least one type of electronic means  However, only 7 consumers actually received such notifications—suggesting a gap between industry notification practices and notification preferences for these consumers  Auto manufacturers: Representatives from 9 of 10 manufacturers we interviewed told us they generally support providing notification using electronic means  Although the NPRM proposes a broad definition of electronic means to give manufacturers flexibility to determine the most effective means, these representatives also shared implementation concerns  For example, representatives from 1 of the 9 manufacturers told us that—although the company collects e- mail addresses from some customers for other purposes—not all customers provide e-mail addresses, and those collected are not always accurate  As we discussed previously, most manufacturers we met with currently use supplemental electronic means notification on a case-by-case basis  Consumer associations: Similarly, both consumer associations we interviewed told us additional electronic notification can help reach consumers who do not complete repairs after receiving initial mailed notification  NHTSA’s proposal would maintain manufacturer reporting requirements, though it may result in additional reporting  This additional information could help the agency evaluate the effectiveness of various means of consumer notification  We previously found that NHTSA may be able to use manufacturers’ data to identify what factors make some recalls more or less successful than others  We recommended that NHTSA use the recall data it collects to analyze particular patterns or trends that may characterize successful recalls and determine whether these factors represent best practices  If the NPRM is finalized, manufacturers would provide NHTSA with representative copies of the newly required electronic notifications, in addition to mailed notifications, and would specify the electronic means used, such as e-mail or text message  According to NHTSA officials, this information could allow the agency to track and evaluate the effectiveness of various notification means used by manufacturers by, for example, comparing completion rates across means—a key step in identifying best practices that could encourage consumers to complete repairs  However, it is too early for NHTSA to conduct such an evaluation, because the agency has not issued a final rule  NHTSA officials told us the agency is working with the administration on NHTSA’s regulatory portfolio and priorities, including this rulemaking  NHTSA has also taken steps to collaborate with industry stakeholders and explore consumer education best practices  For example, in April 2015 NHTSA hosted a day-long workshop that brought together auto industry stakeholders to examine public education of the recall process  During the workshop, participants identified current barriers to the public’s awareness of auto recalls and discussed potential solutions to address them, such as using text messages and social media to communicate with younger consumers and using different delivery methods for recall notices  Similarly, in January 2016 NHTSA and 18 auto manufacturers adopted a set of Proactive Safety Principles to explore and employ new ways to increase safety recall participation rates  For example, NHTSA and auto manufacturers agreed to share industry best practices and policies based on lessons learned from ongoing safety recalls  The Independent Monitor of Takata in conjunction with NHTSA has also issued a set of coordinated communications recommendations based on consumer research, best practices observed during the Takata recall, and discussions with manufacturers  For example, the recommendations encourage manufacturers to: pursue a “multi-touch” communications strategy that employs non- traditional means, such as e-mail and text messages; convey risk in clear, accurate and urgent terms; and include a clear “call to action” designed to facilitate prompt and efficient scheduling of repairs  According to NHTSA officials, the agency relies on auto manufacturers to evaluate the effectiveness of these efforts  However, agency officials told us NHTSA reviews manufacturers communication plans as part of the Takata recall’s Coordinated Remedy Program and provides ongoing recommendations on manufacturers’ communication language, approach, and strategies  With the recent steep increase in safety defect vehicle recalls and continued low recall completion rates, it is vital for consumers to be able to easily access and use publicly available auto recall information  NHTSA has taken important steps to improve its website—which provides safety recall information to consumers—resulting in most consumers in our focus groups finding the website easy to use  However, the difficulties some experienced in attempting to complete essential auto recall tasks demonstrated that NHTSA gov does not always reflect key website usability practices for website design  Although NHTSA plans to conduct a website usability study with consumers after consolidating its websites, it has not determined a completion date for this effort—an essential step for organizations to effectively guide their information technology efforts  Without such a date, the website usability difficulties may persist and limit the effectiveness of NHTSA gov in providing consumers with recall information about their vehicles  By addressing these difficulties in the interim, NHTSA can better assure that consumers obtain this information, which can be vital to their safety  We are making the following two recommendations to NHTSA: The Administrator of NHTSA should determine a completion date for the agency’s website consolidation effort  (Recommendation 1) The Administrator of NHTSA should, while the agency continues its website consolidation effort, take interim steps to improve the usability of the auto recall areas of NHTSA gov by addressing the website usability difficulties we identified  (Recommendation 2) We provided a draft of this report to DOT for review and comment  In its written comments, reproduced in appendix VI, DOT stated that it concurred with our recommendations  The department also provided technical comments, which we incorporated as appropriate  We are sending copies of this report to relevant congressional committees, the Secretary of Transportation, and the Administrator of NHTSA  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2834 or flemings@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix VII  This report examines the use of publicly available auto recall information for safety defects affecting passenger vehicles  The report addresses the following objectives: (1) How do consumers and industry stakeholders use publicly available auto recall information? (2) How easy or difficult to use do consumers find the auto recall areas of NHTSA gov? (3) What steps, if any, has the National Highway Traffic Safety Administration (NHTSA) taken to raise consumer awareness about auto recalls and how has NHTSA evaluated the effectiveness of these steps? We define publicly available auto recall information to include information on the auto recall areas of NHTSA gov, such as examples of notification letters that manufacturers mail to consumers  This report focuses on safety defect vehicle recalls affecting passenger vehicles that are initiated when a defect in a vehicle or vehicle equipment creates an unreasonable safety risk, as determined by NHTSA or a manufacturer  To determine how consumers use publicly available auto recall information, we conducted and analyzed transcripts and questionnaires from 12 consumer focus groups we conducted with used and new vehicle owners who had experienced an auto recall in the last 24 months  Each focus group was split into two sessions: (1) a discussion session to explore participants’ thoughts, experiences, and preferences about auto recall information and (2) a website usability testing session  Also, we administered a questionnaire as part of each of these sessions  For the discussion session, we asked consumers about the recall notification process and how they used the recall information, and for the website usability testing session, we asked consumers to fill in a questionnaire during the session itself as they assessed the usability of the auto recall areas of NHTSA’s website  We conducted the 12 focus groups at six locations across the country, with each group including 7 or 8 consumers for a total of 94 participants  Half of the focus groups were comprised of consumers who had completed the repair and the remaining half included consumers who had not completed the repair  We selected the six focus group locations to provide population and geographic dispersion  To ensure geographic dispersion, we selected at least one location in each U S  Census region (see table 3)  To ensure population dispersion, we selected Metropolitan Statistical Areas representing a range of population sizes based on 2015 U S  Census estimates  To ensure our selection included the perspectives of vehicle owners in geographically distant or isolated communities, we also selected a rural location, which we defined as a city or town that has a population of less than 50,000 inhabitants and is not an urbanized area contiguous and adjacent to a city or town that has a population of greater than 50,000 inhabitants  Using information provided by the participants, we selected focus group participants based on age, income, gender, education level, race, and ethnicity to ensure we collected a range of perspectives on auto recall information use  However, since we did not select a representative sample of participants, focus group results are not generalizable to all vehicle owners  During focus group discussion sessions, we asked participants to discuss factors they considered when deciding whether to repair their recalled vehicle and then to select the single most influential factor  Each of the 12 focus group sessions was audio recorded and transcriptions were created; transcripts served as the record for each group  We then evaluated those transcripts using systematic content analysis to identify the factors consumers considered when deciding whether to complete repairs and any suggested improvements to the auto recall communication process  The analysis was conducted in three steps  First, two analysts independently developed a code framework and then worked together to resolve any discrepancies  Second, each transcript was coded independently by analysts using the framework and any discrepancies were resolved by both analysts agreeing on the coding of the associated statement by a participant  Third, if needed, another analyst adjudicated any continued disagreement between coders  Because the transcripts did not include a unique identifier for each focus group participant, we conducted our analysis of focus group session discussions at the group level (i e , of the 12 focus groups we conducted)  We also administered and analyzed a questionnaire as part of each discussion session to quantify responses regarding consumers’ use of auto recall information, including how they received and preferred to receive auto recall notifications  Our analysis of the questionnaire responses was conducted at the individual consumer level (i e , of the 94 consumers who participated)  These focus group sessions were structured, guided by a moderator who used a standardized list of questions to encourage participants to share their thoughts, experiences, and preferences  We also conducted two pretest focus groups at our headquarters and made some revisions to the focus group guide prior to beginning the sessions with consumers  Methodologically, focus groups are not designed to demonstrate the extent of a problem or to generalize results to a larger population or provide statistically representative samples or reliable quantitative estimates  Instead, they are intended to generate in-depth information about the reasons for the focus group participants’ thoughts, experiences, and preferences on specific topics  The projectability of the information produced by our focus group sessions is limited  For example, the information includes only the responses from the vehicle owners from the 12 selected groups and their individual responses to questions we asked  The experiences and preferences expressed may not reflect other vehicle owners’ thoughts and preferences  In addition, while the composition of the groups was designed to ensure a range of age and education levels, among the other criteria mentioned previously, the groups were not constructed using a random sampling method  To determine how industry stakeholders use auto recall information, we interviewed selected auto manufacturers, selected franchised and independent auto dealerships, NHTSA program officials, and other industry stakeholders  Specifically, we interviewed representatives from the following 10 auto manufacturers, selected based on each manufacturer’s sales market share (small, medium, and large), place of ownership (foreign and domestic), and experience with auto recalls (lower to higher based on the average annual number of auto recall campaigns and average market share of each manufacturer from 2010 to 2014) to collect a range of perspectives on how manufacturers use auto recall information: Tesla Motors, Inc  To understand the perspective of auto dealers, we interviewed four franchised dealerships, one in each of the four U S  Census regions where we conducted focus groups with consumers  We also interviewed three independent auto dealerships in two U S  Census regions  The results of these interviews are not generalizable to all auto manufacturers and dealerships, but provide insights about how some industry stakeholders use auto recall information  We conducted interviews with NHTSA program officials to understand NHTSA’s role in the auto recall process  In addition, we interviewed other stakeholders, including the Independent Monitor of Takata, which assists NHTSA in overseeing the Takata recall, as well as officials from consumer associations and other industry groups (see table 4)  To evaluate how easy or difficult consumers find the auto recall areas of NHTSA gov to use, we reviewed various website usability resources to understand federal and industry key website usability practices for making websites easy to use, such as focusing on design and how easily users can find information  In addition, we reviewed federal standards for internal control related to communicating quality information externally  During our usability testing sessions, we asked consumers to attempt to complete auto recall tasks—the primary means NHTSA gov provides for consumers to access information about auto recalls affecting their vehicles—and discuss their experiences  We then compared consumers’ experiences with the usability of the website against these practices  To identify key website usability practices, we analyzed guidance documents from NHTSA and other federal agencies  For example, we analyzed the General Services Administration’s (GSA) and the Department of Health and Human Services’ Research-Based Web Design & Usability Guidelines, which includes quantified, peer-reviewed guidelines intended to help federal agencies improve the design and usability of their information-based websites  We also analyzed GSA’s Requirements for Federal Websites and Digital Services, and the U S  Digital Services Playbook to identify key practices for making websites easy to use  Identified key practices are: (1) design and content— focusing on the layout, headers, and design; (2) navigation—how easily users can find information; (3) clarity—the ability to read and digest content; (4) identity and purpose—whether the site clearly presents its purpose; and (5) accessibility—the ability of people with physical or mental disabilities to use the site  To analyze the results of focus group website testing sessions, we performed a systematic content analysis of the session transcripts using the same content analysis methods described above and an analysis of the questionnaire we administered to each participant during the website usability sessions  Specifically, we analyzed the transcripts from the website usability testing sessions to account for consumers’ experiences, including their initial impressions of the website and any suggested usability improvements  We also analyzed the results of the questionnaire that each participant completed where participants were asked to mark responses regarding their experience including an assessment of the usability of the auto recall areas of NHTSA gov  Our analysis of the results from the questionnaire responses was conducted at the individual consumer level (i e , of the 94 consumers who participated) while our analysis of focus group session discussions was conducted at the group level (i e , of the 12 focus groups we conducted)  To corroborate the results of usability testing sessions we conducted with the consumers in our focus groups, we requested that five website usability professionals from GSA’s Federal User Experience Community conduct an independent evaluation of the auto recall areas of NHTSA gov against federal and industry key website usability practices (described above)  The website usability professionals developed a website usability evaluation form, which they used to individually evaluate the auto recall areas of NHTSA’s website  The website usability professionals then met to form a consensus and provided us with one final group evaluation of the website usability of the auto recall areas of NHTSA gov  Also, although neither our usability testing nor the website usability evaluation conducted by website usability professionals directly addressed accessibility, we interviewed responsible agency officials about how the agency assesses the accessibility of NHTSA gov  We also requested and analyzed website data provided by NHTSA to understand how consumers access and use NHTSA gov  Requested data included the number of subscribers to NHTSA’s Recall Notification E-mail System Sign-up; the number of weekly vehicle identification number (VIN) searches performed on NHTSA gov from August 2014 through May 2017; and NHTSA gov usage data by device (i e , usage by mobile devices, tablets, and desktop computers)  We assessed the reliability of these data by reviewing any supporting documents provided by the agency and interviewing responsible NHTSA officials, and concluded the data were sufficiently reliable for our reporting purposes  While we did not independently review the usability of auto manufacturers’ auto recall websites, we requested and reviewed the results of any audits that NHTSA performed of these websites, including whether the websites met statutory and regulatory requirements for providing auto recall information to the public  We then corroborated any audit findings by reviewing the auto recall websites of the selected auto manufacturers that we interviewed  To determine any steps NHTSA has taken to raise consumer awareness about auto recalls and how NHTSA evaluates the effectiveness of any steps, we reviewed relevant statutes, regulations, and proposed rules, including the Fixing America’s Surface Transportation Act and a Notice of Proposed Rulemaking related to recall notification methods  We also reviewed agency and other documents that describe or evaluate NHTSA’s public awareness activities  For example, we analyzed NHTSA’s strategic planning documents—such as NHTSA’s Strategic Plan 2016–2020—to identify ongoing public awareness activities along with their related goals, objectives, or performance metrics  Similarly, we requested and analyzed any documents NHTSA uses to evaluate the effectiveness of its public awareness activities, including performance reports for NHTSA’s ongoing Safe Cars Save Lives campaign  To assess the reliability of data included in these performance reports— such as VIN searches performed—we reviewed agency documentation and interviewed agency officials about the reliability, accuracy, and completeness of the data and determined the data were sufficiently reliable for our reporting purposes  We reviewed performance management practices as provided in the Government Performance and Results Act of 1993 (GPRA), the GPRA Modernization Act of 2010, and standards for internal control in the federal government to identify any opportunities for improvement  We also performed a literature review to identify any related published articles and research studies  To understand how NHTSA implements and evaluates any public awareness activities, we also interviewed responsible agency officials from NHTSA’s Office of Communications and Consumer Information and other offices  In addition, we discussed NHTSA’s public awareness efforts during interviews with industry stakeholders, including selected auto manufacturers, selected franchised and independent auto dealerships, and other industry stakeholders  We analyzed the results of these interviews along with the focus group discussions we conducted with consumers (discussed above) to identify perspectives on the effectiveness of NHTSA’s public awareness steps  We conducted this performance audit from October 2016 to December 2017 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  The National Highway Traffic Safety Administration (NHTSA) is required to conduct a biennial analysis of vehicle safety recall completion rates and submit the results of its analysis in a report to certain congressional committees  The report must include, among other things, the annual recall completion rate by vehicle type and vehicle component (such as brakes, fuel systems, and air bags) for each of the 5 years preceding the year the report is submitted  According to NHTSA’s May 2017 report, completion rates for all vehicles combined ranged between 63 percent and 67 percent between calendar year 2011 and calendar year 2014 (see table 5)  However, NHTSA reported wider variation when the recall completion rates are broken down by vehicle type  Similarly, the report found that completion rates for most component categories fall within a range of 60 percent to 75 percent (see table 6)  The annual completion rate is a volume-based, weighted metric, such that the more vehicles affected by the recall, the more weight or influence it has on the computed rate  Focus group participants responded to a questionnaire we administered to collect information on consumers’ auto recall notification preferences during our discussion sessions  Table 7 shows participants’ responses to the administered questionnaire, by age group  We present these responses by age group, because consumers’ notification preferences may vary according to their ages  Focus group participants responded to a questionnaire we administered to collect information on the usability of NHTSA gov during our usability testing sessions  Table 8 shows focus group participants’ responses to the administered questionnaire, by age group  We present these responses by age group, because consumers’ website usability needs or preferences may vary according to their ages  In addition to the individual named above, H  Brandon Haller (Assistant Director); Katherine Blair; Jason Blake; Melissa Bodeau; Alicia Cackley; William Colwell (Analyst in Charge); Lacey Coppage; Elizabeth Dretsch; Jaci Evans; Marcia Fernandez; Sarah Kaczmarek; Malika Rice; Todd Schartung; and Andrew Stavisky made key contributions to this report 

ground_truth: To help address long-standing financial management system deficiencies, DHS initiated its TRIO project, which has focused on migrating three of its components to a modernized financial management system provided by IBC, an OMB-designated, federal SSP. House Report Number 3128 included a provision for GAO to assess the risks of DHS using IBC in connection with its modernization efforts. This report examines (1) the extent to which DHS and the TRIO components followed best practices in analyzing alternatives, and the key factors, metrics, and processes used in their choice of a modernized financial management system; (2) the extent to which DHS managed the risks of using IBC for its TRIO project consistent with risk management best practices; and (3) the key factors and challenges that have impacted the TRIO project and DHS's plans for completing remaining key priorities. GAO interviewed key officials, reviewed relevant documents, and determined whether DHS followed best practices identified by GAO as necessary characteristics of a reliable, high-quality AOA process and other risk management best practices. The Department of Homeland Security's (DHS) TRIO project represents a key effort to address long-standing financial management system deficiencies. During 2012 and 2013, the TRIO components—U.S. Coast Guard (Coast Guard), Transportation Security Administration (TSA), and Domestic Nuclear Detection Office (DNDO)—each completed an alternatives analysis (AA) to determine a preferred alternative for modernizing its financial management system. GAO found that DNDO's AA substantially met the four characteristics—well-documented, comprehensive, unbiased, and credible—that GAO previously identified for a reliable, high-quality analysis of alternatives (AOA) process. However, Coast Guard's and TSA's AAs did not fully or substantially meet three of these characteristics, and DHS guidance for conducting AAs did not substantially incorporate certain best practices, such as identifying significant risks and mitigation strategies and performing an independent review to help validate the AOA process. Based on these analyses and other factors, the TRIO components determined that migrating to a federal shared service provider (SSP) represented the best alternative, and in 2014, DHS selected the Department of the Interior's Interior Business Center (IBC) as the federal SSP for the project. However, because Coast Guard's and TSA's AAs did not fully or substantially reflect all of the characteristics noted above, they are at increased risk that the alternative selected may not achieve mission needs. DHS also did not fully follow best practices for managing project risks related to its use of IBC on the TRIO project. Specifically, DHS followed three of seven risk management best practices, such as determining risk sources and categories and establishing a risk management strategy. However, it did not fully follow four best practices for defining risk parameters, identifying risks, developing risk mitigation plans, and implementing these plans largely because its guidance did not sufficiently address these best practices. For example, although DHS created joint teams with IBC and provided additional resources to IBC to help address risk mitigation concerns, it did not always develop sufficiently detailed risk mitigation plans that also included contingency plans for selected critical risks. As a result, although IBC's capacity and experience for migrating large agencies the size of Coast Guard and TSA was identified as a risk in July 2014, a contingency plan working group to address this concern was not established until January 2017. By not fully following risk management best practices, DHS is at increased risk that potential problems may not be identified or properly mitigated. DHS, IBC, Office of Management and Budget (OMB), and other federal oversight agencies identified various challenges that have impacted the TRIO project and contributed to a 2-year delay in the implementation of Coast Guard's and TSA's modernized solutions. These challenges include the lack of sufficient resources, aggressive schedule, complex requirements, increased costs, and project management and communication concerns. To help address these challenges, DHS and IBC established review teams and have taken other steps to assess potential mitigating steps. In May 2017, DHS determined that migrating the solution from IBC to a DHS data center represented the best option and initiated discovery efforts to further assess this as its path forward for the TRIO project. GAO recommends that DHS more fully follow best practices for conducting an AOA process and managing risks. DHS concurred with GAO's recommendations and described actions it will take, or has taken, in response.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: Since DHS’s creation in 2003, significant internal control and financial management system deficiencies have hampered its ability to reasonably assure effective financial management and to manage operations  These deficiencies contributed to our decision to designate DHS’s management functions, including financial management, as high risk  To help address these deficiencies, DHS initiated a decentralized approach to upgrade or replace legacy financial management systems and has been evaluating various options for modernizing them, including the use of SSPs  DHS initiated three projects for modernizing the systems of selected DHS components, including its TRIO modernization project  The TRIO project has focused on migrating the financial management systems of Coast Guard, DNDO, and TSA to a modernized solution provided by IBC  DHS’s efforts to effectively assess and manage risks associated with this project are essential to DHS’s realizing its modernization goals  In 2013, OMB issued a memorandum directing agencies to consider federal SSPs as part of their AAs  Also, in May 2014, Treasury and OMB designated IBC as one of four federal SSPs for financial management to provide core accounting and other services to federal agencies  This designation was based on Treasury and OMB’s evaluation of the four service providers’ ability to assist federal agencies in meeting their accounting and financial management needs, including experience with implementing financial management systems and providing other financial management services to customers, cost of services provided, compliance with financial management and internal control requirements, commitment to shared services, capacity, and long-term growth strategy  FIT’s responsibilities related to the governance and oversight of federal SSPs were subsequently transferred to USSM after USSM was established in October 2015  Because of concerns that its Core Accounting System (CAS) Suite was outdated, inefficient, and did not reliably meet requirements, Coast Guard completed an AA in January 2012 to assist in developing a path forward for modernizing its financial management system  In August 2012, Coast Guard established its CAS Replacement project team to further evaluate two of the alternatives considered in its AA and develop a recommended course of action  In addition, Coast Guard determined that hosting, owning, operating, and managing a financial management system were not among its core competencies  Because TSA and DNDO also relied on CAS as their primary accounting system, they also conducted AAs to identify the best alternative for transitioning to a modernized financial management system solution  The AAs conducted by the TRIO components during 2012 and 2013 considered the use of federal and commercial SSPs and other options  In addition, Coast Guard completed additional market research including further analysis of commercial SSPs in June 2013  In July 2013, the TRIO components determined that migrating to a federal SSP was the best course of action and subsequently conducted discovery phase efforts with IBC from November 2013 through May 2014 to further explore the functional requirements for procurement, asset, and financial management services  Based on these efforts, in July 2014, the TRIO components recommended that they proceed with implementation of the IBC shared services solution  In August 2014, FIT and OMB concurred with this recommendation, and DHS entered into an interagency agreement (IAA) with IBC for implementation  Figure 1 shows a timeline of these key events  The IAA for implementation and related performance work statement included a description of the services that IBC is to provide and the roles and responsibilities of DHS, the TRIO components, and IBC  The IAA also required IBC to prepare a detailed project management plan describing how the requirements would be managed and updated and an integrated master schedule (IMS) for identifying tasks to be completed, duration, percentage completed, dependencies, critical path, and milestones  According to the February 2015 project management plan, DNDO, TSA, and Coast Guard were expected to go-live on the IBC solution in the first quarter of fiscal years 2016, 2017, and 2018, respectively  However, in May 2016, DHS and IBC determined that TSA’s and Coast Guard’s planned implementation dates were not viable because of various challenges impacting the TRIO project and recommended a 1-year delay for their respective implementation dates  Figure 2 summarizes planned and completed key implementation events for the TRIO project as of May 2016  GAO, SEI, and other entities have developed and identified best practices to help guide organizations in effectively planning and managing various activities, including acquisitions of major information technology systems  These include GAO’s identified best practices for the AOA process and best practices identified by SEI for risk management  GAO-identified best practices for AOA process  GAO identified 22 best practices for a reliable, high-quality AOA process that can be applied to a wide range of activities in which an alternative must be selected from a set of possible options, as well as to a broad range of capability areas, projects, and programs  These practices can provide a framework to help ensure that entities consistently and reliably select the project alternative that best meets mission needs  Not conforming to these best practices may lead to an unreliable process, and the entity will lack assurance that the preferred alternative best meets the mission needs  Appendix II provides additional details on GAO’s identified AOA process best practices and how they can be applied to a wide range of activities in which an alternative must be selected from a set of possible options, as well as to a broad range of capability areas, projects, and programs  SEI’s risk management practices  SEI’s practices for the risk management process area call for the identification of potential problems before they occur so that risk-handling activities can be planned throughout the life of a project to mitigate adverse impacts on achieving objectives  These practices are determining risk sources and categories, defining parameters used to analyze and categorize risks and to control the risk management effort, establishing and maintaining the strategy to be used for risk identifying and documenting risks, evaluating and categorizing each identified risk using defined risk categories and parameters and determining its relative priority, developing a risk mitigation plan in accordance with the risk monitoring the status of each risk periodically and implementing the risk mitigation plan as appropriate  Although the TRIO components conducted AAs to identify the preferred alternative for modernizing their financial management systems, their efforts did not always follow best practices  For example, Coast Guard’s and TSA’s AAs supporting their selection of migrating to a federal SSP for modernizing their financial management systems did not fully or substantially meet all four characteristics of a reliable, high-quality AOA process  In addition, we found that DHS guidance did not fully or substantially incorporate five of GAO’s identified best practices for conducting an AOA process  The TRIO components’ AAs included descriptions of the key factors, such as scores for each alternative against the selection criteria used to assess it  Based on these AAs, DHS and the TRIO components selected the federal SSP alternative as their preferred choice and subsequently selected IBC as their federal SSP  However, because Coast Guard’s and TSA’s AAs did not fully or substantially meet all four characteristics of a reliable, high-quality AOA process, they are at increased risk regarding their decision on the solution that represents the best alternative for meeting their mission needs  Based on the extent to which the DHS TRIO components followed the GAO-identified 22 best practices for conducting an AOA process, we found that DNDO’s AA process substantially met the four characteristics of a reliable, high-quality AOA process while the Coast Guard and TSA AA processes both substantially met one and partially met three of these four characteristics  For example, we found that TSA’s AA partially met the “well-documented” characteristic, in part, because risk mitigation strategies, assumptions, and constraints associated with each alternative were not discussed in its AA  In addition, we found that Coast Guard’s AA partially met the “credible” characteristic, in part, because there was no indication that it contained sensitivity analyses, an evaluation of the impact of changing assumptions on its overall costs or benefits analyses  Our overall assessment is summarized in table 1  Appendix III provides additional details on our assessment of the TRIO components’ AAs for each of the GAO-identified 22 AOA best practices  Further, in comparing DHS AOA and AA guidance to the GAO-identified 22 AOA process best practices, we found that although DHS’s guidance for conducting both AOAs and AAs fully or substantially incorporated 17 of the identified best practices, the guidance did not fully or substantially incorporate 5 of these practices  For example, although the guidance addressed risk management in general terms, it did not detail the need to document risk mitigation strategies for each alternative  Not documenting the risks and related mitigation strategies for each alternative prevents decision makers from performing a meaningful trade-off analysis necessary to choose a recommended alternative  In addition, while DHS guidance describes the need for an AA or AOA review, it describes reviews conducted within the organizational chain of command and does not address the need for an independent review—one of the most reliable means to validate an AOA process  Further, although the guidance noted that weights for selection criteria may become more subjective when they cannot be derived analytically, additional guidance on weighting selection criteria was limited  Our overall assessment is summarized in table 2  Because of these limitations in guidance, and because Coast Guard and TSA did not fully adhere to the GAO-identified best practices, Coast Guard’s and TSA’s AAs did not fully or substantially reflect all four characteristics of a reliable, high-quality AOA process  As a result, Coast Guard and TSA increased their risk of selecting a solution that may not represent the best alternative for meeting their mission needs  Documentation supporting TRIO components’ AA efforts included descriptions of the key factors, metrics, and processes involved in conducting their analyses, including the (1) alternatives considered, (2) market research conducted, (3) three alternatives evaluated, (4) selection criteria used by each and how the criteria were weighted, (5) scores for each alternative against the selection criteria, and (6) alternatives that scored the best under the AOA evaluation  The TRIO components conducted market research to develop reasonable alternative solutions for consideration  For example, through its market research, TSA identified OMB-designated federal SSPs and commercial entities as potential alternatives for hosting and implementing a modernized and integrated financial management system  According to its AA, TSA was able to gain an understanding of the offerings, capabilities, and related costs associated with these alternatives through reviews of documentation and interviews  After developing a diverse range of financial system modernization alternatives for consideration, each of the TRIO components assessed them for viability using various factors—such as measures of effectiveness, cost, risk, and value—and identified the three top-rated alternatives for further evaluation  For example, Coast Guard identified nine alternatives for consideration and analyzed, scored, and ranked them to determine its top three alternatives for further analysis: incrementally improve the current CAS Suite and remove certain outdated components, host the financial management system internally using software and tools already owned, and use an SSP to host the financial management system  Each component identified its three alternatives for further evaluation and used defined selection criteria to rate them  For example, DNDO’s selection criteria included four categories of operational effectiveness that were weighted according to their level of importance  Based on their evaluations, each component identified the best alternative for its respective financial management system needs  According to Coast Guard’s November 2012 decision memorandum, Coast Guard further narrowed the alternatives it focused on to (1) using an SSP to host its financial management system and (2) hosting the system internally using already-owned software and tools, and it also gathered rough order of magnitude cost estimates for both alternatives  Based on its evaluation, Coast Guard determined that the two alternatives were comparable  According to this memorandum, Coast Guard further determined that owning, hosting, operating, and managing a financial management system were not among its core competencies  Based on this determination, OMB direction to agencies to use (with limited exceptions) shared services, and other factors, Coast Guard decided that migrating to an SSP was the best alternative  TSA found in its February 2013 analysis that the differences between federal and commercial SSP alternatives were not significant and, as a result, recommended that a competitive procurement be conducted to better evaluate each alternative  However, DHS officials told us that TSA subsequently determined that a competitive procurement was not warranted and chose to migrate to a federal SSP  This determination was based on additional OMB guidance issued in March 2013 directing agencies to consider federal SSPs as part of their AAs and stating that commercial SSPs are an appropriate solution and would be funded by OMB only in instances in which the agency’s business case demonstrates that a commercial SSP can provide a better value for the federal government  In addition, DNDO determined that migrating to a federal SSP was its best alternative in May 2013  Because its preliminary research focused primarily on the federal SSP marketplace, Coast Guard conducted additional market research to include a more robust analysis of commercial SSPs  Coast Guard’s June 2013 market research report described the results of this effort, including its evaluation of responses from 11 commercial SSPs  Coast Guard reported that none of the commercial SSPs that responded could meet all 44 specific financial management system requirements and the extent to which they could meet them varied significantly  Based on these results, Coast Guard determined that there was a lack of maturity in the commercial SSP market for federal financial management  According to the report, this overall assessment was based on various considerations of information provided by commercial SSP respondents, including the wide variety of proposed configurations, solutions, prices, and implementation schedules, the lack of federal experience and service for agency-wide capabilities, and insufficient length of service to establish positive trends in audit performance; the lack of similar offerings that implied a lack of strong competition between comparable products that would exert downward pressure on cost; and the lack of like product offerings, which increases the likelihood of higher switching costs in the case of poor performance because of increased difficulty in moving from one “turnkey” service to another  In July 2013, the TRIO components and DHS selected the federal SSP alternative as their preferred choice and subsequently selected IBC as their federal SSP  DHS officials told us that IBC was selected based on (1) DHS’s reliance on OMB and Treasury’s designation of IBC as a federal SSP, (2) OMB guidance to consider the use of federal SSPs, and (3) a review of the availability of the four federal SSPs indicating that IBC was the only one available to meet the requirements and implementation schedule at that time  In August 2013, DHS notified OMB that the TRIO components had performed extensive market research and finalized their respective AAs and independently concluded that migrating to a federal SSP was in the best interests of the government  Also, in August 2013, FIT notified OMB regarding the TRIO components’ AA efforts and that the TRIO components would proceed to the discovery phase with IBC  According to FIT’s notification memorandum to OMB, the TRIO components’ AAs demonstrated that migrating to a federal SSP was the best value to the federal government and that the components identified IBC as a suitable partner based on the results of their market research into federal SSPs  Risk management best practices call for the identification of potential problems before they occur so that risk-handling activities can be planned throughout the life of the project to mitigate adverse impacts on achieving objectives  These best practices involve (1) preparing for risk management, (2) identifying and analyzing risks, and (3) mitigating identified risks  Preparing for risk management involves determining risk sources and categories and developing risk mitigation techniques  Identifying and analyzing risks includes determining those that are associated with cost, schedule, and performance and evaluating identified risks using defined risk parameters  Mitigating risks includes determining the levels and thresholds at which a risk becomes unacceptable and triggers the execution of a risk mitigation plan or contingency plan; determining the costs and benefits of implementing the risk mitigation plan for each risk; monitoring risk status; and providing a method for tracking open risk-handling action items to closure  Based on our evaluation, we found that DHS processes generally reflected three of seven specific risk management best practices and partially reflected the remaining four practices  Table 3 summarizes the extent to which DHS followed these seven best practices for managing TRIO project risks  Additional details on DHS and TRIO component efforts to address these practices are summarized following this table  Prepare for risk management  Key aspects of processes established by DHS and TRIO components related to the three best practices associated with preparing for risk management: Determine risk sources and categories  This practice calls for a basis for systematically examining circumstances that affect the ability of the project to meet its objective and a mechanism for collecting and organizing risks  DHS and the TRIO components established processes that met this best practice  For example, DHS reviewed the integrated master schedule that IBC prepared to identify sources of risk and defined risk categories in TRIO project policies  Define risk parameters  Risk parameters are used to provide common and consistent criteria for comparing risks to be managed  The best practice includes defining criteria for evaluating and quantifying risk likelihood and severity levels and defining thresholds for each risk category to determine whether risk is acceptable or unacceptable and to trigger management action  DHS partially met this best practice  DHS’s risk management program defined rating scales to provide consistent criteria for evaluating and quantifying risk likelihood and severity levels  However, DHS’s Risk Management Planning Handbook and related template for developing risk management plans for projects did not address the need for thresholds relevant to each category of risk to facilitate review of performance metrics in order to determine when risks become unacceptable or to invoke selected risk-handling options when monitored risks exceed defined thresholds  Establish a risk management strategy  A risk management strategy addresses specific actions and the management approach used to apply and control the risk management program, including identifying sources of risk, the scheme used to categorize risks, and parameters used to evaluate and control risks for effective handling  DHS met this best practice  DHS and IBC established risk management policies and plans for the TRIO project based on DHS acquisition guidance, which provided a framework for a risk management program  Collectively, these policies and plans constitute a risk management strategy  DHS and IBC have periodically updated these documents to maintain the scope of the risk management effort; the methods and tools to be used for risk identification, risk analysis, risk mitigation, risk monitoring, and communication; the prioritization of risks; and the allocation of resources for risk mitigation  Identify and analyze risks  Key aspects of processes established by DHS and the TRIO components related to the two best practices associated with identifying and analyzing risks: Identify risks  Risk identification should be an organized, thorough process to seek out probable or realistic risks to achieving objectives  This practice recognizes that risks should be identified and described understandably before they can be analyzed and managed properly  Using categories and parameters developed in the risk management strategy and identified sources of risk guides the identification of risks associated with cost, schedule, and performance  To identify risks, best practice elements include reviewing the work breakdown structure (WBS) and project plan to help ensure that all aspects of the work have been considered  Best practices for documenting risks include documenting the context, conditions, and potential consequences of each risk and identifying the relevant stakeholders associated with each risk  DHS partially met this best practice  DHS’s July 2016 risk register contained a wide range of risks associated with defined risk categories  It also reflected DHS’s review of the TRIO project’s integrated master schedule that IBC prepared based on the WBS and work plans that IBC also developed  The risk register documented the context, conditions, potential consequences, and relevant stakeholders associated with each risk  However, DHS’s documented risk management processes did not identify all significant risks or reflect its efforts to revisit risks that had previously been closed  For example, DHS officials told us that IBC was unable to provide sufficient, reliable cost and schedule information for project monitoring; however, a risk reflecting these concerns was not included on its July 2016 risk register  Further, the risk register included certain closed risks related to the need for a governance structure and strategy for ensuring that IBC met performance, cost, and schedule objectives  Although DHS had ongoing concerns about its ability to ensure that IBC met these objectives, the risk register did not reflect efforts to revisit these risks to determine whether their status needed revision or if other risks should be included on the risk register to address its accountability concerns  In addition, DHS did not always take timely action to document its consideration of risks identified by its independent verification and validation (IV&V) contractor for potential inclusion on its risk register  For example, the IV&V contractor identified a risk related to inefficiencies in DHS’s document review process in June 2015 that was not included on DHS’s risk register until February 2016  DHS officials indicated that a crosswalk between the DHS risk register and IV&V contractor risk management observations was performed weekly; however, results of these weekly reviews were not documented  Evaluate, categorize, and prioritize risks  Risk assessment uses defined categories and parameters to determine the priority of each risk to assist in determining when appropriate management attention is required  Best practices for analyzing risks include categorizing risks according to defined risk categories, evaluating identified risks using defined risk parameters, and prioritizing risks for mitigation  DHS’s processes met this practice  For example, the documented risk management program included application of defined risk categories and parameters for all identified risks, providing a means for reviewing risks and determining the likelihood and severity of risks being realized  The TRIO project’s Joint Risk Management Integrated Project Team provided consistency to the application of parameters by reviewing risk assessments when risks were first identified  By determining exposure ratings for each identified risk, DHS prioritized risks for monitoring and allocation of resources for risk mitigation  Mitigate risks  Key aspects of processes established by DHS and the TRIO components related to the two best practices associated with mitigating risks: Develop risk mitigation plans  Risk mitigation plans are developed in accordance with the risk management strategy and include a recommended course of action for each critical risk  The risk mitigation plan for a given risk includes techniques and methods used to avoid, reduce, and control the probability of risk occurrence; the extent of damage incurred should the risk occur; or both  Elements of this practice include determining the levels and thresholds that define when a risk becomes unacceptable and triggers the execution of a risk mitigation plan or contingency plan, identifying the person or group responsible for addressing each risk, determining the costs and benefits of implementing the risk mitigation plan for each risk, developing an overall risk mitigation plan for the work to orchestrate the implementation of individual risk mitigation plans, and developing contingency plans for selected critical risks in the event impacts associated with the risks are realized  DHS partially met this best practice  DHS’s risk management program documentation reflected the development of risk response plans for most risks, including all those determined to be of medium and high exposure level  DHS identified those responsible for addressing each risk  However, DHS and IBC did not always develop sufficiently detailed risk mitigation plans including specific risk-handling action items, determination of the costs and benefits of implementing the risk mitigation plan for each risk, and developing contingency plans for selected critical risks in the event that their impacts are realized  For example, a risk associated with IBC’s capacity and experience for migrating large agencies the size of Coast Guard and TSA was identified in July 2014  Although DHS developed plans to help mitigate this risk, a contingency plan was not developed prior to realizing the adverse impact of not implementing Coast Guard and TSA on IBC’s modernized solution  Rather, a contingency plan working group (CPWG) to address this and other concerns was established in January 2017, over 2 years after the risk was initially identified  Further, thresholds were not used within the risk management program to define when a risk becomes unacceptable, triggering the execution of a risk mitigation plan or contingency plan  Implement risk mitigation plans  Risk mitigation plans are implemented to facilitate a proactive program to regularly monitor risks and the status and results of risk-handling actions to effectively control and manage risks during the work effort  Best practice elements include revisiting and reevaluating risk status at regular intervals to support the discovery of new risks or new risk-handling options that can require reassessment of risks and re-planning of risk mitigation efforts  Elements also include providing a method for tracking open risk-handling action items to closure, establishing a schedule or period of performance for each risk-handling activity, invoking selected risk-handling options when monitored risks exceed defined thresholds, and providing a continued commitment of resources for each risk mitigation plan  DHS partially met this best practice  Risk monitoring of the TRIO project consisted of reviews performed by DHS and TRIO component officials responsible for risk management and oversight functions  These reviews considered significant risks, risks approaching realization events, and the effect of management intervention on the resolution of risks  These reviews also relied, in part, on data contained in DHS’s risk register, which represents the official repository of TRIO project risks and information on the status of risks and related risk mitigation efforts  However, other aspects of DHS’s efforts to implement risk mitigation plans did not fully adhere to certain elements associated with this best practice  For example, we identified certain issues that raised questions concerning the accuracy of data contained in the risk register, such as (1) the lack of clear markings indicating when the accuracy of data on each risk was last confirmed, including risk records that had not been modified in the previous 3 months, and (2) certain risks for which the estimated risk impact date had already occurred but its status risk according to DHS’s risk register did not reflect that it had been realized and become an issue  In addition, DHS officials stated that IBC did not provide sufficiently detailed, reliable cost and schedule information that could have been used to monitor TRIO project risks more effectively  DHS’s ability to monitor cost, schedule, and other performance metrics was also limited because of the lack of thresholds for management involvement, as noted above  DHS’s implementation of risk monitoring plans was further limited by other issues, including (1) a period of performance for each risk-handling activity, which includes a start date and anticipated completion date to control and monitor risk mitigation efforts, was not always established and (2) an inability to fully track open risk-handling action items to closure existed because of the lack of sufficient detail on specific risk-handling activities in the DHS risk register  According to DHS officials, DHS relied heavily on IBC to manage risks associated with the TRIO project and, in particular, those for which IBC was assigned as the risk owner  They also acknowledged DHS’s responsibility for overseeing IBC’s TRIO project risk management efforts and described various actions taken to address growing concerns regarding IBC’s efforts  For example, DHS created the Joint Risk Management Integrated Project Team, in part, to provide a forum in which IBC could obtain assistance in developing risk responses and discuss DHS’s risk mitigation concerns  Further, to help reduce exposure of underlying risks, DHS offered assistance to IBC’s project management functions, such as developing the integrated master schedule and performing quality control checks on project deliverables  Despite these efforts, DHS officials stated that challenges associated with the IAA structure and terms of the performance work statement with IBC on the TRIO project limited DHS’s visibility into IBC’s overall cost, schedule, and performance controls and ability to oversee IBC’s risk management efforts  For example, they stated that the performance work statement did not specify the level of reporting to be provided by IBC on cost, schedule, and performance in sufficient detail to effectively monitor progress on achieving key project objectives  Further, the limitations to managing risks related to the best practices we assessed as partially met were largely attributable to limitations in DHS and TRIO project guidance and policies  For example, DHS’s Risk Management Planning Handbook and related template for developing risk management plans for projects does not address the need to define thresholds to facilitate review of performance metrics to determine when risks become unacceptable  Also, TRIO project policies did not address the need to periodically revisit consideration of risk sources other than IMS-related milestones, specify periods of performance for specific risk- handling activities, or define an interval for updating and certifying risk statuses  In addition, DHS guidance and TRIO project policies did not describe the need to consider and document risks specifically related to the lack of sufficient, reliable cost and schedule information to properly manage and oversee the project or for timely disposition of risks that its IV&V contractor identified  Further, TRIO project risk management policies and management tools used to implement them address best practice elements such as determination of the costs and benefits of implementing risk mitigation plans, developing contingency plans, and developing specific risk-handling action items  However, these policies do not require, and the risk register was not designed to specifically capture, these elements in documented risk mitigation plans  By not adopting important elements of risk management best practices into project guidance, DHS and the TRIO components increase the risk that potential problems would not be identified before they occur and that activities to mitigate adverse impacts would not be effectively planned and initiated  Although DHS has taken various actions to manage the risks of using IBC for the TRIO project, including some that were consistent with best practices, the TRIO project has experienced challenges raising concerns regarding the extent to which its objectives will be achieved  In connection with these challenges, the TRIO components notified DHS during April 2016 through January 2017 that certain baseline cost and schedule objectives had not been, or were projected to not be, achieved as planned  According to these notifications and DHS officials we interviewed, several key factors and challenges significantly impacted DHS’s and IBC’s ability to achieve TRIO project objectives as intended  In addition, IBC, FIT, and USSM officials identified similar issues impacting the TRIO project  In connection with these challenges, DHS and IBC began contingency planning efforts in January 2017 to identify and assess viable options for improving program performance and addressing key TRIO project priorities  Plans for DHS’s path forward on the TRIO project, as of May 2017, involve significant changes, such as transitioning away from using IBC and a 2-year delay in completing Coast Guard and TSA’s migration to a modernized solution  We grouped the key factors and challenges impacting the TRIO project that DHS, IBC, FIT, and USSM officials and OMB staff identified into five broad categories: (1) project resources, (2) project schedule, (3) complex requirements, (4) project costs, and (5) project management and communications  The key factors and challenges related to each category are summarized below  Project resources: Concerns about IBC’s experience and its capacity to handle a modernization project involving agencies the size of Coast Guard and TSA were identified as significant risks in July 2014, resulting from discovery phase efforts completed prior to DHS and IBC’s entering the implementation phase in August 2014  According to DHS officials, status reports, and other documentation, key TRIO project challenges related to resources included concerns that (1) IBC encountered federal employee hiring challenges and was unable to ramp up and deploy the resources necessary to meet required deliverables, and (2) IBC experienced significant turnover of key stakeholders which adversely impacted its ability to achieve TRIO project objectives  In connection with DHS’s decision to use IBC for the TRIO project, DHS officials told us that DHS relied heavily on OMB and Treasury’s designation of IBC as a federal SSP and their related assessment of IBC’s capacity and experience  DHS officials also told us that DHS relied on FIT’s federal agency migration evaluation model during discovery phase efforts that focused on assessing the functionality of the software rather than assessing IBC’s (1) capacity, experience, and capability; (2) ability to address more complex software configurations and interfaces associated with large agencies; and (3) cost, schedule, and performance metrics  DHS officials stated that issues related to IBC’s capacity and experience represented the most significant challenge impacting the TRIO project  IBC officials acknowledged that IBC was unable to ramp up its resources until after the project had begun and that the IBC project team experienced significant turnover in key leadership and TRIO project positions over the course of the project  IBC officials also acknowledged that during its early efforts on the TRIO project, assigned IBC staff lacked the experience and expertise necessary for managing large-scale projects and, as a result, many of the risks initially identified were not effectively addressed  FIT and USSM officials and OMB staff also acknowledged that resource challenges significantly impacted the TRIO project  A FIT official acknowledged that assessing software functionality, rather than implementation, was emphasized during the discovery process  Although DHS relied on OMB and Treasury’s designation of IBC as a federal SSP, this FIT official also told us that because agencies’ specific needs can vary significantly, agencies are responsible for conducting sufficient due diligence to assess a federal SSP’s ability to meet their requirements  Project schedule: DHS, IBC, FIT, and USSM officials acknowledged that migrating the TRIO components to IBC within original time frames was a significant challenge given the overall magnitude and complexity of the TRIO project  According to DHS officials and TRIO project documentation, DHS identified delays in completing various tasks and milestones including providing design phase technical documentation and design processing proposed change requests; meeting proposed baseline schedules for implementing Coast Guard and TSA on the modernized IBC solution; and achieving initial operating capability requirements and stabilizing the production environment after DNDO’s migration to IBC because of various issues related to reporting, invoice payment processing, contract management processes, and resolving help desk tickets in a timely manner  DHS officials also stated that IBC did not consistently update the IMS to ensure that it accurately reflected all required tasks, the completion status, and the resources required to complete them  Concerns related to meeting milestones and updating the IMS were discussed during periodic status update meetings that included DHS, IBC, OMB, FIT, and USSM officials  IBC and DHS officials acknowledged that processes for communicating and resolving issues were not always efficient and contributed to schedule delays  In addition, in November 2016, USSM noted several concerns based on its review of a draft IMS supporting TSA’s re-planning efforts to go-live in October 2017  USSM’s concerns included an incomplete project scope and schedule and need for additional discovery to determine cost and level of effort, an extremely aggressive schedule with very limited contingencies for the lack of interim checkpoints or oversight on tasks exceeding 30 days, the need for a resource-loaded IMS that incorporates an appropriate level of detail, and the need for an expedited program governance strategy and escalation path that DHS and IBC leadership could use to make program decisions within the time allotted on the schedule  Complex requirements: DHS, IBC, FIT, and USSM officials acknowledged the overall complexity of the TRIO project and that the lack of a detailed understanding of the components’ requirements earlier in the project impacted IBC’s and DHS’s ability to satisfy the requirements as planned  For example, USSM and FIT officials told us that under the shared services model, the approach for onboarding new customers usually involves migrating to a proven configuration of a solution that is already being used by the provider’s existing customers  However, rather than taking this approach, DHS and IBC agreed to implement a more recent version of Oracle Federal Financial software (version 12 2) with integrated contract life cycle and project modules  Under this approach, IBC’s plans included migrating other existing customers to this upgraded environment  USSM officials told us that migrating TRIO components to a new solution that required configuring new software and related applications and developing related interfaces introduced additional complexities that contributed to issues on the TRIO project  According to a FIT official, the functionality of this more recent version of software is very different than that of the version IBC’s existing customers used  This official stated that IBC did not have the needed government personnel with knowledge and experience associated with this new software, a condition that likely contributed to the challenges experienced on the TRIO project  IBC officials acknowledged that IBC’s lack of familiarity with Oracle 12 2 increased the complexity of the TRIO project  In addition, DHS and IBC perspectives on the need for changes differed because of the lack of clarity regarding TRIO project requirements  DHS officials told us that many change requests on the TRIO project reflected the need for required functionality based on previously stated requirements  They also told us that they did not consider DNDO-related requirements to be overly complex when compared to those associated with IBC’s similarly sized customers  However, DHS officials stated that as of June 2017, IBC has not yet met DNDO’s needs to deliver a functioning travel system interface and other requirements  According to IBC officials, TRIO project change requests to address components’ requirements were extensive and included significant customizations to meet unique requirements that were not aligned with the federal shared service model  IBC officials noted additional challenges in addressing TRIO project requirements related to DHS’s efforts to address certain organizational change management and business process reengineering responsibilities  According to IBC officials, in some instances, the TRIO components provided conflicting requirements related to the same process that would have been more consistent had DHS completed more of its business process reengineering efforts prior to providing them to IBC  Project costs: According to the July 2014 discovery report, proposed implementation costs for the TRIO project totaled $89 9 million  However, according to DHS officials and TRIO project documentation, estimated costs significantly increased because of schedule delays, unanticipated complexities, and other challenges  In January 2017, DHS prepared a summary of estimated TRIO project implementation costs associated with its IAA with IBC  According to this summary, estimated IBC-related TRIO project implementation costs through fiscal year 2017 increased by approximately $42 8 million (54 percent) from the $79 2 million provided in the original August 2014 IAA with IBC as a result of modifications required, in part, to address challenges impacting the project  DHS officials also expressed concerns regarding increases in estimated operations and maintenance costs for the IBC solution  For example, according to a December 2016 memorandum to DHS on action items associated with failing to meet the baseline schedule date for initial operational capability, DNDO stated that IBC’s updated projected costs of operations and maintenance of its system were unaffordable  In connection with these costs, DHS officials also stated that IBC determined that separate, rather than shared, help desk resources were required to support the TRIO project because it was significantly different from the solution that IBC’s existing customers used  As a result, the officials indicated that these costs were more than originally expected  However, IBC officials told us that a portion of the increase in help desk- related costs was also due to DNDO employees not using the system properly because they were not sufficiently trained on it before it was implemented  In addition, challenges impacting the TRIO project have contributed to significant changes in the path forward on the project; as a result, the extent to which overall TRIO project modernization costs will be impacted going forward has not yet been determined  Project management and communication: According to DHS officials, various program management-related challenges impacted the TRIO project  For example, they expressed concerns regarding the effectiveness of IBC’s project management efforts including cost, schedule, and change management as well as IBC’s allocation of resources and slow decision-making process  They also stated that DHS provided significant time and resources to make up for fundamental project management activities that were under IBC’s control and not performed  In addition, DHS officials identified limitations associated with (1) poorly defined service level agreements and program performance metrics, (2) poor quality control plan, and (3) the lack of mechanisms for measuring delivery and addressing concerns regarding IBC’s performance  DHS officials told us that although various mechanisms can be used to hold commercial vendors accountable—such as cure notices, quality assurance surveillance plans, and incentives or disincentives to monitor performance—few mechanisms are available to hold federal agency service providers accountable for performance concerns  DHS officials also acknowledged challenges in their project management and communication efforts and identified lessons learned to help improve future efforts, including the need to establish a performance-based contract to determine objective and enforceable activity level metrics; be more prepared for organizational changes; improve vendor, project, and schedule management efforts; better understand SSP resource plans and monitor SSP efforts to help ensure that sufficient resources are secured timely; and centralize program management for financial system modernization functions, rather than continuing with the structure used on the TRIO project—for example, the TRIO project’s program management structure consisted of program management offices at the component level performing cost, schedule, and technical monitoring activities with DHS headquarters’ involvement focused on governance and oversight, resulting in duplicate efforts across components  IBC officials acknowledged challenges concerning IBC’s lack of sufficient resources and turnover, as described above  However, they told us that DHS’s approach to project management often resulted in duplicative meetings and a lengthy decision-making process involving several officials and multiple review and approval processes  According to USSM officials, the TRIO project team focused an unbalanced portion of its efforts on the delivery of technology at the expense of organizational change management, communication management, and other project management areas  For example, the failure to incorporate lessons learned from DNDO’s deployment adversely affected subsequent TRIO project implementation efforts, as change management activities did not address previously encountered risks  An OMB staff member concurred with the lessons learned that DHS identified, including those indicating the need for stronger project management  While the project is ongoing, the OMB staff member noted the importance of DHS having well-defined requirements for the project and better coordination to achieve the desired outcomes  In connection with TRIO project challenges, DHS officials told us that IBC notified DHS in April 2016 that it would not be able to meet the planned October 2016 implementation date for TSA  In response, DHS and IBC established the TSA Replan Tiger Team to perform a detailed assessment of potential courses of action  According to DHS officials, DHS and IBC subsequently took various actions to help address these and other challenges impacting the TRIO project, as summarized below  May 2016: IBC requested additional funding for fiscal year 2016 for 14 additional IBC and contractor personnel to strengthen program coordination and management support  According to DHS officials, DHS provided this requested funding along with additional funding to establish a business integration office to help strengthen cross organizational communication  DHS determined that plans for migrating TSA and Coast Guard to IBC during the first quarter of fiscal years 2017 and 2018, respectively, were not viable  As a result, their planned migrations were each extended an additional year  June 2016: DHS and IBC developed a comprehensive remediation plan to track progress on efforts to resolve numerous issues associated with DNDO’s production environment that continued to hamper its stability since going live in November 2015  According to DHS officials, these issues related to invoice payment and interest accruals, contract life cycle management, reporting, and other activities and have required numerous work-arounds to execute business processes  August to October 2016: DHS, Coast Guard, and IBC determined that a similar replanning effort was needed for Coast Guard’s successful migration to IBC  According to DHS officials, IBC indicated that it was unable to simultaneously provide DNDO production and TSA implementation support while also addressing the complexities related to Coast Guard  DHS officials told us that another Tiger Team established to address Coast Guard issues failed to complete the scope of its charter, and as a result, Coast Guard was forced to assume a minimum of a 2- year delay (rather than the 1-year delay previously determined in May 2016) and that this significantly increased program costs  They further stated that some of the team’s deliverables have not been initiated or remain outstanding as of June 2017  December 2016: IBC communicated to DHS that it cannot support the discovery phase with DHS’s CUBE modernization project  In addition, DHS approved the establishment of a Joint Program Management Office to serve as the overarching program management for DHS financial systems modernization projects  According to DHS officials, using a department-wide approach will enable DHS to more effectively leverage the resources and expertise across all modernization projects  January 2017: IBC communicated to DHS that it cannot support Coast Guard implementation in October 2018, and DHS and IBC established a joint CPWG to assess viable options for improving program performance and addressing stakeholder concerns and key TRIO project priorities  February 2017: DHS and IBC issued a joint memorandum to provide an update on contingency planning discussions  DHS and IBC shared commitments and determinations included (1) stabilizing the DNDO production environment and executing TSA implementation activities, (2) delivering the best value for the government and ensuring mutual success to the greatest extent possible, (3) preserving and protecting the current investment, and (4) making TSA implementation the first priority  In addition, DHS and IBC presented two options as representing the best opportunities for success in improving program performance and addressing stakeholder concerns: (1) continue with the status quo plan for Coast Guard implementation in October 2019, with significant improvements to program management and overall support capability and capacity, or (2) platform replacement  Platform replacement was presented as the preferred path toward meeting the needs of both DHS and IBC  Under this option, DHS and IBC would proceed with TSA implementation and work toward an orderly transition of TRIO components to an alternate service provider, hosting location, or both  March 2017: According to DHS officials, DHS, IBC, and USSM officials met to review certain critical success criteria for TSA’s implementation  Based on these discussions, it was determined that TSA would not go live with IBC in fiscal year 2018 given the high-risk schedule and critical criteria involved and the Coast Guard implementation would also be delayed accordingly  Further, TSA release 3 0 would be delivered in October 2017 or as soon as possible thereafter  In addition, the CPWG would continue working to identify an alternative path forward, and DHS and IBC would identify and evaluate critical transition activities and timelines  April 2017: The CPWG recommended moving away from IBC to a commercial service provider leveraging the cloud as the best course of action to complete TRIO project implementation and as the most fiscally responsible approach from a long-term sustainment and cost perspective  The CPWG’s recommendation was based on its analysis of six options and proposed a transition timeline, including key activities, as shown in figure 3  May 2017: During its May 3, 2017 briefing of the Financial Systems Modernization Executive Steering Committee, DHS indicated that two of the options that the CPWG considered were no longer viable, including the CPWG’s recommendation to transition to a commercial cloud service provider because the software was not yet cloud-ready  DHS ranked the remaining four options using 13 OMB risk factors as selection criteria and determined that migrating the solution to a DHS data center represented the best option going forward  In addition, DHS decided to move forward with discovery efforts related to this option  According to its briefing presentation and DHS officials, the notional timeline of planned key events for the TRIO project included various items, as shown in figure 4  DHS officials indicated that DHS expects to present the findings and recommendations resulting from discovery efforts associated with this new path forward to USSM and OMB for concurrence  As of August 2017, results of this effort were under review by DHS leadership  The TRIO project represents a key element of DHS’s efforts to address long-standing deficiencies in its financial management systems and further improve financial management  Following best practices to manage risks effectively can help provide increased assurance that large, complex projects—such as the TRIO project—will achieve planned objectives  DNDO’s AA process substantially met the four characteristics of a reliable, high-quality AOA process  However, Coast Guard’s and TSA’s AAs substantially met one and partially met three of these four characteristics  Further, DHS did not always follow best practices for managing the risks of using IBC for the TRIO project  As a result, TRIO components faced an increased risk that the solution they chose would not represent the best alternative for meeting their mission needs and that the risks impacting the TRIO project would not be effectively managed to mitigate adverse impacts  In addition, significant challenges have impacted the TRIO project, raising concerns about the extent to which objectives will be achieved as planned  Plans for DHS’s path forward on the TRIO project, as of May 2017, involve significant changes, such as transitioning away from IBC and a 2-year delay in completing Coast Guard’s and TSA’s migration to a modernized solution  Without greater adherence to best practices for analyzing alternatives and managing project risks, DHS continues to face increased risk that its financial management system modernization project will not provide reasonable assurance of achieving its mission objectives  We are making the following two recommendations to DHS: The DHS Under Secretary for Management should develop and implement effective processes and improve guidance to reasonably assure that future AAs fully follow AOA process best practices and reflect the four characteristics of a reliable, high-quality AOA process  (Recommendation 1) The DHS Under Secretary for Management should improve the Risk Management Planning Handbook and other relevant guidance for managing risks associated with financial management system modernization projects to fully incorporate risk management best practices, including defining thresholds to facilitate review of performance metrics to determine when risks become unacceptable; identifying and analyzing risks to include periodically reconsidering risk sources, documenting risks specifically related to the lack of sufficient, reliable cost and schedule information needed to help properly manage and oversee the project, and timely disposition of IV&V contractor-identified risks; developing risk mitigation plans with specific risk-handling activities, the costs and benefits of implementing them, and contingency plans for selected critical risks; and implementing risk mitigation plans to include establishing periods of performance for risk-handling activities and defining time intervals for updating and certifying the accuracy and completeness of information on risks in DHS’s risk register  (Recommendation 2) We provided a draft of this product to DHS and the Department of the Interior for comment  In its comments, reprinted in appendix IV, DHS concurred with our recommendations and provided details on its implementation of the recommendations as discussed below  In addition, DHS provided technical comments, which we incorporated as appropriate  The Department of the Interior only provided technical comments, which we incorporated as appropriate  DHS stated that it remains committed to its financial system modernization program  Specifically, regarding our first recommendation to develop and implement effective processes and improve guidance to reasonably assure that future AAs fully follow AOA process best practices and reflect the four characteristics of a reliable, high-quality AOA process, DHS stated that it agrees that effective processes and guidance are necessary to assure best practices  DHS also stated that it is important to note that the GAO-identified best practices were published more than 2 years after the TRIO components’ AAs were completed  While this is the case, as discussed in our report, these best practices are based on long- standing, fundamental tenets of sound decision making and economic analysis and were identified by compiling and reviewing commonly mentioned AOA policies and guidance that are known to and have been used by government and private sector entities  DHS also stated that it has already implemented this recommendation through its issuance of guidance and instructions in 2016 and that a copy of this additional guidance and instructions was provided to GAO  However, the documentation provided by DHS does not fully address our recommendation  As part of our recommendation follow-up process, we will coordinate with DHS to obtain additional information on its efforts to address our recommendation  With regard to our second recommendation to improve the Risk Management Planning Handbook and other relevant guidance, DHS stated that it concurred and agreed that the Risk Management Planning Handbook required updating to fully incorporate risk management best practices  In addition, DHS described actions it will take, and has taken, to revise and publish an updated handbook  We are sending copies of this report to the appropriate congressional committees, the Acting Secretary of Homeland Security, the DHS Under Secretary for Management, the Acting DHS Chief Financial Officer, the Secretary of the Interior, and the Director of the Interior Business Center  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staffs have any questions about this report, please contact me at (202) 512-9869 or khana@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  Key contributors to this report are listed in appendix V  To determine the extent to which the Department of Homeland Security (DHS) followed best practices in analyzing the alternatives used in choosing the preferred alternative for modernizing TRIO components’ financial management systems, we reviewed information that the TRIO components provided as part of their alternatives analysis (AA) process, referred to as the AA body of work, which includes the AA and other supporting documentation that is not specifically included in the AA  In addition, we discussed the DHS AA process with the TRIO components and DHS officials  We evaluated each TRIO component’s AA body of work and assessed this information against the GAO-identified 22 analysis of alternatives (AOA) process best practices  We then scored each AA against those best practices  In appendix II, these GAO- identified best practices are described in detail  Our evaluation comprised the following steps: (1) two GAO analysts separately examined the AA information received for each component, providing a score for each of 18 best practices; (2) a third GAO analyst resolved any differences between the two analysts’ initial scoring; and (3) a GAO specialist on AOA best practices, independent of the audit team, reviewed the team’s AA documentation, scores, and analyses for consistency  The GAO specialist also assessed the four best practices related to cost estimating  We used the average scores for each best practice to determine an overall score for four summary characteristics—well-documented, comprehensive, unbiased, and credible—of a reliable, high-quality AOA process at each TRIO component  Next, we shared our preliminary analysis with the TRIO components and DHS, and requested their technical comments and any additional information for our further consideration  For those characteristics of the AA process that received a score of partially met or below, we met with TRIO component and DHS officials to discuss potential reasons that an AA did not always conform to best practices  Finally, using the same methodology and scoring process explained above, we performed a final assessment based on our preliminary analysis and the comments and additional information received  The best practices were not used to determine whether DHS made the correct decision in selecting Department of the Interior’s Interior Business Center (IBC) to implement the financial management systems modernization solution or whether the TRIO project would have arrived at a different conclusion had it more fully conformed to these best practices  We also reviewed DHS guidance for conducting AOAs and AAs against the GAO-identified 22 AOA process best practices using the same methodology described above for reviewing the TRIO components’ AAs  In the course of applying these best practices to a TRIO component’s AA and to DHS guidance for the AA process, we assessed the reasonableness of the information we collected  We determined that the information from the DHS AA process was sufficiently reliable to use in assessing the TRIO components’ AAs and DHS guidance against these 22 best practices  To determine the key factors, metrics, and processes used by the TRIO components in developing and evaluating DHS’s alternative solutions and final choice for financial system modernization, we reviewed each component’s AA, including a description of (1) the alternatives considered, (2) the market research conducted, (3) the three alternatives evaluated, (4) the selection criteria used and how the criteria were weighted, (5) how each alternative scored against the selection criteria, and (6) the alternative that scored the best according to the component’s evaluation  To determine the extent to which DHS managed the risks of using IBC consistent with risk management best practices, we reviewed DHS’s and TRIO components’ risk management guidance and other documentation supporting their risk management efforts, including risk registers, mitigation plans, status reports, and risk management meeting minutes  We also met with officials to gain an understanding of the key processes and documents used for managing and reporting on TRIO project risks  We assessed the processes against best practices that the Software Engineering Institute (SEI) identified  The practices we selected are fundamental to effective risk management activities  These practices are identified in SEI’s Capability Maturity Model® Integration (CMMI®) for Acquisition, Version 1 3  In particular, the key best practices for preparing for risk management are determine risk sources and categories, define risk parameters, and establish a risk management strategy  The key best practices for identifying and analyzing risks are evaluate, categorize, and prioritize risks  The key best practices for mitigating identified risks are develop risk mitigation plans and implement risk mitigation plans  We applied the criteria from the CMMI risk management process area to determine the extent to which the expected practices were implemented, or future activities were planned for, by the program office  The rating system we used is as follows: (1) meets, or generally satisfies all elements of the specific practice; (2) partially meets, or generally satisfies a portion of specific practice elements; and (3) does not meet, or does not satisfy specific practice elements  In the context of the best practices methodology, we assessed the reliability of TRIO project risk data contained in DHS’s risk register  We interviewed officials on how the risk register was developed and maintained, including key control activities used to provide reasonable assurance of the accuracy of the information reported in the register  We reviewed DHS’s July 2016 risk register and minutes from risk management committee meetings (one meeting per quarter, randomly selected)  Of 120 TRIO project risks on the July 2016 risk register, we found 13 risks with missing data  Of 47 active risks identified, 28 risk records had not been modified in the previous 3 months and the register did not indicate when their accuracy was last confirmed and 35 risks were beyond their indicated impact dates but had not been marked as issues  We concluded that the pervasiveness of these data reliability problems decreased the usefulness of the risk register in connection with managing TRIO project risks  To determine the key factors or challenges that have impacted the TRIO project and DHS’s plans for completing remaining key priorities, we met with DHS, IBC, Office of Financial Innovation and Transformation, and Unified Shared Services Management office officials and Office of Management and Budget staff to obtain their perspectives  In addition, we reviewed documentation provided by these officials, including TRIO project status reports and memorandums, leadership briefings, and other presentations  We conducted this performance audit from March 2016 to September 2017 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  Many guides describe an approach to an analysis of alternatives (AOA); however, there is no single set of practices for the AOA process that has been broadly recognized by both the government and private sector entities  GAO has previously identified 22 best practices for an AOA process by (1) compiling and reviewing commonly mentioned AOA policies and guidance used by different government and private sector entities and (2) incorporating experts’ comments on a draft set of practices to develop a final set of practices  These practices are based on longstanding, fundamental tenets of sound decision making and economic analysis  In addition, these practices can be applied to a wide range of activities in which an alternative must be selected from a set of possible options, as well as to a broad range of capability areas, projects, and programs  These practices can provide a framework to help ensure that entities consistently and reliably select the project alternative that best meets mission needs  The guidance below is an overview of the key principles that lead to a successful AOA process and not as a “how to” guide with detailed instructions for each best practice identified  The 22 best practices that GAO identified are grouped into the following five phases: 1  Initialize the AOA process: Includes best practices that are applied before starting the process of identifying, analyzing, and selecting alternatives  This includes determining the mission need and functional requirements, developing the study time frame, creating a study plan, and determining who conducts the analysis  2  Identify alternatives: Includes best practices that help ensure that the alternatives to be analyzed are sufficient, diverse, and viable  3  Analyze alternatives: Includes best practices that compare the alternatives to be analyzed  The best practices in this category help ensure that the team conducting the analysis uses a standard, quantitative process to assess the alternatives  4  Document and review the AOA process: Includes best practices that would be applied throughout the AOA process, such as documenting all steps taken to initialize, identify, and analyze alternatives and to select a preferred alternative in a single document  5  Select a preferred alternative: Includes a best practice that is applied by the decision maker to compare alternatives and to select a preferred alternative  The five phases address different themes of analysis necessary to complete the AOA process, and comprise the beginning of the AOA process (defining the mission needs and functional requirements) through the final step of the AOA process (selecting a preferred alternative)  We also identified four characteristics that relate to a reliable, high-quality AOA process—that the AOA process is well-documented, comprehensive, unbiased, and credible  Table 4 shows the four characteristics and their relevant AOA best practices  Conforming to the 22 best practices helps ensure that the preferred alternative selected is the one that best meets the agency’s mission needs  Not conforming to the best practices may lead to an unreliable AOA process, and the agency will not have assurance that the preferred alternative best meets mission needs  The Department of Homeland Security’s TRIO components—the U S  Coast Guard (Coast Guard), Transportation Security Administration (TSA), and Domestic Nuclear Detection Office (DNDO)—conducted alternatives analyses (AA) during 2012 and 2013 to determine the best alternative for transitioning to a modernized financial management system solution  We evaluated the TRIO components’ AA processes against analysis of alternatives (AOA) best practices GAO identified as necessary characteristics of a reliable, high-quality AOA process (described in app  II)  GAO’s assessment of the extent to which Coast Guard’s, TSA’s, and DNDO’s AAs met each of the 22 best practices is detailed in tables 5, 6, and 7  In addition to the contact named above, James Kernen (Assistant Director), William Brown, Courtney Cox, Eric Essig, Valerie Freeman, Matthew Gardner, Jason Lee, Jennifer Leotta, and Madhav Panwar made key contributions to this report 


Report 2:  This longer time frame is required because the estimated contribution rate increases as the projection horizon increases and eventually converges to a steady state  If the projection period is of insufficient length, the steady-state level of contribution is not attained, and the necessary contribution rate is understated  We simulated variables used to estimate the pension contribution rate using the approach summarized in table 3  This approach is similar to the approach we have used in prior model updates  Future growth in the number of state and local government retirees— many of whom will be entitled to pension and health care benefits—is largely driven by the size of the workforce in earlier years  We simulated the number of state and local government retirees by assuming that the growth rate in the number of retirees is a weighted average of the growth rates in lagged general government and government enterprise employment  We estimated the weights using a regression of the percent change in the number of retirees on the percent change in employment 1, 6, 11, 16, 21, 26, 31, 36, and 41 years in the past  The coefficients on the past percentage changes in employment were constrained to be non- negative and to sum to 1  For this regression, we removed cyclical swings in employment using the Hodrick-Prescott filter  Similarly, future changes in the real amount of pension benefits will be a function of past changes in real wages and salaries  As indicated in table 3, we used a weighted average of past values of the state and local government employment cost index to simulate the employment cost index for state and local government retirees  We chose the weights to reflect changes in the share and average real benefit level of three subsets of the retiree population over time: (1) new retirees entering the beneficiary pool, (2) deceased retirees leaving the pool, and (3) continuing retirees from the previous year  We simulated the weight for new retirees in a year as the number of retirees less the number of continuing retirees divided by the number of retirees  We simulated the weight for deceased retirees as the mortality rate multiplied by last year’s retirees divided by this year’s retirees  We simulated the weight for continuing retirees as last year’s retirees divided by this year’s retirees  Finally, we simulated the employment cost index for state and local government retirees as the sum of the weight on new retirees multiplied by the state and local government employment cost index and the weight on continuing retirees multiplied by the state and local government employment cost index 8 years prior, less the weight on deceased retirees multiplied by the state and local government employment cost index 21 years prior  As discussed above, we started with historical data for 2017, or the most recent year available, simulated all of the variables in table 3 over the long run, and then used the consumer price index (CPI) and the real return on pension assets to calculate the total present value of wages and salaries for state and local government general government and government enterprise employees, the total present value of real pension benefits paid to state and local government retirees, and the total present value of state and local government employees’ pension contributions  Then, we calculated the total present value of state and local governments’ pension liabilities as the total present value of real pension benefits paid to state and local government retirees less the total present value of state and local government employees’ pension contributions, and the value of assets in state and local government defined benefit pension funds in 2017  Finally, we estimated state and local governments’ pension contribution rate as the ratio of the total present value of their pension liabilities to the total present value of wages and salaries for state and local government employees  Table 4 summarizes the approach we used to simulate interest rates on state and local government financial assets and liabilities  This approach is similar to the approach we have used in prior model updates  Table 5 summarizes our approach to simulating state and local government receipts  This approach is similar to the approach we have used in prior model updates  These variables track state and local government receipts in table 1 above as follows: State and local government personal income tax revenue is the sum of state personal income tax revenue and local personal income tax revenue; State and local government personal tax revenue is the sum of personal income tax revenue and other personal tax revenue; State and local government revenue from taxes on production and imports is the sum of general sales tax revenue, excise tax revenue, property tax revenue, and revenue from other taxes on production and imports; State and local government current tax revenue is the sum of personal tax revenue, revenue from taxes on production and imports, and corporate income tax revenue; State and local government current transfer receipts are equal to federal Medicaid grants minus Medicare Part D payments to the federal government, plus other federal grants (excluding investment grants), transfer receipts from businesses, and transfer receipts from persons; State and local government current receipts are the sum of current tax revenue, current transfer receipts, income on assets, social insurance contributions, and government enterprise surplus; State and local government capital transfer receipts are the sum of federal investment grants and estate and gift tax revenue; and State and local government total receipts are the sum of current receipts and capital transfer receipts  Our general approach to simulating state and local government expenditures is to assume that state and local governments maintain the current level of public goods and services provision per capita (see table 6)  Thus, we generally assume that expenditures keep up with U S  population growth and some measure of inflation, where the relevant rate of inflation varies depending on the specific type of expenditure  However, we use alternative approaches—described below—to simulate depreciation, interest payments, and social benefits for health care  This approach is similar to the approach we have used in prior model updates  These variables correspond to state and local government expenditures in table 1 as follows: Employee compensation is the sum of wages and salaries, pension contributions, health benefits for current employees, health benefits for retirees, and other compensation, for state and local government general government employees  Consumption expenditures are the sum of employee compensation, general government fixed capital consumption, and other general government consumption expenditures  Social benefit payments are the sum of Medicaid benefits, non- Medicaid health benefits, and non-health social benefits  Current expenditures are the sum of consumption expenditures, social benefit payments, interest payments, and subsidy payments  Total expenditures are the sum of current expenditures, gross investment, capital transfer payments, and purchases of nonproduced assets, minus general government and government enterprise fixed capital consumption  Table 7 summarizes our approach for simulating state and local government financial assets and liabilities  This approach is similar to the approach we have used in prior model updates  Our method for simulating the sectors’ short-term debt outstanding leverages the fact that for any entity, there is a direct relationship between budget outcomes and changes in financial position  Specifically, if expenditures exceed receipts, the gap needs to be financed by some combination of changes in financial assets and changes in financial liabilities  If governments spend more than they take in, they must pay for it by issuing debt, cashing in assets, or some combination of the two  Conversely, if receipts exceed expenditures and the sector is a net lender, its net financial investment (the net change in financial assets minus the net change in financial liabilities) must equal the budget surplus  The relationship between budget outcomes and the sector’s financial position is shown in the following accounting identity: total receipts – total expenditures = change in financial assets – change in financial liabilities  The sector’s financial liabilities include short-, medium-, and long-term debt; trade payables; and loans from the federal government, so the accounting identity can be rewritten as follows: total receipts – total expenditures = change in financial assets – change in medium- and long-term debt – change in trade payables – change in federal government loans – change in short term debt  For a given difference between total receipts and total expenditures, various combinations of changes in financial assets and changes in financial liabilities can satisfy this identity  However, we assumed that financial assets grow at the same rate as U S  GDP, that medium- and long-term debt outstanding is determined using the historical relationship described in table 7, that federal government loans to state and local governments are determined using the historical relationship described in table 7, and that trade payables grow at the same rate as other state and local government consumption spending  If the first four terms on the right hand side of the identity are already determined, then only the fifth term— the change in short-term debt—is free to satisfy this identity  As discussed above, our indicators of fiscal balance are operating balance, net saving, and net lending or borrowing  This approach is similar to the approach we have used in prior model updates  Recall that we defined operating balance as follows: operating balance = total receipts – (gross investment + capital transfer payments + net purchases of nonproduced assets – medium- and long-term debt issuance) – (current expenditures – consumption of general government fixed assets) – current surplus of state and local government enterprises – net social insurance fund balance  By substituting for total receipts and current expenditures using the relationships described above and rearranging terms, we can also calculate operating balance using a formula that more easily identifies its revenue components—the items in the first set of parentheses—and expenditure components—the items in the second set of parentheses: operating balance = (current tax revenues + estate and gift tax revenues + social insurance fund contributions + income receipts from assets + current transfers + federal investment grants + medium- and long-term debt issuance) – (compensation of general government employees + social benefit payments + interest payments + gross investment + capital transfer payments + net purchases of nonproduced assets + other general government consumption expenditures + subsidy payments + net social insurance fund balance)  Some of our simulations are based on estimated historical relationships between pairs of variables: Elasticity of real personal consumption expenditures less food and services with respect to real wages and salaries; Elasticity of the real U S  market value of real estate with respect to Relationship between effective interest rates on financial assets and Relationship between state and local government bond yields and 10- year Treasury rates; Relationship between effective interest rates on long-term state and local government debt and federal government loans and state and local government bond yields; Elasticity of real state personal income tax revenue with respect to Elasticity of real state and local government excise tax revenue with respect to real wages and salaries; Relationship between long-term debt issuance as a fraction of gross investment and nonproduced asset purchases in excess of federal investment grants and the change in state and local government bond yields; and Relationship between real federal government lending to state and local governments and real U S  GDP  To estimate each of these historical relationships, we used the following approach: first, we assessed the order of integration of both variables using unit root tests of the levels and the first differences, where a variable is integrated of order 0 (I(0) or stationary) if we rejected the null hypothesis of a unit root in the levels at standard significance levels, and is integrated of order 1 (I(1) or first-order nonstationary) if we could not reject the null hypothesis of a unit root in the levels but we could do so for the first differences  For relationships between variables that were both stationary, we estimated an autoregressive distributed lag model, where y is the dependent variable, x is the independent variable, and ε is an independent, identically distributed error term  The long-run impact on y of a one unit change in x is given by ∑   We initially chose the number of lags based on the Bayesian Information Criteria and then added additional lags of the dependent variable, if needed, until the residuals were consistent with a white noise process at standard significance levels  For relationships between variables that were both first-order nonstationary, we used the same approach but also used the Pesaran, Shin, and Smith bounds test for the existence of a cointegrating (long-run equilibrium) relationship  We concluded that the variables were cointegrated if we rejected the null hypothesis of no relationship at standard significance levels  Tables 8 and 9 summarize the estimated regression models as well as the results of the unit root, white noise, and cointegration tests  We simulated the model for the 75-year period from 2018 through 2092, and we used the results to calculate the operating balance for the state and local government sector as a percentage of U S  GDP  Our results suggest that if the sector maintains current policy and continues to provide current per capita levels of public goods and services, then its operating balance will decline from about -1 percent of U S  GDP to about -3 percent of U S  GDP over the next 50 years  To shed light on how maintaining the operating balance at or above zero would affect the state and local government sector, we used the model to simulate the level of total expenditures that would keep the operating balance greater than or equal to zero  We then calculated the difference between the present value of total expenditures simulated assuming the sector maintains balance, and the present value of total expenditures simulated assuming the sector maintains current policies, both as a percentage of the present value of total expenditures assuming the sector maintains current policies, and as a percentage of the present value of U S  GDP  We calculated all of the present values for the 50-year period from 2018 through 2067, and we used a discount rate equal to the average of the 3-month Treasury rate and the 10-year Treasury rate for each year  Our results suggest that the difference between the present value of total expenditures that maintain balance and the present value of total expenditures that maintain current policies is about -14 7 percent of the present value of total expenditures that maintain current policies, or about -2 4 percent of the present value of U S  GDP  That is, our simulations suggest that maintaining balance would require the sector to spend about 14 7 percent less than it would spend each year to maintain current policies  We note that a similar exercise based on simulating total revenues required to maintain the operating balance at or above zero would generate a similar result  Our approach has a number of limitations and the results should be interpreted with caution: The state and local government fiscal model is not designed for certain types of analyses  The simulations are not intended to provide precise predictions  Even though we know that these governments regularly make changes to tax laws and expenditures, the model essentially holds current policy in place and analyzes the fiscal future for the sector as if those policies were maintained because it would be highly speculative to make any assumptions about future policy adjustments  Fiscal outcomes, as related to the state and local government sector’s financial position and solvency, may not reflect all aspects of the sector’s fiscal health  Other indicators include economic indicators that go beyond the sector’s financial position to include economic growth, income, or distributional equity, as well as indicators of the quality of services provided by the sector, including education, health care, infrastructure, and other public goods and services  Our unit of analysis is the state and local government sector as a whole, so our results provide an assessment of the sector’s fiscal outlook  However, individual state and local governments likely exhibit significant heterogeneity in their expenditure and revenue patterns, so their fiscal outlooks will likely differ from that for the sector  Nevertheless, it is informative to assess the overall fiscal outlook of the sector because doing so reveals the outlook for the average state or local government  In addition, aggregate data on the sector are available on a more timely basis than data for individual state and local governments  This allows for a better assessment of the sector’s current fiscal outlook  Our results for the sector also provide a baseline from which to view the experiences of individual state and local governments  Finally, assessing the fiscal outlook of the sector as a whole can help mitigate the tendency to extrapolate from the most visible, but potentially not representative, experiences of individual states or localities  Our baseline approach to simulating the fiscal outlook for the state and local government sector is described in appendix I  As part of our simulation approach, we used five variables with values for the simulation period—the period from 2018 through 2092—that are projected outside the model and that do not rely on maintaining historical relationships: U S  population, real U S  gross domestic product (GDP) growth, national health care excess cost growth, Medicaid excess cost growth, and the real rate of return on pension assets  U S  population  For our baseline simulations, we used the Board of Trustees of the Federal Old-Age and Survivors Insurance and Federal Disability Insurance Trust Funds’ (OASDI Trustees) intermediate population projections  Real U S  GDP  For our baseline simulations, we projected real U S  GDP to grow at the same rate as Congressional Budget Office (CBO) projections for the period from 2018 through 2028 and to grow at the same rate as the OASDI Trustees’ intermediate projections of real U S  GDP growth for the period from 2029 through 2092  National health expenditures excess cost growth  For our baseline simulations, we used Centers for Medicare & Medicaid Services’ (CMS) baseline projection of national health expenditures excess cost growth  Medicaid excess cost growth  For our baseline simulations, for the period from 2029 through 2092, we used Medicaid excess cost growth derived from CMS’s baseline projections  Real rate of return on state and local government pension assets  For our baseline simulations, we assumed a 5 percent real rate of return on state and local government pension assets  To assess the sensitivity of our results to changes in these baseline projections, we selected two alternative projections of each of these variables, one associated with a faster growth rate or rate of return and one associated with a slower growth rate or rate of return  U S  population  For our alternative simulations, we used the OASDI Trustees’ high cost and low cost population projections  Real U S  GDP  For our alternative simulations, we used the OASDI Trustees’ high cost and low cost projections of real U S  GDP growth  National health expenditures excess cost growth  For our alternative simulations, we used CMS’s alternative projection of national health expenditures excess cost growth  As another alternative, we simulated the model assuming excess cost growth for national health expenditures is zero  Medicaid excess cost growth  For our alternative simulations, for the period from 2029 through 2092, we used Medicaid excess cost growth derived from CMS’s alternative projections for the period from 2029 through 2092  As another alternative, we simulated the model assuming Medicaid excess cost growth is zero for the period from 2029 through 2092  Real rate of return on state and local government pension assets  For our sensitivity analysis, we used real rates of return of 2 5 percent and 7 5 percent  Table 10 shows the average annual growth rate or rate of return associated with the baseline and alternative projections of each variable for the simulation period  For our simulations based on alternative assumptions about U S  population growth and real U S  GDP growth, as well as simulations based on alternative assumptions about real pension asset returns, we simulated the model changing one variable at a time and leaving the others fixed at their baseline values  For example, for one simulation we used the slower assumption for real U S  GDP growth and the baseline assumptions for all other variables  For our simulations based on alternative assumptions about excess cost growth for national health expenditures and for Medicaid, we changed both variables in the same direction and left the others fixed at their baseline values  For example, for one simulation we used zero excess cost growth for both national health expenditures and for Medicaid, and made the baseline assumption for the other variables  Thus, our sensitivity analysis is in the spirit of a partial equilibrium comparative statics analysis that sheds light on how each of the individual variables may affect the state and local government sector’s fiscal outlook  However, these variables are likely to be correlated so future changes in one would likely be associated with changes in others  State and Local Governments’ Fiscal Outlook: December 2016 Update, GAO-17-213SP  Washington, D C : Dec  8, 2016  State and Local Governments’ Fiscal Outlook: December 2015 Update, GAO-16-260SP  Washington, D C : Dec  16, 2015  State and Local Governments’ Fiscal Outlook: December 2014 Update, GAO-15-224SP  Washington, D C : Dec  17, 2014  State and Local Governments’ Fiscal Outlook: April 2013 Update, GAO-13-546SP  Washington, D C : Apr  29, 2013  State and Local Governments’ Fiscal Outlook: April 2012 Update, GAO-12-523SP  Washington, D C : Apr  5, 2012  State and Local Government Pension Plans: Economic Downturn Spurs Efforts to Address Costs and Sustainability, GAO-12-322  Washington, D C : Mar  2, 2012  State and Local Governments’ Fiscal Outlook: April 2011 Update, GAO-11-495SP  Washington, D C : Apr  6, 2011  State and Local Governments: Knowledge of Past Recessions Can Inform Future Federal Fiscal Assistance, GAO-11-401  Washington, D C : Mar  31, 2011  State and Local Governments: Fiscal Pressures Could Have Implications for Future Delivery of Intergovernmental Programs, GAO-10-899  Washington, D C : July 30, 2010  State and Local Governments’ Fiscal Outlook: March 2010 Update, GAO-10-358  Washington, D C : Mar  2, 2010  Update of State and Local Government Fiscal Pressures, GAO-09-320R  Washington, D C : Jan  26, 2009  State and Local Fiscal Challenges: Rising Health Care Costs Drive Long- term and Immediate Pressures, GAO-09-210T  Washington, D C : Nov  19, 2008  State and Local Governments: Growing Fiscal Challenges Will Emerge during the Next 10 Years, GAO-08-317  Washington, D C : Jan  22, 2008  Our Nation’s Long-Term Fiscal Challenge: State and Local Governments Will Likely Face Persistent Fiscal Challenges in the Next Decade, GAO-07-1113CG  Washington, D C : July 18, 2007  State and Local Governments: Persistent Fiscal Challenges Will Likely Emerge within the Next Decade, GAO-07-1080SP  Washington, D C : July 18, 2007  In addition to the contacts listed above, Brenda Rabinowitz and Courtney LaFountain (Assistant Directors), David Aja, Brett Caloia, Ann Czapiewski, Joe Silvestri, Stewart Small, Andrew J  Stephens, Frank Todisco, Walter Vance, and Chris Woika made significant contributions to this report 

ground_truth: Fiscal sustainability presents a national challenge shared by all levels of government. Since 2007, GAO has published simulations of long-term fiscal trends in the state and local government sector, which have consistently shown that the sector faces long-term fiscal pressures. While most states have requirements related to balancing their budgets, deficits can arise because the planned annual revenues are not generated at the expected rate, demand for services exceeds planned expenditures, or both, resulting in a near-term operating deficit. This report updates GAO's state and local fiscal model to simulate the fiscal outlook for the state and local government sector. This includes identifying the components of state and local expenditures likely to contribute to the sector's fiscal pressures. In addition, this report identifies considerations related to federal policy and other factors that could contribute to uncertainties in the state and local government sector's long-term fiscal outlook. GAO's model uses the Bureau of Economic Analysis's National Income and Product Accounts as the primary data source and presents the results in the aggregate for the state and local sector as a whole. The model shows the level of receipts and expenditures for the sector until 2067, based on current and historical spending and revenue patterns. In addition, the model assumes that the current set of policies in place across state and local government remains constant to show a simulated long-term outlook. GAO's simulations suggest that the state and local government sector will likely face an increasing difference between revenues and expenditures during the next 50 years as reflected by the operating balance--a measure of the sector's ability to cover its current expenditures out of its current receipts. While both expenditures and revenues are projected to increase as a percentage of gross domestic product (GDP), a difference between the two is projected to persist because expenditures are expected to grow faster than revenues throughout the simulation period. GAO's simulations also suggest that growth in the sector's overall spending is largely driven by health care expenditures--in particular, Medicaid spending and spending on health benefits for state and local government employees and retirees. These expenditures are projected to grow as a share of GDP during the simulation period. GAO's simulations also suggest that revenues from personal income taxes and federal grants to states and localities will increase during the simulation period. However, revenues will grow more slowly than expenditures such that the sector faces a declining fiscal outlook. GAO also identified federal policy changes that could affect the state and local government sector's fiscal outlook. For example, the effects of the recently-enacted Tax Cuts and Jobs Act will likely depend on how states incorporate the Act into their state income tax rules. In addition, other factors, such as economic growth and rates of return on pension assets, could shift future fiscal outcomes for the sector.