prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.



Report 2: A high-quality, reliable cost estimate is a key tool for budgeting, planning, and managing the 2020 Census  According to OMB, programs must maintain current and well-documented estimates of program costs, and these estimates must encompass the full life-cycle of the program  Among other things, OMB states that generating reliable program cost estimates is a critical function necessary to support OMB’s capital programming process  Without this capability, agencies are at risk of experiencing program cost overruns, missed deadlines, and performance shortfalls  A reliable cost estimate is critical to the success of any federal government program  With the information from reliable estimates, managers can: make informed investment decisions, allocate program resources, measure program progress, proactively correct course when warranted, and ensure overall accountability for results  To be considered reliable, a cost estimate must meet the criteria for each of the four characteristics outlined in our Cost Estimating and Assessment Guide  According to our analysis, a cost estimate is considered reliable if the overall assessment ratings for each of the four characteristics are substantially or fully met  If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable  Those characteristics are: Well-documented: An estimate is thoroughly documented, including source data and significance, clearly detailed calculations and results, and explanations of why particular methods and references were chosen  Data can be traced to their source documents  Accurate: An estimate is unbiased, the work is not overly conservative or overly optimistic, and is based on an assessment of most likely costs  Few, if any, mathematical mistakes are present  Credible: Any limitations of the analysis because of uncertainty or bias surrounding data or assumptions are discussed  Major assumptions are varied, and other outcomes are recomputed, to determine how sensitive they are to changes in the assumptions  Risk and uncertainty analysis is performed to determine the level of risk associated with the estimate  The estimate’s results are cross- checked, and an independent cost estimate (ICE) is conducted to see whether other estimation methods produce similar results  Comprehensive: An estimate has enough detail to ensure that cost elements are neither omitted nor double counted  All cost-influencing ground rules and assumptions are detailed in the estimate’s documentation  Meeting best practices outlined in our Cost Estimating and Assessment Guide for a reliable cost estimate has been a long-standing challenge for the Bureau  In 2008 we reported that the 2010 Census cost estimate was not reliable because it lacked documentation and was not comprehensive, accurate, or credible  For example, in our 2008 report on the Bureau’s cost estimation process, Bureau officials were unable to provide documentation that supported the assumptions for the initial 2001 life-cycle cost estimate as well as the updates  Consequently, we recommended that the Bureau establish guidance, policies, and procedures for estimating costs that would meet best practices criteria  The Bureau agreed with the recommendation and said at the time that it already had efforts underway to improve its future cost estimation methods and systems  Moreover, weaknesses in the life-cycle cost estimate were one reason we designated the 2010 Census a GAO High- Risk Area in 2008  In 2012 we reported that, while the Bureau was taking steps to strengthen its life-cycle cost estimates, it had not yet established guidance for developing cost estimates  We recommended that the Bureau finalize its guidance, policies, and procedures for cost estimation in accordance with best practices  The Bureau agreed with the overall theme of the report but did not comment on the recommendation  During this review we found that the Bureau took steps to address this recommendation, which is discussed later in this report  Such guidance can help to institutionalize best practices and ensure consistent processes and operations for producing reliable estimates  In a 2016 report we found that the October 2015 version of the Bureau’s life-cycle cost estimate for the 2020 Census was not reliable  Overall, we reported that the 2020 Census life-cycle cost estimate partially met two of the characteristics of a reliable cost estimate (comprehensive and accurate) and minimally met the other two (well-documented and credible)  We recommended that the Bureau take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate  The Bureau agreed with this recommendation, and took steps to improve the reliability of its cost estimate, which we focus on later in this report  Consequently, an unreliable life-cycle cost estimate is one of the reasons we designated the 2020 Census a GAO High-Risk Area in 2017  In October 2015, the Bureau estimated the cost of the 2020 Census to be $12 3 billion  According to the Bureau, the October 2015 version was the Bureau’s first attempt to model the life-cycle cost of its planned 2020 Census, in contrast to its earlier 2011 estimate, which the Bureau said was intended to produce an approximation of potential savings and to begin developing the methodology for producing decennial life-cycle cost estimates covering all phases of the decennial life cycle  To help control costs while maintaining accuracy, the Bureau introduced significant change to how it conducts the decennial census in 2020  Its planned innovations include reengineering how it builds its address list, improving self-response by encouraging the use of the Internet and telephone, using administrative records to reduce field work, and reengineering field operations using technology to reduce manual effort and improve productivity  In contrast to the estimated $12 3 billion in 2015, the 2020 Census would cost $17 8 billion in constant 2020 dollars if the Bureau repeated the 2010 Census design and methods, according to the Bureau’s estimates  In October 2017, Commerce announced that it had updated the October 2015 life-cycle cost estimate, projecting the life-cycle cost of the 2020 Census to be $15 6 billion, an increase of over $3 billion (27 percent) over its 2015 estimate  (See figure 1 ) In developing the 2017 version of the cost estimate, Bureau cost estimators identified cost inputs, their ranges for possible outcomes, and overall cost estimating relationships (i e , logical or mathematical formulas, or both)  To identify cost inputs and the ranges of potential outcomes, the Bureau worked with subject matter experts and used historical data to support assumptions and generate inputs  The Bureau’s cost estimation team used a software tool to generate the cost estimate  Because cost estimates predict future program costs, uncertainty is always associated with them  For example, data from the past (such as fuel prices) may not always be relevant in the future  Risk and uncertainty refer to the fact that because a cost estimate is a forecast, there is always a chance that the actual cost will differ from the estimate  One way to determine whether a program is realistically budgeted is to perform an uncertainty analysis, so that the probability associated with achieving its point estimate can be determined, usually relying on simulations such as those of Monte Carlo methods  This can be particularly useful in portraying the uncertainty implications of various cost estimates  Consistent with cost estimation practices outlined in our Cost Estimating and Assessment Guide, the estimate was compared with two independent cost estimates (ICE), developed by Commerce’s Office of Acquisition Management (OAM) and the Bureau’s Office of Cost Estimation, Analysis, and Assessment  The offices producing the ICEs and the cost estimate team worked together to examine the process each used, an effort known as the reconciliation process  Through this reconciliation, the Bureau identified areas where discrepancies existed and elements that could require additional review and possible improvement  According to Bureau documentation the estimate will be updated as the program meets milestones and to reflect changes in technical or program assumptions  Figure 2 details the Bureau’s cost estimation process  OAM was involved extensively in the development of the 2017 estimate, an increased involvement compared to 2015, according to Bureau officials  OAM participated in regular review meetings throughout the development of the estimate and also developed an independent cost estimate, as shown in the figure below  End-to-end system testing activities for the 2020 Census are currently underway in Providence, Rhode Island  According to the Bureau, information collected from the test, such as overall response rates and the use of administrative records to inform census records, will inform future versions of the life-cycle cost estimate  Some updates from the test will be incorporated into the next cost estimate, which will be available in the first quarter of the coming fiscal year  Since our June 2016 report, in which we reviewed the Bureau’s 2015 version of the cost estimate, the Bureau has made significant progress  For example, the Bureau has put into place a work breakdown structure (WBS) that defines the work, products, activities, and resources necessary to accomplish the 2020 Census and is standardized for use in budget planning, operational planning, and cost estimation  However, the Bureau’s October 2017 cost estimate for the 2020 Census does not fully reflect characteristics of a high-quality estimate as described in our Cost Estimating and Assessment Guide and cannot be considered reliable  Our Cost Estimating and Assessment Guide describes best practices for developing reliable cost estimates  For our reporting needs, we collapsed these best practices into four characteristics for sound cost estimating— comprehensive, well-documented, accurate, and credible—and identified specific best practices for each characteristic  To be considered reliable, an organization must meet or substantially meet each characteristic  Our review found the Bureau met or substantially met three out of the four characteristics of a reliable cost estimate, while it partially met one characteristic: well-documented  When compared to the October 2015 estimate, the 2017 estimate shows considerable improvement  (See figure 3 below ) Cost estimates are considered valid if they are well-documented to the point they can be easily repeated or updated and can be traced to original sources through auditing, according to best practices  The Bureau only partially met the criteria for well-documented, as set forth in our Cost Estimating and Assessment Guide  A cost estimate that does not fully meet the criteria for well-documented cannot be used by management to make informed and effective implementation decisions  The well-documented characteristic comprises five best practices  The Bureau substantially met two out of five best practices (as shown in figure 4)  First, the estimate describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost, and the cost estimate had been reviewed by management  Since cost estimates can inform key decisions and budget requests, it is vital that management review and understand how the estimate was developed, including risks associated with the underlying data and methods  The cost estimate only partially met three best practices for the characteristic of being well-documented  In general, some documentation was missing, inconsistent, or difficult to understand  First, we found that source data did not always support the information described in the basis of estimate document or could not be found in the files provided for two of the Bureau’s largest field operations: Address Canvassing and Non- Response Follow-Up (NRFU)  For example, the cost estimate documentation referred to actual data from the 2010 Census and information obtained from experts as sources for address canvassing rework rates  However, the folder source documents provided as support for the basis of estimate did not include this information  Next, in several cases, we could not replicate calculations, such as for mileage costs, using the description provided  Lastly, we found that some of the cost elements did not trace clearly to supporting spreadsheets and assumption documents  Failure to document an estimate in enough detail makes it more difficult to replicate calculations, or to detect possible errors in the estimate; reduces transparency of the estimation process; and can undermine the ability to use the information to improve future cost estimates or even to reconcile the estimate with another independent cost estimate  The Bureau told us it would continue to make improvements to ensure the estimate is well- documented  For the estimate to be considered well-documented, the Bureau will need to address these issues  An accurate cost estimate supports measurement of program progress by providing unbiased and correct data, which can help management ensure accountability for scheduled results  We found the Bureau’s cost estimate substantially met the criteria for accuracy  As shown in figure 5, and in line with best practices outlined in our Cost Estimating and Assessment Guide, the estimate was not overly optimistic; appeared to be free of errors; was based on historical data or input from subject matter experts; and, according to Bureau officials, is updated regularly as information becomes available  The Bureau can enhance the accuracy of their estimate by increasing the level of detail included in the documentation, such as detail on specific inflation indices used, and by monitoring actual costs against estimates  We identified areas for improvement, which, according to Bureau officials, will be addressed as part of its ongoing efforts  For example, while the basis of estimate document describes different inflation indexes, it was not clear exactly which indexes were applied to the various cost elements in the estimate  Also, evidence of how variances between estimated costs and actual expenses would be tracked over time was not available at the time of our analysis  Tools to track variance enable management to measure progress against planned outcomes  Bureau officials stated that they already have systems in place that can be adapted for tracking estimated and actual costs  All estimates include a certain amount of informed judgment about the future  Assumptions made at the start of a program can turn out to be inaccurate  Credible cost estimates identify limitations due to uncertainty or bias surrounding data or assumptions, and control for these uncertainties by identifying and quantifying cost elements that represent the most risk  We found that the Bureau’s cost estimate substantially met the criteria for credible, as shown in figure 6 below  The Bureau’s cost estimate clearly identifies risks and uncertainties, and describes approaches taken to mitigate them  In line with best practices outlined in our Cost Estimating and Assessment Guide, the Bureau did the following: Sensitivity analysis  The Bureau conducted sensitivity analysis to identify possible changes to estimated costs for the 2020 Census based on varying major assumptions, parameters, and data inputs  For example, the Bureau calculated the likely cost implications for a range of possible response rates to identify a range of projected costs and to calculate appropriate reserves for risk  Bureau officials stated that they also identified the estimate input parameters that contributed the most to estimate uncertainty  Risk and uncertainty analysis  A cost estimate is a forecast, and as such, there is always a chance that the actual cost will differ from the estimate  Uncertainty is the indefiniteness about the outcome of a situation  Uncertainty is assessed in cost estimate models to estimate the risk (or probability) that a specific funding level will be exceeded  We found the Bureau performed an uncertainty analysis on a portion of the estimate to determine whether estimated costs were realistic and to establish the probability of achieving projections outlined in the estimate  The Bureau used a combination of modeling based on Monte Carlo analysis and allocations of funding for risks  The Monte Carlo simulation was performed on a portion of the estimate to account for uncertainty around various operational parameters for which a range of outcomes was possible, including Internet response rates and the extent to which data collection issues might be resolved using administrative records  To account for the inherent uncertainty of assumptions included within the life-cycle cost estimate, the Bureau added funding to the cost estimate totaling approximately $292 million to account for risks based on the results of the Monte Carlo analysis  For other risks, such as acquisition lead time and the possibility of delays in information technology (IT) development, contingency funding was added to the estimate to reflect the potential cost of resolving these issues, through use of a backup system or an alternative approach  These are described as “special risks” in the Bureau’s basis of estimate, and total approximately $171 million  Based on additional sensitivity analysis, the Bureau added approximately $965 million to the cost estimate to reflect discrete risks outlined in the risk register as well as those associated with (1) variability in self-response rates, (2) the effect of fluctuations in the size and wage rate of the temporary workforce on the cost of field operations, and (3) the potential need to reduce the enumerator-to- manager staffing ratio in case expected efficiencies in field operations are not realized  In addition to these provisions, the Secretary of Commerce added a contingency amount of about $1 2 billion to account for what the Bureau refers to as unknown-unknowns  Bureau documentation states that conducting a decennial census is an extremely complex, high-risk operation  In order to mitigate some of the risk, contingency funding must be available to initiate ad hoc activities necessary to overcome unforeseen issues  According to Bureau documentation these include such risks as natural disasters or cyber-attacks  The Bureau provides a description of how the general risk contingency is calculated  However, this description does not clearly link calculated amounts to the risks themselves  In our June 2016 report we reported the Bureau had not properly accounted for risk and recommended the Bureau, in part; improve control over how risk and uncertainty are accounted for  We continue to believe the prior recommendation from our June 2016 report remains valid and should be addressed: that the Bureau properly account for risk in the 2020 Census cost estimate, among other things  As such, risks need to be linked to the $1 2 billion general risk contingency fund  Independent cost estimate  According to best practices outlined in our Cost Estimating and Assessment Guide, an independent cost estimate should be performed to determine whether alternate estimate approaches produce similar results  The Bureau compared their estimate with two independent cost estimates, developed by Commerce’s Office of Acquisition Management and the Bureau’s Office of Cost Estimation and Assessment  As part of their process for finalizing the cost estimate, Bureau officials reconciled differences between the estimates in discussions with the two offices, resulting in more conservative assumptions by the Bureau around risk and uncertainty in both cases  In addition to implementing our recommendation to properly account for risk, going forward, while the Bureau substantially met the credibility characteristic it will be important for them to also integrate regular cross-checks of methodology into their cost estimation process  In our analysis we observed that no specific cross-checks of cost methodology were performed  According to the Bureau, cross- checks were not performed because the Bureau considered the independent cost estimates as overall cross-checks on the reliability of their methodology and did not conduct additional cross-checks  The main purpose of cross-checking is to determine whether alternative methods for specific cost elements within the cost estimate could produce similar results  An independent cost estimate, though important for the credibility of an estimate, does not fulfill the same function as a targeted cross-check of individual elements  Comprehensive estimates have enough detail to ensure that cost elements are neither omitted nor double-counted, all cost-influencing assumptions are detailed in the estimate’s documentation, and a work breakdown structure is defined  Our analysis of the 2017 cost estimate demonstrates improvement over the 2015 cost estimate when the Bureau’s cost estimate only partially met the criteria for comprehensive  We found the Bureau met or substantially met all four best practices for the comprehensive characteristic, as shown in figure 7  For example, all life-cycle costs are included in the estimate along with a complete description of the 2020 Census program and current schedule  We also found that the Bureau substantially met criteria for documenting cost influencing ground rules and assumptions  A standardized WBS (as detailed in table 1) with supporting dictionary outlines the major work of the program and describes the activities and deliverables at the project level where costs are tracked  In 2016, the Bureau’s WBS did not contain sufficient detail and we found significant differences in the presentation of the work between sources  In 2017, based on our review of Bureau documentation and interviews with Bureau officials, we found that the WBS is standardized and cost elements are presented in detail  The WBS is a necessary program management tool because it provides a basic framework for a variety of related tasks like estimating costs, developing schedules, identifying resources, determining where risks may occur, and providing the means for measuring program status  Although the Bureau’s updated life-cycle cost estimate reflects three of the four characteristics of a reliable cost estimate, we are not making any new recommendations to the Bureau in this report  We continue to believe the prior recommendation, made in 2016, remains relevant: that the Secretary of Commerce ensure that the Bureau finalizes the steps needed to fully meet the characteristics of a high-quality estimate, most notably in the well-documented area  The Bureau told us it has used our best practices for cost estimation to develop their cost estimate, and will focus on those best practices that require attention moving forward  Without a reliable cost estimate, the Bureau is limited in its ability to make informed decisions about program resources and to effectively measure progress against operational objectives  OMB, in its guidance for preparing and executing agency budgets, cites that credible cost estimates are vital for sound management decision making and for any program or capital project to succeed  A well- developed cost estimate serves as a tool for program development and oversight, supporting management to make informed decisions  According to the Bureau, the 2020 Census cost estimate is used as a management tool to guide decision making  Bureau officials stated the cost estimate is used to examine the cost impact of program changes  For example, the cost estimate served as the basis for the fiscal year 2019 funding request developed by the Bureau  The Bureau also said it used the 2020 Census life-cycle cost estimate to establish cost controls during budget formulation activities and to monitor spending levels for fiscal year 2019 activities  According to the Bureau, as detailed operational and implementation plans are defined, the 2020 Census life- cycle cost estimate has been and will continue to be used to support ongoing “what-if” analyses in determining the cost impacts of design decisions  Specifically, using the cost estimate to model the impact of changes on overall cost, the Bureau adjusted the scope of the Census Enterprise Data Collection and Processing (CEDCaP) operation  The processes for developing and updating estimates are designed to inform management about program progress and the use of program resources, supporting cost-driven planning efforts and well-informed decision making  Our work has identified a number of best practices for use in developing guidance related to cost estimation and analysis that are the basis of effective program cost estimating and should result in reliable and valid cost estimates that management can use for making informed decisions  In 2012 we reported that the Bureau had not yet established guidance for developing cost estimates  We recommended that the Bureau establish guidance, policies, and procedures for developing cost estimates that would meet best practice criteria  The Bureau agreed with the theme of the report but did not specifically agree with the recommendation  Moreover, in June 2016, we also reported that the cost estimation team did not record how and why it changed assumptions that were provided to it and did not document the sources of all data it used  The documentation of these changes to assumptions did not happen because the Bureau lacked written guidance and procedures for the cost estimation team to follow  During this review we found the Bureau has since established reliable guidance, processes, and policies for developing cost estimates and managing the cost estimation process  The following documents, shown in table 2, establish roles and responsibilities for oversight and approval of cost estimation processes, provide a detailed description of the steps taken to produce a high-quality cost estimate, and clarify the process for updating the cost estimate and associated documents over the life of a project  The Decennial Census Program’s Cost Estimate and Analysis Process, which provides a detailed description of the steps taken to produce a high-quality estimate, is reliable as it met the criteria for 8 steps and substantially met the criteria for 4 steps of the 12 best steps outlined in our Cost Estimating and Assessment Guide, as shown below in figure 8  To avoid cost overruns and to support high performance, it will be important for the Bureau to abide by their newly developed policies and guidance and continue to use the life-cycle cost estimate as a management tool  The 2017 life-cycle cost estimate includes significantly higher costs than those included in the 2015 estimate  In 2015, the Bureau estimated that they could conduct the operation at a cost of $12 3 billion in constant 2020 dollars  The Bureau’s latest cost estimate, announced in October 2017, reflects the same design, but at an expected cost of $15 6 billion  Figure 9 below shows the change in cost by WBS category for 2015 and 2017  The largest increases occurred in the Response, Managerial Contingency, and Census/Survey Engineering categories  Increased costs of $1 3 billion in the response category (costs related to collecting, maintaining, and processing survey response data) were in part due to reduced assumptions for self-response rates, leading to increases in the amount of data collected in the field, which is more costly to the Bureau  Contingency allocations increased overall from $1 35 billion in 2015 to $2 6 billion in 2017, as the Bureau gained a greater understanding of risks facing the 2020 Census  Increases of $838 million in the Census/Survey Engineering category were due mainly to the cost of an IT contract for integrating decennial survey systems that was not included in the 2015 cost estimate  Bureau officials attribute a decrease of $551 million in estimated costs for Program Management to changes in the categorization of costs associated with risks: In the 2017 version of the estimate, estimated costs related to program risks were allocated to their corresponding WBS element  More generally, factors that contributed to cost fluctuations between the 2015 and 2017 cost estimates include: changes in assumptions for census operations, improved ability to anticipate and quantify risk, an overall increase in IT costs, and more defined contract requirements  Several assumptions for the implementation of the 2020 Census have changed since the 2015 cost estimate  Some assumptions contributing to cost changes, mainly in the Response (related to collecting and processing response data) and Frame (the mapping and collecting addresses to frame enumeration activities) categories, include the following: Self-response rates  Changes in assumptions for expected self- response rates contributed to increases in the response category, as the assumed rate decreased from 63 5 percent in 2015 to 60 5 percent in 2017, thereby increasing the anticipated percentage and associated cost of nonresponse follow-up  When the Bureau does not receive responses by mail, phone, or Internet, census enumerators visit each nonresponding household to obtain data  Thus, reduced self-response rates lead to increases in the amount of data collected in the field, which is more costly to the Bureau  Bureau officials attributed this decrease to a forecasted reduction in Internet response due to added authentication steps at log in and the elimination of the function allowing users to save their responses and return later to complete the survey  Productivity rates  The productivity of enumerators collecting data for NRFU is another variable in the cost estimate that was updated, contributing to cost increases in the response category  Expected productivity rates for NRFU decreased from the 2015 estimate of 4 attempts per hour to 2 9  According to Bureau documentation, this more conservative estimate is based on historical data, rather than research and test data  In-office address canvassing rates  The Bureau will not go door-to- door to conduct in-field address canvassing across the country to update address and map information for every housing unit, as it has in prior decennial censuses  Rather, some areas would only need a review of their address and map information using computer imagery and third-party data sources—what the Bureau calls “in-office” address canvassing procedures  However, in March 2017, citing budget uncertainty the Bureau decided to discontinue one of the phases of in-office review address canvassing for the 2020 Census  The cancellation of that phase of in-office review is expected to increase the number of housing units canvassed in-field by 5 percent (from 25 to 30 percent of all canvassed housing units)  In-field canvassing is more labor intensive compared to in office procedures  The 2017 version of the cost estimate reflects this increase in workload for in-field address canvassing, though overall changes in estimated costs for the Frame category, of which Address Canvassing is a part, were minimal  Staffing  Updated analysis resulted in changes to several staffing assumptions, which resulted in decreases across WBS categories  Changes included reduced pay rates for field data collection staff based on current labor market conditions and reductions in the length of staff engagement  In general, contingency allocations increased overall from $1 35 billion in 2015 to $2 6 billion in 2017  This increase in contingency can be attributed, in part, to the Bureau gaining a clearer understanding of risk and uncertainty in the 2020 Census as it approaches  The Bureau developed some of its contingency based on proven risk management techniques, including Monte Carlo analysis and allocated funding for known risk scenarios  The 2017 estimate includes close to $1 4 billion in estimated costs for these risks, almost three times the amount included in the 2015 estimate  The basis of estimate contains detail on the various risks and the process for calculating the associated contingency  The 2017 version also includes a contingency amount of $1 2 billion for general risks, or unknown-unknowns, such as natural disasters and cyber-attacks  Contingency amounts were reallocated within the WBS to more closely reflect the nature of the risk: Bureau officials attribute a decrease from the 2015 estimate of $551 million in estimated costs for program management to changes in the categorization of costs associated with risks  Officials stated that, in 2015, discrete program risks were previously consolidated as program management costs  In 2017, these discrete costs were reallocated to associate risks with the appropriate WBS element  For example, contingency amounts related to the likelihood of achieving a certain response rate previously included in the program management work breakdown category are now a part of the “response” work breakdown category  Increases in IT costs, totaling $1 59 billion, represented almost 50 percent of the total cost increase from 2015 to 2017  The total share of IT costs as a percentage of total census costs increased from 28 percent in 2015 to 32 percent in 2017, or from $3 41 billion to approximately $5 billion  Increases in IT costs are spread across seven cost categories  Figure 10 shows the IT and non-IT cost by WBS for the 2017 cost estimate  IT costs in infrastructure, response data, and census/survey WBSs account for the majority of the approximately $5 billion  The Bureau’s October 2015 cost estimate included IT costs for, among other things, system engineering, test and evaluation, and infrastructure, as well as for a portion of the Census Enterprise Data Collection and Processing (CEDCaP) program  The 2017 estimated IT cost increases were due, in large part, to the Bureau (1) updating the cost estimate for CEDCaP; (2) including an estimate for technical integration services that contributed to increases in the Census and Survey Engineering category; and (3) updating costs related to other major contracts (such as mobile device as a service, field IT services, and payroll systems)  Bureau documents described an overall improvement in the Bureau’s ability to define and specify contract requirements  This resulted in updated estimates for several contracts, including for the Census Questionnaire Assistance (CQA) contract  Assumptions regarding call volume to the CQA were increased by 5 percent to account for expected response by phone after the elimination of the option to save Internet responses and return to complete the form later  The Bureau also cited updated cost data and the results of reconciliation with independent cost estimates as factors contributing to the increased costs of other major contracts, including for the procurement of data collection devices  The Secretary of Commerce provided comments on a draft of this report on August 2, 2018  The comments are reprinted in appendix II  The Department of Commerce generally agreed with our findings regarding the improvements the Census Bureau has made in its cost estimates  However, Commerce did not agree with our assessment that the Bureau’s 2017 lifecycle cost estimate is “not reliable ” Commerce noted that it had conducted two independent cost analyses and was satisfied that the cost estimate was reliable  The Bureau also provided technical comments that we incorporated, as appropriate  We maintain that, to be considered reliable, a cost estimate must meet or substantially meet the criteria for each of the four characteristics outlined in our Cost Estimating and Assessment Guide  These characteristics are derived from measures consistently applied by cost estimating organizations throughout the federal government and industry and are considered best practices for the development of reliable cost estimates  Without a reliable cost estimate, the Bureau is limited in its ability to make informed decisions about program resources and to effectively measure progress against operational objectives  Thus, while the Bureau has made considerable progress in all four of the characteristics, it has only partially met the criteria for the characteristic of being well-documented  Until the Bureau meets or substantially meets the criteria for this characteristic, the cost estimate cannot be considered reliable  As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date  At that time, we will send copies of the report to the appropriate congressional committees, the Secretary of Commerce, the Under Secretary of Economic Affairs, the Acting Director of the U S  Census Bureau, and other interested parties  In addition, this report is available at no charge on the GAO website at http://gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2757 or goldenkoffr@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix III  The purpose of our review was to evaluate the reliability of the Census Bureau’s (Bureau) life-cycle cost estimate using our Cost Estimating and Assessment Guide  We reviewed (1) the extent to which the Bureau’s life-cycle cost estimate and associated guidance met our best practices for cost estimation using documentation and information obtained in discussions with the Bureau related to the 2020 life-cycle cost estimate and (2) compared the 2015 and 2017 life-cycle cost estimates to describe key drivers of cost growth  For both objectives we reviewed documentation from the Bureau on the 2020 life-cycle cost estimate and interviewed Bureau and Department of Commerce officials  For the first objective, we relied on our Cost Estimating and Assessment Guide as criteria  Our cost specialists assessed measures consistently applied by cost-estimating organizations throughout the federal government and industry and considered best-practices for developing reliable cost estimates  We analyzed the cost estimating practices used by the Bureau against these best practices and evaluated them in four categories: comprehensive, well-documented, accurate, and credible  Comprehensive  The cost estimate should include both government and contractor costs of the program over its full life-cycle, from inception of the program through design, development, deployment, and operation and maintenance to retirement of the program  It should also completely define the program, reflect the current schedule, and be technically reasonable  Comprehensive cost estimates should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted  Specifically, the cost estimate should be based on a product-oriented work breakdown structure (WBS) that allows a program to track cost and schedule by defined deliverables, such as hardware or software components  Finally, where information is limited and judgments are made, the cost estimate should document all cost-influencing assumptions  Well-documented  A good cost estimate—while taking the form of a single number—is supported by detailed documentation that describes how it was derived and how the expected funding will be spent in order to achieve a given objective  Therefore, the documentation should capture in writing such things as the source data used, the calculations performed and their results, and the estimating methodology used to derive each WBS element’s cost  Moreover, this information should be captured in such a way that the data used to derive the estimate can be traced back to, and verified against, their sources so that the estimate can be easily replicated and updated  The documentation should also discuss the technical baseline description and how the data were normalized  Finally, the documentation should include evidence that the cost estimate was reviewed and accepted by management  Accurate  The cost estimate should provide for results that are unbiased, and it should not be overly conservative or optimistic  An estimate is accurate when it is based on an assessment of most likely costs; adjusted properly for inflation; and contains few, if any, minor mistakes  In addition, a cost estimate should be updated regularly to reflect significant changes in the program—such as when schedules or other assumptions change—and actual costs, so that it is always reflecting current status  During the update process, variances between planned and actual costs should be documented, explained, and reviewed  Among other things, the estimate should be grounded in a historical record of cost estimating and actual experiences on other comparable programs  Credible  The cost estimate should discuss any limitations of the analysis because of uncertainty or biases surrounding data or assumptions  Major assumptions should be varied, and other outcomes recomputed to determine how sensitive they are to changes in the assumptions  Risk and uncertainty analysis should be performed to determine the level of risk associated with the estimate  Further, the estimate’s cost drivers should be cross-checked, and an independent cost estimate conducted by a group outside the acquiring organization should be developed to determine whether other estimating methods produce similar results  If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high- quality estimate and cannot be considered reliable  We also analyzed the Bureau’s cost estimation and analysis guidance and evaluated them against a 12-step process outlined in our Cost Estimation and Assessment Guide  A high-quality cost estimating process integrates the following: 1  Define estimate’s purpose  2  Develop estimating plan  3  Define program characteristics  4  Determine estimating structure  5  Identify ground rules and assumptions  6  Obtain data  7  Develop point estimate and compare it to an independent cost estimate  8  Conduct sensitivity analysis  9  Conduct risk and uncertainty analysis  10  Document the estimate  11  Present estimate to management for approval  12  Update the estimate to reflect actual costs and changes  These 12 steps, when followed correctly, should result in reliable and valid cost estimates that management can use for making informed decisions  If any of the steps in the Bureau’s process do not meet, minimally meet, or partially meet the 12 steps, then the cost estimate guidance does not fully reflect best practices for developing a high-quality estimate and cannot be considered reliable  Lastly, to describe key drivers of cost growth, we compared cost information included in the 2015 and 2017 cost estimates  We analyzed both summary and detailed cost information to assess key changes in totals overall, by WBS category, and by information technology (IT) vs  Non-IT costs  We used this analysis in conjunction with information received from the Bureau during interviews and through document transfers to describe overall changes in the cost estimate from 2015 to 2017  We conducted this performance audit from December 2017 to August 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the contact named above, Lisa Pearson (Assistant Director), Karen Cassidy (Analyst in Charge), Brian Bothwell, Jackie Chapin, Ann Czapiewski, Jason Lee, Ty Mitchell, Kayla Robinson, and Tim Wexler made significant contributions to this report 
Report 2:  The various workout arrangements were eligible for favorable CRA consideration  Banks are unlikely to receive CRA consideration for originating subprime mortgages going forward  The Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 (Dodd-Frank Act; P L  111-203 ) requires lenders to consider consumers' ability to repay before extending them mortgage credit, and one way for lenders to comply is to originate qualified mortgages (QMs) that satisfy various underwriting and product-feature requirements  For example, QMs may not have any negative amortization features, interest-only payments, or points and fees that exceed specified caps of the total loan amount; in most cases, borrowers' debt-to-income ratios shall not exceed 43%  QM originations will give lenders legal protections if the required income verification and other proper underwriting procedures were followed  Given the legal protections afforded to QMs, some banks might show greater reluctance toward making non-QM loans  With this in mind, the federal banking regulators announced that banks choosing to make only or predominately QM loans should not expect to see an adverse effect on their CRA evaluations; however, the regulators did not indicate that CRA consideration would be given for non-QMs  Arguably, the federal banking regulators appear less inclined to use the CRA to encourage lending that could be subject to greater legal risks  Banks have demonstrated interest in providing financial services such as small dollar cash advances, which are similar to payday loans, in the form of subprime credit cards, overdraft protection services, and direct deposit advances  However, banks are discouraged from engaging in payday and similar forms of lending  Legislation, such as the Credit Card Accountability Responsibility and Disclosure Act of 2009 ( P L  111-24 ), placed restrictions on subprime credit card lending  In addition, federal banking regulators expressed concern when banks began offering deposit advance products due to the similarities to payday loans  Specifically, on April 25, 2013, the OCC, FDIC, and Federal Reserve expressed concerns that the high costs and repeated extensions of credit could add to borrower default risks and issued final supervisory guidance regarding the delivery of these products  Many banks subsequently discontinued offering deposit advances  In general, these legislative and regulatory efforts explicitly discourage banks from offering high-cost consumer financial products and thus such products are unlikely to receive CRA consideration  When various financial products are deemed unsound by bank regulators and not offered by banks, a possible consequence may be that some customers migrate to nonbank institutions willing to provide these higher-cost products  Accordingly, the effectiveness of the CRA diminishes if more individuals choose to seek financial products from nonbank institutions  In general, it can be difficult to determine the extent to which banks' financial decisions are motivated by CRA incentives, profit incentives, or both  Compliance with CRA does not require banks to make unprofitable, high-risk loans that would threaten the financial health of the bank  Instead, CRA loans have profit potential; and bank regulators require all loans, including CRA loans, to be prudently underwritten  As evidenced below, it may be difficult to determine whether banks have made particular financial decisions in response to profit or CRA incentives in cases where those incentives exist simultaneously  For example, banks increased their holdings of municipal bonds in 2009  Although banks may receive CRA consideration under the investment test for purchasing state and local municipal bonds that fund public and community development projects in their designated assessment areas, banks may choose this investment for reasons unrelated to CRA  During recessions, for example, banks may reduce direct (or primary market) lending activities and increase their holdings of securities in the wake of declining demand for and supply of direct loan originations that occur during economic slowdowns and early recovery periods  In addition, a provision of the American Recovery and Reinvestment Act of 2009 ( P L  111-5 ) provided banks with a favorable tax incentive to invest in municipal bonds in the wake of the 2007-2009 recession  Hence, determining whether banks increased their municipal holdings because of a turn to securities markets for higher yields following a recession, a favorable tax incentive, or the CRA incentive is challenging  Similarly, banks increased their investments in Small Business Investment Corporations (SBICs, defined in the Appendix ) in 2010  Investments in SBICs allow banks to provide subordinate financing (rather than senior debt) to businesses  Senior lenders have first claims to the business's assets in case of failure; however, subordinate financiers provide funds in the form of mezzanine capital or equity, requiring a higher return because they are repaid after senior lenders  Banks generally are not allowed to act as subordinate financiers because they are not allowed to acquire ownership interests in private equity funds, unless such investments promote public welfare  Hence, attributing community development financing activities, such as SBIC investments, to CRA incentives may arguably be easier (relative to other financing activities) because the ability to engage in subordinate financing activities typically represents a CRA exemption from ordinary permissible banking activities  Following the 2007-2009 recession, however, U S  interest rates dropped to historically low levels for an abnormally long period of time  Because low-yielding interest rate environments squeeze profits, banks were likely to search for higher-yielding and larger-sized lending opportunities, such as investments in SBICs  Hence, it remains difficult to determine whether a particular bank's decision to increase SBIC financing activities was driven by normal profit or CRA-related incentives  Between June 2016 and June 2017, more than 1700 U S  bank branches were closed  Many branch closings occurred primarily in rural and low-income tract areas, raising concerns that banks would be able to circumvent their CRA obligation to lend and be evaluated in these areas  A traditional bank business model, however, relies primarily on having access to core deposits , a stable source of funds used to subsequently originate loans  Banks value geographic locations with greater potential to attract high core deposit volumes, which is also consistent with the CRA's requirement that assessment areas include at least one MSA or contiguous political subdivision (as previously discussed)  Furthermore, using FDIC and U S  Census Bureau data, the Federal Reserve noted that the number of branches per capita in 2017 was higher than two decades ago  Hence, determining whether branch closures reflect a bank's intentions to circumvent CRA compliance or to facilitate its ability to attract core deposits is challenging  On April 3, 2018, the U S  Department of Treasury (Treasury) released recommendations to modernize CRA in a memorandum to the federal banking regulators (OCC, FCIC, and the Fed)  Treasury highlighted four of its recommendations, summarized below  When the CRA was enacted in 1977, banks received deposits and made loans primarily through geographical branches  Assessment areas defined geographically arguably may not fully reflect the community served by a bank because of technology developments, such as the internet and mobile phone banking, prompting Treasury to call for revisiting the approach for determining banks' assessment areas  In 2016, the banking regulators issued Interagency Questions and Answers (Q&As) to provide banks guidance pertaining to CRA-eligible activities; however, Treasury noted that each regulator provides its examiners with additional guidance  Also, the Interagency Q&As illustrate past CRA-qualifying activities, but Treasury noted that no formal process currently exists to help determine whether potential (complex, innovative, or innovative) activities would qualify for CRA credit  Treasury recommends establishing clearer standards for CRA-qualifying activities and flexibility (expanding the types of loans, investments, and services that qualify for CRA credit), which may encourage banks to venture beyond activities that typically receive CRA credit  Treasury reports that each bank regulator follows a different examination schedule; the examinations are lengthy; and delays associated with the release of performance evaluations may limit the time banks can react to recommendations before their next CRA examination  Treasury recommends increasing the timeliness of the CRA examination process  Treasury recommends incorporating performance incentives that might result in more efficient lending activities  For example, CRA-qualifying loans may receive credit in the year of origination, but equity investments may receive credit each year that the investment is held  Treasury recommends consistent treatment of loans and investments, which may encourage banks to make more long-term loans (rather than sequences of short-term loans for the sake of being awarded CRA credits at each CRA examination)  On August, 28, 2018, the OCC released an Advance Notice of Proposed Rulemaking (ANPR) to seek comments on ways to modernize the CRA framework  The ANPR solicited comments on the issues raised by Treasury among other things  The OCC's ANPR does not propose specific changes, but its content and the questions posed suggest that the OCC is exploring the possibility of adopting a quantitative metric-based approach to CRA performance evaluation, changing how assessment areas are defined, expanding CRA-qualifying activities, and reducing the complexity, ambiguity, and burden of the regulations on the bank industry  When the comment period closed on November 19, 2018, the OCC had received 1584 comments  The Federal Reserve and the FDIC did not join the OCC in releasing the ANPR  The Federal Reserve System, however, did host research symposiums around the country to gather comments pertaining to CRA reform  As reported by the Federal Reserve, some banking industry comments suggested, among other things, the need for consistency of the CRA examinations to facilitate CRA compliance  Yet some tailoring may still be necessary with respect to determining assessment areas that better reflect each bank's business models, particularly for models that use technology to deliver products and services  The regulators also heard from community and consumer groups  While expressing the need to retain focus on the historical context of the CRA, these groups highlighted the need to address issues pertaining to banking deserts in underserved communities  Community development investments (CDIs) that meet public welfare investment (PWI) requirements are those that promote the public welfare, primarily resulting in economic benefits for low- and moderate-income (LMI) individuals  This appendix provides examples of CDI activities that would qualify for consideration under the CRA investment test  In many cases, covered banks are more likely to take advantage of these optional vehicles to obtain CRA credits if they perceive the underlying investment opportunities to have profit potential  Loan Participations Banks and credit unions often use participation (syndicated) loans to jointly provide credit  When a financial firm (e g , bank, credit union) originates a loan for a customer, it may decide to structure loan participation arrangements with other institutions  The loan originator often retains a larger portion of the loan and sells smaller portions to other financial institutions willing to participate  Suppose a financial firm originates a business or mortgage loan in a LMI neighborhood  A bank may receive CRA investment credit consideration by purchasing a participation, thus becoming a joint lender to the LMI borrower  An advantage of loan participations is that the default risk is divided and shared among the participating banks (as opposed to one financial firm retaining all of the risk)  CRA consideration is possible if the activity occurs within the designated assessment area  For all participating banks to receive credit, some overlap in their designated assessment areas must exist  An exception is made for participations made to benefit designated disaster areas, in which all participating banks would receive CRA consideration regardless of location  State and Local Government Bonds State and local governments issue municipal bonds, and the proceeds are used to fund public projects, community development activities, and other qualifying activities  The interest that nonbank municipal bondholders receive is exempt from federal income taxes to encourage investment in hospitals, schools, infrastructure, and community development projects that require state and local funding  Legislative actions during the 1980s eliminated the tax-exempt status of interest earned from holdings of municipal bonds for banks  Although banks no longer have a tax incentive to purchase municipal bonds, they still consider the profitability of holding these loans, as they do with all lending opportunities  Furthermore, banks receive CRA investment consideration when purchasing state and local municipal bonds that fund public and community development projects in their designated assessment areas  CRA-Targeted Secondary Market Instruments Secondary market financial products have been developed to facilitate the ability of banks to participate in lending activities eligible for CRA consideration, such as purchasing mortgage-backed securities (MBSs) or shares of real estate investment trusts (REITs)  A MBS is a pool of mortgage loans secured by residential properties; a multifamily MBS is a pool of mortgage loans secured by multifamily properties, consisting of structures designed for five or more residential units, such as apartment buildings, hospitals, nursing homes, and manufactured homes  CRA-MBSs are MBSs consisting of loans that originated in specific geographic assessment areas, thereby allowing bank purchases into these pools to be eligible for CRA consideration under the investment test  Similarly, REITs may also pool mortgages, mortgage MBSs, and real estate investments (e g , real property, apartments, office buildings, shopping malls, hotels)  Investors purchase shares in REIT pools and defer the taxes  Banks may only invest in mortgage REITs and MBS REITs  Similar to the CRA-MBSs, the REITs must consist of mortgages and MBSs that would be eligible for CRA consideration  The Community Development Trust REIT is an example of a REIT that serves as a CRA-qualified investment for banks  Community Development Financial Institutions and Equity Equivalent Investments The Community Development Financial Institutions (CDFI) Fund was created by the Riegle Community Development Regulatory Improvement Act of 1994 (the Riegle Act; P L  103-325 )  The CDFI Fund was established to promote economic development for distressed urban and rural communities  The CDFI Fund, currently located within the U S  Department of the Treasury, is authorized to certify banks, credit unions, nonprofit loan funds, and (for-profit and nonprofit) venture capital funds as designated CDFIs  In other words, a bank may satisfy the requirements to become a CDFI, but not all CDFIs are banks  The primary focus of institutions with CDFI certification is to serve the financial needs of economically distressed people and places  The designation also makes these institutions eligible to receive financial awards and other assistance from the CDFI Fund  In contrast to non-CDFI banks, some CDFI banks have greater difficulty borrowing funds and then transforming them into loans for riskier, economically distressed consumers  The lack of loan level data for most CDFI banks causes creditors to hesitate in making low-cost, short-term loans to these institutions  Specifically, the lack of information on loan defaults and prepayment rates on CDFI banking assets is likely to result in limited ability to sell these loan originations to secondary loan markets  Consequently, the retention of higher-risk loans, combined with limited access to low-cost, short-term funding, makes CDFI banks more vulnerable to liquidity shortages  Hence, CDFIs rely primarily on funding their loans (assets) with net assets , which are proceeds analogous to the equity of a traditional bank or net worth of a credit union  CDFI net assets are often acquired in the form of awards or grants from the CDFI Fund or for-profit banks  Funding assets with net assets is less expensive for CDFIs than funding with longer-term borrowings  Banks may obtain CRA investment credit consideration by making investments to CDFIs, which provides CDFIs with net assets (equity)  Under PWI authority, banks are allowed to make equity investments in specialized financial institutions, such as CDFIs, as long as they are considered by their safety and soundness regulator to be at least adequately capitalized   Furthermore, the final Basel III notification of proposed regulation (NPR) allows for preferential capital treatment for equity investments made under PWI authority, meaning equity investments to designated CDFIs may receive more favorable capital treatment  Consequently, banks often provide funds to CDFIs through equity equivalent investments (EQ2s), which are debt instruments issued by CDFIs with a continuous rolling (indeterminate) maturity  EQ2s, from a bank's perspective, are analogous to holding convertible preferred stock with a regularly scheduled repayment  Hence, banks may view EQ2s as a potentially profitable opportunity to invest in other specialized financial institutions and receive CRA consideration, particularly when the funds are subsequently used by CDFIs to originate loans in the banks' assessment areas  Small Business Investment Companies The Small Business Administration (SBA) was established in 1953 by the Small Business Act of 1953 (P L  83-163) to support small businesses' access to capital in a variety of ways  Although issuing loan guarantees for small businesses is a significant component of its operations, the SBA also has the authority to facilitate the equity financing of small business ventures through its Small Business Investment Company (SBIC) program, which was established by the Small Business Investment Act of 1958 (P L  85-699)  SBICs that are licensed and regulated by the SBA may provide debt and equity financing and, although not a program requirement, educational (management consulting) resources for businesses that meet certain SBA size requirements  Banks may act as limited partners if they choose to provide funds to SBICs, which act as general partners  Banks may establish their own SBICs, jointly establish SBICs (with other banks), or provide funds to existing SBICs  SBICs subsequently use bank funding to invest in the long-term debt and equity securities of small, independent (SBA-eligible) businesses, and banks may receive CRA investment consideration if the activities benefit their assessment areas  Community banks invest in SBICs because of the profit potential as well as the opportunity to establish long-term relationships with business clients during their infancy stages  Banks that are considered by their regulators to be adequately capitalized are allowed to invest in these specialized financial institutions under PWI authority, but the investments still receive risk-based capital treatment  SBIC assets, similar to CDFIs, are illiquid given the difficulty to obtain credit ratings for SBIC investments; thus, they cannot be sold in secondary markets  Because banks risk losing the principal of their equity investments, they are required to perform the proper due diligence associated with prudent underwriting  Tax Credits The low-income housing tax credit (LIHTC) program was created by the Tax Reform Act of 1986 ( P L  99-514 ) to encourage the development and rehabilitation of affordable rental housing  Generally speaking, government (federal or state) issued tax credits may be bought and, in many cases, sold like any other financial asset (e g , stocks and bonds)  Owners of tax credits may reduce their tax liabilities either by the amount of the credits or by using the formulas specified on those credits, assuming the owners have participated in the specified activities that the government wants to encourage  For LIHTCs, banks may use a formula to reduce their federal tax liabilities when they provide either credit or equity contributions (grants) for the construction and rehabilitation of affordable housing  If a bank also owns a LIHTC, then a percentage of the equity grant may be tax deductible if the CDFI uses the funds from the grant to finance affordable rental housing  Furthermore, banks may receive consideration for CRA-qualified investment credits  After a domestic corporation or partnership receives designation as a Community Development Entity (CDE) from the CFDI Fund, it may apply for New Markets Tax Credits (NMTCs)  Encouraging capital investments in LMI communities is the primary mission of CDEs, and CDFIs and SBICs automatically qualify as CDEs  Only CDEs are eligible to compete for NMTCs, which are allocated by the CDFI Fund via a competitive process  Once awarded an allocation of NMTCs, the CDE must obtain equity investments in exchange for the credits  Then, the equity proceeds raised must either be used to provide loans or technical assistance or deployed in eligible community investment activities  Only for-profit CDEs, however, may provide NMTCs to their investors in exchange for equity investments  Investors making for-profit CDE equity investments can use the NMTCs to reduce their tax liabilities by a certain amount over a period of years  As previously discussed, a bank may receive CRA credit for making equity investments in nonprofit CDEs and for-profit subsidiaries, particularly if the investment occurs within the bank's assessment area  Furthermore, banks may be able to reduce their tax liabilities if they can obtain NMTCs from the CDEs in which their investments were made 

ground_truth: Acquisition management has been a long-standing challenge at NASA, although GAO has reported on improvements the agency has made in recent years. Three space telescope projects are the key enablers for NASA to achieve its astrophysics' science goals, which include seeking to understand the universe. In its fiscal year 2018 budget request, NASA asked for about $697 million for these three projects, which represents over 50 percent of NASA's budget for its astrophysics' major projects. In total, these projects represent an expected investment of at least $12.4 billion. This statement reflects preliminary observations on (1) the current status and cost of NASA's major telescope projects and (2) lessons learned that can be applied to NASA's management of its telescope projects. This statement is based on ongoing work on JWST and ongoing work on the status of NASA's major projects. Both reports are planned to be published in Spring 2018. This statement is also based on past GAO reports on JWST and NASA's acquisitions of major projects, and NASA input. The National Aeronautics and Space Administration's (NASA) current portfolio of major space telescopes includes three projects that vary in cost, complexity, and phase of the acquisition life cycle. GAO's ongoing work indicates that these projects are each making progress in line with their phase of the acquisition cycle but also face some challenges. For example, the current launch date for the James Webb Space Telescope (JWST) project reflects a 57-60-month delay from the project's original schedule. GAO's preliminary observations indicate this project still has significant integration and testing to complete, with very little schedule reserve remaining to account for delays. Therefore, additional delays beyond the delay of up to 8 months recently announced are likely, and funding available under the $8 billion Congressional cost cap for formulation and development may be inadequate. There are a number of lessons learned from its acquisitions that NASA could consider to increase the likelihood of successful outcomes for its telescope projects, as well as for its larger portfolio of projects, such as its human spaceflight projects. For example, twice in the history of the JWST program, independent reviews found that the program was not holding adequate cost and schedule reserves. GAO has found that NASA has not applied this lesson learned to all of its large projects, and similar outcomes to JWST have started to emerge. For example, NASA did not incorporate this lesson with its human spaceflight programs. In July 2016 and April 2017, GAO found that these programs were holding inadequate levels of cost and schedule reserves to cover unexpected cost increases or delays. In April 2017, GAO recommended that NASA reassess the date of the programs' first test flight. NASA concurred and, in November 2017, announced a launch delay of up to 19 months. GAO is not making any recommendations in this statement, but has made recommendations in prior reports to strengthen NASA's acquisition management of its major projects. NASA has generally agreed with GAO's recommendations and taken steps to implement them.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: NASA’s mission is to drive advances in science, technology, aeronautics, and space exploration, and contribute to education, innovation, our country’s economic vitality, and the stewardship of the Earth  To accomplish this mission, NASA establishes programs and projects that rely on complex instruments and spacecraft  NASA’s portfolio of major projects ranges from space satellites equipped with advanced sensors to study the Earth to a telescope intended to explore the universe to spacecraft to transport humans and cargo to and beyond low-Earth orbit  Some of NASA’s projects are expected to incorporate new and sophisticated technologies that must operate in harsh, distant environments  The life cycle for NASA space flight projects consists of two phases— formulation, which takes a project from concept to preliminary design, and implementation, which includes building, launching, and operating the system, among other activities  NASA further divides formulation and implementation into phase A through phase F  Major projects must get approval from senior NASA officials at key decision points before they can enter each new phase  Figure 1 depicts NASA’s life cycle for space flight projects  Formulation culminates in a review at key decision point C, known as project confirmation, where cost and schedule baselines are established and documented in a decision memorandum  To inform those baselines, each project with a life-cycle cost estimated to be greater than $250 million must also develop a joint cost and schedule confidence level (JCL)  The JCL initiative, adopted in January 2009, is a point-in-time estimate that, among other things, includes all cost and schedule elements, incorporates and quantifies known risks, assesses the impacts of cost and schedule to date, and addresses available annual resources  NASA policy requires that projects be baselined and budgeted at the 70 percent confidence level  The agency baseline commitment established at key decision point C includes cost and schedule reserves held at the project—those within the project manager’s control—and NASA headquarters level  Cost reserves are for costs that are expected to be incurred—for instance, to address project risks—but are not yet allocated to a specific part of the project  Schedule reserves are extra time in project schedules that can be allocated to specific activities, elements, and major subsystems to mitigate delays or address unforeseen risks  NASA’s current portfolio of major space telescopes includes three projects—WFIRST, TESS, and JWST—that vary in cost, complexity, and phase of the acquisition life cycle  WFIRST, a project that entered the concept and technology development phase and established preliminary cost and schedule estimates in February 2016, is in the earliest stages of the acquisition life cycle  With preliminary cost estimates ranging from $3 2 billion to $3 8 billion, this project is an observatory designed to perform wide-field imaging and survey of the sky at near-infrared wavelengths to answer questions about the structure and evolution of the universe and to expand our knowledge of planets beyond our solar system  The current design includes a 2 4 meter telescope that was built and qualified for another federal agency over 10 years ago; the project is evaluating which components to reuse and which to modify, refurbish, or build new  TESS—a smaller project whose latest cost estimate is approximately $337 million—is targeted to launch in March 2018 and will be used to conduct the first extensive survey of the sky from space for transiting exoplanets  And finally, JWST, with a life-cycle cost estimate of $8 835 billion, is one of NASA’s most complex projects and top priorities  The telescope is designed to help understand the origin and destiny of the universe, the creation and evolution of the first stars and galaxies, and the formation of stars and planetary systems  With a 6 5-meter primary mirror, JWST is expected to operate at about 100 times the sensitivity of the Hubble Space Telescope  JWST’s science instruments are to detect very faint infrared sources and, as such, are required to operate at extremely cold temperatures  To help keep these instruments cold, a multi-layered tennis-court-sized sunshield is being developed to protect the mirrors and instruments from the sun’s heat  We have reported for several years on the JWST project, which has experienced significant cost increases and schedule delays  Prior to being approved for development, cost estimates for JWST ranged from $1 billion to $3 5 billion, with expected launch dates ranging from 2007 to 2011  Before 2011, early technical and management challenges, contractor performance issues, low levels of cost reserves, and poorly phased funding levels caused JWST to delay work after confirmation, which contributed to significant cost and schedule overruns, including launch delays  The Chair of the Senate Subcommittee on Commerce, Justice, Science, and Related Agencies requested from NASA an independent review of JWST in June 2010  In response, NASA commissioned the Independent Comprehensive Review Panel, which issued its report in October 2010  The panel concluded that JWST was executing well from a technical standpoint, but that the baseline cost estimate did not reflect the most probable cost with adequate reserves in each year of project execution, resulting in an unexecutable project  Following this review, Congress in November 2011 placed an $8 billion cap on the formulation and development costs for the project and NASA rebaselined JWST with a life-cycle cost estimate of $8 835 billion that included additional money for operations and a planned launch in October 2018  The new baseline represented a 78 percent increase to the project’s life-cycle cost from the original baseline and a launch date in October 2018, a delay of 52 months  The revised life-cycle cost estimate included a total of 13 months of funded schedule reserve  Our ongoing work indicates that these three projects are each making progress in line with their phase of the acquisition cycle, but also face challenges in execution  Some of these challenges are unique to the projects themselves and some are common among the projects in NASA’s portfolio  For example, when projects enter the integration and test phase, unforeseen challenges can arise and affect the cost and schedule for the project  Table 1 provides more details about the current acquisition phase, cost, and schedule status of NASA’s major space telescope projects based on our ongoing work  WFIRST  NASA’s preliminary cost and schedule estimates for the WFIRST project are currently under review as the project responds to findings in the WFIRST Independent External Technical/Management/Cost Review  This independent review was conducted to ensure the mission’s scope and required resources are well understood and executable  NASA initiated this review in April 2017 to address the National Academies’ concerns that WFIRST cost growth could endanger the balance of NASA’s astrophysics program and negatively affect other scientific priorities  The review found that the mission scope is understood, but not aligned with the resources provided and concluded that the mission is not executable without adjustments and/or additional resources  For example, the study team found that NASA’s current forecasted funding profile for the WFIRST project would require the project to slow down activities starting in fiscal year 2020, which would result in an increase in development cost and schedule  NASA agreed with the study team’s results and directed the project to reduce the cost and complexity of the design in order to maintain costs within the $3 2 billion cost target  The project is currently identifying potential ways to reduce the scope of planned activities (called “descopes”), assessing the science impact of those descopes, and then developing recommendations for the Astrophysics Division leadership  An example of a descope that may be considered is the requirement for WFIRST to be “star-shade ready,” which means the design must be compatible with a star-shade device that is positioned between it and the star being observed to block out starlight while allowing the light emitted by the planet through  TESS  The TESS project is currently holding cost and schedule reserves consistent with NASA center requirements, but there are no longer headquarters-held cost reserves to cover a delay if the project cannot launch as planned in March 2018  According to a project official, the project is holding 16 days of schedule reserve to its target March 2018 launch readiness date, which includes 6 days for the completion of integration and test, and 10 days for launch operations  The project previously used schedule reserves to accommodate the delayed delivery of its Ka-band transmitter, which is essential for TESS as it transmits the mission data back to Earth, due to continued performance and manufacturing issues  The two main risks to the March 2018 launch date are if: 1) SpaceX requires additional time past December 2017 for NASA’s Launch Services Program to certify that TESS can fly on its upgraded launch vehicle—certification is necessary because it will be the first time that NASA will use this version of the vehicle—and 2) any issues are identified during the remainder of environmental testing  The project is also conducting additional testing on its spare camera at temperatures seen in space to better understand expected camera performance on orbit  TESS will use four identical, wide field-of-view cameras to conduct the first extensive survey of the sky from space for transiting exoplanets  However, during thermal testing, the project found that the substance attaching the lenses to the camera barrel places pressure on the lenses and causes the cameras to be slightly out of focus  In June 2017, NASA directed the project to proceed with integrating the cameras—as they are expected to meet TESS’s top level science requirements even with the anomaly  At its most recent key decision review in August 2017, NASA reallocated $15 million of TESS’s headquarters-held reserves to the WFIRST project  While this had the effect of decreasing life cycle costs for TESS, it also increased risk as the project no longer has any additional headquarters-held cost reserves to cover a launch delay past March 2018  JWST  The JWST project continues to make progress towards launch, but the program is encountering technical challenges that require both time and money to fix and may lead to additional delays, beyond a delay recently announced  While the project has made much progress on hardware integration and testing over the past several months, it also used all of its remaining schedule reserves to address various technical issues, particularly on the spacecraft element  In September 2017, the JWST project requested from the European Space Agency—who will contribute the Ariane V launch vehicle—a launch window from March to June 2019, or 5 to 8 months later than the planned October 2018 launch readiness date, established in 2011  The project based this request on the results of a schedule risk assessment that incorporated inputs from the contractor on expected durations of ongoing spacecraft element integration work and other challenges that were expected to increase schedule  With the later launch window to June 2019, the project expected to have up to 4 months of new schedule reserves  However, shortly after requesting the revised launch window, the project learned from its contractor that up to another 3 months of schedule reserve use is likely, due to lessons learned from conducting deployment exercises of the sunshield, such as reach and access limitations on the flight hardware  As a result, and pending further examination of the schedule, the project now has approximately one month of schedule reserve to complete environmental testing of the spacecraft element and the final integration phase  The final integration phase is where the instruments and telescope will be integrated with the spacecraft and sunshield to form the completed observatory  As I previously noted, our work has shown the integration and test is the riskiest phase of development, where problems are most likely to be found and schedules slip  Given the risks associated with the integration and test work ahead, coupled with a level of schedule reserves that is currently well below the level stated in the procedural requirements issued by the NASA center responsible for managing JWST, additional delays to the project’s revised launch readiness date of June 2019 are likely  As a result, the funding available under the Congressional cost cap of $8 billion may be inadequate as the contractor will need to continue to retain higher workforce levels for longer than expected to prepare the mission for a delayed launch  As Congress, NASA, and the science community consider future telescope efforts, it will be exceedingly important to shape and manage new programs in a manner that minimizes cost overruns and schedule delays  This is particularly important for the largest programs as even small cost increases can have reverberating effects  NASA’s telescope and other science projects will always have inherent technical, design, and integration risks because they are complex, specialized, and often push the state of the art in space technology  But too often, our reports find that management and oversight problems—which can include poor planning, optimistic cost estimating, funding gaps, lax oversight, and poor contractor performance, among other issues—are the real drivers behind cost and schedule growth  To its credit, NASA has taken significant steps, partly in response to our past recommendations, to reduce acquisition risk from both a technical and management standpoint, including actions to enhance cost and schedule estimating, provide adequate levels of reserves to projects, establish better processes and metrics to monitor projects, and expand the use of earned value management to better monitor contractor performance  For example, in November 2012, we found that NASA employee skill sets available to analyze and implement earned value management vary widely from center to center, and we recommended that NASA conduct an earned value management skills gap analysis to identify areas requiring augmented capability across the agency, and, based on the results of the assessment, develop a workforce training plan to address any deficiencies  NASA concurred with this recommendation and developed an earned value management training plan in 2014 based on the results of an earned value management skills gap analysis that was conducted in 2013  Moreover, in recent years, we have found that many of the projects within the agency’s major project portfolio have improved their cost and schedule performance  Nevertheless, the extent to which NASA has adopted some of the following lessons learned within its portfolio of major projects is mixed, and NASA has an opportunity to strengthen its program management of major acquisitions, including its space telescopes, by doing so  Manage Cost and Schedule Performance for Large Projects to Limit Implications for Entire Portfolio  In 2013, following JWST’s cost increases and schedule growth, we found that though cost and schedule growth can occur on any project, increases associated with NASA’s most costly and complex missions can have cascading effects on the rest of the portfolio  For example, we found that the JWST cost growth would have reverberating effects on the portfolio for years to come and required the agency to identify $1 4 billion in additional resources over fiscal years 2012 through 2017, according to Science Mission Directorate officials  NASA identified approximately half of this required funding from the four science divisions within the Science Mission Directorate account  The majority of the cuts were related to future high priority missions, missions in the operations and sustainment phase, and research and analysis  In essence, NASA had to mortgage future high priority missions and research to address JWST’s additional resource needs  Similarly, the National Academy of Sciences has concluded in the past that it is important for NASA to have a clearly articulated and consistently applied method for prioritizing why and how its scarce fiscal resources are apportioned with respect to the science program in general and on a more granular level among component scientific disciplines  The academy noted that failure to do so could result in a loss of capacity, capability, and human resources in a number of scientific disciplines and technological areas that may take a generation or more to reconstitute once eliminated  NASA’s establishment of the WFIRST Independent External Technical/Management/Cost Review that I previously discussed is a step in the right direction to help ensure the Astrophysics Division incorporates this lesson learned  Establish Adequate Cost and Schedule Reserves to Address Risks  Twice in the history of the JWST program, independent reviewers found that the program’s planned cost reserves were inadequate  First, in April 2006, an Independent Review Team confirmed that the project’s technical content was complete and sound, but expressed concern over the project’s reserve funding, reporting that it was too low and phased in too late in the development lifecycle  The review team reported that for a project as complex as JWST, 25 to 30 percent total reserve funding was appropriate  The team cautioned that low reserve funding compromised the project’s ability to resolve issues, address risk areas, and accommodate unknown problems  As I previously mentioned, following additional cost increases and schedule threats, NASA commissioned the Independent Comprehensive Review Panel  In 2010, the panel again concluded JWST was executing well from a technical standpoint, but that the baseline cost estimate did not reflect the most probable cost with adequate reserves in each year of project execution, resulting in an unexecutable project  NASA heeded these lessons when it established a new baseline for JWST in 2011  For example, the revised schedule included more reserves than required by the procedural requirements issued by the NASA center responsible for managing JWST  We have found, however, that NASA has not applied this lesson learned to all of its large projects— most notably with its human spaceflight projects, including the Space Launch System, Orion Crew Capsule, and associated ground systems— and similar outcomes to the JWST project have started to emerge with these projects  We previously reported that all three of these programs were operating with limited cost reserves, which limited each program’s ability to address risks and unforeseen technical challenges  For example, we found in July 2016 that the Orion program planned to maintain very low levels of annual cost reserves until 2018  The lack of available cost reserves in the near term led to the program deferring work to address technical issues to stay within budget, and put the program’s future cost reserves at risk of being overwhelmed by deferred work  In April 2017, we also found that all three programs faced development challenges in completing work, and each had little to no schedule reserve remaining to the launch date—meaning they would have to complete all remaining work with minimal delay during the most challenging stage of development  We found that it was unlikely that the programs would achieve the planned launch readiness date and recommended that NASA reassess the date  NASA agreed with this recommendation and stated that it would establish a new launch readiness date  In November 2017, NASA announced that a review of the possible manufacturing and production schedule risks indicated a launch date of June 2020—a delay of 19 months—but the agency will manage to a December 2019 launch date because, according to NASA, they have put in mitigation strategies for those risks  We will follow-up on those mitigation strategies as part of future work on the human space exploration programs  Regularly and Consistently Update Project JCLs to Provide Realistic Estimates to Decision Makers  In 2009, NASA began requiring that programs and projects with estimated life-cycle costs greater than $250 million develop a JCL prior to project confirmation  This was a positive step for NASA to help ensure that cost and schedule estimates are realistic and projects are thoroughly planning for anticipated risks  This is because a JCL assigns a confidence level, or likelihood, of a project meeting its cost and schedule estimates  Our cost estimating best practices recommend that cost estimates should be updated to reflect changes to a program or be kept current as a program moves through milestones  As new risks emerge on a project, an updated cost and schedule risk analysis can provide realistic estimates to decision-makers, including the Congress  This is especially true for NASA’s largest projects as updated estimates may require the Congress to consider a variety of actions  However, there is no requirement for NASA projects to update their JCLs, and our prior work has found that projects—including JWST—do not regularly update cost risk analyses to take into account newly emerged risks  Our ongoing work indicates that of the 16 major projects currently in NASA’s portfolio that have developed JCL estimates, only 2 have reported updating their JCLs (other than required due to a rebaseline)  For example, the Interior Exploration using Seismic Investigations, Geodesy, and Heat Transport Project (InSight), a Mars lander, updated its JCL after the project missed its committed launch date  As a result, the project was able to provide additional information to decision makers about the probability that it will meet its revised cost and schedule estimates  As a project reaches the later stages of development, especially integration and testing, the types of risks the project will face may change  An updated project JCL would provide both project and agency management with data on relevant risks that can guide the project decisions  For example, in December 2012, we recommended the JWST project update its JCL  NASA concurred with this recommendation; however, we recently closed the recommendation because NASA had not taken steps to implement it and the amount of time remaining before launch would not have allowed the benefit of implementing the recommendation to be realized  An updated JCL may have portended the current schedule delays, which could have been proactively addressed by the project  Enhance Oversight of Contractors to Improve Project Outcomes  In December 2012, we found that the JWST project had taken steps to enhance communications with and oversight of its contractors  According to project officials, the increased communication allowed them to better identify and manage project risks by having more visibility into contractors’ activities  The project reported that a great deal of communication existed across the project prior to the Independent Comprehensive Review Panel; however, additional improvements were made  For example, the project increased its presence at contractor facilities as necessary to provide assistance; this included assigning two engineers on a recurring basis at a Lockheed Martin facility to assist in solving problems with an instrument  The JWST project also assumed full responsibility for the mission system engineering functions from Northrop Grumman in March 2011  NASA and Northrop Grumman officials both said that NASA is better suited to perform these tasks  We continue to see instances in our ongoing work that highlight the importance of implementing this lesson learned from JWST  For example, we found in 2017 that the Space Network Ground Segment Sustainment project—a project that plans to develop and deliver a new ground system for one Space Network site that provides essential communications tracking services to NASA and non-NASA missions—exceeded its original cost baseline by at least $401 7 million and been delayed by 27 months  The project has attributed some of the cost overruns and schedule delays to the contractor’s incomplete understanding of its requirements, which led to poor contractor plans and late design changes  The project also took steps to assign a new NASA project manager, increase physical presence at the contractor facility, and have more staff focused on validation and verification activities  In summary, NASA continues to make progress developing its space telescopes to help understand the universe and our place in it  But much like other major projects that NASA is developing, there continues to be an opportunity for NASA to learn from JWST and other projects that have suffered from cost overruns and schedule delays  Key project management tools and prior GAO recommendations that I have highlighted here today, could help to better position these large, complex, and technically challenging efforts for a successful outcome  We look forward to continuing to work with NASA and this subcommittee in addressing these issues  Chairman Babin, Ranking Member Bera, and Members of the Subcommittee, this completes my prepared statement  I would be pleased to respond to any questions that you may have at this time  If you or your staff have any questions about this testimony, please contact Cristina T  Chaplain, Director, Acquisition and Sourcing Management at (202) 512-4841 or chaplainc@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement  GAO staff who made key contributions to this statement include Molly Traci, Assistant Director; Richard Cederholm, Assistant Director; Carrie Rogers; Lisa Fisher; Laura Greifner; Erin Kennedy; and Jose Ramos  This is a work of the U S  government and is not subject to copyright protection in the United States  The published product may be reproduced and distributed in its entirety without further permission from GAO  However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately 


Report 2:  The various workout arrangements were eligible for favorable CRA consideration  Banks are unlikely to receive CRA consideration for originating subprime mortgages going forward  The Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 (Dodd-Frank Act; P L  111-203 ) requires lenders to consider consumers' ability to repay before extending them mortgage credit, and one way for lenders to comply is to originate qualified mortgages (QMs) that satisfy various underwriting and product-feature requirements  For example, QMs may not have any negative amortization features, interest-only payments, or points and fees that exceed specified caps of the total loan amount; in most cases, borrowers' debt-to-income ratios shall not exceed 43%  QM originations will give lenders legal protections if the required income verification and other proper underwriting procedures were followed  Given the legal protections afforded to QMs, some banks might show greater reluctance toward making non-QM loans  With this in mind, the federal banking regulators announced that banks choosing to make only or predominately QM loans should not expect to see an adverse effect on their CRA evaluations; however, the regulators did not indicate that CRA consideration would be given for non-QMs  Arguably, the federal banking regulators appear less inclined to use the CRA to encourage lending that could be subject to greater legal risks  Banks have demonstrated interest in providing financial services such as small dollar cash advances, which are similar to payday loans, in the form of subprime credit cards, overdraft protection services, and direct deposit advances  However, banks are discouraged from engaging in payday and similar forms of lending  Legislation, such as the Credit Card Accountability Responsibility and Disclosure Act of 2009 ( P L  111-24 ), placed restrictions on subprime credit card lending  In addition, federal banking regulators expressed concern when banks began offering deposit advance products due to the similarities to payday loans  Specifically, on April 25, 2013, the OCC, FDIC, and Federal Reserve expressed concerns that the high costs and repeated extensions of credit could add to borrower default risks and issued final supervisory guidance regarding the delivery of these products  Many banks subsequently discontinued offering deposit advances  In general, these legislative and regulatory efforts explicitly discourage banks from offering high-cost consumer financial products and thus such products are unlikely to receive CRA consideration  When various financial products are deemed unsound by bank regulators and not offered by banks, a possible consequence may be that some customers migrate to nonbank institutions willing to provide these higher-cost products  Accordingly, the effectiveness of the CRA diminishes if more individuals choose to seek financial products from nonbank institutions  In general, it can be difficult to determine the extent to which banks' financial decisions are motivated by CRA incentives, profit incentives, or both  Compliance with CRA does not require banks to make unprofitable, high-risk loans that would threaten the financial health of the bank  Instead, CRA loans have profit potential; and bank regulators require all loans, including CRA loans, to be prudently underwritten  As evidenced below, it may be difficult to determine whether banks have made particular financial decisions in response to profit or CRA incentives in cases where those incentives exist simultaneously  For example, banks increased their holdings of municipal bonds in 2009  Although banks may receive CRA consideration under the investment test for purchasing state and local municipal bonds that fund public and community development projects in their designated assessment areas, banks may choose this investment for reasons unrelated to CRA  During recessions, for example, banks may reduce direct (or primary market) lending activities and increase their holdings of securities in the wake of declining demand for and supply of direct loan originations that occur during economic slowdowns and early recovery periods  In addition, a provision of the American Recovery and Reinvestment Act of 2009 ( P L  111-5 ) provided banks with a favorable tax incentive to invest in municipal bonds in the wake of the 2007-2009 recession  Hence, determining whether banks increased their municipal holdings because of a turn to securities markets for higher yields following a recession, a favorable tax incentive, or the CRA incentive is challenging  Similarly, banks increased their investments in Small Business Investment Corporations (SBICs, defined in the Appendix ) in 2010  Investments in SBICs allow banks to provide subordinate financing (rather than senior debt) to businesses  Senior lenders have first claims to the business's assets in case of failure; however, subordinate financiers provide funds in the form of mezzanine capital or equity, requiring a higher return because they are repaid after senior lenders  Banks generally are not allowed to act as subordinate financiers because they are not allowed to acquire ownership interests in private equity funds, unless such investments promote public welfare  Hence, attributing community development financing activities, such as SBIC investments, to CRA incentives may arguably be easier (relative to other financing activities) because the ability to engage in subordinate financing activities typically represents a CRA exemption from ordinary permissible banking activities  Following the 2007-2009 recession, however, U S  interest rates dropped to historically low levels for an abnormally long period of time  Because low-yielding interest rate environments squeeze profits, banks were likely to search for higher-yielding and larger-sized lending opportunities, such as investments in SBICs  Hence, it remains difficult to determine whether a particular bank's decision to increase SBIC financing activities was driven by normal profit or CRA-related incentives  Between June 2016 and June 2017, more than 1700 U S  bank branches were closed  Many branch closings occurred primarily in rural and low-income tract areas, raising concerns that banks would be able to circumvent their CRA obligation to lend and be evaluated in these areas  A traditional bank business model, however, relies primarily on having access to core deposits , a stable source of funds used to subsequently originate loans  Banks value geographic locations with greater potential to attract high core deposit volumes, which is also consistent with the CRA's requirement that assessment areas include at least one MSA or contiguous political subdivision (as previously discussed)  Furthermore, using FDIC and U S  Census Bureau data, the Federal Reserve noted that the number of branches per capita in 2017 was higher than two decades ago  Hence, determining whether branch closures reflect a bank's intentions to circumvent CRA compliance or to facilitate its ability to attract core deposits is challenging  On April 3, 2018, the U S  Department of Treasury (Treasury) released recommendations to modernize CRA in a memorandum to the federal banking regulators (OCC, FCIC, and the Fed)  Treasury highlighted four of its recommendations, summarized below  When the CRA was enacted in 1977, banks received deposits and made loans primarily through geographical branches  Assessment areas defined geographically arguably may not fully reflect the community served by a bank because of technology developments, such as the internet and mobile phone banking, prompting Treasury to call for revisiting the approach for determining banks' assessment areas  In 2016, the banking regulators issued Interagency Questions and Answers (Q&As) to provide banks guidance pertaining to CRA-eligible activities; however, Treasury noted that each regulator provides its examiners with additional guidance  Also, the Interagency Q&As illustrate past CRA-qualifying activities, but Treasury noted that no formal process currently exists to help determine whether potential (complex, innovative, or innovative) activities would qualify for CRA credit  Treasury recommends establishing clearer standards for CRA-qualifying activities and flexibility (expanding the types of loans, investments, and services that qualify for CRA credit), which may encourage banks to venture beyond activities that typically receive CRA credit  Treasury reports that each bank regulator follows a different examination schedule; the examinations are lengthy; and delays associated with the release of performance evaluations may limit the time banks can react to recommendations before their next CRA examination  Treasury recommends increasing the timeliness of the CRA examination process  Treasury recommends incorporating performance incentives that might result in more efficient lending activities  For example, CRA-qualifying loans may receive credit in the year of origination, but equity investments may receive credit each year that the investment is held  Treasury recommends consistent treatment of loans and investments, which may encourage banks to make more long-term loans (rather than sequences of short-term loans for the sake of being awarded CRA credits at each CRA examination)  On August, 28, 2018, the OCC released an Advance Notice of Proposed Rulemaking (ANPR) to seek comments on ways to modernize the CRA framework  The ANPR solicited comments on the issues raised by Treasury among other things  The OCC's ANPR does not propose specific changes, but its content and the questions posed suggest that the OCC is exploring the possibility of adopting a quantitative metric-based approach to CRA performance evaluation, changing how assessment areas are defined, expanding CRA-qualifying activities, and reducing the complexity, ambiguity, and burden of the regulations on the bank industry  When the comment period closed on November 19, 2018, the OCC had received 1584 comments  The Federal Reserve and the FDIC did not join the OCC in releasing the ANPR  The Federal Reserve System, however, did host research symposiums around the country to gather comments pertaining to CRA reform  As reported by the Federal Reserve, some banking industry comments suggested, among other things, the need for consistency of the CRA examinations to facilitate CRA compliance  Yet some tailoring may still be necessary with respect to determining assessment areas that better reflect each bank's business models, particularly for models that use technology to deliver products and services  The regulators also heard from community and consumer groups  While expressing the need to retain focus on the historical context of the CRA, these groups highlighted the need to address issues pertaining to banking deserts in underserved communities  Community development investments (CDIs) that meet public welfare investment (PWI) requirements are those that promote the public welfare, primarily resulting in economic benefits for low- and moderate-income (LMI) individuals  This appendix provides examples of CDI activities that would qualify for consideration under the CRA investment test  In many cases, covered banks are more likely to take advantage of these optional vehicles to obtain CRA credits if they perceive the underlying investment opportunities to have profit potential  Loan Participations Banks and credit unions often use participation (syndicated) loans to jointly provide credit  When a financial firm (e g , bank, credit union) originates a loan for a customer, it may decide to structure loan participation arrangements with other institutions  The loan originator often retains a larger portion of the loan and sells smaller portions to other financial institutions willing to participate  Suppose a financial firm originates a business or mortgage loan in a LMI neighborhood  A bank may receive CRA investment credit consideration by purchasing a participation, thus becoming a joint lender to the LMI borrower  An advantage of loan participations is that the default risk is divided and shared among the participating banks (as opposed to one financial firm retaining all of the risk)  CRA consideration is possible if the activity occurs within the designated assessment area  For all participating banks to receive credit, some overlap in their designated assessment areas must exist  An exception is made for participations made to benefit designated disaster areas, in which all participating banks would receive CRA consideration regardless of location  State and Local Government Bonds State and local governments issue municipal bonds, and the proceeds are used to fund public projects, community development activities, and other qualifying activities  The interest that nonbank municipal bondholders receive is exempt from federal income taxes to encourage investment in hospitals, schools, infrastructure, and community development projects that require state and local funding  Legislative actions during the 1980s eliminated the tax-exempt status of interest earned from holdings of municipal bonds for banks  Although banks no longer have a tax incentive to purchase municipal bonds, they still consider the profitability of holding these loans, as they do with all lending opportunities  Furthermore, banks receive CRA investment consideration when purchasing state and local municipal bonds that fund public and community development projects in their designated assessment areas  CRA-Targeted Secondary Market Instruments Secondary market financial products have been developed to facilitate the ability of banks to participate in lending activities eligible for CRA consideration, such as purchasing mortgage-backed securities (MBSs) or shares of real estate investment trusts (REITs)  A MBS is a pool of mortgage loans secured by residential properties; a multifamily MBS is a pool of mortgage loans secured by multifamily properties, consisting of structures designed for five or more residential units, such as apartment buildings, hospitals, nursing homes, and manufactured homes  CRA-MBSs are MBSs consisting of loans that originated in specific geographic assessment areas, thereby allowing bank purchases into these pools to be eligible for CRA consideration under the investment test  Similarly, REITs may also pool mortgages, mortgage MBSs, and real estate investments (e g , real property, apartments, office buildings, shopping malls, hotels)  Investors purchase shares in REIT pools and defer the taxes  Banks may only invest in mortgage REITs and MBS REITs  Similar to the CRA-MBSs, the REITs must consist of mortgages and MBSs that would be eligible for CRA consideration  The Community Development Trust REIT is an example of a REIT that serves as a CRA-qualified investment for banks  Community Development Financial Institutions and Equity Equivalent Investments The Community Development Financial Institutions (CDFI) Fund was created by the Riegle Community Development Regulatory Improvement Act of 1994 (the Riegle Act; P L  103-325 )  The CDFI Fund was established to promote economic development for distressed urban and rural communities  The CDFI Fund, currently located within the U S  Department of the Treasury, is authorized to certify banks, credit unions, nonprofit loan funds, and (for-profit and nonprofit) venture capital funds as designated CDFIs  In other words, a bank may satisfy the requirements to become a CDFI, but not all CDFIs are banks  The primary focus of institutions with CDFI certification is to serve the financial needs of economically distressed people and places  The designation also makes these institutions eligible to receive financial awards and other assistance from the CDFI Fund  In contrast to non-CDFI banks, some CDFI banks have greater difficulty borrowing funds and then transforming them into loans for riskier, economically distressed consumers  The lack of loan level data for most CDFI banks causes creditors to hesitate in making low-cost, short-term loans to these institutions  Specifically, the lack of information on loan defaults and prepayment rates on CDFI banking assets is likely to result in limited ability to sell these loan originations to secondary loan markets  Consequently, the retention of higher-risk loans, combined with limited access to low-cost, short-term funding, makes CDFI banks more vulnerable to liquidity shortages  Hence, CDFIs rely primarily on funding their loans (assets) with net assets , which are proceeds analogous to the equity of a traditional bank or net worth of a credit union  CDFI net assets are often acquired in the form of awards or grants from the CDFI Fund or for-profit banks  Funding assets with net assets is less expensive for CDFIs than funding with longer-term borrowings  Banks may obtain CRA investment credit consideration by making investments to CDFIs, which provides CDFIs with net assets (equity)  Under PWI authority, banks are allowed to make equity investments in specialized financial institutions, such as CDFIs, as long as they are considered by their safety and soundness regulator to be at least adequately capitalized   Furthermore, the final Basel III notification of proposed regulation (NPR) allows for preferential capital treatment for equity investments made under PWI authority, meaning equity investments to designated CDFIs may receive more favorable capital treatment  Consequently, banks often provide funds to CDFIs through equity equivalent investments (EQ2s), which are debt instruments issued by CDFIs with a continuous rolling (indeterminate) maturity  EQ2s, from a bank's perspective, are analogous to holding convertible preferred stock with a regularly scheduled repayment  Hence, banks may view EQ2s as a potentially profitable opportunity to invest in other specialized financial institutions and receive CRA consideration, particularly when the funds are subsequently used by CDFIs to originate loans in the banks' assessment areas  Small Business Investment Companies The Small Business Administration (SBA) was established in 1953 by the Small Business Act of 1953 (P L  83-163) to support small businesses' access to capital in a variety of ways  Although issuing loan guarantees for small businesses is a significant component of its operations, the SBA also has the authority to facilitate the equity financing of small business ventures through its Small Business Investment Company (SBIC) program, which was established by the Small Business Investment Act of 1958 (P L  85-699)  SBICs that are licensed and regulated by the SBA may provide debt and equity financing and, although not a program requirement, educational (management consulting) resources for businesses that meet certain SBA size requirements  Banks may act as limited partners if they choose to provide funds to SBICs, which act as general partners  Banks may establish their own SBICs, jointly establish SBICs (with other banks), or provide funds to existing SBICs  SBICs subsequently use bank funding to invest in the long-term debt and equity securities of small, independent (SBA-eligible) businesses, and banks may receive CRA investment consideration if the activities benefit their assessment areas  Community banks invest in SBICs because of the profit potential as well as the opportunity to establish long-term relationships with business clients during their infancy stages  Banks that are considered by their regulators to be adequately capitalized are allowed to invest in these specialized financial institutions under PWI authority, but the investments still receive risk-based capital treatment  SBIC assets, similar to CDFIs, are illiquid given the difficulty to obtain credit ratings for SBIC investments; thus, they cannot be sold in secondary markets  Because banks risk losing the principal of their equity investments, they are required to perform the proper due diligence associated with prudent underwriting  Tax Credits The low-income housing tax credit (LIHTC) program was created by the Tax Reform Act of 1986 ( P L  99-514 ) to encourage the development and rehabilitation of affordable rental housing  Generally speaking, government (federal or state) issued tax credits may be bought and, in many cases, sold like any other financial asset (e g , stocks and bonds)  Owners of tax credits may reduce their tax liabilities either by the amount of the credits or by using the formulas specified on those credits, assuming the owners have participated in the specified activities that the government wants to encourage  For LIHTCs, banks may use a formula to reduce their federal tax liabilities when they provide either credit or equity contributions (grants) for the construction and rehabilitation of affordable housing  If a bank also owns a LIHTC, then a percentage of the equity grant may be tax deductible if the CDFI uses the funds from the grant to finance affordable rental housing  Furthermore, banks may receive consideration for CRA-qualified investment credits  After a domestic corporation or partnership receives designation as a Community Development Entity (CDE) from the CFDI Fund, it may apply for New Markets Tax Credits (NMTCs)  Encouraging capital investments in LMI communities is the primary mission of CDEs, and CDFIs and SBICs automatically qualify as CDEs  Only CDEs are eligible to compete for NMTCs, which are allocated by the CDFI Fund via a competitive process  Once awarded an allocation of NMTCs, the CDE must obtain equity investments in exchange for the credits  Then, the equity proceeds raised must either be used to provide loans or technical assistance or deployed in eligible community investment activities  Only for-profit CDEs, however, may provide NMTCs to their investors in exchange for equity investments  Investors making for-profit CDE equity investments can use the NMTCs to reduce their tax liabilities by a certain amount over a period of years  As previously discussed, a bank may receive CRA credit for making equity investments in nonprofit CDEs and for-profit subsidiaries, particularly if the investment occurs within the bank's assessment area  Furthermore, banks may be able to reduce their tax liabilities if they can obtain NMTCs from the CDEs in which their investments were made 

ground_truth: In October 2017, the Department of Commerce (Commerce) announced that the projected life-cycle cost of the 2020 Census had climbed to $15.6 billion, a more than $3 billion (27 percent) increase over its 2015 estimate. A high-quality, reliable cost estimate is a key tool for budgeting, planning, and managing the 2020 Census. Without this capability, the Bureau is at risk of experiencing program cost overruns, missed deadlines, and performance shortfalls. GAO was asked to evaluate the reliability of the Bureau's life-cycle cost estimate. This report evaluates the reliability of the Bureau's revised life-cycle cost estimate for the 2020 Census and the extent to which the Bureau is using it as a management tool, and compares the 2015 and 2017 cost estimates to describe key drivers of cost growth. GAO reviewed documentary and testimonial evidence from Bureau officials responsible for developing the 2020 Census cost estimate and used its cost assessment guide ( GAO-09-3SP ) as criteria. Since 2015, the Census Bureau (Bureau) has made significant progress in improving its ability to develop a reliable cost estimate. While improvements have been made, the Bureau's October 2017 cost estimate for the 2020 Census does not fully reflect all the characteristics of a reliable estimate. (See figure.) Specifically, for the characteristic of being well-documented, GAO found that some of the source data either did not support the information described in the cost estimate or was not in the files provided for two of its largest field operations. In GAO's assessment of the 2015 version of the 2020 Census cost estimate, GAO recommended that the Bureau take steps to ensure that each of the characteristics of a reliable cost estimate is met. The Bureau agreed and has taken steps, but has not fully implemented this recommendation. A reliable cost estimate serves as a tool for program development and oversight, helping management make informed decisions. During this review, GAO found the Bureau used the cost estimate to inform decision making. Factors that contributed to cost fluctuations between the 2015 and 2017 cost estimates include: Changes in assumptions. Among other changes, a decrease in the assumed rate for self-response from 63.5 percent in 2015 to 60.5 percent in 2017 increased the cost of collecting responses from nonresponding housing units. Improved ability to anticipate and quantify risk. In general, contingency allocations designed to address the effects of potential risks increased overall from $1.3 billion in 2015 to $2.6 billion in 2017. An overall increase in information technology (IT) costs. IT cost increases, totaling $1.59 billion, represented almost 50 percent of the total cost increase from 2015 to 2017. GAO is not making any new recommendations but maintains its earlier recommendation—that the Secretary of Commerce direct the Bureau to take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate. In its response to this report, Commerce generally agreed with the findings related to cost estimation improvements, but disagreed that the cost estimate was not reliable. However, until GAO's recommendation is fully implemented the cost estimate cannot be considered reliable.