prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.



Report 2: According to NRC’s website, the higher the radiation dose, the sooner the effects of radiation will appear, and the higher the probability of death  Radiation doses such as those received by survivors of the atomic bombs in Japan can cause cancers such as leukemia and colon cancer and, if levels are high enough, acute radiation syndrome  The symptoms of this syndrome range from nausea, fatigue, and vomiting to death within days or weeks  In contrast, the effects of low-dose radiation are more difficult to detect  In particular, below about 100 millisieverts (mSv) (10 rem)—the level below which the National Academies of Sciences, Engineering, and Medicine’s (National Academies) 2006 report on radiation and human health considered radiation to be low dose—data do not definitively establish the dose-response relationship between cancer and radiation exposure  In developing and applying radiation protection requirements and guidance for workers and the public—specifically, limits on dose or increased health risk and guidance levels on exposure—EPA, NRC, DOE, and FDA have generally taken the advice of scientific advisory bodies  In particular, they have relied on the advice of the International Commission on Radiological Protection, the National Council on Radiation Protection and Measurements, and the National Academies’ Nuclear and Radiation Studies Board  This advice includes the use of the linear no-threshold model, which assumes that the risk of cancer increases with every incremental increase in radiation exposure  For example, the National Academies published a report in 2006 stating that the balance of evidence from various types of studies tends to favor a simple proportionate relationship between radiation at low doses and cancer risk  According to the National Academies, the availability of new and more extensive data since the publication of its previous report in 1990 strengthened confidence in the 2006 report’s estimates of cancer risk  The advisory bodies have recognized challenges in accurately estimating cancer risks from very low doses of radiation exposure when using the linear no-threshold model  For example, much of the data on health effects of radiation exposure come from non-U S  populations, such as Japanese atomic bomb survivors  These individuals received a large exposure to radiation over a short period of time (an acute exposure), and there is uncertainty about the extent to which the health effects for these populations can be extrapolated to a U S  population that is regularly (chronically) exposed to low-dose radiation  Nevertheless, NRC officials told us that, in the absence of convincing evidence that there is a dose threshold below which low levels of radiation are beneficial or not harmful, NRC will continue to follow the recommendations of scientific advisory bodies to use the linear no- threshold model  Similarly, officials from EPA told us that they would consider changing the use of the linear no-threshold model as the basis of their requirements and guidance only if there were a strong recommendation from scientific advisory bodies on radiation protection as well as an endorsement of the change by the National Academies  Under this model, federal regulations set dose limits for radiation exposure that are below the level in the National Academies’ 2006 report on radiation and human health for defining low-dose radiation  For example, NRC’s annual dose limit for members of the public (excluding natural, or background, sources of radiation) from operation of nuclear power plants is a hundredth of the level the National Academies considers low dose  NRC based the dose limit on an advisory body recommendation that the cancer risk to the general public from exposure to radiation should be comparable to the public’s risk from everyday activities, such as taking public transportation  The low-dose radiation limits and guidance that federal agencies have developed and applied vary depending on the settings in which exposure can occur  For example, NRC has established limits on occupational dose that apply to nuclear power-plant workers; these limits are higher than NRC’s annual dose limit for members of the public but are still below the level the National Academies considers low dose  In keeping with advisory body recommendations, NRC also applies the principle that doses should be kept as low as reasonably achievable (ALARA)  NRC defines ALARA to mean making every reasonable effort to maintain exposures to radiation as far below dose limits as is practical  At a nuclear power plant we visited as part of our work, representatives told us that under their ALARA plan, the plant set its own dose limit for workers at 40 percent of the NRC’s regulatory limit  Moreover, officials at the plant told us that they have been able to keep exposures below the plant’s own limit by continuously seeking opportunities to reduce unnecessary worker exposure to radiation, such as using robots to perform maintenance work in radiation areas  In contrast to radiation exposure received from nuclear power plants, FDA officials stated that the agency regulates the maximum radiation output of medical equipment, instead of setting limits on the total amount of radiation exposure to patients  According to FDA officials, FDA does not generally have the authority to regulate the total amount of radiation exposure a patient receives from medical imaging equipment  However, in keeping with the principle that radiation exposure should be kept as low as reasonably achievable, FDA encourages voluntary measures by health care providers, such as to investigate and determine whether it is possible to reduce radiation exposure to patients from the use of medical- imaging equipment  From fiscal year 2012 through fiscal year 2016, seven federal agencies obligated $209 6 million for research on the health effects of low-dose radiation, but they did not use a collaborative mechanism to address overall research priorities in this area  DOE and NIH accounted for most of the funding, with DOE obligating $116 3 million and NIH obligating $88 6 million, or about 56 percent and 42 percent of the total, respectively  The five other agencies—NRC, NASA, DOD, EPA, and CDC—obligated the remaining $4 7 million, or about 2 percent of the total  DOE has two offices that have funded research on the health effects of low-dose radiation—the Office of Science and the Office of Environment, Health, Safety and Security—according to funding information DOE provided  The Office of Science established the Low Dose Radiation Research Program in 1998 and funded it through fiscal year 2016  A primary focus of this program was radiobiological research, which examines molecular and cellular responses to radiation exposure  According to DOE’s website for the program, the program provided data and information about the low-dose range of exposure, producing 737 peer-reviewed publications as of March 2012  The Office of Environment, Health, Safety and Security provided funding for epidemiological studies, including studies involving Japanese atomic bomb survivors  NIH has funded and conducted both epidemiological and radiobiological studies on low-dose radiation, according to NIH officials  The officials stated that the studies are conducted through the National Cancer Institute’s internal research program for radiation epidemiology, as well as through NIH’s research programs for external funding of investigator- initiated research  Other institutes of NIH, including the National Institute of Environmental Health Sciences, also fund research related to the health effects of radiation exposure as part of NIH’s overall mission to fund medical research  Among the other agencies that provided some funding to low-dose radiation studies, several provided funding to the Epidemiological Study of One Million U S  Radiation Workers and Veterans (Million Person Study)—an ongoing study headed by the National Council on Radiation Protection and Measurements  DOE also provided funding for this study  In fiscal years 2012 through 2016, the seven agencies who provided funding for research on health effects of low-dose radiation collectively decreased their annual funding obligations in this area by 48 percent, from $57 9 million in fiscal year 2012 to $30 4 million in fiscal year 2016  DOE accounted for a large portion of this overall decrease in annual funding  Specifically, over this 5-year period, DOE reduced its annual funding obligations for this area of research by 45 percent—from $32 6 million in fiscal year 2012 to $18 0 million in fiscal year 2016  According to DOE, the decrease was primarily due to DOE’s reduction in funding for its Low Dose Radiation Research Program  According to DOE officials, decreases in funding for the program reflected a shift toward bioenergy and environmental research  Similarly, over the 5-year period, NIH’s funding for low-dose radiation research decreased by 48 percent—from $23 1 million in fiscal year 2012 to $12 0 million in fiscal year 2016  NIH officials explained that funding levels for a particular disease or research area can fluctuate depending on several factors, including the number and quality of research proposals submitted and the outcome of NIH’s peer reviews of the proposals, as well as the overall research budget  The seven agencies that funded research on health effects of low-dose radiation for fiscal years 2012 through 2016 collaborated on particular research projects through various mechanisms, including joint funding of individual projects, but they did not use a collaborative mechanism to address overall research priorities  As previously noted, the 2016 report of DOE’s Biological and Environmental Research Advisory Committee provided information about research needs in low-dose radiation and found that further research could decrease uncertainty in predicting cancer risk from low-dose radiation  The report stated that other agencies—including NRC, NIH, EPA, DOD, and NASA—could benefit from the reduction in uncertainty that could be obtained by this research  In our September 2017 report, we recommended that the Secretary of Energy lead the development of a mechanism for interagency collaboration to determine roles and responsibilities for addressing priorities related to research on the health effects of low-dose radiation  We made this recommendation because our previous work has shown that collaborative mechanisms can serve multiple purposes, such as leading interagency efforts to develop and coordinate sound science and technology policies across the federal government  Although collaborative mechanisms differ in complexity and scope, they all benefit from certain key features, such as leadership  We directed this recommendation to DOE for several reasons  In the past, DOE took a leading role in advocating for greater communication and coordination between the fields of radiation biology and epidemiology  In addition, DOE is the federal agency that currently has primary responsibility under the Atomic Energy Act of 1954 for research related to the protection of health during activities that can result in exposure to radiation  DOE is well positioned to lead an effort to ensure that federal agencies have a mechanism for interagency collaboration to address overall research priorities related to low-dose radiation health effects because of the agency’s past experience as a leader in this area of research  Such an effort could help DOE and the collaborating agencies determine roles and responsibilities, including leadership when addressing shared research priorities  DOE did not agree with our recommendation  In particular, DOE stated that EPA and NRC also have legal mandates to research low-dose radiation exposure and that these agencies establish their research priorities in accordance with their respective budget authorities and recommendations from independent advisory bodies  DOE stated that as a result, it would not be appropriate for DOE to lead the development of a mechanism for interagency collaboration  We believe that DOE’s concerns stem from a misinterpretation of our recommendation, and we made several changes to our report and our recommendation to clarify DOE’s role  We noted that we did not recommend that a mechanism for interagency collaboration serve as a replacement for agencies’ legal mandates, budget authorities, and recommendations from independent advisory bodies  Instead, this mechanism would help agencies address shared research priorities  In making our recommendation, we did not specify the coordinating mechanism that agencies should use and instead left it to DOE to lead the development of an appropriate mechanism  We continue to believe that an interagency coordination mechanism for low-dose research is needed and that DOE is in the best position to lead agencies in developing the most appropriate mechanism  Chairman Weber, Ranking Member Veasey, and Members of the Subcommittee, this concludes my prepared statement  I would be pleased to respond to any questions that you may have at this time  If you or your staff have any questions about this statement, please contact John Neumann at (202) 512-3841 or neumannj@gao gov  In addition, contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement  Individuals who made key contributions to the report on which this testimony is based include Allen Chan, Kendall Childers, Joseph Cook, Richard Johnson, Cynthia Norris, Josie Ostrander, Amber Sinclair, and Jack Wang  This is a work of the U S  government and is not subject to copyright protection in the United States  The published product may be reproduced and distributed in its entirety without further permission from GAO  However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately 
Report 2: S  military  It has had to consider, among other things, the Services’ mission requirements, resource constraints, and the myriad demands on servicemembers and their families during their frequent relocations  Anything that further complicates a relocation—such as not receiving the required family support services for family members with special needs—potentially affects readiness or, at a minimum, makes an already stressful situation worse  By providing little direction on how the Services should provide family support or what the scope of family support services should be, some servicemembers get more—or less—from the EFMP each time they relocate, including when a servicemember from one Service is assigned to a joint base led by another Service  By largely deferring to the Services to design, implement, and monitor their EFMPs’ performance, DOD cannot, as required by the NDAA for Fiscal Year 2010, fully determine the adequacy of the Services’ EFMPs in serving families with special needs, including any gaps in services these families receive, because it has not built a systematic process to do so  Instead, it relies on the Services to self-monitor and address, within each Service, the results of monitoring activities  However, because servicemembers relocate frequently and often depend on the EFMP of a Service other than their own, a view of EFMP performance across all of the Services is essential to ensuring, for example, that relocating servicemembers get consistent EFMP service delivery no matter where they are stationed  Evaluating and developing program improvements based on the results of the Services’ monitoring would help DOD ensure the Services’ EFMPs achieve the desired outcomes and improve its ability to assess the overall effectiveness of the program  We are making the following three recommendations to DOD: We recommend the Secretary of Defense direct the Office of Special Needs (OSN) to assess the extent to which each Service is (1) providing sufficient resources for an appropriate number of family support providers, and (2) developing services plans for each family with special needs, and to include these results as part of OSN’s analysis of any gaps in services for military families with special needs in each annual report issued by the Department to the congressional defense committees  (Recommendation 1) We recommend that the Secretary of Defense direct the Office of Special Needs (OSN) to develop common performance metrics for assignment coordination and family support, in accordance with leading practices for performance measurement  (Recommendation 2) We recommend that the Secretary of Defense implement a systematic process for evaluating the results of monitoring activities conducted by each Service’s EFMP  (Recommendation 3) We provided a draft of this report to the Department of Defense (DOD) for comment  DOD provided written comments, which are reproduced in appendix IV  DOD also provided technical comments, which we incorporated as appropriate  DOD agreed with all three of our recommendations  In its written comments, DOD stated that additional performance metrics need to be developed for assignment coordination and that it is in the process of measuring families’ satisfaction with family support provided through the EFMP  DOD also stated that it is developing plans for evaluating the results of each Service’s monitoring activities for the EFMP  We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and Education, and other interested parties  The report also is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (617) 788-0580 or nowickij@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  The National Defense Authorization Act (NDAA) for Fiscal Year 2017 includes a provision for GAO to assess the effectiveness of the Department of Defense’s (DOD) Exceptional Family Member Programs (EFMP)  This report focuses on the assignment coordination and family support components of the EFMP for dependents with special needs and examines: (1) the extent to which each Service has provided family support as required by DOD, and (2) the extent to which the Services monitor and DOD evaluates assignment coordination and family support  To address these objectives, we used a variety of data collection methods  Key methods are described in greater detail below  For both objectives, we reviewed relevant federal laws, regulations, and DOD guidance and documentation that pertain to the EFMP, including the following: The NDAA for Fiscal Year 2010, which established the Office of Special Needs and defined program requirements for assisting families with special needs, including assignment coordination and family support  DOD’s guidance for administering the EFMP  We assessed how DOD implements the requirements in the NDAA for Fiscal Year 2010; how each Service implements assignment coordination and family support; and how the Services and DOD monitor assignment coordination and family support using performance measures  Specially, we reviewed DOD Instruction 1315 19 - Exceptional Family Member Program; Service-specific guidance and related documents from the Air Force, Army, Marine Corps, and Navy; and DOD Instruction 1342 22 - Military Family Readiness  Standards for internal control in the federal government related to the documentation of responsibilities through policies, performance measures, and evaluating the results of monitoring activities  We compared each Service’s procedures for monitoring assignment coordination and family support to these standards  To determine the extent of the Services’ EFMP family support, we obtained and analyzed fiscal year 2016 EFMP data (the most recent available) for each Service  We reviewed DOD policy to identify data variables that each Service maintains related to its EFMP  We used these data to summarize key characteristics of each Service’s EFMP  The selected variables provided Service-wide and installation-specific EFMP information on, the number of continental United States (CONUS) and outside the continental United States (OCONUS) installations; the number of servicemembers (sponsors) enrolled in the EFMP; the number of family members with special needs enrolled in the EFMP; the number of EFMP family support personnel; and the number of services plans created for families with special needs enrolled in the EFMP  We determined that the selected data variables from each Service are sufficiently reliable for the purposes of providing summary results about family support for fiscal year 2016  To learn more about how the Services implement their EFMPs, we visited seven installations in five states  We selected the seven installations based on their location in states with the largest number of military- connected students in school year 2012-2013 (the most recent available and reliable data) or in states with the largest percentage of students enrolled in U S  DOD schools as of May 2017, as well as their status as a joint base  At each installation, we interviewed installation officials, EFMP managers, selected family support personnel, and family members and caregivers enrolled in the program  In states we visited that had the largest number of military-connected students, the EFMP personnel we interviewed collectively served 66 percent of students who attend local public schools and 42 percent of the students attending U S  DOD schools  To obtain illustrative examples about how the EFMP serves families with special needs, we conducted seven group interviews with EFMP-enrolled family members and caregivers (one at each of the seven installations we visited)  Using a prepared script, we asked participants to describe how they were identified and enrolled in the EFMP, how they were assigned to new installations, and the types of family support services they received  We also asked about how these services aligned with their family member’s EFMP-eligible condition, the benefits and challenges they experienced, as well as their overall satisfaction  A total of 38 self- selected volunteers participated in the seven group discussions  While the participants in these groups included a variety of family members and caregivers, the number of participants and groups were very small relative to the total number of family members enrolled in the EFMP  Their comments are not intended to represent all EFMP-enrolled family members or caregivers  Other EFMP-enrolled family members and caregivers may have had other experiences with the program during the same period  Finally, for both objectives, we conducted interviews with a variety of DOD, Service-level, and nonfederal officials  We spoke with DOD officials from the Office of the Assistant Secretary of Defense–Offices of Manpower and Reserve Affairs, Military Community and Family Policy, Military Family Readiness Policy, and Special Needs  We also spoke with EFMP Managers from Air Force, Army, Marine Corps, and Navy headquarters  We also met with officials from selected national military family advocacy organizations including the National Military Family Association; the Military Family Advisory Network; and the Military Officers Association of America to discuss the EFMP  We conducted this performance audit from February 2017 to May 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  Each Service has an Exceptional Family Member Program (EFMP) that provides support to military families with special needs  The tables below present the following information on selected EFMP and family support categories for each Service’s program at continental United States (CONUS) and outside the continental United States (OCONUS) installations in fiscal year 2016: City, state or country; Number of exceptional family members; Number of family support providers (by Full-Time Equivalent); Number of family support provider vacancies; Number of services plans; Number of indirect contacts; and Number of direct contacts  The information below is listed sequentially in alphabetical order by Service  We held small group discussions with Exceptional Family Member Program (EFMP) participants at the seven military installations we visited  Family members and caregivers who attended each session reported they had children or spouses with EFMP-eligible conditions  The discussion group participants were self-selected; and their comments are not intended to represent all EFMP -enrolled family members or caregivers in fiscal year 2016  In addition, other EFMP -enrolled family members and caregivers may have had different experiences with the program during the same period  There were a total of 38 participants representing all the Services  The following issues were discussed by one or more participants during the small group discussions at the installations we visited  The issues that emerged relate to the current and future overall effectiveness of the EFMP  Overall Satisfaction with EFMP (Discussed by 30 of 38 participants): Measure of participants’ approval of the family support services offered and experience with the EFMP  Many participants expressed overall satisfaction with the EFMP  Several participants expressed dissatisfaction with the EFMP  A participant expressed dissatisfaction with the lack of consistency in the provision of family support services (i e , special education advocacy) across installations  School Liaison Officers (Discussed by 20 of 38 participants): Serve as the primary point of contact for school-related matters as well as assist military families with school issues  Several participants noted that they received no response to their request for assistance from their School Liaison Officer or they only received general information  Several participants said School Liaison Officers were not helpful  Some participants found School Liaison Officers were helpful  Some participants were unaware of School Liaison Officers being available at their installation and the service(s) they provide  A few participants said School Liaison Officers did not follow up on requests for information  A participant noted there seems to be a disconnect between family support services provided through the EFMP and services provided by School Liaison Officers  Family Support Personnel (Discussed by 12 of 38 participants): Provide information and referral to military families with special needs  Some participants at one installation noted that the EFMP was understaffed  Some participants at one installation noted high turnover of family support personnel  Some participants noted family support personnel did not provide support for their family with special needs  Stigma (Discussed by 12 of 38 participants): A perception that participating in the EFMP may limit a soldier’s assignment opportunities and/or compromise career advancement  Several participants believe there is still stigma associated with participating in the EFMP  Some participants said participating in the EFMP has not affected career advancement  Assignment Coordination (Discussed by 10 of 38 participants): The assignment of military personnel in a manner consistent with the needs of armed forces that considers locations where care and support for family members with special needs are available  Some participants found the assignment coordination process challenging  Some participants described limitations with the assignment coordination process  A few participants noted there is a lack of information among families with special needs regarding how to express the need for stabilization and /or continuity of care  A few participants cited the challenges of assignment coordination as contributing to their decision to retire  One participant commented that the opinion of a medical professional was not reflected in the assignment coordination process  Special Education Services (Discussed by 10 of 38 participants): The provision of staff capable of assisting families with special needs with special education and disability law advice and/or assistance and attendance at individualized education program (IEP) meetings where appropriate  Several participants who had a family support provider assist them with preparing for or attending a school-based meeting, including IEP meetings, spoke positively of their experience(s)  Some participants at one installation agreed that assistance from family support providers during meetings with school officials regarding special education services is helpful  A few participants who were unable to get assistance with special education services from the EFMP sought the services of private attorneys at their own expense  Family Support Services (Discussed by 9 of 38 participants): The non-clinical case management delivery of information and referral for families with special needs, including the development and maintenance of a services plan  Some participants found that family support providers were helpful  Some participants could not identify needed resources or were unaware of the resources or services available to them  One participant noted that the family support provider had minimal contact  One participant said navigating the system can be challenging  Surveys (Discussed by 8 of 38 participants): The process of collecting data from a respondent using a structured instrument and survey method to ensure the accurate collection of data  Several participants noted that they had not or rarely had the opportunity to evaluate the family support services provided through the EFMP  One participant noted that comment cards used by each service are not effective for evaluating the EFMP  Warm hand-off (Discussed by 6 of 38 participants): Assistance to identify needed supports or services and facilitating the initial contact or meeting with the next program  Many participants at one installation agreed that the warm hand-off process worked well for them  Several participants said they found the warm hand-off process helpful when moving from one installation to the next  Outreach (Discussed by 5 of 38 participants): Developing partnerships with military and civilian agencies and offices (local, state, and national), improving program awareness, providing information updates to families, and hosting and participating in EFMP family events  Some participants found it difficult to obtain information regarding the types of family support services that are available  A few participants noted that communications regarding the EFMP were not targeted to address their needs  A few participants noted communications regarding the EFMP are untimely, (e g , newsletters not issued periodically)  Joint Base Family Support Services (Discussed by 1 of 38 participants): Family support services provided by the lead Service of the Joint Base that is different from that of the servicemember enrolled in the EFMP  One participant said that using family support services on joint bases may pose a challenge as each Service has different rules and procedures and as a result provides different types of family support services  In addition to the contact named above, Bill MacBlane (Assistant Director), Brian Egger (Analyst-in-Charge), Patricia Donahue, Holly Dye, Robin Marion, James Rebbe, Shelia Thorpe, and Walter Vance made significant contributions to this report  Also contributing to this report were Lucas Alvarez, Bonnie Anderson, Connor Kincaid, Brian Lepore, Daniel Meyer, and Mimi Nguyen 

ground_truth: In October 2017, the Department of Commerce (Commerce) announced that the projected life-cycle cost of the 2020 Census had climbed to $15.6 billion, a more than $3 billion (27 percent) increase over its 2015 estimate. A high-quality, reliable cost estimate is a key tool for budgeting, planning, and managing the 2020 Census. Without this capability, the Bureau is at risk of experiencing program cost overruns, missed deadlines, and performance shortfalls. GAO was asked to evaluate the reliability of the Bureau's life-cycle cost estimate. This report evaluates the reliability of the Bureau's revised life-cycle cost estimate for the 2020 Census and the extent to which the Bureau is using it as a management tool, and compares the 2015 and 2017 cost estimates to describe key drivers of cost growth. GAO reviewed documentary and testimonial evidence from Bureau officials responsible for developing the 2020 Census cost estimate and used its cost assessment guide ( GAO-09-3SP ) as criteria. Since 2015, the Census Bureau (Bureau) has made significant progress in improving its ability to develop a reliable cost estimate. While improvements have been made, the Bureau's October 2017 cost estimate for the 2020 Census does not fully reflect all the characteristics of a reliable estimate. (See figure.) Specifically, for the characteristic of being well-documented, GAO found that some of the source data either did not support the information described in the cost estimate or was not in the files provided for two of its largest field operations. In GAO's assessment of the 2015 version of the 2020 Census cost estimate, GAO recommended that the Bureau take steps to ensure that each of the characteristics of a reliable cost estimate is met. The Bureau agreed and has taken steps, but has not fully implemented this recommendation. A reliable cost estimate serves as a tool for program development and oversight, helping management make informed decisions. During this review, GAO found the Bureau used the cost estimate to inform decision making. Factors that contributed to cost fluctuations between the 2015 and 2017 cost estimates include: Changes in assumptions. Among other changes, a decrease in the assumed rate for self-response from 63.5 percent in 2015 to 60.5 percent in 2017 increased the cost of collecting responses from nonresponding housing units. Improved ability to anticipate and quantify risk. In general, contingency allocations designed to address the effects of potential risks increased overall from $1.3 billion in 2015 to $2.6 billion in 2017. An overall increase in information technology (IT) costs. IT cost increases, totaling $1.59 billion, represented almost 50 percent of the total cost increase from 2015 to 2017. GAO is not making any new recommendations but maintains its earlier recommendation—that the Secretary of Commerce direct the Bureau to take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate. In its response to this report, Commerce generally agreed with the findings related to cost estimation improvements, but disagreed that the cost estimate was not reliable. However, until GAO's recommendation is fully implemented the cost estimate cannot be considered reliable.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: A high-quality, reliable cost estimate is a key tool for budgeting, planning, and managing the 2020 Census  According to OMB, programs must maintain current and well-documented estimates of program costs, and these estimates must encompass the full life-cycle of the program  Among other things, OMB states that generating reliable program cost estimates is a critical function necessary to support OMB’s capital programming process  Without this capability, agencies are at risk of experiencing program cost overruns, missed deadlines, and performance shortfalls  A reliable cost estimate is critical to the success of any federal government program  With the information from reliable estimates, managers can: make informed investment decisions, allocate program resources, measure program progress, proactively correct course when warranted, and ensure overall accountability for results  To be considered reliable, a cost estimate must meet the criteria for each of the four characteristics outlined in our Cost Estimating and Assessment Guide  According to our analysis, a cost estimate is considered reliable if the overall assessment ratings for each of the four characteristics are substantially or fully met  If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable  Those characteristics are: Well-documented: An estimate is thoroughly documented, including source data and significance, clearly detailed calculations and results, and explanations of why particular methods and references were chosen  Data can be traced to their source documents  Accurate: An estimate is unbiased, the work is not overly conservative or overly optimistic, and is based on an assessment of most likely costs  Few, if any, mathematical mistakes are present  Credible: Any limitations of the analysis because of uncertainty or bias surrounding data or assumptions are discussed  Major assumptions are varied, and other outcomes are recomputed, to determine how sensitive they are to changes in the assumptions  Risk and uncertainty analysis is performed to determine the level of risk associated with the estimate  The estimate’s results are cross- checked, and an independent cost estimate (ICE) is conducted to see whether other estimation methods produce similar results  Comprehensive: An estimate has enough detail to ensure that cost elements are neither omitted nor double counted  All cost-influencing ground rules and assumptions are detailed in the estimate’s documentation  Meeting best practices outlined in our Cost Estimating and Assessment Guide for a reliable cost estimate has been a long-standing challenge for the Bureau  In 2008 we reported that the 2010 Census cost estimate was not reliable because it lacked documentation and was not comprehensive, accurate, or credible  For example, in our 2008 report on the Bureau’s cost estimation process, Bureau officials were unable to provide documentation that supported the assumptions for the initial 2001 life-cycle cost estimate as well as the updates  Consequently, we recommended that the Bureau establish guidance, policies, and procedures for estimating costs that would meet best practices criteria  The Bureau agreed with the recommendation and said at the time that it already had efforts underway to improve its future cost estimation methods and systems  Moreover, weaknesses in the life-cycle cost estimate were one reason we designated the 2010 Census a GAO High- Risk Area in 2008  In 2012 we reported that, while the Bureau was taking steps to strengthen its life-cycle cost estimates, it had not yet established guidance for developing cost estimates  We recommended that the Bureau finalize its guidance, policies, and procedures for cost estimation in accordance with best practices  The Bureau agreed with the overall theme of the report but did not comment on the recommendation  During this review we found that the Bureau took steps to address this recommendation, which is discussed later in this report  Such guidance can help to institutionalize best practices and ensure consistent processes and operations for producing reliable estimates  In a 2016 report we found that the October 2015 version of the Bureau’s life-cycle cost estimate for the 2020 Census was not reliable  Overall, we reported that the 2020 Census life-cycle cost estimate partially met two of the characteristics of a reliable cost estimate (comprehensive and accurate) and minimally met the other two (well-documented and credible)  We recommended that the Bureau take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate  The Bureau agreed with this recommendation, and took steps to improve the reliability of its cost estimate, which we focus on later in this report  Consequently, an unreliable life-cycle cost estimate is one of the reasons we designated the 2020 Census a GAO High-Risk Area in 2017  In October 2015, the Bureau estimated the cost of the 2020 Census to be $12 3 billion  According to the Bureau, the October 2015 version was the Bureau’s first attempt to model the life-cycle cost of its planned 2020 Census, in contrast to its earlier 2011 estimate, which the Bureau said was intended to produce an approximation of potential savings and to begin developing the methodology for producing decennial life-cycle cost estimates covering all phases of the decennial life cycle  To help control costs while maintaining accuracy, the Bureau introduced significant change to how it conducts the decennial census in 2020  Its planned innovations include reengineering how it builds its address list, improving self-response by encouraging the use of the Internet and telephone, using administrative records to reduce field work, and reengineering field operations using technology to reduce manual effort and improve productivity  In contrast to the estimated $12 3 billion in 2015, the 2020 Census would cost $17 8 billion in constant 2020 dollars if the Bureau repeated the 2010 Census design and methods, according to the Bureau’s estimates  In October 2017, Commerce announced that it had updated the October 2015 life-cycle cost estimate, projecting the life-cycle cost of the 2020 Census to be $15 6 billion, an increase of over $3 billion (27 percent) over its 2015 estimate  (See figure 1 ) In developing the 2017 version of the cost estimate, Bureau cost estimators identified cost inputs, their ranges for possible outcomes, and overall cost estimating relationships (i e , logical or mathematical formulas, or both)  To identify cost inputs and the ranges of potential outcomes, the Bureau worked with subject matter experts and used historical data to support assumptions and generate inputs  The Bureau’s cost estimation team used a software tool to generate the cost estimate  Because cost estimates predict future program costs, uncertainty is always associated with them  For example, data from the past (such as fuel prices) may not always be relevant in the future  Risk and uncertainty refer to the fact that because a cost estimate is a forecast, there is always a chance that the actual cost will differ from the estimate  One way to determine whether a program is realistically budgeted is to perform an uncertainty analysis, so that the probability associated with achieving its point estimate can be determined, usually relying on simulations such as those of Monte Carlo methods  This can be particularly useful in portraying the uncertainty implications of various cost estimates  Consistent with cost estimation practices outlined in our Cost Estimating and Assessment Guide, the estimate was compared with two independent cost estimates (ICE), developed by Commerce’s Office of Acquisition Management (OAM) and the Bureau’s Office of Cost Estimation, Analysis, and Assessment  The offices producing the ICEs and the cost estimate team worked together to examine the process each used, an effort known as the reconciliation process  Through this reconciliation, the Bureau identified areas where discrepancies existed and elements that could require additional review and possible improvement  According to Bureau documentation the estimate will be updated as the program meets milestones and to reflect changes in technical or program assumptions  Figure 2 details the Bureau’s cost estimation process  OAM was involved extensively in the development of the 2017 estimate, an increased involvement compared to 2015, according to Bureau officials  OAM participated in regular review meetings throughout the development of the estimate and also developed an independent cost estimate, as shown in the figure below  End-to-end system testing activities for the 2020 Census are currently underway in Providence, Rhode Island  According to the Bureau, information collected from the test, such as overall response rates and the use of administrative records to inform census records, will inform future versions of the life-cycle cost estimate  Some updates from the test will be incorporated into the next cost estimate, which will be available in the first quarter of the coming fiscal year  Since our June 2016 report, in which we reviewed the Bureau’s 2015 version of the cost estimate, the Bureau has made significant progress  For example, the Bureau has put into place a work breakdown structure (WBS) that defines the work, products, activities, and resources necessary to accomplish the 2020 Census and is standardized for use in budget planning, operational planning, and cost estimation  However, the Bureau’s October 2017 cost estimate for the 2020 Census does not fully reflect characteristics of a high-quality estimate as described in our Cost Estimating and Assessment Guide and cannot be considered reliable  Our Cost Estimating and Assessment Guide describes best practices for developing reliable cost estimates  For our reporting needs, we collapsed these best practices into four characteristics for sound cost estimating— comprehensive, well-documented, accurate, and credible—and identified specific best practices for each characteristic  To be considered reliable, an organization must meet or substantially meet each characteristic  Our review found the Bureau met or substantially met three out of the four characteristics of a reliable cost estimate, while it partially met one characteristic: well-documented  When compared to the October 2015 estimate, the 2017 estimate shows considerable improvement  (See figure 3 below ) Cost estimates are considered valid if they are well-documented to the point they can be easily repeated or updated and can be traced to original sources through auditing, according to best practices  The Bureau only partially met the criteria for well-documented, as set forth in our Cost Estimating and Assessment Guide  A cost estimate that does not fully meet the criteria for well-documented cannot be used by management to make informed and effective implementation decisions  The well-documented characteristic comprises five best practices  The Bureau substantially met two out of five best practices (as shown in figure 4)  First, the estimate describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost, and the cost estimate had been reviewed by management  Since cost estimates can inform key decisions and budget requests, it is vital that management review and understand how the estimate was developed, including risks associated with the underlying data and methods  The cost estimate only partially met three best practices for the characteristic of being well-documented  In general, some documentation was missing, inconsistent, or difficult to understand  First, we found that source data did not always support the information described in the basis of estimate document or could not be found in the files provided for two of the Bureau’s largest field operations: Address Canvassing and Non- Response Follow-Up (NRFU)  For example, the cost estimate documentation referred to actual data from the 2010 Census and information obtained from experts as sources for address canvassing rework rates  However, the folder source documents provided as support for the basis of estimate did not include this information  Next, in several cases, we could not replicate calculations, such as for mileage costs, using the description provided  Lastly, we found that some of the cost elements did not trace clearly to supporting spreadsheets and assumption documents  Failure to document an estimate in enough detail makes it more difficult to replicate calculations, or to detect possible errors in the estimate; reduces transparency of the estimation process; and can undermine the ability to use the information to improve future cost estimates or even to reconcile the estimate with another independent cost estimate  The Bureau told us it would continue to make improvements to ensure the estimate is well- documented  For the estimate to be considered well-documented, the Bureau will need to address these issues  An accurate cost estimate supports measurement of program progress by providing unbiased and correct data, which can help management ensure accountability for scheduled results  We found the Bureau’s cost estimate substantially met the criteria for accuracy  As shown in figure 5, and in line with best practices outlined in our Cost Estimating and Assessment Guide, the estimate was not overly optimistic; appeared to be free of errors; was based on historical data or input from subject matter experts; and, according to Bureau officials, is updated regularly as information becomes available  The Bureau can enhance the accuracy of their estimate by increasing the level of detail included in the documentation, such as detail on specific inflation indices used, and by monitoring actual costs against estimates  We identified areas for improvement, which, according to Bureau officials, will be addressed as part of its ongoing efforts  For example, while the basis of estimate document describes different inflation indexes, it was not clear exactly which indexes were applied to the various cost elements in the estimate  Also, evidence of how variances between estimated costs and actual expenses would be tracked over time was not available at the time of our analysis  Tools to track variance enable management to measure progress against planned outcomes  Bureau officials stated that they already have systems in place that can be adapted for tracking estimated and actual costs  All estimates include a certain amount of informed judgment about the future  Assumptions made at the start of a program can turn out to be inaccurate  Credible cost estimates identify limitations due to uncertainty or bias surrounding data or assumptions, and control for these uncertainties by identifying and quantifying cost elements that represent the most risk  We found that the Bureau’s cost estimate substantially met the criteria for credible, as shown in figure 6 below  The Bureau’s cost estimate clearly identifies risks and uncertainties, and describes approaches taken to mitigate them  In line with best practices outlined in our Cost Estimating and Assessment Guide, the Bureau did the following: Sensitivity analysis  The Bureau conducted sensitivity analysis to identify possible changes to estimated costs for the 2020 Census based on varying major assumptions, parameters, and data inputs  For example, the Bureau calculated the likely cost implications for a range of possible response rates to identify a range of projected costs and to calculate appropriate reserves for risk  Bureau officials stated that they also identified the estimate input parameters that contributed the most to estimate uncertainty  Risk and uncertainty analysis  A cost estimate is a forecast, and as such, there is always a chance that the actual cost will differ from the estimate  Uncertainty is the indefiniteness about the outcome of a situation  Uncertainty is assessed in cost estimate models to estimate the risk (or probability) that a specific funding level will be exceeded  We found the Bureau performed an uncertainty analysis on a portion of the estimate to determine whether estimated costs were realistic and to establish the probability of achieving projections outlined in the estimate  The Bureau used a combination of modeling based on Monte Carlo analysis and allocations of funding for risks  The Monte Carlo simulation was performed on a portion of the estimate to account for uncertainty around various operational parameters for which a range of outcomes was possible, including Internet response rates and the extent to which data collection issues might be resolved using administrative records  To account for the inherent uncertainty of assumptions included within the life-cycle cost estimate, the Bureau added funding to the cost estimate totaling approximately $292 million to account for risks based on the results of the Monte Carlo analysis  For other risks, such as acquisition lead time and the possibility of delays in information technology (IT) development, contingency funding was added to the estimate to reflect the potential cost of resolving these issues, through use of a backup system or an alternative approach  These are described as “special risks” in the Bureau’s basis of estimate, and total approximately $171 million  Based on additional sensitivity analysis, the Bureau added approximately $965 million to the cost estimate to reflect discrete risks outlined in the risk register as well as those associated with (1) variability in self-response rates, (2) the effect of fluctuations in the size and wage rate of the temporary workforce on the cost of field operations, and (3) the potential need to reduce the enumerator-to- manager staffing ratio in case expected efficiencies in field operations are not realized  In addition to these provisions, the Secretary of Commerce added a contingency amount of about $1 2 billion to account for what the Bureau refers to as unknown-unknowns  Bureau documentation states that conducting a decennial census is an extremely complex, high-risk operation  In order to mitigate some of the risk, contingency funding must be available to initiate ad hoc activities necessary to overcome unforeseen issues  According to Bureau documentation these include such risks as natural disasters or cyber-attacks  The Bureau provides a description of how the general risk contingency is calculated  However, this description does not clearly link calculated amounts to the risks themselves  In our June 2016 report we reported the Bureau had not properly accounted for risk and recommended the Bureau, in part; improve control over how risk and uncertainty are accounted for  We continue to believe the prior recommendation from our June 2016 report remains valid and should be addressed: that the Bureau properly account for risk in the 2020 Census cost estimate, among other things  As such, risks need to be linked to the $1 2 billion general risk contingency fund  Independent cost estimate  According to best practices outlined in our Cost Estimating and Assessment Guide, an independent cost estimate should be performed to determine whether alternate estimate approaches produce similar results  The Bureau compared their estimate with two independent cost estimates, developed by Commerce’s Office of Acquisition Management and the Bureau’s Office of Cost Estimation and Assessment  As part of their process for finalizing the cost estimate, Bureau officials reconciled differences between the estimates in discussions with the two offices, resulting in more conservative assumptions by the Bureau around risk and uncertainty in both cases  In addition to implementing our recommendation to properly account for risk, going forward, while the Bureau substantially met the credibility characteristic it will be important for them to also integrate regular cross-checks of methodology into their cost estimation process  In our analysis we observed that no specific cross-checks of cost methodology were performed  According to the Bureau, cross- checks were not performed because the Bureau considered the independent cost estimates as overall cross-checks on the reliability of their methodology and did not conduct additional cross-checks  The main purpose of cross-checking is to determine whether alternative methods for specific cost elements within the cost estimate could produce similar results  An independent cost estimate, though important for the credibility of an estimate, does not fulfill the same function as a targeted cross-check of individual elements  Comprehensive estimates have enough detail to ensure that cost elements are neither omitted nor double-counted, all cost-influencing assumptions are detailed in the estimate’s documentation, and a work breakdown structure is defined  Our analysis of the 2017 cost estimate demonstrates improvement over the 2015 cost estimate when the Bureau’s cost estimate only partially met the criteria for comprehensive  We found the Bureau met or substantially met all four best practices for the comprehensive characteristic, as shown in figure 7  For example, all life-cycle costs are included in the estimate along with a complete description of the 2020 Census program and current schedule  We also found that the Bureau substantially met criteria for documenting cost influencing ground rules and assumptions  A standardized WBS (as detailed in table 1) with supporting dictionary outlines the major work of the program and describes the activities and deliverables at the project level where costs are tracked  In 2016, the Bureau’s WBS did not contain sufficient detail and we found significant differences in the presentation of the work between sources  In 2017, based on our review of Bureau documentation and interviews with Bureau officials, we found that the WBS is standardized and cost elements are presented in detail  The WBS is a necessary program management tool because it provides a basic framework for a variety of related tasks like estimating costs, developing schedules, identifying resources, determining where risks may occur, and providing the means for measuring program status  Although the Bureau’s updated life-cycle cost estimate reflects three of the four characteristics of a reliable cost estimate, we are not making any new recommendations to the Bureau in this report  We continue to believe the prior recommendation, made in 2016, remains relevant: that the Secretary of Commerce ensure that the Bureau finalizes the steps needed to fully meet the characteristics of a high-quality estimate, most notably in the well-documented area  The Bureau told us it has used our best practices for cost estimation to develop their cost estimate, and will focus on those best practices that require attention moving forward  Without a reliable cost estimate, the Bureau is limited in its ability to make informed decisions about program resources and to effectively measure progress against operational objectives  OMB, in its guidance for preparing and executing agency budgets, cites that credible cost estimates are vital for sound management decision making and for any program or capital project to succeed  A well- developed cost estimate serves as a tool for program development and oversight, supporting management to make informed decisions  According to the Bureau, the 2020 Census cost estimate is used as a management tool to guide decision making  Bureau officials stated the cost estimate is used to examine the cost impact of program changes  For example, the cost estimate served as the basis for the fiscal year 2019 funding request developed by the Bureau  The Bureau also said it used the 2020 Census life-cycle cost estimate to establish cost controls during budget formulation activities and to monitor spending levels for fiscal year 2019 activities  According to the Bureau, as detailed operational and implementation plans are defined, the 2020 Census life- cycle cost estimate has been and will continue to be used to support ongoing “what-if” analyses in determining the cost impacts of design decisions  Specifically, using the cost estimate to model the impact of changes on overall cost, the Bureau adjusted the scope of the Census Enterprise Data Collection and Processing (CEDCaP) operation  The processes for developing and updating estimates are designed to inform management about program progress and the use of program resources, supporting cost-driven planning efforts and well-informed decision making  Our work has identified a number of best practices for use in developing guidance related to cost estimation and analysis that are the basis of effective program cost estimating and should result in reliable and valid cost estimates that management can use for making informed decisions  In 2012 we reported that the Bureau had not yet established guidance for developing cost estimates  We recommended that the Bureau establish guidance, policies, and procedures for developing cost estimates that would meet best practice criteria  The Bureau agreed with the theme of the report but did not specifically agree with the recommendation  Moreover, in June 2016, we also reported that the cost estimation team did not record how and why it changed assumptions that were provided to it and did not document the sources of all data it used  The documentation of these changes to assumptions did not happen because the Bureau lacked written guidance and procedures for the cost estimation team to follow  During this review we found the Bureau has since established reliable guidance, processes, and policies for developing cost estimates and managing the cost estimation process  The following documents, shown in table 2, establish roles and responsibilities for oversight and approval of cost estimation processes, provide a detailed description of the steps taken to produce a high-quality cost estimate, and clarify the process for updating the cost estimate and associated documents over the life of a project  The Decennial Census Program’s Cost Estimate and Analysis Process, which provides a detailed description of the steps taken to produce a high-quality estimate, is reliable as it met the criteria for 8 steps and substantially met the criteria for 4 steps of the 12 best steps outlined in our Cost Estimating and Assessment Guide, as shown below in figure 8  To avoid cost overruns and to support high performance, it will be important for the Bureau to abide by their newly developed policies and guidance and continue to use the life-cycle cost estimate as a management tool  The 2017 life-cycle cost estimate includes significantly higher costs than those included in the 2015 estimate  In 2015, the Bureau estimated that they could conduct the operation at a cost of $12 3 billion in constant 2020 dollars  The Bureau’s latest cost estimate, announced in October 2017, reflects the same design, but at an expected cost of $15 6 billion  Figure 9 below shows the change in cost by WBS category for 2015 and 2017  The largest increases occurred in the Response, Managerial Contingency, and Census/Survey Engineering categories  Increased costs of $1 3 billion in the response category (costs related to collecting, maintaining, and processing survey response data) were in part due to reduced assumptions for self-response rates, leading to increases in the amount of data collected in the field, which is more costly to the Bureau  Contingency allocations increased overall from $1 35 billion in 2015 to $2 6 billion in 2017, as the Bureau gained a greater understanding of risks facing the 2020 Census  Increases of $838 million in the Census/Survey Engineering category were due mainly to the cost of an IT contract for integrating decennial survey systems that was not included in the 2015 cost estimate  Bureau officials attribute a decrease of $551 million in estimated costs for Program Management to changes in the categorization of costs associated with risks: In the 2017 version of the estimate, estimated costs related to program risks were allocated to their corresponding WBS element  More generally, factors that contributed to cost fluctuations between the 2015 and 2017 cost estimates include: changes in assumptions for census operations, improved ability to anticipate and quantify risk, an overall increase in IT costs, and more defined contract requirements  Several assumptions for the implementation of the 2020 Census have changed since the 2015 cost estimate  Some assumptions contributing to cost changes, mainly in the Response (related to collecting and processing response data) and Frame (the mapping and collecting addresses to frame enumeration activities) categories, include the following: Self-response rates  Changes in assumptions for expected self- response rates contributed to increases in the response category, as the assumed rate decreased from 63 5 percent in 2015 to 60 5 percent in 2017, thereby increasing the anticipated percentage and associated cost of nonresponse follow-up  When the Bureau does not receive responses by mail, phone, or Internet, census enumerators visit each nonresponding household to obtain data  Thus, reduced self-response rates lead to increases in the amount of data collected in the field, which is more costly to the Bureau  Bureau officials attributed this decrease to a forecasted reduction in Internet response due to added authentication steps at log in and the elimination of the function allowing users to save their responses and return later to complete the survey  Productivity rates  The productivity of enumerators collecting data for NRFU is another variable in the cost estimate that was updated, contributing to cost increases in the response category  Expected productivity rates for NRFU decreased from the 2015 estimate of 4 attempts per hour to 2 9  According to Bureau documentation, this more conservative estimate is based on historical data, rather than research and test data  In-office address canvassing rates  The Bureau will not go door-to- door to conduct in-field address canvassing across the country to update address and map information for every housing unit, as it has in prior decennial censuses  Rather, some areas would only need a review of their address and map information using computer imagery and third-party data sources—what the Bureau calls “in-office” address canvassing procedures  However, in March 2017, citing budget uncertainty the Bureau decided to discontinue one of the phases of in-office review address canvassing for the 2020 Census  The cancellation of that phase of in-office review is expected to increase the number of housing units canvassed in-field by 5 percent (from 25 to 30 percent of all canvassed housing units)  In-field canvassing is more labor intensive compared to in office procedures  The 2017 version of the cost estimate reflects this increase in workload for in-field address canvassing, though overall changes in estimated costs for the Frame category, of which Address Canvassing is a part, were minimal  Staffing  Updated analysis resulted in changes to several staffing assumptions, which resulted in decreases across WBS categories  Changes included reduced pay rates for field data collection staff based on current labor market conditions and reductions in the length of staff engagement  In general, contingency allocations increased overall from $1 35 billion in 2015 to $2 6 billion in 2017  This increase in contingency can be attributed, in part, to the Bureau gaining a clearer understanding of risk and uncertainty in the 2020 Census as it approaches  The Bureau developed some of its contingency based on proven risk management techniques, including Monte Carlo analysis and allocated funding for known risk scenarios  The 2017 estimate includes close to $1 4 billion in estimated costs for these risks, almost three times the amount included in the 2015 estimate  The basis of estimate contains detail on the various risks and the process for calculating the associated contingency  The 2017 version also includes a contingency amount of $1 2 billion for general risks, or unknown-unknowns, such as natural disasters and cyber-attacks  Contingency amounts were reallocated within the WBS to more closely reflect the nature of the risk: Bureau officials attribute a decrease from the 2015 estimate of $551 million in estimated costs for program management to changes in the categorization of costs associated with risks  Officials stated that, in 2015, discrete program risks were previously consolidated as program management costs  In 2017, these discrete costs were reallocated to associate risks with the appropriate WBS element  For example, contingency amounts related to the likelihood of achieving a certain response rate previously included in the program management work breakdown category are now a part of the “response” work breakdown category  Increases in IT costs, totaling $1 59 billion, represented almost 50 percent of the total cost increase from 2015 to 2017  The total share of IT costs as a percentage of total census costs increased from 28 percent in 2015 to 32 percent in 2017, or from $3 41 billion to approximately $5 billion  Increases in IT costs are spread across seven cost categories  Figure 10 shows the IT and non-IT cost by WBS for the 2017 cost estimate  IT costs in infrastructure, response data, and census/survey WBSs account for the majority of the approximately $5 billion  The Bureau’s October 2015 cost estimate included IT costs for, among other things, system engineering, test and evaluation, and infrastructure, as well as for a portion of the Census Enterprise Data Collection and Processing (CEDCaP) program  The 2017 estimated IT cost increases were due, in large part, to the Bureau (1) updating the cost estimate for CEDCaP; (2) including an estimate for technical integration services that contributed to increases in the Census and Survey Engineering category; and (3) updating costs related to other major contracts (such as mobile device as a service, field IT services, and payroll systems)  Bureau documents described an overall improvement in the Bureau’s ability to define and specify contract requirements  This resulted in updated estimates for several contracts, including for the Census Questionnaire Assistance (CQA) contract  Assumptions regarding call volume to the CQA were increased by 5 percent to account for expected response by phone after the elimination of the option to save Internet responses and return to complete the form later  The Bureau also cited updated cost data and the results of reconciliation with independent cost estimates as factors contributing to the increased costs of other major contracts, including for the procurement of data collection devices  The Secretary of Commerce provided comments on a draft of this report on August 2, 2018  The comments are reprinted in appendix II  The Department of Commerce generally agreed with our findings regarding the improvements the Census Bureau has made in its cost estimates  However, Commerce did not agree with our assessment that the Bureau’s 2017 lifecycle cost estimate is “not reliable ” Commerce noted that it had conducted two independent cost analyses and was satisfied that the cost estimate was reliable  The Bureau also provided technical comments that we incorporated, as appropriate  We maintain that, to be considered reliable, a cost estimate must meet or substantially meet the criteria for each of the four characteristics outlined in our Cost Estimating and Assessment Guide  These characteristics are derived from measures consistently applied by cost estimating organizations throughout the federal government and industry and are considered best practices for the development of reliable cost estimates  Without a reliable cost estimate, the Bureau is limited in its ability to make informed decisions about program resources and to effectively measure progress against operational objectives  Thus, while the Bureau has made considerable progress in all four of the characteristics, it has only partially met the criteria for the characteristic of being well-documented  Until the Bureau meets or substantially meets the criteria for this characteristic, the cost estimate cannot be considered reliable  As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date  At that time, we will send copies of the report to the appropriate congressional committees, the Secretary of Commerce, the Under Secretary of Economic Affairs, the Acting Director of the U S  Census Bureau, and other interested parties  In addition, this report is available at no charge on the GAO website at http://gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2757 or goldenkoffr@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix III  The purpose of our review was to evaluate the reliability of the Census Bureau’s (Bureau) life-cycle cost estimate using our Cost Estimating and Assessment Guide  We reviewed (1) the extent to which the Bureau’s life-cycle cost estimate and associated guidance met our best practices for cost estimation using documentation and information obtained in discussions with the Bureau related to the 2020 life-cycle cost estimate and (2) compared the 2015 and 2017 life-cycle cost estimates to describe key drivers of cost growth  For both objectives we reviewed documentation from the Bureau on the 2020 life-cycle cost estimate and interviewed Bureau and Department of Commerce officials  For the first objective, we relied on our Cost Estimating and Assessment Guide as criteria  Our cost specialists assessed measures consistently applied by cost-estimating organizations throughout the federal government and industry and considered best-practices for developing reliable cost estimates  We analyzed the cost estimating practices used by the Bureau against these best practices and evaluated them in four categories: comprehensive, well-documented, accurate, and credible  Comprehensive  The cost estimate should include both government and contractor costs of the program over its full life-cycle, from inception of the program through design, development, deployment, and operation and maintenance to retirement of the program  It should also completely define the program, reflect the current schedule, and be technically reasonable  Comprehensive cost estimates should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted  Specifically, the cost estimate should be based on a product-oriented work breakdown structure (WBS) that allows a program to track cost and schedule by defined deliverables, such as hardware or software components  Finally, where information is limited and judgments are made, the cost estimate should document all cost-influencing assumptions  Well-documented  A good cost estimate—while taking the form of a single number—is supported by detailed documentation that describes how it was derived and how the expected funding will be spent in order to achieve a given objective  Therefore, the documentation should capture in writing such things as the source data used, the calculations performed and their results, and the estimating methodology used to derive each WBS element’s cost  Moreover, this information should be captured in such a way that the data used to derive the estimate can be traced back to, and verified against, their sources so that the estimate can be easily replicated and updated  The documentation should also discuss the technical baseline description and how the data were normalized  Finally, the documentation should include evidence that the cost estimate was reviewed and accepted by management  Accurate  The cost estimate should provide for results that are unbiased, and it should not be overly conservative or optimistic  An estimate is accurate when it is based on an assessment of most likely costs; adjusted properly for inflation; and contains few, if any, minor mistakes  In addition, a cost estimate should be updated regularly to reflect significant changes in the program—such as when schedules or other assumptions change—and actual costs, so that it is always reflecting current status  During the update process, variances between planned and actual costs should be documented, explained, and reviewed  Among other things, the estimate should be grounded in a historical record of cost estimating and actual experiences on other comparable programs  Credible  The cost estimate should discuss any limitations of the analysis because of uncertainty or biases surrounding data or assumptions  Major assumptions should be varied, and other outcomes recomputed to determine how sensitive they are to changes in the assumptions  Risk and uncertainty analysis should be performed to determine the level of risk associated with the estimate  Further, the estimate’s cost drivers should be cross-checked, and an independent cost estimate conducted by a group outside the acquiring organization should be developed to determine whether other estimating methods produce similar results  If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high- quality estimate and cannot be considered reliable  We also analyzed the Bureau’s cost estimation and analysis guidance and evaluated them against a 12-step process outlined in our Cost Estimation and Assessment Guide  A high-quality cost estimating process integrates the following: 1  Define estimate’s purpose  2  Develop estimating plan  3  Define program characteristics  4  Determine estimating structure  5  Identify ground rules and assumptions  6  Obtain data  7  Develop point estimate and compare it to an independent cost estimate  8  Conduct sensitivity analysis  9  Conduct risk and uncertainty analysis  10  Document the estimate  11  Present estimate to management for approval  12  Update the estimate to reflect actual costs and changes  These 12 steps, when followed correctly, should result in reliable and valid cost estimates that management can use for making informed decisions  If any of the steps in the Bureau’s process do not meet, minimally meet, or partially meet the 12 steps, then the cost estimate guidance does not fully reflect best practices for developing a high-quality estimate and cannot be considered reliable  Lastly, to describe key drivers of cost growth, we compared cost information included in the 2015 and 2017 cost estimates  We analyzed both summary and detailed cost information to assess key changes in totals overall, by WBS category, and by information technology (IT) vs  Non-IT costs  We used this analysis in conjunction with information received from the Bureau during interviews and through document transfers to describe overall changes in the cost estimate from 2015 to 2017  We conducted this performance audit from December 2017 to August 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the contact named above, Lisa Pearson (Assistant Director), Karen Cassidy (Analyst in Charge), Brian Bothwell, Jackie Chapin, Ann Czapiewski, Jason Lee, Ty Mitchell, Kayla Robinson, and Tim Wexler made significant contributions to this report 


Report 2: S  military  It has had to consider, among other things, the Services’ mission requirements, resource constraints, and the myriad demands on servicemembers and their families during their frequent relocations  Anything that further complicates a relocation—such as not receiving the required family support services for family members with special needs—potentially affects readiness or, at a minimum, makes an already stressful situation worse  By providing little direction on how the Services should provide family support or what the scope of family support services should be, some servicemembers get more—or less—from the EFMP each time they relocate, including when a servicemember from one Service is assigned to a joint base led by another Service  By largely deferring to the Services to design, implement, and monitor their EFMPs’ performance, DOD cannot, as required by the NDAA for Fiscal Year 2010, fully determine the adequacy of the Services’ EFMPs in serving families with special needs, including any gaps in services these families receive, because it has not built a systematic process to do so  Instead, it relies on the Services to self-monitor and address, within each Service, the results of monitoring activities  However, because servicemembers relocate frequently and often depend on the EFMP of a Service other than their own, a view of EFMP performance across all of the Services is essential to ensuring, for example, that relocating servicemembers get consistent EFMP service delivery no matter where they are stationed  Evaluating and developing program improvements based on the results of the Services’ monitoring would help DOD ensure the Services’ EFMPs achieve the desired outcomes and improve its ability to assess the overall effectiveness of the program  We are making the following three recommendations to DOD: We recommend the Secretary of Defense direct the Office of Special Needs (OSN) to assess the extent to which each Service is (1) providing sufficient resources for an appropriate number of family support providers, and (2) developing services plans for each family with special needs, and to include these results as part of OSN’s analysis of any gaps in services for military families with special needs in each annual report issued by the Department to the congressional defense committees  (Recommendation 1) We recommend that the Secretary of Defense direct the Office of Special Needs (OSN) to develop common performance metrics for assignment coordination and family support, in accordance with leading practices for performance measurement  (Recommendation 2) We recommend that the Secretary of Defense implement a systematic process for evaluating the results of monitoring activities conducted by each Service’s EFMP  (Recommendation 3) We provided a draft of this report to the Department of Defense (DOD) for comment  DOD provided written comments, which are reproduced in appendix IV  DOD also provided technical comments, which we incorporated as appropriate  DOD agreed with all three of our recommendations  In its written comments, DOD stated that additional performance metrics need to be developed for assignment coordination and that it is in the process of measuring families’ satisfaction with family support provided through the EFMP  DOD also stated that it is developing plans for evaluating the results of each Service’s monitoring activities for the EFMP  We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and Education, and other interested parties  The report also is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (617) 788-0580 or nowickij@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  The National Defense Authorization Act (NDAA) for Fiscal Year 2017 includes a provision for GAO to assess the effectiveness of the Department of Defense’s (DOD) Exceptional Family Member Programs (EFMP)  This report focuses on the assignment coordination and family support components of the EFMP for dependents with special needs and examines: (1) the extent to which each Service has provided family support as required by DOD, and (2) the extent to which the Services monitor and DOD evaluates assignment coordination and family support  To address these objectives, we used a variety of data collection methods  Key methods are described in greater detail below  For both objectives, we reviewed relevant federal laws, regulations, and DOD guidance and documentation that pertain to the EFMP, including the following: The NDAA for Fiscal Year 2010, which established the Office of Special Needs and defined program requirements for assisting families with special needs, including assignment coordination and family support  DOD’s guidance for administering the EFMP  We assessed how DOD implements the requirements in the NDAA for Fiscal Year 2010; how each Service implements assignment coordination and family support; and how the Services and DOD monitor assignment coordination and family support using performance measures  Specially, we reviewed DOD Instruction 1315 19 - Exceptional Family Member Program; Service-specific guidance and related documents from the Air Force, Army, Marine Corps, and Navy; and DOD Instruction 1342 22 - Military Family Readiness  Standards for internal control in the federal government related to the documentation of responsibilities through policies, performance measures, and evaluating the results of monitoring activities  We compared each Service’s procedures for monitoring assignment coordination and family support to these standards  To determine the extent of the Services’ EFMP family support, we obtained and analyzed fiscal year 2016 EFMP data (the most recent available) for each Service  We reviewed DOD policy to identify data variables that each Service maintains related to its EFMP  We used these data to summarize key characteristics of each Service’s EFMP  The selected variables provided Service-wide and installation-specific EFMP information on, the number of continental United States (CONUS) and outside the continental United States (OCONUS) installations; the number of servicemembers (sponsors) enrolled in the EFMP; the number of family members with special needs enrolled in the EFMP; the number of EFMP family support personnel; and the number of services plans created for families with special needs enrolled in the EFMP  We determined that the selected data variables from each Service are sufficiently reliable for the purposes of providing summary results about family support for fiscal year 2016  To learn more about how the Services implement their EFMPs, we visited seven installations in five states  We selected the seven installations based on their location in states with the largest number of military- connected students in school year 2012-2013 (the most recent available and reliable data) or in states with the largest percentage of students enrolled in U S  DOD schools as of May 2017, as well as their status as a joint base  At each installation, we interviewed installation officials, EFMP managers, selected family support personnel, and family members and caregivers enrolled in the program  In states we visited that had the largest number of military-connected students, the EFMP personnel we interviewed collectively served 66 percent of students who attend local public schools and 42 percent of the students attending U S  DOD schools  To obtain illustrative examples about how the EFMP serves families with special needs, we conducted seven group interviews with EFMP-enrolled family members and caregivers (one at each of the seven installations we visited)  Using a prepared script, we asked participants to describe how they were identified and enrolled in the EFMP, how they were assigned to new installations, and the types of family support services they received  We also asked about how these services aligned with their family member’s EFMP-eligible condition, the benefits and challenges they experienced, as well as their overall satisfaction  A total of 38 self- selected volunteers participated in the seven group discussions  While the participants in these groups included a variety of family members and caregivers, the number of participants and groups were very small relative to the total number of family members enrolled in the EFMP  Their comments are not intended to represent all EFMP-enrolled family members or caregivers  Other EFMP-enrolled family members and caregivers may have had other experiences with the program during the same period  Finally, for both objectives, we conducted interviews with a variety of DOD, Service-level, and nonfederal officials  We spoke with DOD officials from the Office of the Assistant Secretary of Defense–Offices of Manpower and Reserve Affairs, Military Community and Family Policy, Military Family Readiness Policy, and Special Needs  We also spoke with EFMP Managers from Air Force, Army, Marine Corps, and Navy headquarters  We also met with officials from selected national military family advocacy organizations including the National Military Family Association; the Military Family Advisory Network; and the Military Officers Association of America to discuss the EFMP  We conducted this performance audit from February 2017 to May 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  Each Service has an Exceptional Family Member Program (EFMP) that provides support to military families with special needs  The tables below present the following information on selected EFMP and family support categories for each Service’s program at continental United States (CONUS) and outside the continental United States (OCONUS) installations in fiscal year 2016: City, state or country; Number of exceptional family members; Number of family support providers (by Full-Time Equivalent); Number of family support provider vacancies; Number of services plans; Number of indirect contacts; and Number of direct contacts  The information below is listed sequentially in alphabetical order by Service  We held small group discussions with Exceptional Family Member Program (EFMP) participants at the seven military installations we visited  Family members and caregivers who attended each session reported they had children or spouses with EFMP-eligible conditions  The discussion group participants were self-selected; and their comments are not intended to represent all EFMP -enrolled family members or caregivers in fiscal year 2016  In addition, other EFMP -enrolled family members and caregivers may have had different experiences with the program during the same period  There were a total of 38 participants representing all the Services  The following issues were discussed by one or more participants during the small group discussions at the installations we visited  The issues that emerged relate to the current and future overall effectiveness of the EFMP  Overall Satisfaction with EFMP (Discussed by 30 of 38 participants): Measure of participants’ approval of the family support services offered and experience with the EFMP  Many participants expressed overall satisfaction with the EFMP  Several participants expressed dissatisfaction with the EFMP  A participant expressed dissatisfaction with the lack of consistency in the provision of family support services (i e , special education advocacy) across installations  School Liaison Officers (Discussed by 20 of 38 participants): Serve as the primary point of contact for school-related matters as well as assist military families with school issues  Several participants noted that they received no response to their request for assistance from their School Liaison Officer or they only received general information  Several participants said School Liaison Officers were not helpful  Some participants found School Liaison Officers were helpful  Some participants were unaware of School Liaison Officers being available at their installation and the service(s) they provide  A few participants said School Liaison Officers did not follow up on requests for information  A participant noted there seems to be a disconnect between family support services provided through the EFMP and services provided by School Liaison Officers  Family Support Personnel (Discussed by 12 of 38 participants): Provide information and referral to military families with special needs  Some participants at one installation noted that the EFMP was understaffed  Some participants at one installation noted high turnover of family support personnel  Some participants noted family support personnel did not provide support for their family with special needs  Stigma (Discussed by 12 of 38 participants): A perception that participating in the EFMP may limit a soldier’s assignment opportunities and/or compromise career advancement  Several participants believe there is still stigma associated with participating in the EFMP  Some participants said participating in the EFMP has not affected career advancement  Assignment Coordination (Discussed by 10 of 38 participants): The assignment of military personnel in a manner consistent with the needs of armed forces that considers locations where care and support for family members with special needs are available  Some participants found the assignment coordination process challenging  Some participants described limitations with the assignment coordination process  A few participants noted there is a lack of information among families with special needs regarding how to express the need for stabilization and /or continuity of care  A few participants cited the challenges of assignment coordination as contributing to their decision to retire  One participant commented that the opinion of a medical professional was not reflected in the assignment coordination process  Special Education Services (Discussed by 10 of 38 participants): The provision of staff capable of assisting families with special needs with special education and disability law advice and/or assistance and attendance at individualized education program (IEP) meetings where appropriate  Several participants who had a family support provider assist them with preparing for or attending a school-based meeting, including IEP meetings, spoke positively of their experience(s)  Some participants at one installation agreed that assistance from family support providers during meetings with school officials regarding special education services is helpful  A few participants who were unable to get assistance with special education services from the EFMP sought the services of private attorneys at their own expense  Family Support Services (Discussed by 9 of 38 participants): The non-clinical case management delivery of information and referral for families with special needs, including the development and maintenance of a services plan  Some participants found that family support providers were helpful  Some participants could not identify needed resources or were unaware of the resources or services available to them  One participant noted that the family support provider had minimal contact  One participant said navigating the system can be challenging  Surveys (Discussed by 8 of 38 participants): The process of collecting data from a respondent using a structured instrument and survey method to ensure the accurate collection of data  Several participants noted that they had not or rarely had the opportunity to evaluate the family support services provided through the EFMP  One participant noted that comment cards used by each service are not effective for evaluating the EFMP  Warm hand-off (Discussed by 6 of 38 participants): Assistance to identify needed supports or services and facilitating the initial contact or meeting with the next program  Many participants at one installation agreed that the warm hand-off process worked well for them  Several participants said they found the warm hand-off process helpful when moving from one installation to the next  Outreach (Discussed by 5 of 38 participants): Developing partnerships with military and civilian agencies and offices (local, state, and national), improving program awareness, providing information updates to families, and hosting and participating in EFMP family events  Some participants found it difficult to obtain information regarding the types of family support services that are available  A few participants noted that communications regarding the EFMP were not targeted to address their needs  A few participants noted communications regarding the EFMP are untimely, (e g , newsletters not issued periodically)  Joint Base Family Support Services (Discussed by 1 of 38 participants): Family support services provided by the lead Service of the Joint Base that is different from that of the servicemember enrolled in the EFMP  One participant said that using family support services on joint bases may pose a challenge as each Service has different rules and procedures and as a result provides different types of family support services  In addition to the contact named above, Bill MacBlane (Assistant Director), Brian Egger (Analyst-in-Charge), Patricia Donahue, Holly Dye, Robin Marion, James Rebbe, Shelia Thorpe, and Walter Vance made significant contributions to this report  Also contributing to this report were Lucas Alvarez, Bonnie Anderson, Connor Kincaid, Brian Lepore, Daniel Meyer, and Mimi Nguyen 

ground_truth: This testimony summarizes the information contained in GAO's September 2017 report, entitled Low-Dose Radiation: Interagency Collaboration on Planning Research Could Improve Information on Health Effects ( GAO-17-546 ). The Department of Energy (DOE), Nuclear Regulatory Commission (NRC), Environmental Protection Agency (EPA), and Food and Drug Administration generally used the advice of scientific advisory bodies to develop and apply radiation protection requirements and guidance for workers and the public in the radiation exposure settings that GAO reviewed. These settings were: (1) the operation and decommissioning of nuclear power plants; (2) the cleanup of sites with radiological contamination; (3) the use of medical equipment that produces radiation; and (4) accidental or terrorism-related exposure to radiation. Specifically, the agencies relied on the advice of three scientific advisory bodies that supported the use of a model that assumes the risk of cancer increases with every incremental radiation exposure. Accordingly, the agencies have set regulatory dose limits and issued guidance to confine exposure to levels that reduce the risk of cancer, while recognizing that scientific uncertainties occur in estimating cancer risks from low-dose radiation. For example, NRC requires nuclear power plants to consider measures for limiting workers' exposure below NRC's regulatory dose limit, such as by using robots for maintenance work in radiation areas. GAO identified seven federal agencies that funded research on low-dose radiation's health effects. In fiscal years 2012 to 2016, DOE, NRC, EPA, and four other federal agencies obligated about $210 million for such research . Although the agencies have collaborated on individual projects on radiation's health effects, they have not established a collaborative mechanism to set research priorities. GAO's previous work has shown that federal agencies can use such mechanisms to implement interagency collaboration to develop and coordinate sound science policies. In the past, DOE took a leading role in this area because DOE provided stable funding and advocated for greater coordination on research on low-dose radiation's health effects. However, since fiscal year 2012, DOE has phased out funding for one of its main research programs in this area. This has created a void in coordination efforts among federal agencies, and no other agency has stepped forward to fill this void. Because of DOE's prior experience as a leader in this area of research and its research responsibility under the Atomic Energy Act of 1954, it could play an important role in helping federal agencies establish a coordinating mechanism for low-dose radiation research. Dollars are in millions and have not been adjusted for inflation Source: GAO analysis of agency data. | GAO-17-546