OMB’s Uniform Guidance was issued on December 26, 2013, and became effective on December 26, 2014. The Uniform Guidance establishes the principles for defining, calculating, and negotiating ICRs for federally funded research. The guidance describes the classification and types of allowable indirect costs; methods of allocating such costs; reasonableness of claimed costs; and exclusions and descriptions of unallowable cost elements, such as alcohol and bad debt. The Uniform Guidance also provides the basis for OMB to systematically collect information from federal agencies on all their federal financial assistance programs and establishes federal policies for providing this information to the public. NSF implements the Uniform Guidance using its Proposal and Award Policies and Procedures Guide and Grant General Conditions. The guide consists of NSF’s proposal preparation and submission guidelines as well as NSF’s policy and procedures used to award, administer, and monitor grants and cooperative agreements. NSF awards are subject to the Grant General Conditions, which list awardee responsibilities, obligations, and rights, and other conditions for awards. In addition, NSF’s CAAR staff use the Indirect Cost Rate Proposal Review Standing Operating Guidance as a guide for reviewing and negotiating indirect cost proposals from the awardees for which NSF has cognizance. The internal guidance includes descriptions of the three stages of ICR proposal review. Briefly these stages are: 1. Intake and Adequacy Review: This stage consists of steps for processing an incoming proposal including verifying that NSF has rate cognizance for the awardee and entering the proposal into the MTD, conducting initial reviews, and assigning the proposal to staff for review and rate negotiation. 2. Proposal Analysis and Rate Negotiation: This stage consists of steps for reconciling total expenditures in the financial statements with the total expenditures in the indirect cost proposal, reviewing indirect costs, verifying unallowable costs, preparing a trend analysis (including the organization’s indirect costs and rates for the previous 2 years, if available), and negotiating the indirect rate with the awardee. 3. Rate Approval and Issuance: This stage consists of steps for approving the indirect rate, preparing and transmitting the rate agreement to the awardee for signature, and closing out the proposal review by updating the MTD and filing working papers in appropriate files. Additionally, NSF’s internal guidance includes tools to assist awardees in preparing ICR proposals and to expedite NSF’s review process. These tools, as explained in further detail later in the report, are to be used in coordination with federal regulation and NSF guidance. For fiscal year 2000 through fiscal year 2016, the percentage of total annual award funding that NSF budgeted for indirect costs varied from year to year. This variation was based on various factors, such as by the types of activities supported by the awards and the types of organizations receiving the awards. Budgeted indirect costs on NSF awards ranged from 16 to 24 percent of the total annual amounts the agency awarded for fiscal years 2000 through 2016. The percentage fluctuated during this period, though it has generally increased since reaching a low point in 2010. In fiscal year 2016, NSF awards included approximately $1.3 billion budgeted for indirect costs, or about 22 percent of the total $5.8 billion that NSF awarded. Budgeted indirect costs were 16 percent of total annual amounts twice during the 17-year period—in fiscal years 2002 and 2010—and reached their highest point of 24 percent of total annual amounts in fiscal year 2015. Figure 1 illustrates annual funding for direct and indirect costs over the 17-year period. Indirect costs on individual awards within a given year varied more widely than the year-to-year variation for all NSF awards. Most NSF awards included indirect costs in their budgets—for example, the budgets for about 90 percent of the 12,013 awards that NSF made in fiscal year 2016 included indirect costs. Our analysis indicated that the funding for indirect costs ranged from less than 1 percent to 59 percent of the total award. Among the organizations whose ICR files we examined at NSF, examples of awards with indirect costs of less than 1 percent included awards to organizations that provided funding for (1) student travel to a networking workshop for women on computer and network systems and (2) a university workshop on modeling magnetic interactions between stars and planets. In contrast, examples of awards with indirect costs that ranged from 50 to 59 percent included awards to (1) a small business to study the atmosphere and improve current models of air quality and climate change and (2) a policy research organization to study how to broaden participation of underrepresented groups in the academic disciplines of science, technology, engineering, and mathematics. NSF officials told us that the overall composition of NSF’s awards portfolio varies from year to year and that indirect costs as a percentage of budgeted total award costs for any given year will reflect that variation. In particular, officials said that variation in indirect costs among individual awards—and thus variation in total award costs from year to year—can be due to several factors: Type of award supported activity: NSF’s awards support various types of activities, and NSF allows awardees to budget for indirect costs on these activities to varying degrees. For example, it does not allow indirect costs on its awards for stipends and travel of participants in NSF-sponsored conferences or training projects. In contrast, NSF allows administrative and clerical salaries to be allowed as indirect costs. However, these salaries may be considered direct costs if, for example, administrative or clerical services are integral to an activity or the costs are not recovered as an indirect costs. Type of research: NSF supports a range of research activities, some of which require investment in expensive infrastructure such as a telescope to study the universe. Which research activities are funded each year depends on a variety of considerations including the types of proposals submitted, the objectives of scientific research, the outcome of the merit review process, and available funding. Type of disciplinary field: NSF supports research in biological sciences, engineering, and social sciences, among others, and the level of indirect costs associated with awards in these fields can vary, according to NSF officials. In our analysis of awards made in fiscal year 2016, we found that the indirect costs varied among the NSF directorates that focus on different disciplinary fields. For example, budgeted indirect costs as a percentage of total annual amounts of awards were 22.5 percent in the directorate for engineering and 26.3 percent in the directorate for geosciences. Type of organization: NSF’s data categorized awardees as federal, industry, small business, university, or other—a category that includes nonprofits and individual researchers. Figure 2 illustrates our analysis on the average percentage of total awards budgeted for indirect costs in fiscal year 2016, by type of awardee. As shown in the figure, university awardees had the highest average indirect costs— about 27 percent of the total amount of awards—and federal awardees had the lowest average indirect costs—about 8 percent of the total amount of awards. According to NSF officials, certain types of projects, such as those carried out at universities, typically involve more indirect costs than others. They said that this outcome is because, for example, of the universities’ expense for maintaining scientific research facilities, which may be included as an indirect cost in awards. Universities accounted for about 91 percent of the approximately $1.3 billion NSF budgeted for indirect costs in fiscal year 2016. Awards to organizations for which NSF had cognizance (e.g., non-profits, professional societies, museums, and operators of large shared-use facilities) averaged lower budgeted indirect costs than awards to organizations for which other federal agencies had cognizance (e.g., universities for which HHS or DOD have cognizance). According to NSF officials, this variance resulted from differences in the organizations and the types of awards they receive and does not reflect differences in how agencies negotiate ICRs. As shown in figure 3, our analysis of NSF data indicates that on average, NSF budgeted about 23 percent of award amounts for indirect costs on awards to organizations for which NSF did not have indirect cost cognizance and about 11 percent for indirect costs on awards to organizations for which NSF had cognizance. In fiscal year 2016, NSF made over 90 percent ($5.4 billion of $5.8 billion) of its awards to organizations for which it did not have cognizance. NSF has developed internal guidance for setting ICRs, but NSF staff have not consistently followed it. In addition, the internal guidance on supervisory review does not include certain details and procedures. Specifically, the guidance does not include the criteria to be used by the supervisor for risk assessment and mitigation and the steps for reviewing and documenting the work performed by NSF staff when setting ICRs. It also does not include procedures for implementing new provisions issued under the Uniform Guidance and for performing oversight of the ICRs set by Interior on NSF’s behalf. Standards for Internal Control in the Federal Government states that management should design control activities, such as controls to ensure the accuracy and completeness of an entity’s data, and implement the controls through guidance, including guidance on each office’s responsibility for an agency’s operational processes. NSF has designed control activities for setting ICRs and has implemented them through internal guidance, such as Indirect Cost Rate Proposal Review Standing Operating Guidance. Our review of a nongeneralizable sample of seven NSF ICR agreement files showed that NSF staff followed many parts of the agency’s internal guidance. For example, as required by the internal guidance, NSF staff reviewed awardees’ cost policy statements to obtain an understanding of awardees’ accounting systems, cost allocation methods, and types of costs charged as either direct or indirect prior to setting ICRs. However, NSF staff did not follow two aspects of NFS’s internal guidance for setting ICRs: Use of tools and templates for setting ICRs: In accordance with NSF’s internal guidance, NSF has developed tools and templates along with procedures for using these tools to help staff conduct consistent reviews of the ICR proposals. For example, NSF’s internal guidance includes procedures for NSF staff to use a standard document checklist to verify that the awardee’s ICR proposal package is complete and that all required documents have been submitted. However, in our review of seven NSF ICR agreement files, we found that in all cases, staff did not use the standard document checklist to verify that the awardee had submitted all required documents. Instead, NSF staff used five different versions of the document checklist to track the receipt of the documents. Although these five versions of the document checklist contained similar awardee documentation, we also identified differences. For example, some checklists required certifications of lobbying costs and indirect costs, and others did not. According to the Uniform Guidance, awardees are required to submit these certifications with their proposals for ICRs. The staff were not consistently using the tools and templates NSF had developed because, according to NSF officials, NSF had not yet required them to do so. Without fully implementing this aspect of their guidance by requiring staff to use the standard tools and templates, the agency does not have assurance that NSF staff are consistently collecting required documentation from awardees and that ICRs are being set in accordance with federal requirements. Updating the Monitoring Tracking Database with current information: We identified instances where NSF staff did not consistently follow internal guidance for updating the MTD with current data about (1) the awardees for which NSF has cognizance and (2) the status of ICR proposals. NSF’s internal guidance requires staff to verify in the MTD that NSF has cognizance over the awardee prior to negotiating an ICR and to update information such as the date of receipt of the ICR proposal. In reviewing MTD reports, we identified 6 of 102 awardees for which NSF was the cognizant agency that were not included on a list of awardees for which NSF had cognizance. NSF officials confirmed that NSF was the cognizant agency for the six awardees and that the MTD had not been updated, which resulted in the awardees being incorrectly omitted from the database report. Additionally, we identified instances where NSF staff had not followed guidance to update the current status of awardees’ proposals, including instances where the MTD was missing either the received date or both the received and closed dates. NSF officials said that such errors resulted from either the agency’s incomplete reconciliation of the database, which NSF normally conducts on an annual basis after the end of the fiscal year, or from data entry errors. In order to achieve the agency’s objectives and adhere to requirements in federal internal control standards, it is essential for management to have accurate and complete ICR operational data. With accurate operational data for management to use in their decision making process, NSF could better ensure that it is managing the process for setting ICRs efficiently and in accordance with its internal guidance. NSF’s internal guidance includes various details on its control activities for setting ICRs, such as details on confirming the mathematical accuracy of the rates proposed and reconciling the total costs in the proposal, both allowable and unallowable, to the total costs shown on audited financial statements. As described in Standards for Internal Control in the Federal Government, including an appropriate level of such details allows for effective monitoring of an organization’s control activities. Specifically, through monitoring, management can assess the quality of work performed and promptly resolve any issues identified. However, we identified two areas of supervisory activities in which the procedures established by NSF did not include this level of detail. In particular, NSF’s existing internal guidance on supervisory activities, which are a key part of the agency’s control activities for setting ICRs, did not include details on (1) the criteria to be used by the supervisor to assess the risk level of a proposal and determine the types of review steps to be performed by staff for each risk level and (2) the steps that the supervisor needs to take when reviewing and documenting the work performed by NSF staff to set ICRs. According to NSF officials, risk assessments are typically performed by the supervisor for each proposal submitted and are intended to provide the basis for determining the scope of the ICR review, including the steps to be performed to mitigate the identified risks. NSF’s internal guidance includes procedures for performing risk assessment, such as reviewing past issues in the ICR negotiation for an awardee and changes in the awardee’s accounting system. According to NSF officials, part of conducting the risk assessment includes the supervisor’s categorizing the identified risk in levels of high, medium, or low and determining steps for staff to perform to mitigate risks at each level. In our sample of seven NSF ICR agreement files we reviewed, the supervisor used various criteria for assessing risks, such as NSF’s funding levels and the awardee’s rate history. Depending on the risk level, the supervisor directed staff to perform additional review steps, such as transaction testing. However, NSF’s internal guidance does not include these details on the criteria for the supervisor to use when categorizing the level of risk and the steps for mitigating the risks at each level. Supervisory review is a type of control activity that aids in providing reasonable assurance that staff follow agency procedures for setting ICRs. NSF’s internal guidance includes broad procedures for supervisory review, such as requiring the supervisor to review the staff’s completed proposal package and the applicable rates and document any concerns identified during the review; however, the guidance does not include details on what steps the supervisor needs to perform when (1) reviewing the completed proposal package and applicable rates, and (2) annotating the results of the review in the working papers. For example, the internal guidance does not include procedures requiring the supervisor to ensure that staff have adequately performed and documented key controls identified in the internal guidance, such as analyzing trends in awardees indirect costs. In our review of seven NSF ICR agreement files, we found that the files included documents supporting key controls such as trend analysis; however, we did not find any documentation that the supervisor had reviewed the work performed by staff and annotated the results of the reviews. In a September 2016 report, we identified similar issues in which agencies lacked detailed supervisory procedures, resulting in supervisors approving rates that were set by staff who did not perform control activities required by agencies’ internal guidance. NSF officials explained that the office that sets ICRs is relatively small— consisting of a single supervisor and several staff who review ICR proposals—and that as a result, supervisory activities are not as fully documented in guidance as they would be in a larger office. Additionally, the officials stated that because of the complexity of the ICR setting process, the supervisor directly discusses any concerns about a completed proposal package with the staff instead of documenting such concerns. However, more detailed internal guidance on supervisory activities could help NSF management ensure that ICRs are set consistently and in accordance with federal guidance and decrease the risk that the supervisor could approve rates that were not properly executed by staff—for example, when a new person assumes the supervisory position, as occurred in 2016. We found that NSF internal guidance does not include (1) procedures for implementing new provisions issued under the Uniform Guidance and (2) procedures in its internal guidance for performing oversight of the ICRs set by Interior on NSF’s behalf. The Uniform Guidance became effective for grants awarded on or after December 26, 2014, and NSF implemented the Uniform Guidance through its policies and procedures. The Uniform Guidance included several new provisions for research organizations, such as an option to apply for a onetime extension of an ICR for up to 4 years and an option to use a de minimis ICR of 10 percent if the organization has not previously had a negotiated ICR with the cognizant agency. However, NSF’s internal guidance for setting ICRs does not include specific procedures that NSF staff should perform to implement certain aspects of the Uniform Guidance’s new provisions. For example, the internal guidance does not specify criteria for determining whether an awardee is eligible for an extension. NSF officials stated that they updated the internal guidance to include Internet links to the Uniform Guidance and that they expect NSF staff to speak to a supervisor for clarification on questions about applying the new provisions. However, the links in the internal guidance directed staff to outdated OMB guidance rather than to the Uniform Guidance. NSF officials stated that they had incorrectly added links in the guidance. By adding procedures in its internal guidance for implementing the new provisions, NSF could better ensure that NSF staff will apply the provisions correctly in accordance with the Uniform Guidance. Standards for Internal Control in the Federal Government states that management should establish and operate monitoring activities for its internal control system and evaluate the results. In addition, when an entity engages an external party to perform certain operational processes, management retains responsibility for monitoring the effectiveness of internal control activities performed by the external party. NSF has engaged an external party to provide assistance in negotiating ICRs. Specifically, in 2009, NSF entered into an interagency agreement with Interior to negotiate ICRs for a portion of the awardees for which NSF has cognizance. For the first 10 months of fiscal year 2017, Interior had negotiated ICRs for approximately 33 percent of the awardees for which NSF had cognizance. According to NSF officials, Interior, acting as an agent for NSF, has the responsibility for reviewing ICR proposals and setting ICRs directly with awardees, following OMB requirements. In addition, NSF’s internal guidance states that Interior will set ICRs in accordance with NSF protocols. However, according to NSF officials and internal guidance, Interior’s approval and signing of ICRs are largely independent of NSF, and in addition, NSF conducts limited monitoring of the ICRs that Interior negotiates. For example, NSF officials said that they review monthly summaries of rates negotiated by Interior and completed rate agreements and that on request, they meet with Interior’s supervisors and participate in problem resolution. However, they do not review Interior’s ICR files, such as checking the adequacy of documentation submitted by awardees or the accuracy and reasonableness of the calculations and resulting rates proposed by the awardee. NSF officials stated that it was their understanding that by entering into the interagency agreement, they delegated the authority for performing ICR proposal reviews and setting ICRs to Interior. However, under Standards for Internal Control in the Federal Government, NSF still retains cognizance and oversight responsibilities for the ICRs set by Interior. By including procedures in its internal guidance for overseeing work performed by Interior, NSF could ensure that the ICRs set by Interior comply with federal guidance and NSF protocols. Setting ICRs in accordance with federal guidance is important to ensuring that federal agencies do not pay more than their share of awardees’ indirect costs. NSF has developed internal guidance to help ensure that ICRs are set appropriately, and NSF staff follow many parts of the guidance. However, staff have not consistently followed the guidance for using certain tools and templates for setting ICRs or guidance for updating the agency’s database to reflect the status of awardees and their ICR proposals—for example, because NSF has not yet required them to do so. Details on supervisory activities are also not included in the guidance, including the criteria used by the supervisor for assessing the risk level of a proposal and determining specific steps for mitigating risks, and steps supervisors take for their reviews of work performed to set ICRs. Additionally, the guidance does not include procedures necessary to carry out new provisions of the Uniform Guidance and to monitor the ICRs set by Interior on NSF’s behalf to ensure that they comply with federal guidance and NSF protocols. NSF officials described ways that staff implement these procedures even though the procedures are not fully detailed or included in guidance. However, including the missing details and procedures in NSF’s internal guidance, and requiring staff to follow the guidance, could help NSF ensure that staff properly and consistently negotiate ICRs and that the rates negotiated comply with applicable federal guidance, which in turn would help ensure that the funding provided for indirect costs does not unnecessarily limit the amount available for research. We are making the following three recommendations to NSF: The Director of NSF should require staff to follow written internal guidance for (1) using tools and templates NSF has developed for the process for setting indirect cost rates and (2) updating the agency’s database to reflect the status of awardees for which NSF has cognizance and of indirect cost rate proposals. (Recommendation 1) The Director of NSF should add details to NSF’s internal guidance for setting indirect cost rates specifying (1) the criteria to be used by the supervisor for assessing the level of risk and steps for mitigating the risks at each level and (2) the steps for supervisory review of the process for setting indirect cost rates and documentation of the results of the review. (Recommendation 2) The Director of NSF should add procedures to NSF’s internal guidance for (1) implementing the applicable new provisions of the Uniform Guidance, including updating links to OMB guidance, and (2) monitoring the indirect cost rates that the Department of Interior sets on NSF’s behalf. (Recommendation 3) We provided a draft of this report to NSF and Interior for review and comment. In its comments, reproduced in appendix I, NSF concurred with our recommendations and described actions it would take to address them. These actions include updating and adding details and procedures to its internal guidance for setting ICRs, reviewing and updating the tracking database on a quarterly basis, and working with Interior to establish procedures for monitoring ICRs set by Interior on NSF’s behalf. NSF stated that these actions will improve NSF’s protocols for negotiating ICRs. Interior stated that it did not have comments on our draft report. We are sending copies of this report to the appropriate congressional committees; the Director of the National Science Foundation; Secretary of the Department of the Interior; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact John Neumann at (202) 512-3841 or neumannj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. In addition to the contacts named above, key contributors to this report were Joseph Cook, Assistant Director; Kim McGatlin, Assistant Director; Rathi Bose; Ellen Fried; Cindy Gilbert; Ruben Gzirian; Terrance Horner, Jr.; Diana Lee; David Messman; Kathryn Smith; and Sara Sullivan. NIH Biomedical Research: Agencies Involved in the Indirect Cost Rate- Setting Process Need to Improve Controls. GAO-16-616. Washington, D.C.: September 2016. Federal Research Grants: Opportunities Remain for Agencies to Streamline Administrative Requirements. GAO-16-573. Washington, D.C.: June 2016. Grants Management: Programs at HHS and HUD Collect Administrative Cost Information but Differences in Cost Caps and Definitions Create Challenges. GAO-15-118. Washington, D.C.: December 2014. Biomedical Research: NIH Should Assess the Impact of Growth in Indirect Costs on Its Mission. GAO-13-760. Washington, D.C.: September 2013. University Research: Policies for the Reimbursement of Indirect Costs Need to be Updated. GAO-10-937. Washington, D.C.: September 2010. National Institutes of Health Extramural Research Grants: Oversight of Cost Reimbursements to Universities. GAO-07-294R. Washington, D.C.: January 2007.

Summary: NSF awards billions of dollars to institutions of higher education (universities), K-12 school systems, industry, science associations, and other organizations to promote scientific progress by supporting research and education. NSF reimburses awardees for direct and indirect costs incurred for most awards. Direct costs, such as salaries and equipment, can be attributed to a specific project that receives an NSF award. Indirect costs, such as the costs of operating and maintaining facilities, are not directly attributable to a specific project but are necessary for the general operation of an awardee's organization. For certain organizations, NSF also negotiates ICR agreements, which are then used for calculating reimbursements for indirect costs. ICR negotiations and reimbursements are to be done in accordance with federal guidance and regulation and NSF policy. GAO was asked to review the amount of NSF funding for indirect costs and NSF's negotiation of ICRs. This report examines (1) what is known about indirect costs on NSF awards over time, and (2) the extent to which NSF has implemented guidance for setting ICRs for organizations over which it has cognizance. GAO reviewed relevant regulations, guidance, and agency documents; analyzed budget data and a nongeneralizable sample of nine ICR files from fiscal year 2016 selected based on award funding; and interviewed NSF officials. For National Science Foundation (NSF) awards during fiscal years 2000 through 2016, budgeted indirect costs varied from 16 to 24 percent of the total annual amounts the agency awarded. The percentage fluctuated during this period, though this percentage generally has increased since reaching a low point in 2010. The variation from year-to-year was based on various factors such as by the types of activities supported by the awards and the types of awardee organizations receiving the awards. Note: Award funding has not been adjusted for inflation. NSF has developed internal guidance for setting indirect cost rates (ICR) but has not consistently implemented this guidance and has not included certain details and procedures, in particular: NSF has not consistently implemented its guidance because it has not yet required NSF staff to follow aspects of its guidance, such as using a documentation checklist that NSF developed to verify that an awardee's ICR proposal package is complete. NSF did not include details on supervisory activities, such as the criteria to be used by the supervisor of the ICR process for assessing an ICR proposal's risk level and mitigating risks at each level. NSF did not include certain procedures, such as for implementing new provisions of federal guidance on setting ICRs. NSF officials described ways that staff implement procedures even though the procedures are not fully detailed or included in guidance. Nevertheless, with complete guidance that includes the missing details and procedures and that is consistently followed, NSF could better ensure that ICRs are set consistently and in accordance with federal guidance on indirect costs and with federal internal control standards. GAO recommends that NSF take three actions to improve its guidance for setting ICRs, including adding certain details and procedures. NSF concurred with GAO's recommendations and described plans to address them.