National Science Foundation: Steps Taken to Improve Contracting Practices, but Opportunities Exist to Do More
Published: 2013-03-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The NSF spends more than $400 million of its $7 billion annual budget acquiring goods and services in support of its mission to promote science and engineering. Much of this spending involves exploration activities in remote locations throughout the world, such as the Arctic and Antarctic. GAO examined the extent to which NSF uses key contracting practices in three phases of the acquisition process: (a) acquisition planning, (b) contract award, and (c) post-award contract monitoring. GAO selected and reviewed a nongeneralizable sample of 11 contracts or orders with at least $3 million in funding obligations for fiscal year 2011, which accounted for about 70 percent of NSF's total contract obligations for that year. Although all 11 contracts and orders received funding during fiscal year 2011, some were awarded more than 7 years ago. Some were awarded more recently. We reviewed each of the 11 contracts to determine the extent to which they reflected the use of key contracting practices based on the Federal Acquisition Regulation, our prior work, and NSF-OIG findings. GAO also reviewed NSF contracting policies and met with NSF contracting and program officials.
section_title: What GAO Found
paragraphs: For the contracts GAO reviewed, the National Science Foundation (NSF) generally used key contracting practices in each of the three phases of the acquisition process, but the agency needs additional guidance on early acquisition planning as well as arrangements for contract audits.
The contracts GAO reviewed all involved some degree of acquisition planning, but NSF's guidance does not address appropriate time frames for early planning activities. Without such guidance, NSF contract and program officials said they could not convince their colleagues of the need to initiate early planning activities. Delays in these activities can lead to further delays later. For example, NSF had to extend one order on a non-competitive basis for more than a year to complete planning tasks for the follow-on order. In another case, the delayed award of an order compressed the data collection period for a report with firm deadlines, which could lead to higher overall costs. Further, having sufficient time for early planning may facilitate an increased use of lower risk contracting approaches.
Contract documentation showed that NSF generally followed key practices in the award phase. An NSF corrective action plan, in response to NSF's Office of Inspector General's (NSF-OIG) 2009 financial statement audits, clarifies the agency's procedures for reviewing contractors' accounting practices and financial disclosure statements to better align with key practices. Contract file documentation shows NSF improved in this area, with most of the negotiated contracts having documentation of accounting system reviews. Further, NSF generally documents price reasonableness determinations.
NSF updated its guidance and took steps to incorporate key contract monitoring practices. NSF-OIG's 2009 financial statement audits recommended that NSF obtain incurred cost submissions and audits for its largest cost-reimbursable contracts to ensure the validity of costs billed to NSF. Around the same time, the NSF-OIG and the NSF Office of the Director signed a memorandum of understanding (MOU) that provides a process for arranging for contract audits. Audits for one of the ocean drilling contracts completed in 2012 resulted in $1.5 million in recovered funds. The NSF Director and NSF-OIG have both identified additional audits of this contract as a top priority. However, despite the terms of the MOU, and the agreement between NSF and the NSF-OIG on the need for further audits, arrangements have not been made to conduct additional audits of this contract for more recent fiscal years, according to officials. Similarly, despite requests from the contracting officer, NSF has not made arrangements for incurred cost audits for another large contract GAO reviewed.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Director of NSF (1) supplement existing guidance on acquisition planning to address the time needed for the early stages of the process, and (2) arrange for audits to be performed on major contracts, consistent with the terms of the memorandum of understanding with NSF-OIG. NSF agreed with the recommendations.

section_title: Letter
paragraphs: The National Science Foundation (NSF) spends more than $400 million  annually acquiring the goods and services it needs to carry out its mission  to promote science and engineering. Much of NSF’s acquisition spending  involves exploration activities in remote and austere locations throughout  the world. Our work at several federal agencies has highlighted the  importance of sound contracting practices—such as adequate time for  planning, sufficient market research, and effective contract monitoring— as critical to a strong foundation for successful acquisition outcomes. But  in recent years, we and others, including the NSF Office of the Inspector  General (NSF-OIG), have identified various challenges regarding the  contracting practices at NSF, particularly involving the agency’s use of  cost-type contracts.
You requested that we assess NSF’s contracting practices. Specifically,  our objective was to assess the extent to which NSF incorporates key  contracting practices in the three major phases of the acquisition process:  (a) acquisition planning, (b) contract award, and (c) post-award contract  monitoring. To do so, we reviewed a nongeneralizable sample of 11  contracts and contract orders with funding obligations of at least $3  million each in fiscal year 2011, the latest year for which data were  available when we began our work. We selected contracts and orders to  reflect a mix of program offices and a variety of contract types, such as  fixed-price and cost-reimbursement. The sample we used is not  generalizable to the universe of all contracts at NSF, but the 11 contracts  and orders represent 70 percent of the total dollars obligated by NSF on  all its contracts and orders during 2011. The 11 contracts and orders  consisted of 4 contracts for which NSF used the negotiation process set  forth in Part 15 of the Federal Acquisition Regulation (FAR) and 7 orders  under existing contracts, which involved the use of streamlined  procedures described in other parts of the FAR. Although all 11 contracts  were active during the time of our review, some of the selected contracts  were awarded more than 7 years ago—before NSF updated its  contracting manual— and some more recently. As more fully described in  appendix I, we determined the extent to which each of the contracts and  orders in our sample reflected the use of selected key contracting  practices. We selected the key practices based on requirements and  principles in the FAR, practices identified in our prior work, or matters  raised in NSF-OIG recommendations. In our view, each of the selected  practices is critical to the foundation of a successful acquisition.
We conducted this performance audit from February 2012 to March 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The acquisition process at federal agencies generally consists of three  phases: (1) acquisition planning; (2) contract award; and (3) contract  monitoring. Each phase involves a number of key activities, as shown in  figure 1:  In the acquisition planning phase, agencies establish their requirements  and develop a plan to meet those requirements. Both program and  contracting officials participate in acquisition planning activities. During  this phase, agencies conduct market research to determine what  products or services are available and on what terms. They select a  contracting approach best suited to the nature of the acquisition,  addressing among other things, the availability of existing contracts,  extent of competition required, and the most appropriate contract type,  such as cost-reimbursable or fixed-price.
In the award phase, agencies solicit bids, quotes, or proposals from  prospective vendors, depending on the contracting method selected. In  negotiated acquisitions, they evaluate the submissions from vendors  under established evaluation criteria in the solicitation and award a  contract to the vendor representing the best value to the government,  based on a combination of technical and cost factors. Agencies follow a  similar process when ordering from the Federal Supply Schedule, where  quotes from contractors are evaluated using stated evaluation criteria and  orders are awarded to the contractor that would provide the best value  and offers the lowest overall cost alternative.
In the contract monitoring phase, agencies engage in a range of activities  intended to ensure that the contractor delivers according to the terms of  the contract. These activities often are described in detail in a contract  surveillance plan, sometimes called a quality assurance surveillance plan.  For cost-reimbursement contracts, agencies may arrange for an audit of  costs incurred by the contractor. These audits may be performed by  entities such as the agency inspector general or the Defense Contract  Audit Agency (DCAA).
NSF spends most of its annual budget of about $7 billion to fund grants to  universities and other research entities, but the agency also spent more  than $446 million in fiscal year 2011 acquiring goods and services in  support of its mission. The largest of these acquisitions involved contracts  for logistics support of scientific missions in the Arctic and Antarctica, as  well as ocean-drilling projects in various locations. For these types of  large-scale projects, NSF uses the negotiated contracting procedures of  Part 15 of the FAR. NSF uses negotiated contracting methods for about  66 percent of its contract spending, as shown in figure 2. For another 32  percent of its contract spending, NSF uses a variety of more streamlined  contracting methods allowed under the FAR. These include placing  orders under Federal Supply Schedule contracts awarded by the General  Services Administration or other pre-existing contracts. Placing orders  under existing contracts is often a more simplified approach than  awarding a new contract. The remaining 1 percent or so of NSF contract  spending is through various other methods, such as interagency  agreements with the U.S. Navy for deep sea research vessel certification.
The Division of Acquisition and Cooperative Support (DACS) at NSF is  responsible for the solicitation, negotiation, award, and administration of  the agency’s contracts for NSF’s research facilities and major programs.  DACS oversees NSF procurement systems, contracts policy, processes  and guidance. This Division is under the Office of Budget, Finance, and  Award Management which reports to the Office of the Director. The Office  of Inspector General provides independent oversight of the agency’s  programs and operations, including contracts. The NSF-OIG is  responsible for promoting efficiency and effectiveness in agency  programs and for preventing and detecting fraud, waste, and abuse. By  statute, the NSF-OIG is under the general supervision of the National  Science Board and reports to the Board and Congress.
Much of NSF’s contracting activity is for recurring needs, such as logistics  support for its facilities in the polar regions, data collection, or surveys.  For example, the National Survey of Recent College Graduates began in  1973 and continues today. In our prior work on acquisition planning  practices, we found that documenting decisions, particularly when there is  frequent staff turnover, is key to providing insight for subsequent  contracts. Specifically, we found that documenting cost estimates is  particularly important to help ensure the information is available when  planning for follow-on contracts. Incorporating lessons learned from prior  acquisitions can help further refine requirements and strategies when  planning for future acquisitions.
NSF officials must decide on a contract pricing arrangement for every  contract or order. The major categories of pricing arrangements NSF  uses are fixed-price, time-and-materials, and cost-reimbursement. Under  a fixed-price contract, the government generally pays a firm price and  may also pay an award or incentive fee related to performance. In a time- and-materials contract or order, the government pays a set amount for  every hour of service the contractor provides, plus the cost of any  materials used. Because the number of hours to be provided is  dependent on a number of factors, this type of contract requires an  enhanced level of government oversight. When using a cost- reimbursement contract, the government agrees to reimburse all the  allowable costs incurred by the contractor as prescribed in the contract.  These types of contracts can be risky because the government agrees to  pay for costs incurred regardless of the outcome achieved. Cost-type  contracts that exceed certain dollar thresholds generally are subject to the  cost allocation rules of the government’s Cost Accounting Standards  (CAS), and in these cases the contractor generally is required to disclose  its cost accounting practices in a CAS Disclosure Statement. We  previously reported on the use of cost-reimbursement contracts at several  agencies, including NSF, finding that agencies frequently did not  document why they selected this type of contract.
Financial statement audits performed by an independent accounting firm  on behalf of the NSF-OIG for fiscal years 2009 and 2010 identified  significant deficiencies related to the use and monitoring of cost- reimbursement contracts at NSF. Specifically, the audits found that NSF  did not ensure the adequacy of contractor accounting systems prior to  award or the validity of costs incurred on the contract. In 2011, however,  the same firm concluded that the concern had been addressed through  the adoption of new policies and procedures. While we were conducting  our audit work, NSF was in the process of conducting a self-assessment  of its acquisition function in accordance with the Office of Management  and Budget (OMB) Circular A-123. The agency also retained a consulting  firm to review its self-assessment. In July 2012, the firm issued a report  summarizing its findings. We did not assess the methodology, findings, or  conclusions of either the NSF self-assessment or the consulting firm’s  review. In October 2012, NSF updated its contracting manual to  incorporate a number of changes. For example, NSF reorganized the  manual to align it with the FAR and added additional guidance to address  the deficiencies identified in the financial audits. All of the contract  activities in our review were subject to prior versions of the contracting  manual.
section_title: Acquisition Planning Practices Conducted to Varying Degrees, but NSF Lacks Guidance on Time Needed for Early Planning Activities
paragraphs: The NSF contract files we reviewed reflected the use of selected key  acquisition planning practices to varying degrees, but the agency has not  provided guidance on the time needed to complete early planning phase  activities. Allowing sufficient time to plan procurements may facilitate an  increased use of lower risk contracting vehicles by providing time for the  contracting officer to consider including more fixed-priced elements. Our  observations of the use of some of the key practices for acquisition  planning activities are summarized in table 1, and explained in more  detail below.
The acquisitions we reviewed all involved some degree of acquisition  planning, but the time spent planning and the content of planning  documents varied. Planning for the negotiated acquisitions ranged from a  few months to more than 6 years, while many of the streamlined  acquisitions in our sample had more abbreviated planning periods.  Contracting and program officials responsible for one program office told  us they often copy planning documents from predecessor orders to  compensate for abbreviated planning periods. This practice, however,  does not allow for incorporation of new guidance or changing contract  requirements. In addition, some of the individual contract acquisition  plans for the earlier contracts in our sample did not include details on how  the agency planned to evaluate the proposals from competing vendors.  Documenting a decision regarding the plan for proposal evaluation is an  important component of the acquisition planning phase.
Contracting guidance at NSF does not identify the range of time needed  to conduct acquisition planning activities for the types of acquisitions  methods it employs. Currently, the guidance states that the process of  acquisition planning should begin as soon as a program need is identified  and it is determined that the need must be met through the use of  resources from outside the government. The guidance does not provide  any detail, however, on the expected range of time needed to conduct  planning activities in the earliest stages of an acquisition when key  documents are prepared, such as the statement of work and a cost  estimate. Acquisition planning usually occurs in three phases, and while  NSF has established expected time frames for the latter stages of  acquisition planning, the agency has not established such expectations  for the earliest planning phase. Figure 3 depicts what we found at NSF.
Allowing sufficient time to plan procurements may provide agencies a  better opportunity to clearly define contract requirements, outline source  selection procedures, conduct market research to support competition,  estimate costs, and consider opportunities for increased use of lower-risk  contracting vehicles containing more fixed-priced elements. Conversely,  the lack of sufficient time for planning may have adverse effects, such as  unplanned delays. For example, NSF had to extend one streamlined  order in our review on a non-competitive basis for more than a year and a  half in order to complete planning tasks for the follow-on order. The  contracting officer used the additional planning time to conduct the  analysis needed to incorporate more fixed-priced elements into the new  order. Planning for the earlier order did not include documentation of a  price history analysis, which, according to contracting officials, may have  helped expedite the follow-on planning and was likely due to short  planning time frames for the earlier order. In another case, the delayed  award of one of the orders in our review caused a compressed period for  data collection for a report with firm deadlines. The schedule risk from  these delays could lead to higher overall costs. Further, officials from two  program offices told us that they would benefit from knowing an expected  time range to complete early planning activities. For example, in the  absence of guidance on the time needed to complete early planning  activities, program and contracting officials responsible for NSF’s largest  contract told us they had difficulty convincing their colleagues of the  appropriate time to initiate contract planning. They added that this  acquisition required a number of changes before a follow-on contract  could be awarded—some based on updates to the FAR and some based  on internal decisions, including the use of a different source selection  strategy.
Market research is a key element in the acquisition planning phase that  provides insight into available sources for the acquisition and may provide  information on estimated costs. We found evidence of market research in  each of the acquisition plans we reviewed, though the link between the  research conducted and the impact on the acquisition strategy was not  always clear. For example, the acquisition plan for one streamlined  acquisition noted concerns about the lack of offerors for past solicitations.  The acquisition plan stated that NSF would use the Federal Supply  Schedule and release the request for quotations to six potential offerors,  but it did not address how market research impacted this decision. By  contrast, NSF engaged in extensive planning for its Integrated Ocean- Drilling Program, including requirements development and market  research to identify potential sources to support its mission. According to  officials, this planning, which occurred over about a 5-year period,  consisted of soliciting interest from more than 30 international institutions  using various techniques such as market surveys and sources sought  notices. NSF used this multi-year planning period to set up the funding  and organizational infrastructure requirements of this complex  international program.
All of the files we reviewed showed that during the planning phase  agency officials had addressed how the contract would be priced.  However, the planning documentation for the cost-reimbursable  acquisitions in our review did not consistently include assessments of the  additional risk and burden these high-risk contracts place on the agency  or an assessment of the potential for firmer pricing in future acquisitions.  Knowing the risk of using a cost-reimbursable contract and identifying  opportunities to use a less risky contract type after experience provides a  basis for firmer pricing is a sound practice identified by our prior work, by  the Department of Defense, and, more recently, in federal regulation.  Despite the risk associated with cost-type contracts, NSF contracting  officials did not document their acknowledgment of this risk for an early  contract for the ocean drilling program or whether they would attempt to  minimize the future use of a cost-type contracts. Further, in a prior report  we noted that NSF’s procurements of data collection and analysis  services for mandated surveys did not consider pricing history and  whether there was a basis to transition to firmer pricing. According to  NSF officials, when re-awarding these types of survey procurements, staff  will make an effort to identify tasks to convert to firmer pricing. In fact, a  contracting officer responsible for the survey-related orders in our sample  told us he has been conducting analysis to determine what tasks could be  transitioned to a fixed-price contract type rather than a time-and-materials  contract type. He stressed that some tasks are less suitable for fixed- pricing due to the unknowns and “what ifs” inherent in the work, but his  goal is to incorporate fixed-pricing into 70 to 80 percent of each survey  order. We identified examples of this transition to firm fixed-price  elements in some of NSF’s more recently awarded streamlined  acquisitions.
section_title: NSF Generally Followed Key Practices in the Award Phase
paragraphs: Contract documentation for negotiated and streamlined acquisitions  showed that NSF generally followed key practices in the award phase.  Table 2 summarizes our findings based on the contracts and orders we  reviewed.
Most of the contracts in our sample included price reasonableness  determinations, as outlined in both federal regulation and NSF guidance  current at the time of our review. For most of the streamlined acquisitions  we reviewed, NSF documented reasonable price determinations,  including an analysis of the contractor’s proposed labor hours and the  level of effort. In one case, contracting staff worked with an offeror to  obtain lower labor rates that were more in line with the government cost  estimate. These actions decreased the cost of the order by approximately  8 percent ($1.2 million).
In recent years, NSF has taken steps to address deficiencies related to  accounting system and disclosure statement reviews identified in its fiscal  year 2009 financial statement audits. Specifically, NSF clarified its CAS  disclosure statement and accounting system review procedures to better  align with sound practices identified by the NSF-OIG and in federal  regulation. Contract file documentation indicates that NSF has improved  in this area, with most of the negotiated contracts we reviewed having  documentation of more recent accounting system and CAS disclosure  statement reviews, and the most recent contract having documentation of  pre-award audits of all contractors in the competitive range. One of the  earlier contracts did not have pre-award audits on file or an accounting  system review prior to award. NSF officials told us that they did not think  this requirement applied. In another earlier case, the contracting officer  waived the requirement for a CAS disclosure statement adequacy  determination prior to award with the expectation that the determination  would be made shortly after award. However, NSF did not have  documentation of the final disclosure statement adequacy determination.
section_title: NSF Followed Plans for Monitoring Performance, but Incurred Costs Audits Were Not Completed Regularly
paragraphs: NSF updated its guidance and took steps to incorporate sound practices  related to contract monitoring, but the agency has not made  arrangements for audits of some of the larger contracts we sampled. Our  findings are summarized in table 3.
Most of the contracts we reviewed included documentation of surveillance  plans outlining how NSF would monitor contractor performance and  costs, although one of the streamlined acquisitions did not have the  surveillance documents called for in the acquisition plan. Further, we  found evidence that at least some monitoring activities occurred for all the  procurements we reviewed, though not always as specified in the  monitoring plans or using deliverables described in the contract or order.  For example, the acquisition plan for a large, information technology (IT)  order states that the contractor shall provide “daily, weekly, and monthly  progress reports” as well as an IT Management Plan and other ad hoc  reports as required, with similar requirements reflected in the order. The  contracting officer for this order was not aware of any daily progress  reports for this order, and added that the monitoring process for these  types of acquisitions depends on the quality of the contractor, noting that  for some contracts with few performance issues, the monitoring is less  rigorous. In another case, the contracting officer noted that despite the  statement of work calling for a Quality Assurance Plan, such a plan would  be too restrictive for an IT support contract due to the frequent changes in  IT systems.
Our prior reports state that without consistent cost surveillance, such as  through incurred cost audits, an agency may be exposed to the  unnecessary risk of overpaying the contractor. Further, NSF-OIG’s fiscal  year 2009 financial statement audits recommended that NSF obtain  incurred cost submissions and audits for its largest cost-reimbursable  contracts, depending on materiality and risk, to assure the validity of costs  billed to NSF. In response, NSF updated its guidance on incurred cost  audits and took the necessary steps to obtain incurred cost audits for its  largest contract.
Around the same time, in August 2009, NSF-OIG and the NSF Office of  the Director signed a memorandum of understanding (MOU) that provides  procedures to ensure appropriate coordination between the NSF-OIG and  NSF for the performance and funding of contract audits. The MOU  indicates that the NSF-OIG will provide, within its resources, appropriated  funds necessary to perform contract audits selected for its annual audit  plan. The NSF-OIG solicits recommendations from NSF per the MOU and  prioritizes its annual audit plan based on this input, its own needs, and a  variety of risk factors. The MOU identifies the following factors the NSF- OIG uses to prioritize contract audits: type of contract, materiality,  whether NSF is the cognizant agency responsible for contractor  oversight, known prior audit concerns, contract administration at other  federal agencies, and whether NSF expects to continue to have a  relationship with the contractor. For audits that NSF determines  necessary that are not in the NSF-OIG audit plan, the MOU states that  “NSF will obtain and fund the services of an outside auditor.” The  contracts branch officials told us that their first option is to ask the NSF- OIG to obtain an audit, and if the NSF-OIG does not complete the  contract audit, the branch tries to obtain alternative funding. NSF officials  told us, however, that alternate funding requires approval at senior  management levels, and contracting staff continue relying on the NSF- OIG as the primary means for obtaining contract audits.
The NSF Director and NSF-OIG identified the need for incurred costs  audits of an ocean drilling contract in our sample. Despite the MOU, the  agency has not made arrangements for these audits of the contract.  Officials stated for earlier years of this contract, the Contracts Branch  identified and provided funds for the contracting officer to initiate audits  for this contract through DCAA. According to officials, an audit of a prime  subcontractor for this contract resulted in $1.5 million in recovered funds.  But at the time of our review, despite agreement on the importance of  additional audits, the findings from the prior year’s audits, and NSF’s  continued relationship with the contractor, the agency had yet to make  arrangements to plan and fund incurred cost audits for more recent fiscal  years for this contract, according to officials. Similarly, despite the  contracting officer requesting incurred cost audits for another major  contract in our review, the audit did not meet the NSF-OIG priorities.  According to officials, NSF has not conducted or planned for audits on  this contract. In addition, audits for another major contract we reviewed  are not scheduled to be completed until fiscal year 2015, which is about  two years after the contract expires. In a recent report, we pointed out  that timely closing of contracts, including completing any necessary  incurred cost audits, can help the government limit its financial risk and  possibly recover improper payments.
section_title: Conclusions
paragraphs: Sound acquisition planning, including cost estimation and identification of  the most cost-effective contract type, is important to establishing a strong  foundation for successful outcomes for the millions of dollars NSF spends  annually on acquisitions. Without sufficient planning time frames to  develop acquisition plans that align with sound acquisition practices NSF  may have a limited ability to develop a strong foundation for its  acquisitions. How long the early acquisition planning activities should take  is not covered in existing NSF guidance and will vary based on the  complexity of the acquisition. However, without a clear understanding of  the time frames needed for the early acquisition planning process,  program officials may not know when to start planning or how long the  planning will take, potentially increasing the likelihood of poorly prepared  documents and contract delays. Better insights into when acquisition  planning should begin would help ensure sufficient time to carry out the  important acquisition planning activities that are designed to facilitate  more successful outcomes.
When an acquisition involves substantial uncertainties and the agency  deems a cost-type contract as the most appropriate vehicle, contract and  program staff need to provide additional oversight to protect the  government’s interests. NSF has taken steps to address NSF-OIG  recommendations to increase contract oversight. NSF has a management  responsibility to ensure that adequate resources are available to enable  contracting officers to determine that costs billed by contractors are  allowable, through incurred cost audits or similar assessments. The  process in place to ensure the necessary audits occur requires  coordination between the NSF-OIG and the NSF Office of the Director;  however, the process has not worked for some of the contracts we  reviewed. Further, the Contracts Branch continues to place a strong  reliance on the NSF-OIG to provide the resources to obtain the audits.  Without a process to ensure audits are conducted in cases when NSF- OIG resources are not available, NSF exposes itself to unnecessary risk  and cannot assure the validity of costs billed.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Director of NSF take the following two actions:
To help ensure good acquisition outcomes through comprehensive  acquisition planning, direct DACS to supplement existing guidance on  the time frames for acquisition planning to include a focus on the early  stages.
Consistent with the terms of the existing MOU with the Office of the  Inspector General, take steps to arrange, and fund as necessary,  timely audits of major contracts.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to NSF for review and comment. In  written comments, NSF agreed with our recommendations.  NSF also  provided technical comments, which we incorporated as appropriate. NSF  comments are reprinted in appendix II.
We are sending a copy of this report to the Director of the National  Science Foundation. In addition, the report is also available at no charge  on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or WoodsW@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: As requested by the Subcommittee on Commerce, Justice, Science, and  Related Agencies, House Committee on Appropriations, we reviewed the  National Science Foundation’s (NSF) contracting practices. Specifically,  we assessed the extent to which the NSF incorporates key contracting  practices in the three major phases of the contracting process: (a)  acquisition planning, (b) contract award, and (c) post-award contract  monitoring.
Within each contracting phase, we focused our work on selected  elements:
Acquisition planning. We focused on the completeness and review of  written acquisition plans, market research, contract type  determinations, and time frames for planning. We selected these  elements because they are critical to the successful planning of a  contract and, in one case, had been identified in the past by the NSF  Office of the Inspector General (NSF-OIG) as a potential concern.
Contract award. We focused on cost and price analyses, cost  accounting system reviews and pre-award audits, and Cost  Accounting Standards (CAS) Statement reviews. We selected these  elements because they were identified by the NSF-OIG as  deficiencies in the past and are essential to determining that the  contractor has the ability to complete the contract cost requirements.
Contract monitoring. We focused on the development of monitoring or  surveillance plans, monitoring activities, and incurred cost audits.  These activities were previously identified by the NSF-OIG as  deficiencies and are key to determining if the contractor is performing  as expected and within allowable costs.
To determine key practices in each of these areas, we relied on prior  reports and findings from the GAO, NSF-OIG, and other agencies. Below  is the list of GAO reports we relied on:
GAO, Standards for Internal Control in the Federal Government,  GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999);
GAO, Contract Management: Trends and Challenges in Acquiring  Services, GAO-01-753T (Washington, D.C. May 22, 2001);
GAO, Defense Contracting: Improved Insight and Controls needed  Over DOD’s Time-and-Materials Contracts, GAO-07-273  (Washington, D.C.: June 29, 2007);
GAO, Contract Management: Extent of Federal Spending under Cost- Reimbursement Contracts Unclear and Key Controls Not Always  Used, GAO-09-921(Washington, D.C.: Sept. 30, 2009); and
GAO, Acquisition Planning: Opportunities to Build Strong Foundations  for Better Services Contracts, GAO-11-672 (Washington, D.C.: Aug.  9, 2011).
We also reviewed internal NSF guidance and the Federal Acquisition  Regulation (FAR) for additional key practices.
To determine the extent to which NSF’s contracting practices incorporate  key practices and address prior NSF-OIG recommendations, we reviewed  a nongeneralizable sample of 11 contracts and orders with funding  obligations over $3 million in fiscal year 2011, the latest year for which  data were available when we began our work. We used a risk-based  approach to select our sample to ensure it included NSF acquisitions with  the highest obligation dollar amount. The 11 contracts and orders  selected for review represent 70 percent of total contract obligations in  fiscal year 2011 and reflect a mix of program offices, a range of obligation  amounts, and a variety of contract types, such as fixed-price and cost- reimbursement. We selected four contracts for which NSF used the  negotiation process set forth in Part 15 of the Federal Acquisition  Regulation and seven orders on existing contracts for which NSF used  streamlined procedures described in other parts of the FAR. The four  negotiated acquisitions in our sample are cost-reimbursement contracts  and represent about 56 percent of NSF’s total fiscal year 2011 contract  obligations and about 80 percent of the obligations in our sample. The  seven streamlined acquisitions represent about 14 percent of NSF’s fiscal  year 2011 contract obligations and 20 percent of the obligations in our  sample. One of the seven streamlined orders is a hybrid contract type  using fixed-price and time-and-materials (T&M) elements; one is a cost-  reimbursable order; and the other five are T&M orders. Although the 11  contracts were active during the time of our review , some of the selected  contracts were awarded more than 7 years ago—before NSF updated its  contracting manual to provide more procedural guidance— and some  more recently.
We reviewed the files for the selected contracts and used practices  identified in the FAR, NSF internal guidance, and prior GAO reports to  assess NSF’s use of key practices and procedures for the acquisition  planning, award, and contract monitoring phases. In addition to contract  file review, we met with contract and program officials to confirm our  understanding of information in the contract files and of NSF’s practices  and procedures as evidenced by the contract files. We also reviewed and  considered additional documentations provided by the program and  contract officials that were not maintained in the contract files.
To assess progress NSF made in response to prior NSF-OIG findings, we  reviewed prior NSF-OIG recommendations and corrective action plans.  We met with NSF-OIG officials to better understand their  recommendations related to our review and used this information to  provide assessments of progress made in response to these findings.
NSF was in the process of a full acquisition system assessment when we  initiated our review. While we were completing our audit work, NSF  issued a review of its acquisition function in July 2012. While we met with  the internal controls officials involved in this review to understand their  process, we did not assess the NSF review as part of this review.
We conducted this performance audit from February 2012 to March 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the National Science Foundation
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Penny Berrier, Assistant Director,  Caryn Kuebler, Margaret Childs, Danielle Greene, Jeffrey Hartnett, Julia  Kennon, Jean McSween, Emily Owens, Ken Patton, Erin Schoening,  Roxanna Sun, and Alyssa Weir also made key contributions to this report.