jurisdiction: US

text: b' Office of Inspector General\n Audit Report\n\n\n FAA\xe2\x80\x99S CONTRACTING PRACTICES ARE\nINSUFFICIENT TO EFFECTIVELY MANAGE ITS\n SYSTEMS ENGINEERING 2020 CONTRACTS\n Federal Aviation Administration\n Report Number: ZA-2012-082\n Date Issued: March 28, 2012\n\x0c U.S. Department of\n Memorandum\n Transportation\n Office of the Secretary\n of Transportation\n Office of Inspector General\n\n\nSubject: ACTION: FAA\xe2\x80\x99s Contracting Practices Are Date: March 28, 2012\n Insufficient To Effectively Manage Its Systems\n Engineering 2020 Contracts\n Report Number ZA-2012-082\n Federal Aviation Administration\n\n From: Mary Kay Langan-Feirson Reply to\n Attn. of: JA-60\n Assistant Inspector General\n for Acquisition and Procurement Audits\n\n To: Federal Aviation Administrator\n\n The National Airspace System handles almost 50,000 flights a day and more than\n 700 million passengers each year. To accommodate the Nation\xe2\x80\x99s air traffic, which\n is expected to triple by 2025, the Federal Aviation Administration (FAA) is\n developing the Next Generation Air Transportation System (NextGen)\xe2\x80\x94an effort\n that involves multibillion-dollar investments from both the Government and the\n airline industry.\n\n To accomplish NextGen and efforts related to maintaining the National Airspace\n System, FAA awarded seven Systems Engineering 2020 (SE-2020) base contracts\n between April and October 2010 for technical and professional support services.\n These contracts have a cumulative maximum value of $7.3 billion\xe2\x80\x94the largest\n award in FAA history. As of August 15, 2011, FAA awarded 104 task orders from\n these base contracts, valued at $319 million.\n\n Given the magnitude of these contracts and NextGen\xe2\x80\x99s technical complexity, the\n Chairmen of the House Committee on Transportation and Infrastructure and its\n Aviation Subcommittee requested that we examine FAA\xe2\x80\x99s SE-2020 contracts.\n Specifically, we assessed whether FAA (1) manages its SE-2020 contract costs\n effectively and (2) uses sound contracting practices to select contractors and\n oversee their performance. We reviewed FAA\xe2\x80\x99s SE-2020 contracts and task orders\n awarded between April and October 2010 to assess compliance with FAA and\n Federal best practices and procurement policies. In addition, we administered a\n survey to FAA\xe2\x80\x99s SE-2020 program office oversight staff. We conducted our audit\n\x0c 2\n\n\nbetween April 2010 and October 2011 in accordance with generally accepted\nGovernment auditing standards. Exhibit A details our scope and methodology.\n\nRESULTS IN BRIEF\nFAA\xe2\x80\x99s Acquisition Management System (AMS) establishes a number of\nrequirements for managing FAA\xe2\x80\x99s contracts; however, the requirements for\nensuring fair and reasonable labor rates are unclear, which has led to unreliable\ncost baselines. Without a reliable baseline, FAA cannot effectively monitor and\ncontrol contract costs. For example, AMS lacks clear requirements for addressing\nsignificant differences between contractors\xe2\x80\x99 proposed contract costs and FAA\xe2\x80\x99s\nestimated contract costs. FAA awarded the SE-2020 contracts using the\ncontractors\xe2\x80\x99 proposed labor rates, which cumulatively were 29 percent lower than\nFAA estimated. FAA also included 18 million more labor hours than needed in the\ncontracts\xe2\x80\x99 ceilings to provide more flexibility during contract administration,\nalthough the contracts already allow for such flexibility. Consequently, FAA\xe2\x80\x99s\nbaseline is overstated as the labor hours included in it exceed FAA\xe2\x80\x99s estimate.\nFurther, AMS does not clearly define requirements for Defense Contract Audit\nAgency (DCAA) audits. As such, FAA did not verify five of seven prime\ncontractors\xe2\x80\x99 proposed labor rates, which may have provided a basis for\ndetermining a reliable cost baseline. Weaknesses in its monitoring tools and\ncritical acquisition data errors further impede FAA\xe2\x80\x99s ability to ensure it does not\noverpay for professional and technical services. For example, FAA predicted that\none of its SE-2020 task orders will experience an overrun of more than\n$55 million, but our calculations showed the task order is actually projected to be\nunder budget by about $10 million.\n\nFAA\xe2\x80\x99s contracting practices for selecting SE-2020 contractors and overseeing\ntheir performance are not sufficient. AMS requires an assessment of contractor\npast performance and emphasizes the use of performance-based acquisitions for\nservice contracts to ensure that contractors provide timely, cost-effective, and\nquality results. However, we found FAA took a narrow approach to evaluating\ncontractors\xe2\x80\x99 past performance when awarding the seven base contracts and had\nnot, at the time of the audit, used performance-based acquisition for its SE-2020\ntask orders. In addition, at the time of our audit, FAA had not taken action to\nensure adequate competition for task orders, identify potential conflicts of interest,\ndocument task order decisions, or ensure that contract oversight staff have needed\nskills. For example, FAA approved the organizational conflict of interest\nmitigation plans for all seven of its SE-2020 prime contractors; yet, FAA\xe2\x80\x99s\ncontracting officer was unaware that one contractor\xe2\x80\x99s impartiality could\npotentially be impaired by its performance on a $1.8 billion NextGen acquisition\nthat may require future SE-2020 services. FAA has recently taken steps to\nimprove its contracting practices, but additional efforts are needed to ensure it\n\x0c 3\n\n\nimplements promised improvements and follows acquisition best practices. For\nexample, while FAA reported that it may award future SE-2020 task orders\ncompetitively when doing so is in the best interest of the Government, it has not\nyet formalized a process for deciding when competition is in the Government\xe2\x80\x99s\nbest interest.\n\nWe made recommendations to improve FAA\xe2\x80\x99s cost management and contracting\npractices and help ensure that the Agency can achieve desired outcomes for SE-\n2020.\n\nBACKGROUND\nIn 2010, FAA awarded its seven SE-2020 base contracts to provide up to 10 years\nof technical and professional support services for its NextGen systems, programs,\nand functions. 1 Prior to contract awards, FAA encouraged contractors to form\nteams to ensure they had the capabilities to provide required services, which\nranged from air traffic management to cost benefit analysis. The final teams\nconsist of 7 prime contractors and 95 subcontractors.\n\nThe SE-2020 contracts are cost-plus-fixed-fee contracts. Each contract establishes\nan estimated total cost for the required work and a cost ceiling that the contractor\nmay not exceed without the Government\xe2\x80\x99s approval. FAA reimburses the\ncontractor the incurred costs allowable under the contract, but the contractor\naccepts the risk for costs that exceed the cost ceiling without approval. Each\ncontract also provides for payment of a negotiated fee to the contractor, which is\nfixed at the contract\xe2\x80\x99s inception. This fee can change as the contract\xe2\x80\x99s scope of\nwork changes. SE-2020 contracts require that the contractor provide a specific\nlevel of effort, or number of labor hours; therefore, if the specified number of\nhours is not achieved at the end of the contract, the fixed fee is adjusted\ndownward. The downward adjustment represents the percentage of labor hours not\nachieved.\n\nIndividual task orders define specific requirements that contractors must\naccomplish within the base contract\xe2\x80\x99s statement of work. Each task order has an\naward value, specifies a period of performance, and establishes the number of\nlabor hours that the contractor will provide.\n\n\n\n\n1\n The 10-year performance period includes a 5-year base with two renewal (option) periods of 3 and 2 years,\n respectively.\n\x0c 4\n\n\n\nFAA LACKS ADEQUATE CONTROLS TO MANAGE ITS SE-2020\nCONTRACT COSTS\nWhile AMS establishes a number of requirements for managing FAA\xe2\x80\x99s contracts,\nsome of the requirements are unclear, which has led to FAA establishing\nunreliable cost baselines and overstated contract values for the SE-2020 contracts.\nWithout a reliable baseline, FAA cannot effectively monitor contract costs.\nFlawed cost monitoring tools and unreliable acquisition data further impede\nFAA\xe2\x80\x99s ability to ensure it does not overpay for professional and technical services.\nThe weaknesses we found in FAA\xe2\x80\x99s contract cost baseline coupled with\ninsufficient cost monitoring tools heighten the risk of FAA not being able to\neffectively manage contract costs.\n\nFAA\xe2\x80\x99s Implementation of Unclear Contract Pricing Policies Has Led to\nUnreliable SE-2020 Cost Baselines\nAMS requires contracting officers to ensure that final contract prices are fair and\nreasonable. To help determine fair and reasonable prices, FAA developed an\nIndependent Government Cost Estimate (IGCE), an internal estimate that\ndescribes how much it could reasonably expect to pay for needed supplies or\nservices. FAA estimated its cost baseline of $7.1 billion prior to contract award\nusing an IGCE but awarded SE-2020 contracts at contractors\xe2\x80\x99 proposed labor\nrates, which cumulatively were 29 percent lower than those used in the IGCE.\nAMS specifically requires a written explanation to the Chief Financial Officer\n(CFO) when IGCEs exceed contractors\xe2\x80\x99 proposals by 15 percent or more.\nHowever, FAA did not specify a due date for submitting this explanation until\nOctober 2010. 2 Despite changes to AMS, at the time of our audit, the SE-2020\nprogram office had yet to submit the required explanation for these significant\ndifferences. As a result, FAA cannot be sure that the contracts\xe2\x80\x99 cost baseline is an\naccurate benchmark for monitoring costs. This is a concern because at the time of\nour review, the contractors were also billing labor rates that were 16 percent\nhigher than proposed.\n\nIn addition, although FAA officials verified that pre-award audits were required on\nthe SE-2020 contracts, FAA did not obtain DCAA audits, which may have\nprovided a basis for determining a realistic cost baseline. We found that AMS\nguidance on obtaining such audits is not clear. 3 Specifically, AMS requires pre-\naward or post-award audits for cost-reimbursement contracts estimated to exceed\n$100 million, which allows audits to be conducted after contract award. In\naddition, AMS does not include guidance on what types of pre-award audits\n\n2\n In October 2010, FAA revised its AMS to require program offices to submit an explanation to the CFO within\n 30 days of contract award when IGCEs exceed contractors\xe2\x80\x99 proposals by 15 percent or more.\n3\n AMS T3.2.3 (A)(1)(f), \xe2\x80\x9cCost and Price Methodology, Pre-and Post-award Audits.\xe2\x80\x9d\n\x0c 5\n\n\nshould be obtained. As such, FAA did not obtain audits for five of seven\ncontractors\xe2\x80\x99 labor rates and two of seven contractors\xe2\x80\x99 accounting systems.\n\nFAA Overstated Its SE-2020 Contract Values, Which Increases Cost\nManagement Risk\nFAA overstated the contract values by including unneeded labor hours in most of\nthe SE-2020 contracts\xe2\x80\x99 ceilings. FAA stated that it needed 40 million labor hours;\nhowever, the contracts cumulatively include 58 million hours. FAA stated that it\ndoes not intend to use more than 40 million hours and that it chose to include the\nadditional hours to provide flexibility to transfer hours between contractors.\nHowever, based on language in the contracts, this flexibility already existed within\nthe original 40 million hours. As a result of including the additional hours at the\ncontractors\xe2\x80\x99 proposed rates, the contract values are significantly overstated by as\nmuch as $2 billion (see table 1), which increases the risks to FAA of not\neffectively managing the SE-2020 contracts\xe2\x80\x99 total costs.\n\nTable 1. FAA\xe2\x80\x99s Increase to Contract Values Resulting From\nInclusion of Extra Labor Hours\n Direct Labor Rates Direct Labor Hours Total Estimated\n Costs plus Fees\nAwarded contract Contractors\xe2\x80\x99 proposed FAA included $7.3 billion\nvalues rates were 29 percent 58 million hours for\n lower than IGCE more flexibility\n\nContract award values Contractors\xe2\x80\x99 proposed 40 million $5.1 billion\nusing only needed labor rates\nhours\n\nSource: FAA data and OIG analysis\n\nAnother consequence of FAA including excess labor hours is that it may award\nhigher fixed fees to contractors. Fixed fees are calculated using the contracts\xe2\x80\x99\nestimated costs. FAA increased the value of the contractors\xe2\x80\x99 estimated costs by\n$2 billion when it included the 18 million unneeded labor hours. The cumulative\n$7.3 billion contract values resulted in $428 million in fixed fees. If FAA had\nbased its calculation of the fixed fees to be earned on the 40 million needed labor\nhours, its fixed fees would have totaled $264 million, which is about $164 million\nless.\n\nThe SE-2020 contracts allow for a downward adjustment of the fixed fees if FAA\ndoes not use all of the 58 million labor hours. However, the unneeded labor hours\nincreased the awarded contract values used to calculate fixed fees; consequently,\nFAA may pay contractors more in fixed fees than it should and may have to\nrecoup these fees at the end of the contracts. For example, if FAA continues to pay\nat the 16 percent higher labor rates for 40 million labor hours, FAA would require\n\x0c 6\n\n\nadditional Government resources to recoup up to $43 million in fixed fees. In\naddition, these funds could be put to better use for other FAA programs.\n\nFlawed Cost Monitoring Tools and Unreliable Acquisition Data\nImpede FAA\xe2\x80\x99s Ability To Monitor SE-2020 Contract Costs\nAMS states that cost-reimbursable contracts require appropriate surveillance to\nminimize cost risks to the Government. However, FAA lacks effective cost\nmonitoring tools to identify potential errors. We identified several miscalculations\nin the cost monitoring spreadsheets that FAA developed and uses to identify and\nprevent potential cost overruns for SE-2020 task orders. FAA calculated each task\norder\xe2\x80\x99s projected total costs incorrectly, and its miscalculations\xe2\x80\x94such as adding\nnumbers that should have been subtracted\xe2\x80\x94resulted in projecting significant\noverruns. For example, FAA\xe2\x80\x99s calculations predicted that one of its $38 million\ntask orders would experience an overrun of more than $55 million. When we\nrecalculated this projection, we found that the task order was actually projected to\nbe under budget by about $10 million. FAA program officials were unaware of\nthese errors until we brought them to their attention. In response to our concerns,\nFAA corrected some, but not all, of its errors.\n\nFAA reports inaccurate SE-2020 cost data in the three databases it uses to fund,\nwrite, and report its procurements: Federal Procurement Database System-Next\nGeneration (FPDS-NG), the Government\xe2\x80\x99s principal repository for acquisition\ninformation; PRISM, FAA\xe2\x80\x99s contract writing system; and Delphi, FAA\xe2\x80\x99s financial\nmanagement system, which is used for accounting for costs and paying\ncontractors. For example, at the time of our review, FAA had awarded a total of\n$239 million in SE-2020 task orders, 4 but reported only $93 million in PRISM and\n$127 million in FPDS-NG (see figure 1), which understates task order values by\nabout 61 and 47 percent, respectively.\n\n\n\n\n4\n We verified this cost figure based on an OIG reconciliation of the contracting officer\xe2\x80\x99s manual spreadsheets against\n SE-2020 task orders.\n\x0c 7\n\n\nFigure 1. Differences Between Actual SE-2020 Task Order Award\nValues and Amounts Reported in FAA\xe2\x80\x99s Acquisition Databases\n\n\n\n\nSource: OIG analysis of FAA data\n\nFAA acknowledges that its acquisition data reporting are inaccurate. The SE-2020\nprogram office has implemented a monthly exceptions report to help contracting\nspecialists reconcile differences among its acquisition databases and enable FAA\nto report its contracting obligations more accurately.\n\nFAA\xe2\x80\x99S CONTRACTING PRACTICES ARE INSUFFICIENT TO\nENSURE EFFECTIVE SELECTION AND OVERSIGHT OF SE-2020\nCONTRACTORS\nAMS requires an assessment of contractor past performance and emphasizes the\nuse of performance-based acquisitions for service contracts to ensure that\ncontractors provide timely, cost-effective, and quality results. 5 However, FAA has\ntaken a narrow approach to evaluating contractors\xe2\x80\x99 past performance and has not\nused performance-based acquisition for its SE-2020 contracts. At the same time,\nFAA has not taken action to ensure adequate competition for task orders, identify\npotential conflicts of interest, document task order decisions, or ensure that\ncontract oversight staff have needed skills. FAA\xe2\x80\x99s weak processes can result in\ncost and schedule overruns and increase the risk of receiving services that do not\nmeet the Agency\xe2\x80\x99s needs. Since the inception of our audit, FAA has taken several\nsteps to improve some of its contracting practices; however, further enhancements\nare needed to ensure that FAA selects and oversees its SE-2020 contractors in the\nmost effective manner possible.\n5\n AMS 3.2.2.2, \xe2\x80\x9cSource Selection, Policy," and AMS 3.8.2.4, \xe2\x80\x9cPerformance-Based Service Contracts.\xe2\x80\x9d\n\x0c 8\n\n\nFAA Performed Limited Past Performance Evaluations of SE-2020\nContractors\nAMS requires procurement teams to evaluate prospective contractors\xe2\x80\x99 past\nperformance but gives teams latitude for the design of evaluation measures. 6 FAA\nopted to focus on personnel recruitment and retention and not on quality of work\nand customer satisfaction for its SE-2020 contractors\xe2\x80\x99 past performance\nevaluations. According to selection officials, FAA focused on recruitment and\nretention because contractors from prior systems engineering contracts had\ndifficulties hiring and retaining qualified workers. However, this narrow approach\nto past performance evaluations prevented the Agency from considering problems\non prior contracts that contractors\xe2\x80\x99 references identified. One reference reported to\nFAA that a contractor had taken 6 to 8 months to fill vacancies, but FAA did not\nconsider this feedback in its scoring of the contractor and awarded the contractor a\n$1.2 billion contract. Ultimately, FAA\xe2\x80\x99s past performance evaluation criteria\nresulted in perfect scores for all contractors, which effectively eliminated past\nperformance as a deciding factor in its SE-2020 base contract awards.\n\nFAA\xe2\x80\x99s Source Selection Official acknowledged that FAA needs to develop a more\nobjective way to assess past performance because poor performing contractors\ncontinue to win contract awards. For example, FAA had to temporarily stop\nawarding task orders to one SE-2020 contractor because it repeatedly submitted\nuntimely staffing plans and proposed using unqualified employees despite FAA\xe2\x80\x99s\nobjections.\n\nFAA Does Not Effectively Use Performance-Based Acquisition on Its\nSE-2020 Contracts\nIn awarding its SE-2020 task orders, FAA also chose not to use performance-\nbased acquisition (PBA)\xe2\x80\x94an approach that Federal regulations, policies, and\nguidance have encouraged agencies to use for over 2 decades to better ensure that\ncontractors provide quality results. PBA calls for requirements to be described in\nterms of desired results with measurable outcomes, instead of a general description\nof how the work will be performed. PBA also stresses the use of financial\nincentives tied to measurable performance criteria if an acquisition is mission-\ncritical or costly.\n\nIn 2002, we recommended that FAA implement performance-based service\ncontracting to reduce contract costs and improve performance. 7 In response to our\nrecommendation, FAA agreed to use one of its service contracts as a PBA pilot\nprogram. However, FAA had problems implementing this approach. For example,\n6\n AMS 3.2.2.2, \xe2\x80\x9cSource Selection, Policy.\xe2\x80\x9d\n7\n OIG Report Number AV-2003-002, \xe2\x80\x9cSupport Contracts: Cost Controls Over the NAS Implementation Support\n Contract Need Significant Strengthening,\xe2\x80\x9d November 15, 2002. OIG reports are available on our Web site:\n www.oig.dot.gov.\n\x0c 9\n\n\nthe performance evaluation criteria used in the PBA pilot lacked measurable\noutcomes. 8\n\nWhen we brought this matter to FAA\xe2\x80\x99s attention, it responded that some of its SE-\n2020 task orders do not lend themselves to PBA because they involve research and\ndevelopment. While 1 of the 14 SE-2020 task orders we reviewed involved\nresearch and development services, the remaining 13 task orders\xe2\x80\x94for follow-on\nwork such as program management and investment analysis 9\xe2\x80\x94could benefit from\nPBA:\n\n \xe2\x80\xa2 Of the 14 task orders, 12 lacked measurable outcomes and performance\n criteria. For example, a $24 million SE-2020 task order stated that FAA would\n evaluate a contractor\xe2\x80\x99s business case analysis and other reports on \xe2\x80\x9cquality,\n accuracy, completeness, cost, and schedule criteria,\xe2\x80\x9d but it did not specify what\n criteria FAA would use.\n\n \xe2\x80\xa2 As of October 2010, FAA had not awarded any task orders with financial\n incentives, even though the contracts allow for them.\n\n \xe2\x80\xa2 Thirteen of SE-2020\xe2\x80\x99s 93 tasks that we reviewed did not have deliverables, 10\n even though AMS requires the Agency to clearly define deliverables in its\n contracts. 11 For example, a $7.3 million task order did not require deliverables\n for safety risk studies and assessments and related activities, such as\n development of appropriate safety documentation to ensure all National\n Airspace System safety aspects are considered for the Network Enabled\n Weather Program.\n\nWhile FAA asserts that it remains committed to PBA and agrees that it should be\nused when appropriate, at the time of our audit, it still had not effectively used\nPBA on SE-2020 tasks or determined which tasks could lend themselves to a\nperformance based approach. FAA has recently indicated that it is in the process\nof determining whether PBA can be used in SE-2020 task orders and will train\nstaff on how to draft and implement PBA task orders. These efforts must be\nsustained and ultimately implemented to ensure future SE-2020 task orders\nprovide quality results.\n\n\n\n8\n OIG Report Number FI-2008-027, \xe2\x80\x9cInterim Report on Award-Fee Criteria for the National Airspace System\n Implementation Support II Contract and Bridge Contract,\xe2\x80\x9d February 27, 2008, and OIG Report Number FI-2009-\n 002, \xe2\x80\x9cInterim Report on Award-Fee Criteria for FAA\xe2\x80\x99s System Engineering and Technical Assistance II (SETA-II)\n Contract,\xe2\x80\x9d October 7, 2008.\n9\n Investment analysis provides decision makers with information about investment opportunities, such as their risks\n and value.\n10\n We could not calculate the value of the 13 tasks because they were not separately priced.\n11\n AMS T3.8.2 (A)(1)(b)(1), \xe2\x80\x9cService Contracting, General Requirements.\xe2\x80\x9d\n\x0c 10\n\n\nFAA\xe2\x80\x99s Criteria for Competing SE-2020 Task Orders Are Not Fully\nDeveloped\nFAA\xe2\x80\x99s ability to achieve the best value for SE-2020 task orders is also limited by a\nlack of competition. FAA initially chose not to award any SE-2020 task orders\ncompetitively but plans to compete select task orders in the future. Counter to\nGovernmentwide guidance, FAA senior officials eliminated an AMS requirement\nto compete task orders valued over $1 million. Therefore, FAA is not required to\nuse competition for such SE-2020 task orders. Officials from FAA\xe2\x80\x99s SE-2020\nprogram office also stated that the SE-2020 contract structure provides \xe2\x80\x9cinherent\ncompetition\xe2\x80\x9d by allowing the transfer of labor hours from one contractor to\nanother, a practice that it believes also incentivizes contractor performance.\nHowever, this structure does not encourage contractors to improve performance or\nlower costs beyond the minimum standards necessary to receive additional work.\nIn response to our concerns, FAA reported that it is working on the competitive\naward of future SE-2020 task orders but states it will only compete tasks when it is\nin the best interest of the Government. However, it has not yet formalized a\nprocess for deciding when competition is in the Government\xe2\x80\x99s best interest.\n\nFAA Has Not Developed Adequate Tools and Techniques To Support\nthe Identification of Potential OCIs on Its SE-2020 Contracts and Task\nOrders\nAMS states that FAA contracting officers must avoid awarding contracts to\ncontractors that have unacceptable organizational conflicts of interest (OCI). 12\nFAA had approved OCI mitigation plans for all seven of its SE-2020 prime\ncontractors. However, it currently lacks adequate mechanisms to supplement its\nAMS that would assist contracting officers in detecting potential OCIs when\nawarding and overseeing task orders. OCIs can arise in various situations, but one\ncommon OCI occurs when an agency hires a contractor to evaluate its own work.\nFor example, at the time of our audit, FAA\xe2\x80\x99s contracting officer was unaware that\none contractor\xe2\x80\x99s impartiality could be impaired by its performance on FAA\xe2\x80\x99s\nAutomatic Dependent Surveillance-Broadcast system, a $1.8 billion NextGen\nacquisition that may require future SE-2020 services. While each organizational\nconflict of interest scenario must be evaluated on a case by case basis, FAA has\nnot developed adequate tools and techniques to implement its AMS guidance and\nsupport the identification of OCIs. For example, FAA lacks a contractor and\nsubcontractor database that would allow FAA to verify whether contractors\xe2\x80\x99 past\nwork could potentially impair objectivity. 13 In response to our concerns, SE-2020\nprogram officials plan to improve their OCI process by developing tools that\nenhance their awareness of potential OCIs prior to SE-2020 task order awards.\n\n12\n AMS T3.1.7, \xe2\x80\x9cOrganizational Conflicts of Interest.\xe2\x80\x9d\n13\n According to AMS T3.1.7, a contractor\xe2\x80\x99s objectivity can be impaired when it provides assessment and evaluation\n findings on itself, a subsidiary, business division, or other entity with which it has a significant financial relationship.\n\x0c 11\n\n\nFAA Has Not Adequately Documented Its SE-2020 Task Order Award\nDecisions\nWhile FAA requires that SE-2020 program managers record important task order\naward decisions, such as potential contractor OCI cases and justification for\ncontractor selections, FAA does not adequately document its SE-2020 task order\naward decisions. Sufficient documentation of task order awards 14 provides a\nhistory that FAA can use when selecting contractors for future task orders.\nHowever, at the time of our audit, SE-2020 program managers did not document\n1 or more key decisions for 8 out of 14 of the task orders we reviewed. Despite\nAMS and Office of Management and Budget (OMB) 15 requirements for sufficient\ncontract documentation, FAA acknowledged that, in its urgency to award task\norders by the end of the fiscal year, it did not thoroughly document its award\ndecisions. In response to our concerns, FAA has agreed to develop more complete\nrecords of its task order award decisions which may help FAA improve its task\norder awards in the future.\n\nFAA Lacks Adequate Contract Oversight Mechanisms for Its SE-2020\nTask Orders\nFederal acquisition policy and regulations emphasize the need for sound oversight\nprinciples to ensure that contractors meet cost, schedule, and performance goals. It\nis critical that FAA train and develop its contract oversight employees because\nthey play an important role in ensuring that SE-2020 contractors provide quality\nand timely services. However, during the course of our audit, we found that FAA\ndid not require its contract oversight staff to receive training or use oversight\nplans. Only 4 of 26 oversight employees 16 who responded to our survey\nmaintained oversight plans for their assigned task orders. 17 In addition, we\nreviewed the four oversight plans that were maintained and found that they did not\noffer detailed methods for assessing contractors\xe2\x80\x99 work results. Instead, the plans\nconsisted mainly of contractor task lists.\n\nIn March 2011, FAA began requiring contract oversight staff to complete monthly\nstandard performance monitoring templates for each task order. While OMB\xe2\x80\x99s\nbest practices state that oversight plans should focus on desired performance\noutcomes, FAA\xe2\x80\x99s new templates do not include measurable criteria to evaluate\ndesired performance outcomes, such as quality, cost, and schedule. 18 FAA also\nbegan requiring all contract oversight staff to be certified Contracting Officers\n14\n AMS T3.10.1 (A)(10)(a)(2 and 3), \xe2\x80\x9cContract Administration, Contract Files.\xe2\x80\x9d AMS does not define \xe2\x80\x9csufficient.\xe2\x80\x9d\n15\n OMB Circular A-123, \xe2\x80\x9cManagement\xe2\x80\x99s Responsibility for Internal Control,\xe2\x80\x9d December 21, 2004.\n16\n We surveyed 85 contract oversight employees assigned to SE-2020 planned and awarded task orders as of\n October 31, 2010.\n17\n One oversight employee responsible for monitoring three task orders, with a cumulative value over $30 million, did\n not establish and maintain oversight plans to oversee the task order contractors.\n18\n OMB\xe2\x80\x99s Office of Federal Procurement Policy (OFPP), \xe2\x80\x9cA Guide to Best Practices for Contract Administration,\xe2\x80\x9d\n October 1994.\n\x0c 12\n\n\nTechnical Representatives (COTR) and complete mandatory COTR oversight\ntraining in April 2011 after we shared our concerns about their lack of training\nrequirements.\n\nCONCLUSION\nImplementing sound procurement practices is critical for FAA to safeguard against\nwasteful spending and help ensure it acquires the best value in professional and\ntechnical services\xe2\x80\x94especially for FAA\xe2\x80\x99s $7 billion SE-2020 contracts portfolio,\nwhich is the largest award in its history. FAA\xe2\x80\x99s ineffective cost management and\ncontracting practices, however, could result in schedule and cost overruns and\nincrease the risk of receiving services that do not meet the Agency\xe2\x80\x99s needs. While\nFAA has responded to many of our concerns and taken steps to improve its\ncontract management, it has faced challenges in implementing procurement\npractices that have been proven to prevent wasteful spending and improve quality\nresults, such as PBA. While FAA is in the early stages of using its SE-2020\ncontracts, it is imperative that it take prompt action to improve its contracting\npractices and sustain efforts on promised improvements to manage and monitor\nfuture SE-2020 performance\xe2\x80\x94especially given the billions of dollars the Agency\nmay yet award. Until FAA takes such actions, it will continue to put FAA\xe2\x80\x99s\nNextGen and its National Airspace efforts\xe2\x80\x94as well as taxpayer funds\xe2\x80\x94at risk.\n\nRECOMMENDATIONS\nWe recommend that FAA\xe2\x80\x99s Vice President of Business and Acquisition Services:\n\n 1. Require the SE-2020 program office to (a) submit to the CFO a written\n reconciliation of the difference between its IGCE and the contractors\xe2\x80\x99\n proposals and (b) use this reconciliation as a basis to develop a reasonable\n cost baseline.\n\n 2. Revise AMS to require that, when IGCEs exceed contractor proposals by\n 15 percent or more, program officials submit an explanation and\n recommended corrective actions to the CFO before contract award.\n\n 3. Revise AMS to (a) specifically require pre-award and post-award audits of\n contracts in excess of $100 million and (b) define the types of pre-award\n audits required, including\xe2\x80\x94at a minimum\xe2\x80\x94direct labor rates, indirect\n rates, and accounting system reviews.\n\n 4. Amend SE-2020 awarded contract values using contractors\xe2\x80\x99 proposed rates\n and FAA\xe2\x80\x99s estimated need for 40 million hours and adjust fixed fees to\n reflect revised contract costs.\n\x0c 13\n\n\n 5. Require the SE-2020 program office to (a) develop policies and procedures\n to ensure timely reconciliations and corrections to acquisition databases and\n (b) revise its cost monitoring spreadsheets to ensure accurate data for\n effective cost control of SE-2020 contracts.\n\n 6. Revise AMS to establish controls that require more comprehensive\n evaluations of contractor past performance.\n\n 7. Require FAA\xe2\x80\x99s contracting and program staff to use performance-based\n acquisition principles in their SE-2020 task orders and ensure staff is\n adequately trained to develop and monitor such awards.\n\n 8. Require the SE-2020 program office to define criteria that specify when\n competing task orders are in the Government\xe2\x80\x99s best interest.\n\n 9. Revise AMS to include guidance on how to identify and mitigate risks of\n potential OCIs prior to contract or task order award.\n\n10. Require FAA contracting personnel to develop, maintain, and use a record\n of active prime contractors and subcontractors to identify and mitigate risks\n of potential OCIs.\n\n11. Require the SE-2020 program office to develop policies and procedures to\n ensure adequate documentation of task order award decisions.\n\n12. Require the SE-2020 program office to amend the standard performance\n monitoring templates to include measurable criteria to evaluate desired\n performance outcomes, such as quality, cost, and schedule.\n\nAGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL\nRESPONSE\nWe sent FAA our draft report on October 31, 2011, for comment, which FAA\nprovided in a letter dated January 20, 2012 (see appendix). In its letter, FAA\nconcurred or partially concurred with 10 of our 12 recommendations and\nrecognized the need to continually seek ways to further strengthen SE-2020\ncontract oversight and program management. Despite its general concurrence,\nFAA raised concerns about the presentation of our findings.\n\nFAA concurred with six recommendations and provided acceptable planned\ncorrective actions and implementation dates for three of these recommendations\n(1, 8, and 12). These three recommendations will remain open pending our\nverification that planned actions are complete and sufficient. However, additional\n\x0c 14\n\n\nactions are needed to ensure FAA meets the intent of the three other\nrecommendations. Specifically:\n\n \xe2\x80\xa2 To meet the intent of recommendation 5(a), FAA needs to submit a formal\n implementation plan for its proposed reconciliations of information and data in\n Delphi, PRISM, and FPDS to ensure they are accurate and consistent. Also, for\n recommendation 5(b), FAA needs to submit a plan to periodically perform\n audits of its spreadsheets used to monitor its SE-2020 contracts and task order\n costs throughout the contracts\xe2\x80\x99 entire period of performance.\n\n \xe2\x80\xa2 To meet the intent of recommendation 7, FAA needs to submit a formal plan to\n ensure the implementation of performance-based contracting on new task\n orders that are suitable for PBA. FAA also needs to include COTRs in its PBA\n training plans for its contracting officers and SE-2020 Program Management\n Office staff. COTRs who are assigned to task orders are part of the program\n office acquiring the work and represent first-line oversight of the contractor\xe2\x80\x99s\n performance. Training such officials to monitor and measure the contractor\xe2\x80\x99s\n performance is critical to minimizing risks.\n\n \xe2\x80\xa2 To meet the intent of recommendation 11, FAA needs to ensure adequate\n documentation of task order award decisions. While FAA stated that it plans to\n conduct biannual reviews of the Adjudication Board\xe2\x80\x99s records to\n institutionalize the adjudication process and provide a thorough documentation\n and evaluation process, these planned actions need to be documented to ensure\n the reviews consistently provide control over the task order award process.\n\nFAA partially concurred with four recommendations and submitted acceptable\nplanned corrective actions and implementation dates for these recommendations.\nFAA already completed actions to address one of these recommendations (3), and\nwe consider it closed. The remaining three recommendations (6, 9, and 10) remain\nopen pending our verification that FAA\xe2\x80\x99s planned actions are complete and\nsufficient. To address recommendation 9, we encourage FAA to document the\nresults of its review of the practices developed by its SE-2020 team to avoid and\nmitigate potential OCIs cited in its response letter. FAA needs to use this\ninformation to improve its tools and techniques to implement its existing AMS\nguidance on OCIs.\n\nFAA did not concur with the remaining two recommendations (2 and 4). First,\nFAA did not agree that program officials should submit to the CFO before\ncontract award an explanation and recommended corrective actions for contractor\nproposals that are 15 percent below IGCEs. FAA states that its current policy\nrequiring such submissions to the CFO within 30 days after award is sufficient and\nthat its CFO and FAA Acquisition Policy offices agreed there is no added value to\n\x0c 15\n\n\nseeking approval prior to award. While providing an explanation after contract\naward can help inform future awards and IGCEs, implementing measures to\nmanage contract costs before contract award is critical to ensure FAA does not\noverpay for professional and technical services. FAA also noted that no additional\nactions were needed \xe2\x80\x9cparticularly in light of the state of the economy.\xe2\x80\x9d We\ndisagree. A weak economy can incentivize contractors to propose unreasonably\nlow bids, which in turn can lead to significant cost over-runs, poor performance, or\nboth\xe2\x80\x94risks that FAA should not undertake in any economy. We request that FAA\nreconsider its position and propose a course of action that would meet the intent of\nour recommendation.\n\nSecond, FAA did not concur with our recommendation to amend SE-2020\nawarded contract values and adjust fixed fees to reflect revised contract costs.\nHowever, it submitted alternative actions that partially meet the intent of the\nrecommendation. Specifically, FAA agreed to (1) reconcile each task order\xe2\x80\x99s total\ncost and fee at task order close out and (2) complete an analysis of fees paid\nthrough the end of calendar year 2011 and provide OIG with the results of the\nanalysis by June 30, 2012. To fully meet the intent of our recommendation, FAA\nmust perform periodic reconciliations of the SE-2020 contracts\xe2\x80\x94which have a\npotential 10-year performance period\xe2\x80\x94to ensure that it does not pay excess fees to\ncontractors that it would need to recoup at a later date. Consequently,\nrecommendation 4 will remain open until FAA provides OIG a schedule requiring\nperiodic reviews of the fees paid to contractors and finalizes resolution of the June\n2012 analysis.\n\nThe FAA response letter acknowledged that it is committed to providing\nmeaningful oversight of its SE-2020 contracts, and we cited in our report a number\nof recent actions FAA noted in its letter. In addition, we also made changes where\nappropriate to address FAA\xe2\x80\x99s comments. However, we remain concerned that\nFAA does not always distinguish planned and recently implemented actions from\nthose that have been fully institutionalized. Planning and implementation without\nenforcement do not constitute strong and effective management controls. For\nexample, FAA revised its AMS policy in October 2010 to require program offices\nto submit an explanation to the CFO within 30 days of contract award when\nIGCEs exceed contractors\xe2\x80\x99 proposals by 15 percent or more. However, as of\nOctober 31, 2011\xe2\x80\x94when we completed our audit work\xe2\x80\x94FAA had yet to submit\nto the CFO an explanation as to why the contractors\xe2\x80\x99 proposed labor rates were\n29 percent lower than the IGCE\xe2\x80\x99s.\n\nWe will continue to work with FAA to better ensure that its management controls\nminimize contract risks and maximize taxpayer dollars.\n\x0c 16\n\n\n\nACTIONS REQUIRED\nIn accordance with Department of Transportation Order 8000.1C, we request that\nyou provide a response to this report within 30 days that clearly indicates how you\nwill resolve recommendations 2, 4, 5, 7, and 11. We request that you provide the\nfollowing: formal implementation plans for the reconciliations of Delphi, PRISM,\nand FPDS and the audits of SE-2020 contracts and task order costs, as well as\nplanned action dates for recommendation 5; a plan to ensure that COTRs assigned\nto performance-based task orders are trained and a planned action date for\nrecommendation 7; documentation of the Adjudication Board record review\nprocess and a planned action date for recommendation 11; and a schedule of\nperiodic reconciliations of fees paid to contractors, final resolution of the June\n2012 analysis, and a planned action date for recommendation 4. For\nrecommendation 2, we are asking the Agency to reconsider its position.\n\nWe appreciate the courtesies and cooperation of FAA representatives during this\naudit. If you have any questions concerning this report, please contact me at (202)\n366-5225; Tony Wysocki, Program Director, at (202) 493-0223; or Dana Short,\nProject Manager, at (202) 366-2089.\n\n #\n\ncc: Martin Gertel, M-1\n Pierre McLeod, AAE-100\n\x0c 17\n\n\n\n\nEXHIBIT A. SCOPE AND METHODOLOGY\nWe conducted this audit between April 2010 and October 2011 in accordance with\ngenerally accepted Government auditing standards. Those standards require that\nwe plan and perform the audit to obtain sufficient, appropriate evidence to provide\na reasonable basis for our findings and conclusions based on our audit objectives.\nWe believe that the evidence obtained provides a reasonable basis for our findings\nand conclusions based on our audit objectives.\n\nTo address our audit objectives, we selected 100 percent of SE-2020 task orders\nvalued at $5 million or more\xe2\x80\x94a total of 14 task orders valued at $153 million, or\n64 percent of the total universe of awarded task orders. We selected these task\norders out of a universe of 77 task orders\xe2\x80\x94valued at $239 million\xe2\x80\x94awarded from\nSE-2020\xe2\x80\x99s 7 base contracts as of October 31, 2010. We assessed whether these\ntask orders comply with FAA and Federal best practices and procurement policies.\nWe also reviewed 100 percent of FAA\xe2\x80\x99s SE-2020 contractor invoices received as\nof November 22, 2010, to identify differences between the contractors\xe2\x80\x99 billed and\nproposed labor rates.\n\nWe assessed FAA\xe2\x80\x99s procurement practices and compliance against criteria and\nbest practices from the following sources:\n\n \xe2\x80\xa2 FAA\xe2\x80\x99s Acquisition Management System (AMS), including 3.0 Procurement\n Policy and Guidance, 5.0 Acquisition Career Program, and FAA Pricing\n Handbook\n \xe2\x80\xa2 Office of Management and Budget (OMB)\n o OMB Circular A-123, \xe2\x80\x9cManagement\xe2\x80\x99s Responsibility for Internal Control,\xe2\x80\x9d\n dated December 21, 2004\n o OMB Memorandum, \xe2\x80\x9cConducting Acquisition Assessments under OMB\n Circular A-123,\xe2\x80\x9d dated May 21, 2008\n o Office of Federal Procurement Policy (OFPP) guidance, \xe2\x80\x9cSeven Steps to\n Performance-Based Service Acquisition\xe2\x80\x9d\n \xe2\x80\xa2 Presidential Memorandum, \xe2\x80\x9cGovernment Contracting,\xe2\x80\x9d dated March 4, 2009\n \xe2\x80\xa2 DOT\xe2\x80\x99s OMB Circular A-123 Acquisition Assessment Report, dated November\n 13, 2008\nWe assessed whether FAA (1) manages its SE-2020 contract costs effectively; and\n(2) uses sound contracting practices to select contractors and oversee their\n\n\n\nE xh i b it A. S co p e an d M et h o d o l o g y\n\x0c 18\n\n\nperformance. We also interviewed FAA program office, contracting, legal, and\nother management personnel.\n\nIn addition, we administered a survey to 100 percent of the 85 program office\noversight staff assigned to SE-2020\xe2\x80\x99s planned and awarded task orders as of\nOctober 31, 2010, to which 26 responded. We analyzed the survey results to\ndetermine if procedures are in place to ensure that task orders are adequately\nmonitored. To assess the reliability of SE-2020 contract data in FAA\xe2\x80\x99s acquisition\ndatabases, we compared data from the following systems: Federal Procurement\nDatabase System-Next Generation (FPDS-NG), the Federal Government\xe2\x80\x99s\ncontract reporting system; PRISM, FAA\xe2\x80\x99s contract writing system; and Delphi,\nFAA\xe2\x80\x99s accounting system.\n\n\n\n\nE xh i b it A. S co p e an d M et h o d o l o g y\n\x0c 19\n\n\n\n\nEXHIBIT B. MAJOR CONTRIBUTORS TO THIS REPORT\n\n\nName Title\nAnthony Wysocki Program Director\n\nDana Short Project Manager\n\nRachel Alderman Senior Auditor\n\nJill Cottonaro Senior Analyst\n\nAngela Hailes Analyst\n\nDavid Lahey Auditor\n\nAaron Malinoff Analyst\n\nPatrick Paradise Analyst\n\nKathryn Novicky Analyst\n\nArnett Sanders Project Manager\n\nAmy Berks Senior Counsel\n\nKaren Sloan Communications Officer\n\nChristina Lee Writer-Editor\n\nSusan Neill Writer-Editor\n\nPetra Swartzlander Statistician\n\n\n\n\nE xh i b it B . M ajo r Co n t rib u t o r s t o T his R ep o rt\n\x0c 20\n\n\n\n\nAPPENDIX. AGENCY RESPONSE\n\n\n Federal Aviation\n Administration\n\nMemorandum\nDate: January 20, 2012\nTo: Mary Kay Langan-Feirson, Assistant Inspector General for Acquisition and\n Procurement Audits\n\nFrom: H. Clayton Foushee, Director, Office of Audit and Evaluation (AAE-1)\nSubject: Office of Inspector General Draft Report on FAA\xe2\x80\x99s Systems Engineering 2020\n Contracts\n\n\n\n\nThe System Engineering 2020 Contract Program (SE-2020) currently has strong and effective\nmanagement controls as well as appropriate oversight in place to ensure that the contracts\nprovide the FAA with a cost efficient means to obtain the services necessary to support Next\nGeneration Air Transportation System (NextGen) initiatives, as well as non-NextGen activities.\nThe SE-2020 support service contracts are a complex and innovative undertaking. From the\nbeginning, the FAA recognized the need to provide meaningful oversight and has continually\nsought ways to further strengthen our management of the program. As a result of these efforts,\nFAA has implemented systematic improvements to the program that were not captured in the\nOIG draft report. As a result it does not convey an accurate representation of the program as it is\ncurrently operating. The following are a number of key points that will be more fully developed\nin the sections to follow. Specifically, FAA:\n \xef\x83\x98 Implemented enhanced and systematic oversight based on sound practices and standard\n operating procedures;\n \xef\x83\x98 Established a reasonable cost baseline, and more importantly, is establishing reliable task\n order cost baselines;\n \xef\x83\x98 Used sound contracting principles to select the SE-2020 contractors;\n \xef\x83\x98 Committed to full and appropriate use of performance based contracting; and\n \xef\x83\x98 Established systems and controls to avoid organizational conflicts of interest.\n\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 21\n\n\nFAA Enhanced SE-2020 Contract Oversight\nThe FAA is committed to the sound management of the SE-2020 contracts and has dedicated\nextensive resources to provide effective contract oversight. In part, this oversight implements\nprocess improvements that will enhance the agency\xe2\x80\x99s management of the SE-2020 contracts. As\na result of FAA\xe2\x80\x99s efforts, significant actions have been implemented since the OIG completed\nthe audit work that is the basis for the draft report. Specifically, the FAA has:\nIncreased Competition for Task Orders - The SE-2020 PMO is currently defining criteria for\nspecifying task order competition. As instructed by Office of Management and Budget (OMB),\nguidelines for best practices for multiple award task order contracts, and supported by the\nGovernment Accountability Office (GAO), it is essential to strike an appropriate balance\nbetween the value inherent in open competition for task orders, the cost and delay that\ncompetition also creates, and the value of retaining well-performing contractors, particularly to\ncontinue or complete work. A process document will be finalized by January 31, 2012 to define\nrequirements that lend themselves to task order competition. Two significant task order\ncompetitions are currently being prepared for release with awards planned for February 2012.\nIncreased Staffing Assigned to Contract Management and Oversight - SE-2020 contracts are\noverseen by eight fully-dedicated and trained contracting officers and specialists. Eight\nContracting Officer\xe2\x80\x99s Technical Representatives (COTRs) are assigned along with a specialized\nFinancial and Cost Team that are dedicated to the monitoring and oversight of costs. In total,\nthere are approximately 45 full-time staff dedicated to the management and oversight of the\ncontracts.\nImplemented Standard Operating Procedures - In August 2011, FAA implemented standard\noperating procedures across SE-2020 task orders that include documentation requirements, User\nGuides, and Work Plans. These procedures were under development for over the past 18\nmonths, and with their implementation, the FAA is using best practices to improve performance\nmeasures, cost controls, guidance and training to all SE-2020 team members.\nImplemented Formal Metrics \xe2\x80\x93 As part of its standard operating procedures, FAA\nimplemented a set of formal metrics to track and evaluate performance on SE-2020 task orders\nand other internal program metrics. Among other things, these metrics track vendor\nperformance, timeliness of vendor deliverable submissions; and process efficiency in areas\nincluding task order awards, task order modifications, resume submissions, and invoice\nsubmissions.\nImplemented Process Enhancements - SE-2020 task orders award processes have been revised\nto further ensure that award decisions are sound, thorough, and well-documented. For example,\nthe FAA enhanced the process for identifying and managing Organizational Conflicts of Interest\n(OCI) that could affect the SE-2020 contracts by implementing additional review procedures to\nmonitor and detect OCIs. These additional review procedures include the incorporation of an\nOCI/Conflict of Interest (COI) Evaluation section as part of the task order award process, the\naddition of an OCI certification in the vendor\xe2\x80\x99s \xe2\x80\x9ctechnical task plan\xe2\x80\x9d response, and enhancement\nof the SE-2020 \xe2\x80\x9cresume review module\xe2\x80\x9d to include a required COI section as part of the resume\nsubmission. In addition, the FAA has assigned a dedicated technical advisor in the SE-2020\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 22\n\n\nProgram Management Office (PMO) to analyze every SE-2020 task order and assist the SE-2020\nContracting Officer identify and assess potential OCI issues.\nThese examples demonstrate the FAA\xe2\x80\x99s commitment to strengthening its processes and\nprocedures to further reduce program costs, increase program controls, and improve performance\nmanagement on the SE-2020 contracts. While the FAA recognizes that there are areas for\nadditional improvement in our management of cost and performance on the SE-2020 contracts,\nwe do not believe that the audit findings support the broad conclusion that the FAA is failing to\neffectively manage costs and performance on these contracts. As a result, we request the OIG\nrevisit its conclusions regarding cost and performance management of SE-2020.\n\nFAA Established a Reasonable Cost Baseline for SE-2020\nThe Independent Government Cost Estimate (IGCE) developed for the SE-2020 competition is a\nreasonable and appropriate baseline against which cost overruns should be measured at a global\ncontract level. The SE-2020 contracts provide FAA program offices with access to a wide\nvariety of research and systems engineering support services to help FAA organizations carryout\nactivities related to NextGen and other National Airspace System (NAS) requirements. The\nIGCE developed for SE-2020 represents the FAA\xe2\x80\x99s independent and rigorous estimate of the\ncosts for the support services that are likely to be acquired by the FAA program offices under the\nSE-2020 contracts. The FAA dedicated extensive resources to develop the IGCE for SE-2020\nincluding: conducting in-depth analysis of costs incurred on a variety of contracts held by the\nFAA and other Federal agencies; obtaining industry input from two market surveys; analyzing\nsalary data obtained from multiple salary data services; and incorporating lessons learned on\nother FAA procurement efforts. These efforts are documented in detail in the SE-2020 Business\nCase which provides a detailed description describing the methodology and assumptions used to\ncreate the SE-2020 IGCE.\nThe SE2020 team will submit data showing the differences between its IGCE and the proposals\nit received to the FAA Chief Financial Officer (CFO). The CFO will consider whether the\noverall cost baseline for this program should be revised. Given the nature of this program,\nhowever, it is more critical to establish cost control mechanisms at the task order level than it is\nto determine the precisely accurate baseline for the overall set of contracts, particularly as these\ncontracts have two quite different purposes. The SE-2020 contracts essentially function as an\n\xe2\x80\x9cumbrella\xe2\x80\x9d contract mechanism that an FAA program office can use to place orders for services\nit requires to support a particular mission. Program office \xe2\x80\x9ccustomers\xe2\x80\x9d use the SE-2020 contracts\nto obtain highly technical support services in a wide range of areas such as research and\ndevelopment, systems engineering, and investment analysis. As it is not possible to establish\ndefinite performance and schedule parameters at the total SE2020 program level, the cost\nbaseline at the SE-2020 program level is not a significant tool for the agency to use to control\ncosts. 19\nThe SE-2020 program works with program office customers to develop and issue task orders for\nthe support services the program office needs to support a project or activity. Each task order\ndescribes, among other things, the scope of the work to be conducted, the period of performance,\nand the cost of the work. Before issuing a task order, the SE-2020 team and the program office\n\n19\n The FAA will continue, however, to monitor the SE2020 program at this level as well as at the task order level.\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 23\n\n\ncustomer work together to analyze the specific work that will be conducted under the task order\nand develop an IGCE specifically tailored for that work. The FAA then asks the contractor to\nsubmit a cost proposal for the task order. When the FAA receives the cost proposal, the SE-2020\nteam and the program office customer analyze the cost proposal in detail and compare it to the\nIGCE to ensure that the proposed cost for the task order is fair and reasonable and that the FAA\nwill not overpay for the services. Before issuing a task order, the FAA also negotiates with the\ncontractor to further control the cost for the task order. The FAA will award the task order only if\nthe agency and the contractor agree on a fair and reasonable cost for the service provided. During\nperformance of the task order, the SE-2020 team and the program office customer closely\nmonitor the costs incurred for the task order and use the task order cost baseline to identify any\ncost overruns for that work.\nThe FAA is effectively baselining, monitoring, and controlling costs for every SE-2020 task\norder. Baselining and monitoring costs at the task order level is a rigorous approach that allows\nthe FAA to control costs with a greater degree of precision than could be achieved by solely\nrelying on the SE-2020 program-level cost baseline which was the focus of OIG concern in its\ndraft report. As a result, the OIG draft report paints only a partial picture of cost baseline\nactivities on the SE-2020 program.\n\nThere is No Risk FAA will Overpay SE2020 Contractors $2 billion in the manner described\nin the draft report\nThe OIG report incorrectly interprets the estimated labor hours listed in Section B of each SE-\n2020 contract. The values of each SE-2020 contract simply represent ceiling amounts over the\nprojected ten-year program and do not guarantee that each contractor will receive task orders\ntotaling these ceilings. The use of a higher ceiling than might actually be needed is not unusual\nacross the Executive Branch as a means to assure that contracts with durations of five years or\nmore have sufficient scope to address unanticipated requirements. As many acquisition\nprofessionals have observed, the ceilings on the Federal Government\xe2\x80\x99s multiple award service\ncontracts are considerably larger than \xe2\x80\x9cany reasonable person\xe2\x80\x9d anticipates will be reached. 20\nAlready in the relatively short duration of the SE2020 program, there have been a number of\nunanticipated changes which might affect the number of hours the agency needs to acquire under\nthese contracts. For example, the Administration has recently selected NextGen as a Presidential\ninfrastructure priority, which when coupled with the variety of potential funding scenarios facing\nthe Federal Government, could increase some of the work needed under the systems engineering\ncontracts.\nThe SE-2020 contracts provide the FAA with the flexibility to order anywhere from zero labor\nhours to the maximum number of hours specified in the contracts. Specifically, Section B.5 of\nthe Contract states in part:\n \xe2\x80\x9cThe Contractor is not authorized to expend any hours or incur any costs\n until the Contractor is in receipt of a fully executed Task Order (TO) or\n other written direction from the Contracting Officer (CO). The FAA\n intends to purchase the level of effort required for which funding is\n\n\n20\n See, e.g., http://washingtontechnology.com/articles/2010/11/15/editors-note-contract-trends.aspx.\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 24\n\n\n appropriated and available. However, the FAA is not obligated to utilize\n this contract, or issue any TOs at any minimum level.\xe2\x80\x9d\nThe SE-2020 contracts clearly specify that labor hours can only be ordered or \xe2\x80\x9cawarded\xe2\x80\x9d by\nissuing a TO. The contracts themselves do not award or otherwise obligate the FAA to order any\nlabor hours. Moreover, the number of hours listed in Section B only represents an estimate of\nthe number of hours the FAA would require, and the contracts expressly recognize that changes\nin the FAA\xe2\x80\x99s requirements could cause the number of hours in Section B to increase or decrease.\nSpecifically, Section B.7(d) of the SE-2020 contracts provides:\n \xe2\x80\x9cThe level of effort delineated in Section B is provided as the\n Government\xe2\x80\x99s best estimate. Changes in programmatic requirements may\n cause an increase or decrease in the number of man-hours identified in\n Section B.\xe2\x80\x9d\nIn summary, the provisions of the SE-2020 contracts specify that: the number of labor hours\nlisted in Section B of the contracts is only an estimate; the contracts do not award or otherwise\nauthorize the contractor to expend any labor hours; and the FAA will only award labor hours by\nissuing task orders. Even if the FAA eventually orders all 58 million labor hours, this would not\nconstitute an overpayment or even a program overrun. Hours will only be ordered if the work is\nneeded. Payment to the SE2020 contractors will be made only for work performed for which\ncosts have been determined to be reasonable, allowable and allocable. As a result, we disagree\nwith the draft report conclusions relating to FAA \xe2\x80\x9cawarding\xe2\x80\x9d unneeded labor hours and do not\nsee how the existence of these ceilings creates a risk of overpaying the SE2020 contractors..\nFurthermore, as the FAA did not award 58 million labor hours, and the SE-2020 contracts do not\ncommit the FAA to ordering 58 million labor hours, the estimate of labor hours does not increase\nthe costs that the FAA will incur to procure services under the contracts. The FAA will only\nincur costs if it orders labor hours through task orders; and the estimated number of labor hours\nin Section B of each contract do not commit the FAA to order any hours beyond what the FAA\nrequires. The basis for the OIG\xe2\x80\x99s finding that the award of unneeded hours \xe2\x80\x9cincreased the cost\xe2\x80\x9d\nof the SE-2020 contracts is not clear. If the concern relates to the amount of fixed fee that the\ncontractors might earn, as the OIG\xe2\x80\x99s draft report recognizes, the contract permits the FAA to\nreduce the fixed fee to reflect the actual number of hours ordered. We further address this\nconcern in the next section.\n\nFAA Will Ensure that Contractor Fixed Fee Payments are Accurate\nThe OIG report identified issues regarding the potential need to recoup fixed fees paid to\ncontractors at the end of the contracts. The FAA agrees that it is important to ensure that\ncontractors are only paid the fixed fees they earn. The FAA currently tracks the total cost\nincurred and fixed fee ceilings at the contract, base period, and task order levels. This tracking\nhelps to ensure that the FAA will not exceed the approved program baseline (or individual task\norder baseline) and that each vendor earns no greater than the awarded fixed fee on each task\norder. To further ensure that the Government does not incur fee at a rate higher than the ceiling\namount of each task order, the FAA intends to reconcile total cost and fee at task order closeout.\nThe FAA will also reconcile the amount of the fixed fee paid at the end of the base period of the\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 25\n\n\ncontract and at the end of each option (if exercised) consistent with the contract provisions.\nThese actions limit the risk to the Government.\nIn addition, the FAA can easily offset any fee to be recouped against the contractors\xe2\x80\x99 invoices\nunder these or other contracts. The FAA continues to explore whether other appropriate\nmeasures can be put in place to reduce the likelihood the Government will need to recoup or\noffset portions of fixed fee payments at the end of the contracts.\n\nFAA Used Sound Practices to Select SE-2020 Contractors\nFAA conducted a rigorous competition in order to award the SE-2020 contracts. The record for\nthe SE-2020 acquisition demonstrates that the FAA utilized sophisticated and sound contracting\npractices to select the SE-2020 contractors. Indeed, the first award decision for the SIR2-FO SE-\n2020 competition was the subject of a comprehensive bid protest filed by a disappointed bidder.\nThat protest specifically challenged the soundness of the FAA\xe2\x80\x99s technical and cost evaluation for\nthis competition. Each of the protester\xe2\x80\x99s claims was independently reviewed, considered, and\nultimately rejected by the FAA\xe2\x80\x99s Office of Dispute Resolution (\xe2\x80\x9cODRA\xe2\x80\x9d), which issued a 105-\npage decision recommending that each of the protest grounds be denied on the merits. See FAA\nOrder No. ODRA-11-578, Protest of Apptis, Inc., Under Solicitation No. DTFAWA-09-R-\nSE2020-SIR2FO (Mar. 31, 2011) adopting and incorporating by reference ODRA Findings and\nRecommendations in Protest of Apptis, Inc., 10-ODRA-00535 (Mar. 25, 2011).\nWhile the OIG has identified a concern with the method the FAA used to evaluate past\nperformance, this evaluation factor played only a limited role among multiple evaluation factors\nused to select the SE-2020 contractors. For example, offerors participating in the second Source\nInformation Request (SIR 2) Full and Open SE-2020 competition were required to submit the\nfollowing 12 proposal submissions with the following weights:\n\n PROPOSAL VOLUME EVALUATION STRUCTURE\nVolume I - Core Capabilities 20%\nVolume II - Past Performance 15%\nVolume III - Technical Capability 25%\nTechnical Scenario #1 Data Communications\nVolume III - Technical Capability 15%\nTechnical Scenario #2 Investment Analysis and Business\nCase Analysis\nVolume IV - Task Order #1 (Program Management) 5%\nVolume IV - Task Order #2 (NAS Enterprise Architecture) 5%\nVolume IV - Task Order #3 (Investment Planning and 5%\nAnalysis)\nVolume V - Management Approach 10%\nVolume VI - Organizational Conflict of Interest Acceptable/Unacceptable/Risk\nVolume VII - Cost/Price Considered as part of best value\n trade off\nVolume VIII - Mentor Prot\xc3\xa9g\xc3\xa9 Program More Favorable\nVolume IX - Small Business Subcontracting Plan Acceptable/Unacceptable/Risk\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 26\n\n\nThe FAA conducted extensive and rigorous evaluations of the offerors\xe2\x80\x99 technical capabilities.\nFor example, for the SIR2 competition the FAA assembled the agency\xe2\x80\x99s leading subject matter\nexperts to evaluate a variety of technical discriminators including:\n\n \xe2\x80\xa2 the offerors\xe2\x80\x99 experience in nearly 30 core areas of technical capabilities (Volume I);\n \xe2\x80\xa2 the quality of the offerors\xe2\x80\x99 responses to two highly complex technical scenarios (Volume\n III);\n \xe2\x80\xa2 the strength of the offeror\xe2\x80\x99s proposed approach for completing three hypothetical task\n orders (Volume IV); and\n \xe2\x80\xa2 the strength of the offeror\xe2\x80\x99s proposed management approach (Volume V);\n\nWith respect to cost proposals, every SIR2 offeror and subcontractor was required to submit cost\nproposal workbooks and narrative descriptions that substantiated the different cost elements in\nthe offeror Workbook submission. In total, the Excel-based portion of the cost proposals\ncomprised approximately 260 Excel workbooks (totaling approximately 1,050 worksheets) and\nincluded a vast number of proposed cost elements. For instance, offerors submitted a total of\napproximately 4,550 direct labor rates in the cost workbooks. In addition to the Excel-based\nsubmissions, approximately 800 Word documents were also submitted as part of the cost\nproposals.\nAs documented in the Cost Evaluation report generated by the Cost Evaluation Team (\xe2\x80\x9cCET\xe2\x80\x9d),\nthe CET analyzed cost submissions by:\n\n \xef\x83\x98 Reviewing in detail the cost narratives for each proposal in order to identify any\n discrepancies between the cost approach described in the narratives and the cost\n elements submitted in the corresponding cost workbooks;\n\n \xef\x83\x98 Conducting a detailed review and analysis of the approximately 4,550 labor category\n position rates submitted in the cost workbook submissions in order to identify\n rates that required further examination and possible clarification. This included a\n detailed comparison to the direct labor rates for the corresponding labor\n categories in the IGCE followed by an additional review of all the direct labor\n rate submissions to identify any patently unreasonable or unrealistic rates.\n\n \xef\x83\x98 Conducting a series of clarification exchanges with the offerors in order to verify and\n substantiate the rates submitted. For instance, in one round of clarifications\n offerors were required to re-confirm the accuracy of the direct labor rates\n submitted. In another round of clarifications, the CET conducted a targeted\n sampling of offerors\xe2\x80\x99 and subcontractors\xe2\x80\x99 direct labor rates and required them to\n provide additional substantiation for approximately 245 direct labor rates.\n\nIn sum, while the OIG believes the FAA used overly narrow criteria to evaluate past\nperformance, the draft report did not disagree with the criteria reflecting 85% of the evaluation.\nIn addition, the one example cited in the draft report to support the opinion that the criteria was\noverly narrow (feedback that a contractor took an overly long time to fill vacancies), appears\ninstead to support the rationale that the team used for focusing the past performance criteria on\nthe contractor\xe2\x80\x99s ability to hire and retain qualified workers. An examination of all of the\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 27\n\n\nresponses the FAA received to evaluate past performance supports the agency\xe2\x80\x99s methodology in\nthis area.\nThe record for this procurement, when taken as a whole, demonstrates that the FAA used sound\ncontracting practices to select the awardees. The limited issue identified with respect to the past\nperformance evaluation does not provide a basis for the broad conclusion in the draft report, as\nenumerated in the section titles, that the FAA\xe2\x80\x99s overall contracting practices are either unsound,\nor do not produce best value for the taxpayer.\nThe FAA agrees that past performance is an important factor to be used not only in awarding\ncontracts, but in awarding task orders. As the draft report acknowledges, when one of the\ncontractors submitted untimely staffing plans and insufficiently qualified employees, the FAA\nstopped placing tasks with that contractor until performance improved.\n\nFAA is Committed to Appropriate Use of Performance Based Acquisition\nThe OIG draft report includes statements indicating that FAA does not believe the Performance\nBased Acquisition (PBA) techniques should be used on SE-2020. This is not the case. On May\n16, 2011, the FAA provided comments to the OIG stating that PBA may not be appropriate for\ncertain of the research and development work under the SIR1 contract portfolio. Specifically,\nthe FAA comments stated:\n \xe2\x80\x9cPBA may not be appropriate for certain services to be procured under the SE-\n 2020 contracts. Indeed, the Office of Procurement Policy has recognized that\n certain \xe2\x80\x9ctypes of services may not lend themselves to outcome-oriented\n requirements\xe2\x80\x9d and, for this reason, excluded Research and Development services\n contracts from reporting requirements relating to agencies\xe2\x80\x99 use of Performance\n Based Service Acquisition methods. This is particularly pertinent to this\n acquisition given that the services to be provided under the SIR1 contracts are\n research and development services.\nHowever, the FAA went on to state:\n \xe2\x80\x9cThe FAA concurs that PBA should be used when appropriate and remains\n committed to implementing best practices in the administration of the SE-2020\n contracts. To that end, the PMO is in the process of engaging a subject matter\n expert in PBA methods to assist the FAA in evaluating where PBA could be used\n appropriately in SE-2020 tasking, and devise guidelines to assist the Agency in\n identifying SE-2020 tasks where PBA should be employed.\xe2\x80\x9d\nAs reflected in the draft report, approximately 86% of the task orders examined by the\nOIG contained deliverables as required by the FAA\xe2\x80\x99s Acquisition Management System.\nThis does not excuse the remainder by any means, but it does mean that the evidence\ndrawn from a sampling of tasks under one program\xe2\x80\x99s contracts is inadequate for the\n\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 28\n\n\nbreadth of the draft report\xe2\x80\x99s conclusion that the FAA is not using sound contracting\npractices to award and oversee its contracts. 21\n\nFAA Avoided Organizational Conflict of Interest on SE-2020 Regarding ADS-B\nWhile an Organizational Conflict of Interest (OCI) can occur when an agency hires a contractor\nto evaluate its own work, FAA does not agree this has occurred or would ever likely occur in the\ninstance of one of the SE-2020 prime contractors in regard to the ADS-B system. The SE-2020\ncontracts are divided into two portfolios, each of which corresponds to different portions of the\nAMS acquisition lifecycle. The SIR1 portfolio covers certain research and development work\nearly in the acquisition lifecycle. To broadly generalize, the SIR1 contracts are used to help the\nFAA research potential solutions, technologies or concepts early in the acquisition lifecycle. The\nSIR 2 portfolio covers certain systems engineering and investment analysis work occurring later\nin the acquisition lifecycle. Further generalizing, SIR2 work is used to evaluate the various\npossible solutions, technologies or concepts previously researched during the R&D phase of the\nlifecycle, and help the agency select the best option.\nThe contractor in question is a SIR1 contractor and as such may be called on by the FAA to\nconduct certain research and development activities. This work will not place the contractor in\nthe position of evaluating its own work under the ADS-B program. Indeed, when structuring the\nSE-2020 acquisition, the FAA intentionally split the SE-2020 work into two portfolios, and\nsegregated the research and development work into the SIR1 portfolio. The agency did this, in\npart, to minimize OCIs and allow full scale development vendors to support the FAA with\nresearch work early in the acquisition lifecycle without generating OCIs that would impair their\nability to participate in full scale development later in the acquisition lifecycle.\nMoreover, even if an impaired objectivity OCI 22 could arise due to the contractor\xe2\x80\x99s work on both\nefforts, an OCI would not arise unless the contractor was actually issued a task under SE-2020\nand that work generated an OCI. In a situation where a task order statement of work indicates\nthe possibility of such a risk, the FAA could elect to award the task order to one of the other\nthree SIR contractors in order to avoid the OCI, one of the major reasons for the award of\nmultiple contracts.\nThe FAA is well-aware that OCIs can arise for the systems engineering contractors in a variety\nof situations given the dynamic nature of contractors\xe2\x80\x99 work on other government contracts.\nHowever, OCIs must be evaluated on a case-by-case basis after considering the particular facts\ninvolved, including a detailed analysis of the scope of work that the contractor is being asked to\nperform.. See Axiom Resource Management, Inc. v. United States, 564 F.3d 1374, 1383 (Fed.\nCir. 2009) (\xe2\x80\x9c[T]he identification of OCIs and the evaluation of mitigation proposals are fact-\n\n21\n We also note that the draft report implies that the FAA is at fault for not following \xe2\x80\x9cgovernment-wide guidance\xe2\x80\x9d to compete\n task orders over $1 million. For other agencies, this \xe2\x80\x9cguidance\xe2\x80\x9d is a legal requirement found in statutes and regulations the\n FAA is prohibited from following. The FAA does generally compete task orders under its multiple award support service\n contracts, and as addressed elsewhere in this response, is developing guidance for competition of task orders under SE2020.\n Congress expressly prohibits the FAA, however, from using the statutes in question in order for the FAA to find an appropriate\n balance among policy factors that will best match the FAA\xe2\x80\x99s unique needs.\n22\n The draft report uses two of the three types of organizational conflicts of interest interchangeably, although in policy, law and\n practice, each type is quite distinct. The confusion of terms makes this portion of the report difficult to understand, particularly\n given the two distinct statements of work involved in SE2020.\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 29\n\n\nspecific inquiries that require the exercise of considerable discretion. The exercise of common\nsense, good judgment, and sound discretion is required in both the decision on whether a\nsignificant potential conflict exists and, if it does, the development of an appropriate means for\nresolving it.")\nThe FAA has established a system to identify OCIs that could be generated by a given task order,\nand addressing those OCIs before the task order is awarded. The FAA conducts OCI evaluations\nprior to the award of each task order. This case-by-case approach is consistent not only with\nAMS Policy \xc2\xa7 3.1.7, and the ODRA\xe2\x80\x99s precedent in Washington Consulting Group, 97-ODRA-\n00059, but with the policy and law followed by the entire Federal Government, which permit a\ncase-by-case approach to impaired objectivity OCIs in the context of an IDIQ contract. See\nAxiom Res. Mgmt., Inc. v. United States, 564 F.3d 1374, 1381 (Fed. Cir. 2009); Protest of Apptis,\nInc., 10-ODRA-00535 at 99 (Mar. 25, 2010).\nThe FAA intentionally awarded multiple contracts under the SIR1 and SIR2 contract portfolios,\nin part, so that the agency could avoid issuing task orders to a contractor if that task order would\ngenerate an OCI (having access to multiple contracts allows the FAA to issue the task order to\nan unconflicted vendor in the portfolio). Section G.7 of the SE-2020 contracts reflects this\napproach and provides that the FAA will consider OCIs as part of its task order award decision.\nThe draft report neither provides any details on how the scope of work under the SE-2020\ncontract in question places the referenced contractor in a position where its impartiality could be\ncompromised, nor does it provide any information on the fact-specific inquiries conducted by the\nOIG to determine whether the contractor\xe2\x80\x99s impartiality could be impaired by its performance on\nFAA\xe2\x80\x99s Automatic Dependent Surveillance-Broadcast (ADS-B) system. If the OIG found that an\nOCI could hypothetically arise if the FAA were to assign certain work to the contractor under its\nSE-2020 contract, this finding should take into consideration that the FAA has placed substantial\nsafeguards in place, including requiring technical review by a designated member of the PMO\nfor OCIs and contractor disclosure of potential OCIs. These safeguards are in addition to review\nby the contracting officer and would allow the FAA to identify the potential OCI before a task\norder is ever issued and mitigate or avoid the OCI.\n\nSE-2020 Contract Officers/Specialists and COTRs Are Required to Undergo Extensive\nTraining\nThe draft report states that FAA did not require its SE-2020 \xe2\x80\x9ccontract oversight staff\xe2\x80\x9d to receive\ntraining. The FAA does not agree with this finding.\nThe SE-2020 contracts are overseen by a mix of contracting oversight staff that includes:\ncontracting officers/specialists, COTRs, and Technical Officer Representatives (\xe2\x80\x9cTORs\xe2\x80\x9d). The\nFAA requires contracting officers/specialists and COTRs to receive extensive contract oversight\ntraining. Of the assigned contracting staff, three team members are Level III certified\nContracting Officers, two are Level II certified and two are Level I certified. The remaining\nteam member is contractor support staff.\nTORs are assigned by program office customers and assist contracting officers and COTRs with\nthe oversight of the customer\xe2\x80\x99s SE-2020 task order. The discussion draft report noted that the\nFAA did not require TORs to receive contract oversight training, but made no findings with\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 30\n\n\nrespect the other contract oversight staff (contracting/officers and COTRs). We understand that\nthe findings in the report regarding training of contract oversight staff are limited to TORs and\ndo not apply to contracting officers/specialists or COTRs. Accordingly, the OIG\xe2\x80\x99s findings do\nnot support the broad statement in the draft report that the FAA did not require any of its SE-\n2020 \xe2\x80\x9ccontract oversight staff\xe2\x80\x9d to receive training.\nIn addition, it is important to note that the OIG found that most TORs had in fact received\ncontract oversight training. Indeed, the OIG\xe2\x80\x99s discussion draft report noted that, while the\nFAA did not require TORs to receive contract oversight training, 88% of TORs that\nresponded to the OIG\xe2\x80\x99s survey had in fact had received COTR training.\n\nOIG Recommendations and FAA Responses\n\nOIG Recommendation 1: Require the SE-2020 Program Office to (a) submit to the CFO a\nwritten reconciliation of the difference between its IGCE and the contractors\xe2\x80\x99 proposals, and (b)\nuse this reconciliation as a basis to develop a reasonable cost baseline.\n\nFAA Response: Concur. The SE-2020 PMO will provide the CFO with a written explanation of\nthe differences between the approved SE-2020 IGCE direct labor rates and the vendor cost\nproposals\xe2\x80\x99 direct labor rates by January 20, 2012. The CFO will determine whether the SE2020\nprogram baseline should be revised by February 29, 2012.\n\nOIG Recommendation 2: Revise AMS to require that when IGCEs exceed contractor\nproposals by 15 percent or more, program officials should submit an explanation and\nrecommended corrective actions to the CFO before contract award.\n\nFAA Response: Non-Concur. AMS Guidance T3.2.1.4 states that if the contractor proposal\nexceeds the IGCE by greater than 15%, a revised estimate and an explanation for the changes\nmust be submitted to the CFO for approval. It further states that if the contractor proposal is\n15% below the IGCE, the program office must submit to the CFO an explanation of why the\ninitial estimate was overstated, within 30 days of award.\n\nWe do not concur, particularly in light of the state of the economy during this period, that\nreceiving prices below the IGCE requires taking corrective action. Proposed costs or prices\nbelow the Government estimate could improve the business case for the procurement; however\nwe need to ensure that lower proposal costs do not increase the FAA\xe2\x80\x99s performance or budgeted\ncost risk. The estimates need to be looked at on a case by case basis and assessed based on the\nindividual circumstances of the procurement. A discussion was held with the CFO office and the\nFAA Acquisition Policy office on January 3, 2012 to review the AMS Guidance referenced\nabove. All offices agree that no change to the AMS is required. In the case where an award\nwould result that is 15% above the approved baseline, the CFO must approve the deviation prior\nto contract award. In the converse, where the proposed award is 15% below the CFO approved\namount, the FAA believes there is no added value to seeking approval prior to award. The AMS\nprovision which requires notification and explanation within 30 days after award is sufficient.\n\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 31\n\n\nOIG Recommendation 3: Revise AMS to (a) specifically require pre-award \xe2\x80\x9cand\xe2\x80\x9d post-award\naudits of contracts in excess of $100 million; and (b) define the types of pre-award audits\nrequired, including, at a minimum, direct labor rates, indirect rates, and accounting system\nreviews.\n\nFAA Response: Partially concur. The AMS provisions for determining whether contractors\xe2\x80\x99\nproposed prices are fair and reasonable are sufficiently defined. AMS policy 3.2.3.3 states \xe2\x80\x9cThe\nCO must request audits on all cost reimbursement contracts exceeding $100 million.\xe2\x80\x9d AMS\nguidance also includes the following:\n\n\xe2\x80\x9cField Pricing Support - The CO may request field pricing support when such support is deemed\nnecessary before negotiating any contracts or modifications. Methods of field pricing support\nmay include the following: (a) rate verifications; (b) third party audits; (c) estimating system\naudits; and (d) proposal analysis.\xe2\x80\x9d\n\nIn addition, the FAA Pricing Handbook, located within the Procurement Toolbox, is an extensive\nreference source for conducting an analysis of contractor\xe2\x80\x99s proposed prices/costs. The tools and\nsources described above are available to the CO when evaluating proposals. The CO is required\nto use good business judgment consistent with AMS when determining price reasonableness. If\nan audit is not obtained, the CO must have a rational basis and must document the decision for\nnot requiring an audit, as well as a sufficient alternative means for assuring that the prices or\ncosts proposed are fair and reasonable.\n\nWe concur that one sentence in the AMS, \xe2\x80\x9cthe CO must request pre-award or post-award\nDefense Contract Audit Agency (DCAA) audits as appropriate on all cost\nreimbursement contracts estimated to exceed $100 million.\xe2\x80\x9d should be clarified to reflect that\npre-award and post-award audits should be obtained as appropriate.\n\nTo date we have received some assistance from DCAA in conducting audits. Some examples\ninclude:\n\n 1) General Dynamics - In June 2010, DCAA assisted with a billing audit under the General\n Dynamics contract. DCAA reviewed five invoices under three task orders and all rates\n were determined acceptable.\n 2) Boeing - DCAA provided a July 2011 disclosure statement review of Boeing Aerospace\n Operations. The report further noted that an accounting system audit will be conducted\n in 2012.\n 3) CSSI - DCAA completed an accounting system review in 2010 to assess their system for\n accumulating and billing costs. The system was deemed adequate.\n\nWe also attempted, unsuccessfully, to obtain other DCAA assistance, but due to limited\nresources and the priority of its DOD work, DCAA is often unable to assist the agency. The\nFAA has contracted with a private auditing firm to provide audits when DCAA is unable to do\nso. The SE-2020 contracting team continues to work with the FAA\xe2\x80\x99s Acquisition Pricing Branch\nto obtain current invoice reviews on all prime contractors, to obtain reviews of each contractor\xe2\x80\x99s\nprovisional billing rates and to obtain cost incurred reviews. The SE-2020 team will continue to\nobtain audits in accordance with AMS prescriptions.\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 32\n\n\nThe AMS provision on pre-award and post-award audits will be clarified by March 31, 2012.\n\nOIG Recommendation 4: Amend SE-2020 awarded contract values using contractors\xe2\x80\x99\nproposed rates and FAA\xe2\x80\x99s estimated need for 40 million hours, and adjust fixed fees to reflect\nrevised contract costs.\n\nFAA Response: Non-Concur. The SE-2020 Program baseline, as approved by the CFO, is not\nto exceed $7.1B and 40 million hours. The individual vendor contracts, in aggregate, are\nstructured to ensure that the total cost of SE-2020 work will not exceed this amount. As noted in\nthe discussion portion of this response, the FAA has not \xe2\x80\x9cawarded\xe2\x80\x9d the referenced labor hours.\n\nUnilaterally amending each vendor\xe2\x80\x99s contract to downwardly adjust its contract value, estimated\nhours, and fixed fees when there has not been a change to the FAA\xe2\x80\x99s requirements reflecting less\nneed for this work than existed at the time the contracts were awarded, could expose the FAA to\ncontract claims and could entitle the contractors to additional compensation. This is distinct\nfrom the situation contemplated by the contract terms, under which if at the end of the contract it\nturns out the FAA did not order all of the hours on which the fee is based, the fee will be\nproportionately reduced.\n\nThe FAA currently tracks the total cost and fixed fee ceilings at the contract, base period, and\ntask order levels. This tracking helps ensure that the FAA will not exceed the approved program\nbaseline and that each vendor earns no greater than the awarded fixed fee on each task order. To\nfurther ensure that the Government does not pay fee in excess of the ceiling amount of each task\norder, the FAA will reconcile total cost and fee at task order closeout. By March 31, 2012 the\nFAA will complete an analysis of all work ordered and invoices paid through the end of calendar\nyear 2011. If we find that adjustments to the method that the FAA uses to pay out the fixed fee\nas the contract is performed is warranted, the FAA will address this subject with the SE2020\ncontractors to revise the method in which fee is paid. We will share the results of this anslysis\nwith the OIG by June 30, 2012.\n\nOIG Recommendation 5: Require the SE-2020 program office to (a) develop policies and\nprocedures to ensure timely reconciliations and corrections to acquisition databases, and (b)\nrevise its cost monitoring spreadsheets to ensure accurate data for effective cost control of SE-\n2020 contracts.\n\nFAA Response: Concur. The FAA will develop a verification procedure to ensure that\ninformation is properly populated in the Federal Procurement Database System (FPDS) and that\nthe data in PRISM, Delphi and FPDS are consistent and accurate. This process will include a\nquarterly reconciliation and correction process and will begin no later than January 31, 2012.\n\nBy February 29, 2012, the SE-2020 Program team will complete an audit of all cuff records to\nensure that all data is current, that spreadsheet formulas are correct, and that data is consistent\nbetween PRISM, Delphi, FPDS and the task order tracking cuff records. All inconsistencies will\nbe resolved by March 31, 2012.\n\nOIG Recommendation 6: Revise AMS to establish controls that require more comprehensive\nevaluations of contractor past performance.\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 33\n\n\nFAA Response: Partial-Concur. The AMS includes comprehensive guidance for evaluating\ncontractor past performance and allows the FAA to tailor its approach for evaluating past\nperformance to the particular circumstances of each procurement. The FAA disagrees that the\ntailoring of past performance for this one procurement could have been better worded and we\nbelieve this is not an adequate basis for the OIG\xe2\x80\x99s conclusion that the FAA does not use sound\ncontracting practices to select its contractors. The FAA concurs, however, that past performance\nevaluations should include, at a minimum, a thorough analysis of the quality of the contractors\xe2\x80\x99\npast performance. The FAA will amend the AMS by February 29, 2012 to add this requirement.\n\nOIG Recommendation 7: Require FAA\xe2\x80\x99s contracting and program staff to use performance-\nbased acquisition principles in their SE-2020 task orders, and ensure staff is adequately trained to\ndevelop and monitor such awards.\n\nFAA Response: Concur. The AMS stipulates that performance-based contracting is the\npreferred method for describing work-in-service contracts and should be used when appropriate.\nWe concur that the FAA must ensure that PBA principles are used on SE-2020 task orders where\nappropriate. We also concur that SE-2020 contracting and PMO staff should be adequately\ntrained to develop, apply and monitor these types of awards.\n\nThe FAA has begun analyzing new SE-2020 task order request packages to identify task orders\nthat are appropriate for performance based contracting principles. The FAA plans to implement\nperformance based contracting on new task orders that are suitable for PBA. This will include\nusing a Quality Assurance Surveillance Plan (QASP) and will include standard templates to\nassess technical compliance of deliverables, quality of services, cost control and other criteria.\nResults of this surveillance will be used as past performance inputs for new task order award\ndecisions.\n\nThe FAA has also begun to train SE-2020 staff to ensure that they are adequately trained to\ndevelop, apply and monitor these types of awards. Two of the SE-2020 contracting officers\nreceived training on performance based contracting within the past year, and another contract\nspecialist is scheduled for training during the first quarter of FY12. The FAA also plans to train\n17 of the SE-2020 PMO staff in two phases: ten PMO team members will complete training in\nFY12; and an additional seven PMO team members will complete training in FY13.\n\nIn the interim, while awaiting formal classroom training of SE-2020 team members, the PMO\nwill conduct a mandatory training session by January 31, 2012 to provide awareness and an\noverview of performance based contracting.\n\nOIG Recommendation 8: Require the SE-2020 program office to define criteria that specifies\nwhen competing task orders are in the government\xe2\x80\x99s best interests.\n\nFAA Response: Concur. The SE-2020 PMO is currently defining criteria for specifying task\norder competition. These criteria will be used to identify requirements that lend themselves to\ntask order competition and will be finalized by January 31, 2012.\n\nOIG Recommendation 9: Include guidance in AMS on how to identify and mitigate risks of\npotential OCIs prior to contract or task order award.\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 34\n\n\nFAA Response: Partially Concur. The FAA believes that the AMS, as currently published,\ncontains appropriate guidance on how to identify and mitigate the risks of potential OCIs.\nSpecifically, the AMS: outlines how to identify and mitigate an OCI; describes scenarios that\nmay lead to an OCI; requires offerors to disclose potential OCIs; and provides instruction on\nappropriate remedies should an offeror or contractor refuse to disclose or misrepresent any\ninformation about a potential OCI. In addition, the AMS instructs Contracting Officers to obtain\nassistance from legal counsel and appropriate technical specialists when evaluating potential\nOCIs.\n\nThe FAA will review the practices developed by the SE2020 team to avoid and mitigate OCIs,\nincluding the database that will be created and used by the program, to determine best practices\nthat should be incorporated into the AMS. The FAA will conduct this review one year after\nimplementation of the SE2020 database (expected to be in place June 2012 so the review is\nexpected to be done June 2013). Best practices identified as a result of this review will be\nincorporated into the AMS three months after completion of this review, or by September 30,\n2013.\n\nOIG Recommendation 10: Require FAA contracting personnel to develop, maintain, and use a\nrecord of active prime contractors and subcontractors to identify and mitigate risks of potential\nOCIs.\n\nFAA Response: Partially Concur. It is neither practical nor desirable for FAA contracting\npersonnel to maintain a database of all active prime contractors and subcontractors performing\nwork for the FAA. The agency employs such a large, and ever-changing, number of prime\ncontractors and subcontractors that the compilation and maintenance of an accurate database\nwould neither be practical, nor cost-effective. Although FPDS captures basic award information,\nsuch as contractor names, location, general product/service line of business, and a short\ndescription of work, it does not include a detailed description of tasks to be performed by the\nprime contractor nor information about subcontractors. To be useful for OCI purposes, the FAA\nwould need to create and maintain a separate database of all contracts and subcontracts, to\ninclude detailed, task-level descriptions of work. Building and maintaining a new database\nwould involve a significant investment in time and scarce funding, and this investment would not\nbe justified by the benefits\xe2\x80\x94the FAA\xe2\x80\x99s current processes are already efficient at detecting OCIs,\nand the OIG\xe2\x80\x99s audit does not provide any basis to conclude that the FAA is failing to detect or\nmitigate actual OCIs, or that an OCI database would materially increase the number of OCIs\nlikely to be detected by the FAA..\n\nThe AMS requires that each contractor performing work for the FAA disclose any potential\nOCIs to the Government before accepting work. Thus, each contractor is required to monitor,\nidentify and disclose potential OCIs and this reduces the risk of OCIs. The AMS also requires\nContracting Officers to analyze planned acquisitions in order to identify OCIs and avoid,\nneutralize, or mitigate potential conflicts before contract award. On task order contracts, such as\nthe SE-2020 contracts, the AMS allows Contracting Officers to address OCIs prior to the award\nof each task order. This case-by-case approach is consistent with AMS Policy \xc2\xa7 3.1.7, and legal\nprecedent. As observed by the Government Accountability Office, contracting officers may\nproperly defer evaluating OCIs until the facts are ripe for review. See Axiom Resource\nManagement, Inc. v. United States, 564 F.3d 1374 (Fed. Cir. 2009) citing Axiom, No. B-\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 35\n\n\n298870.3, 2007 U.S. Comp. Gen. LEXIS 123, 2007 WL 2141694, at *5 (holding that OCIs that\ncould arise if a contractor is awarded certain other work could properly be analyzed as part of\nthose subsequent award decisions).\n\nPrior to awarding a task order under the SE-2020 contracts, the FAA analyzes the task order to\nidentify potential OCIs, and the contractor conducts its own review to identify potential OCIs. In\norder to further reduce the risk of OCIs, the FAA has implemented additional review procedures\nto oversee all of the SE-2020 contractors\xe2\x80\x99 work and detect OCIs. These additional review\nprocedures include the incorporation of the OCI/COI Evaluation section as part of the\nAdjudication Board meeting and record, the addition of the OCI statement in the vendor\xe2\x80\x99s\n\xe2\x80\x9ctechnical task plan\xe2\x80\x9d response, and enhancement of the \xe2\x80\x9cresume review module\xe2\x80\x9d to include a\nrequired COI section as part of the resume submission. In addition, the FAA has designated a\ndedicated point of contact within the PMO responsible for assisting the CO to identify and assess\npotential OCI issues.\n\nThe PMO is also in the process of developing an active prime/subcontractor contract database for\nSE-2020 task orders awarded. This database will allow us to track all potential OCIs that have\nbeen disclosed and/or mitigated under the SE-2020 contracts for ongoing monitoring and\nmitigation. The database is targeted for completion by June 2012. The PMO will use the\nexperience gained with this database to aid in developing a methodology to assist the FAA\xe2\x80\x99s\ncontracting personnel in identifying potential undisclosed OCIs.\n\nOIG Recommendation 11: Require the SE-2020 Program Office to develop policies and\nprocedures to ensure adequate documentation of task order award decisions.\n\nFAA Response: Concur. In February, 2011, the PMO applied the OIG\xe2\x80\x99s recommendations and\nmade updates to the existing Adjudication Board record to incorporate the feedback. Since that\ntime, there have been three additional revisions as a result of further analyses and a desire to\ncontinuously improve the evaluation process and decision tracking. Another review of the\nAdjudication Board record is scheduled for January 2012 and the FAA plans to conduct reviews\non a bi-annual basis thereafter to ensure institutionalization of the adjudication process and a\nthorough documentation and evaluation process. Thus, the FAA requests that this\nrecommendation be closed.\n\nOIG Recommendation 12: Require the SE-2020 Program Office to amend the standard\nperformance monitoring templates to include measurable criteria to evaluate desired performance\noutcomes, such as quality, cost, and schedule.\n\nFAA Response: Concur. The FAA took action in March 2011 to require that contract oversight\nstaff complete monthly standard performance monitoring reports for each task order. This was\npart of an effort by the FAA to implement a formal metrics tracking system to track and evaluate\ntask order performance.\n\nThe FAA will ensure that appropriate updates are made to SE-2020 task order performance\nmonitoring reports reflecting the use of measurable performance outcomes. To this end, the SE-\n2020 PMO is in the process of reviewing the standard performance monitoring template to\n\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c 36\n\n\nidentify necessary and desirable revisions and include performance measurement criteria such as\nquality, cost, and schedule that may provide a more quantifiable measurement of the\nperformance assessment. The review will be completed by April 1, 2012. The FAA also plans\nto use Quality Assurance Surveillance Plans to monitor performance on task orders that\nincorporate performance based acquisition principles.\n\n ********\n\nThe FAA remains committed to working with the OIG to address any remaining questions or\nconcerns that it may have regarding the SE-2020 contracts. Should you have any questions\nregarding the foregoing, please feel free to contact me at 202-267-9440.\n\n\n\n\nAp p en d ix . Ag e n c y C o mm en t s\n\x0c'