Special Inspector General for 

Afghanistan Reconstruction 

SIGAR 17-57 Audit Report

Afghanistan National Defense and Security 

Forces: DOD Spent $457.7 Million on 

Intelligence Capacity-Building Programs, but 

Impact Cannot Be Fully Assessed Because of a 

Lack of Performance Metrics 

SIGAR 17-57-AR/Legacy and ASOM Programs 

SIGAR 

JULY 

2017 
For more information, contact SIGAR Public Affairs at (703) 545-5974 or sigar.pentagon.ccr.mbx.public-affairs@mail.mil. 

WHAT SIGAR REVIEWED 

From 2010 to 2013, the Department of Defense 

(DOD), through the Army Contracting Command 

(ACC), awarded one task order and four 

contracts (referred to in this report as 

“contracts”), valued at $536.1 million, to 

develop the Afghan National Security and 

Defense Forces’ (ANDSF) intelligence 

capabilities. ACC awarded the initial four 

research and development (R&D) contracts, 

valued at $332.8 million, to Jorge Scientific 

Corporation—rebranded as Imperatis 

Corporation in 2013—which subcontracted with 

New Century Consulting Limited (NCC). The R&D 

task order and contracts—Legacy Afghanistan, 

Legacy Kabul, Legacy South, and Legacy East—

became known as the Legacy Afghanistan R&D 

program. In 2013, ACC awarded a $203.3 

million contract to NCC to implement the 

Afghanistan Source Operations Management 

(ASOM) training and mentoring program. By the 

end of the programs, the Legacy and ASOM task 

order and contracts cost $457.7 million. 

ACC appointed the Army Research Laboratory 

(ARL) as the contracting officer’s representative 

and the Combatting Terrorism Technical Support 

Office (CTTSO) as the contracting officer’s 

technical representative for Legacy and ASOM. 

ACC also delegated administrative contracting 

officer responsibility to the Defense Contract 

Management Agency to review and approve 

contractor invoices, which it did with assistance 

from the Defense Contract Audit Agency (DCAA). 

The objectives of this audit were to determine 

the extent to which: (1) Imperatis and NCC 

successfully performed the tasks required by 

the contracts and developed the ANDSF 

intelligence capability; (2) ACC awarded the 

Legacy and ASOM contracts in accordance with 

federal and DOD regulations; and (3) ARL and 

CTTSO properly monitored contract performance 

and the Defense Contract Management Agency 

monitored contract costs. 

WHAT SIGAR FOUND 

DOD awarded the Legacy and ASOM contracts to train and mentor Afghan Ministry 

of Defense (MOD) and Ministry of Interior (MOI) intelligence officers, with the goal 

of improving their intelligence capability. However, the Legacy contracts did not 

have performance metrics to track progress toward that goal. This was because 

DOD awarded the Legacy contracts as R&D contracts, which have fewer oversight 

requirements than standard services contracts. 

Later, with the modification of the Legacy East contract and the award of the 

ASOM services contract, DOD did include some performance metrics to measure 

the progress of the programs; however, SIGAR found that even with this added 

requirement and the regular contractor reports, clear metrics were not available to 

assess the overall success of each contract. Under Legacy East and ASOM, the 

contractor was required to provide continued training and mentoring to Afghan 

intelligence officers at various sites throughout Afghanistan. Under ASOM, contract 

success was measured, in part, by the capabilities and independence of 

intelligence training sites where the contractor had conducted training and 

mentoring to the Afghan government. However, it was left up to the contractor to 

measure each intelligence training site’s readiness to transition to the Afghan 

government. 

In addition, CTTSO’s attempts to assess contractor performance relied, in part, on 

contractor-provided data, such as NCC’s self-assessments created by its mentors 

and compliance officers. SIGAR reviewed the self-assessments and found that 

NCC deemed the Legacy and ASOM programs a success, but provided few 

specifics to support its claim. In addition to the contractor-provided data, CTTSO 

also relied on contractually required status meetings, monthly contracting officer’s 

representative assessments based on checklists (with no narrative), the up-todate status of contractor invoices, oversight from one military officer in each 

regional command, which CTTSO noted, “some were more effective than others,” 

and reports from RAND and ManTech to assess program performance. RAND and 

ManTech compiled their reports using information obtained from site visits, 

interviews, and their analysis of Legacy and ASOM data. However, SIGAR reviewed 

the reports and metrics CTTSO, RAND, and NCC provided and found them to be 

subjective. Because of a lack of performance metrics for the Legacy R&D program 

and a reliance on contractor-provided data for the ASOM program, it is almost 

impossible to gauge the government’s return on investment for the $457.7 million 

spent. 

DOD hired the RAND National Defense Research Institute (RAND) to evaluate the 

Legacy and ASOM programs, and RAND’s evaluations showed mixed results. In 

2012, RAND noted that the program’s model warranted some basis for cautious 

optimism. However, in 2014, RAND stated that the Legacy R&D program was not 

properly planned prior to its implementation. In 2015, in its final Legacy 

assessment, RAND reported mixed results for the program’s overall success and 

expressed concerns related to program implementation and assessment. 

However, the report also noted that the complex nature of the program limited 

RAND’s ability to make a definitive assessment of the program’s success. 

SIGAR 

Special Inspector General for 

Afghanistan Reconstruction 

July 2017 

Afghanistan National Defense and Security Forces: DOD Spent $457.7 

Million on Intelligence Capacity-Building Programs, but Impact Cannot 

Be Fully Assessed Because of a Lack of Performance Metrics 

SIGAR 17-57 AUDIT REPORT 
For more information, contact SIGAR Public Affairs at (703) 545-5974 or sigar.pentagon.ccr.mbx.public-affairs@mail.mil. 

Due to the lack of performance metrics for the Legacy program, SIGAR reviewed the extent to which existing records demonstrated

that individual ANDSF officers completed the required number of courses to be considered intelligence instructors or trainers, as a 

possible indicator of the Legacy program’s success. Although this review showed that 70 of 71 Afghan National Army instructors 

completed all four mandatory courses required to be an instructor, SIGAR found that significant portions of other groups of ANDSF 

intelligence trainers and instructors failed to meet the minimally established training requirements. Specifically, 

 Ten of 24 police intelligence student trainers completed all nine courses required to be a Ministry of the Interior trainer, 

while the 14 remaining trainers completed between one and eight of the required courses. 

 None of the four Afghan National Army student trainers completed all six courses required to be a Ministry of Defense 

trainer. Although all the trainers completed at least three of the required courses, no one completed more than four of the 

courses. 

 Five of six Afghan National Army student instructors did not complete any of the four courses required to be a Ministry of 

Defense instructor, and one student instructor completed only one required course. 

Starting with the modified Legacy East and ASOM contracts, the successful transition of intelligence training sites to the Afghan 

government became the primary way that NCC measured the program’s success. Under ASOM, the capability and independence of 

each intelligence training site were judged by NCC using “aspect ratio scores” on a scale of 0 to 5. A score above 4.0 meant the 

site was ready for transitioning to the Afghan government; a score between 3.5 and 4.0 meant the site could be transitioned, but 

the capability carried a risk of being reversible. A score below 3.5 meant the site was not ready to be transitioned. In August 2013, 

NCC reported on 103 out of 111 MOD and MOI intelligence sites and found that 47 sites had aspect ratio scores above 3.5; 46 

sites had aspect ratio scores below 3.5, but were turned over with handover plans to address deficiencies; and 10 sites were 

assessed as not capable of self-sustainment. When DOD directed the mentoring teams to vacate these 10 sites due to troop 

withdrawals, they were left with an undeveloped intelligence capability. However, NCC stopped mentoring under ASOM at all 111 

intelligence sites and, in August 2014, reported that 58 sites were successfully transferred to the ANDSF, 42 sites were 

transferred with capability handover plans, and the other 11 sites were not transitioned. In addition, NCC reported that 13 of the 

14 General Command of Police Special and Investigative Surveillance sites were not able to transition with handover plans by the 

time the ASOM program ended in February 2016. As a result, a total of 59 of the 125 (47 percent) of Legacy and ASOM 

intelligence training and mentoring sites were fully transitioned to the Afghan government, with the remaining being transitioned 

with varying levels of capability or without capability. 

Although the aspect ratio scores indicate some success in transitioning MOD and MOI intelligence sites, DOD and the Afghan 

government continue to report that there is a need for coalition assistance and the ANSDF is only partial capable with respect to its 

intelligence operations. For example, in its semiannual reports to Congress from 2011 to 2014 on Progress toward Security and 

Stability in Afghanistan, produced while Legacy and ASOM were under way, DOD found that the MOD and MOI intelligence units 

required coalition assistance to accomplish their missions. Similarly, in SIGAR’s January 2016 Quarterly Report to Congress, SIGAR 

stated that based on Afghan assessments in 2015, the “MOD and MOI intelligence capabilities were rated as high as ‘partially 

capable,’ but none were rated as fully operational.” DOD’s June 2016 report on Enhancing Security and Stability in Afghanistan, 

which followed the conclusion of Legacy and ASOM, noted that persistent capability gaps in the Afghan security forces’ intelligence 

collection and dissemination, along with other gaps, “have hampered more rapid improvement in their ability to maintain security 

and stability.” This same report also noted that the Afghan National Army is making progress in intelligence collection and analysis, 

but is still developing its ability to conduct intelligence-driven operations and that the MOI is “progressing in their intelligence 

capabilities, however, there is much room for improvement.” Although these reports focus on the intelligence operations as a 

whole and not solely on human intelligence, which was the focus of the Legacy and ASOM contracts, there was no indication of 

improvement because of the Legacy and ASOM contracts. CTTSO stated that a number of factors undermined the potential for 

program success, such as the difficulty in finding program candidates from the Afghan security forces, which contributed to the

mixed results of the Legacy and ASOM programs. 

SIGAR found that in 2013, ACC awarded NCC the ASOM cost-plus-fixed-fee contract, even though NCC did not have an acceptable 

accounting system, as required. According to ACC, prior to the award, ACC found that NCC’s accounting system to be acceptable 

for determining applicable contract costs by a pre-award accounting system survey. DCAA conducted a post-contract award audit 

of NCC’s accounting system and determined that the system was not acceptable for accumulating and billing costs for U.S. 

government contracts. Based on the audit results, in 2014, DCAA and the Defense Contract Management Agency disapproved 

NCC’s accounting system. The Defense Contract Management Agency’s final determination letter informed NCC “that current 

contracts modified to contain or future contracts that contain [Defense Federal Acquisition Regulation Supplement] clause 

252.242-7005 . . . could be subject to withholding” due to significant deficiencies in NCC’s accounting system. However, SIGAR 

was not provided with evidence showing that the contracting officer sought to modify the ASOM contract to include Defense 
For more information, contact SIGAR Public Affairs at (703) 545-5974 or sigar.pentagon.ccr.mbx.public-affairs@mail.mil. 

Federal Acquisition Regulation Supplement clause 252.242-7005, which would have permitted the government to withhold 

payments to the contractor. Instead, the Defense Contract Management Agency appears to have simply continued approving 

payments to NCC for billed services until the contract ended in February 2016. 

SIGAR found that NCC deployed staff and incurred costs under the Legacy Afghanistan, Kabul, and South contracts either before 

ACC awarded the contracts or before the approved start date for charging program costs. For example, under the Legacy 

Afghanistan contract, which was the first in Afghanistan, Imperatis deployed training and mentoring teams on January 9, 2010, 4

months before the contract was awarded on April 9, 2010, even though it did not authorize precontract costs. According to ACC, 

it modified the Legacy Iraq task order on June 9, 2009, for Imperatis to begin adapting the Legacy Iraq methodology for use in 

Afghanistan. Imperatis used about $7.7 million in funding from the Legacy Iraq task order to pay for this effort until ACC awarded 

the Legacy Afghanistan contract. Similarly, under Legacy Kabul and Legacy South, NCC deployed mentoring teams to Afghanistan 

prior to the base contract’s precontract cost authorization date of July 27, 2010. DCAA conducted an incurred cost audit that 

included these costs; however, the results cannot be publicly released due to DCAA policies. 

Although the Federal Acquisition Regulation allows for the use of precontract cost clauses, they are supposed to be limited to 

costs that would have been allowable if incurred after the contract award date. However, SIGAR’s financial audit and subsequent

review of Imperatis’s monthly billings, along with a DCAA financial audit, identified questionable costs. As noted in SIGAR’s April 

2015 financial audit report on the Legacy East contract, Imperatis did not have the supporting documentation for NCC’s costs, 

preventing SIGAR’s auditors from performing a complete review of subcontractor invoice costs, leading SIGAR to question more 

than $134 million in unsupported costs. SIGAR also found that Imperatis billed, on average, more than $1.8 million per month 

under the Legacy Afghanistan contract for the 10-month period from March 2011 through December 2011, even though the 

training courses it was supposed to conduct were canceled in February 2011. For comparison, the average monthly billings for 

the 5 months prior to the training courses being canceled was less than $180,000. DCAA conducted an incurred cost audit of 

NCC’s subcontractor invoices and supporting documentation for invoiced amounts from August 1, 2007, through December 31, 

2013. During its audit, the agency questioned $51 million in costs incurred under both the Legacy R&D contracts in Afghanistan 

and Iraq. According to ACC, Imperatis failed to monitor and evaluate its subcontractor costs and, therefore, misrepresented some 

costs as allowable and allocable when it submitted invoices for payment. This resulted in DOD paying for costs the U.S. 

government was not legally responsible to pay, thereby increasing those contracts’ costs. 

WHAT SIGAR RECOMMENDS 

SIGAR recommends that the Secretary of Defense (1) review ACC‘s, ARL’s, and CTTSO’s award and oversight of the Legacy and 

ASOM contracts to identify remedies to provide better oversight controls and performance measurements for future R&D 

contracts; and (2) review ongoing ANDSF intelligence training and mentoring contracts, and incorporate into them requirements 

to enable the measurement and verification of contractor performance and contract outcomes, including training and mentoring 

results, and impacts on the ANDSF’s human intelligence capability. In informal comments on a draft of this report, the Office of 

Under Secretary of Defense for Policy concurred with both recommendations. 
July 27, 2017 

The Honorable Jim Mattis 

Secretary of Defense 

General Joseph L. Votel 

Commander, U.S. Central Command 

General John W. Nicholson, Jr. 

Commander, U.S. Forces–Afghanistan and 

 Commander, Resolute Support 

General Gustave F. Perna 

Commanding General, U.S. Army Materiel Command 

Major General James E. Simpson 

Commanding General, U.S. Army Contracting Command 

This report discusses the results of SIGAR’s audit of the Department of Defense’s (DOD) efforts to develop the 

Afghan National Defense and Security Forces’ (ANDSF) intelligence capabilities. In support of the Army 

Research Laboratory (ARL), from 2010 to 2013, the Army Contracting Command (ACC) awarded one task order 

and four contracts, collectively valued at $536.1 million, to develop and implement human intelligence training 

and mentoring programs for the ANDSF. ACC awarded the task order and first three contracts to Imperatis—

formerly Jorge Scientific until 2013—for what became known as the Legacy research and development (R&D) 

training and mentoring program. ACC awarded the fourth contract to New Century Consulting Limited to 

operate the Afghanistan Source Operations Management (ASOM) training and mentoring program. By the end 

of the programs, the total amount spent was $457.7 million. 

We are making two recommendations to DOD. We recommend that the Secretary of Defense (1) review ACC‘s, 

ARL’s, and the Combating Terrorism Technical Support Office’s award and oversight of the Legacy and ASOM 

contracts to identify remedies to provide better oversight controls and performance measurements for future 

R&D contracts; and (2) review ongoing ANDSF intelligence training and mentoring contracts, and incorporate 

into them requirements to enable the measurement and verification of contractor performance and contract 

outcomes, including training and mentoring results, and impacts on the ANDSF’s human intelligence capability. 

In its informal comments on a draft of this report, the Office of the Under Secretary of Defense for Policy 

(OUSD-P) concurred with both of our recommendations. Additionally, OUSD-P and ACC–Aberdeen Proving 

Ground provided technical comments, which we incorporated, as appropriate. ACC–Aberdeen Proving Ground‘s 

comments are reproduced in appendix II. We will publish OUSD-P’s formal comments in a separate product 

when we receive them. 
SIGAR conducted this work under the authority of Public Law No. 110‐181, as amended, and the Inspector 

General Act of 1978, as amended, and in accordance with generally accepted government auditing standards. 

John F. Sopko 

Special Inspector General 

 for Afghanistan Reconstruction 
SIGAR 17-57-AR/Legacy and ASOM Programs Page vi 

TABLE OF CONTENTS

Background .................................................................................................................................................................. 1

Impact of the Legacy and ASOM Programs Is Unknown Because of Lack of Useful Performance Metrics ........... 4

ACC Awarded the ASOM Contract to NCC After Conducting an Assessment of Its Accounting System, Which Was 

Then Found to Have Signifcant Deficiences .............................................................................................................. 9

ARL and CTTSO Did Not Properly Administer the Legacy R&D Contracts and Questions Remain About Contract 

Costs ........................................................................................................................................................................... 10

Conclusion .................................................................................................................................................................. 12

Recommendations .................................................................................................................................................... 13

Agency Comments ..................................................................................................................................................... 13

Appendix I - Scope and Methodology ....................................................................................................................... 14

Appendix II - Comments from the Army Contracting Command–Aberdeen Proving Ground ................................. 16

Appendix III - Acknowledgements ............................................................................................................................. 21

FIGURE 

Figure 1 - Timeline for the Legacy Afghanistan R&D Program and ASOM Contracts .............................................. 3
SIGAR 17-57-AR/Legacy and ASOM Programs Page vii 

ABBREVIATIONS 

ACC Army Contracting Command 

ANDSF Afghan National Defense and Security Forces 

ARL Army Research Laboratory 

ASOM Afghanistan Source Operations Management 

COR contracting officer’s representative 

CTTSO Combating Terrorism Technical Support Office 

DCAA Defense Contract Audit Agency 

DCMA Defense Contract Management Agency 

DOD Department of Defense 

FAR Federal Acquisition Regulation 

MOD Ministry of Defense 

MOI Ministry of Interior 

NCC New Century Consulting Limited 

OUSD-P Office of the Under Secretary of Defense for Policy 

PSTM Professional Standards Transition Model 

R&D research and development 

RAND RAND National Defense Research Institute 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 1 

From 2010 to 2013, the Department of Defense (DOD), through the Army Contracting Command (ACC), 

awarded one task order and four contracts (all referred to as the “contracts” for the purposes of this audit), 

collectively valued at $536.1 million, to develop the Afghan National Security and Defense Forces’ (ANDSF) 

intelligence capabilities. ACC awarded the cost-plus-fixed fee contracts in support of the Army Research 

Laboratory (ARL), which identified the requirement for intelligence training services. ACC awarded the first four 

contracts, known collectively as the Legacy research and development (R&D) training and mentoring program, 

to Jorge Scientific Corporation (rebranded as Imperatis in 2013), with New Century Consulting Limited (NCC) as 

the subcontractor.1 These awards had a combined value of $332.8 million, of which $314.4 million was spent. 

ACC awarded a fifth contract, valued at $203.3 million, to NCC to operate the Afghanistan Source Operations 

Management (ASOM) training and mentoring program. NCC spent $143.3 million on this contract. 

To administer and oversee the contracts, ACC designated the Irregular Warfare Support Program of the 

Combating Terrorism Technical Support Office (CTTSO) as the contracting officer’s technical representative for 

both the Legacy and ASOM programs.2 The Defense Contract Management Agency (DCMA) was appointed as 

the administrative contracting officer to review and approve contractor invoices. 

In April 2015, we reported on the costs Imperatis incurred under the Legacy East contract, the last one 

awarded under the Legacy R&D program. In that report we identified more than $134 million in questioned 

costs because Imperatis failed to retain sufficient supporting documentation of subcontractor costs.3

The objectives of this audit were to determine the extent to which: (1) Imperatis and NCC successfully 

performed the tasks required by the contracts and developed the ANDSF intelligence capability; (2) ACC 

awarded the Legacy and ASOM contracts in accordance with federal and DOD regulations; and (3) ARL and 

CTTSO properly monitored contract performance and DCMA monitored contract costs. 

To accomplish our objectives, we reviewed federal and DOD acquisition regulations and policies. We also 

reviewed DOD contract and budget documents; Imperatis’s and NCC’s weekly, monthly, and final reports; and 

other related data. Further, we reviewed DOD, NCC, and RAND National Defense Research Institute (RAND) 

program assessments, and SIGAR and Defense Contract Audit Agency (DCAA) audit reports on the Legacy 

program and NCC’s accounting system. We analyzed Legacy and ASOM training courses, and training and 

mentoring data. We interviewed DOD, DCAA, and NCC officials. We conducted our work in Washington, D.C., 

and Kabul, Afghanistan, from August 2015 through July 2017 in accordance with generally accepted 

government auditing standards. A more detailed discussion of our scope and methodology is in appendix I. 

BACKGROUND 

In 2007, the U.S. Marine Corps determined that it needed to improve its human intelligence capabilities to 

reduce the number of U.S. military casualties from improvised explosive devices detonated in Iraq. In response, 

ARL issued a broad agency announcement to solicit proposals to “research, develop, test, and evaluate 

information key to success in military conflict, peacekeeping, and humanitarian operations via the collection of 

human intelligence information.”4 On September 27, 2007, ACC awarded an indefinite delivery-indefinite 

 

1 In 2013, Jorge Scientific Corporation was reorganized and rebranded as Imperatis Corporation. We refer to Imperatis 

throughout this report because that was the name of the company when the Legacy and ASOM programs ended. 

2 CTTSO’s mission is to identify and develop capabilities to combat terrorism and irregular adversaries, and to deliver these 

capabilities to DOD components and interagency partners through rapid R&D advanced studies and technical innovation. 

3 See SIGAR, Department of the Army’s Legacy East Project: Jorge Scientific Corporation’s Lack of Supporting 

Documentation Results in about $135 million in Questionable Project Costs, SIGAR 15-43-FA, April 21, 2015. 

4 See broad agency announcements number W911NF-07-R-0001, Section 9.0 and 9.1.1 from fiscal year 2007 to fiscal 

year 2012, and number W911NF-12-R-0011, from May 2012 to March 2017, Engineering and Mathematical Sciences–

Information Fusion. The Federal Acquisition Regulation (FAR) defines a “broad agency announcement” as “a general 

announcement of an agency’s research interest including criteria for selecting proposals and soliciting the participation of 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 2 

quantity base contract to Imperatis for the research and develop technological solutions to address 

intelligence deficiencies in counterinsurgency, pursue the global war on terrorism, and counter transnational 

threats.5 ACC later awarded two task orders under this base contract, with NCC as the subcontractor, for the 

Legacy Iraq and Legacy Gryphon intelligence training and mentoring programs in Iraq. These task orders had a 

combined value of $145.6 million, of which $64.7 million was spent.6

In 2010 and 2011, ACC, using the same broad agency announcement, awarded another four contracts, with a 

combined award value of $332.8 million, to Imperatis, with NCC as a subcontractor, for intelligence training 

and mentoring in Afghanistan. Imperatis spent $314.4 million on the contracts—Legacy Afghanistan, Legacy 

Kabul, Legacy South, and Legacy East—which became known collectively as the Legacy Afghanistan R&D 

program. The program’s objective was to develop a sustainable human intelligence capability and doctrine for 

source operations within the Afghan intelligence services.7 Specifically, the contracts required Imperatis and 

NCC to create courses to train Afghan intelligence officers. The initial R&D contracts were awarded in 2010, 

and the final contract was awarded in 2011. In May 2012, ACC modified the Legacy East contract to combine 

it with Legacy Afghanistan, Kabul, and South, and closed the three earlier contracts. 

Following the expansion of the Legacy East contract in May 2012, ARL requested competitive proposals for a 

new contract that would provide training and mentoring services to help: (1) professionalize the Afghan 

Ministry of Defense’s (MOD) and Ministry of Interior’s (MOI) intelligence units; (2) build a human intelligence 

program; and (3) continue the Legacy R&D program’s work in Afghanistan. Specifically, the contractor was 

required to provide continued training and mentoring to Afghan intelligence officers at various sites throughout 

Afghanistan. In July 2013, ACC competitively awarded NCC a contract valued at $203.3 million to implement 

ASOM. This contract ended in February 2016 and cost $143.3 million. By September 2013, following the 

completion of the Legacy East contract, all training doctrine, courses, personnel, and equipment from the 

Legacy Afghanistan R&D program were transferred to the ASOM program. 

Figure 1 shows the contract award dates, periods of performance, and amount expended for each contract for 

the Legacy Afghanistan R&D program and ASOM from 2007 through 2016. 

 

all offerors capable or satisfying the Government’s needs (see 6.102(d)(2).” FAR 2.101. A broad agency announcement 

can be used as part of a competitive selection procedure for basic and applied research and development. 

5 The contract number is W911QX-07-D-0012. 

6 ACC awarded the task orders for Legacy Iraq and Legacy Gryphon in September 2007 and March 2009, respectively. 

7 An intelligence source management doctrine is a set of curricula to train and mentor personnel on information gathering 

to build specialized intelligence organizations within the Ministries of Defense and Interior to defeat insurgent groups. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 3 

Figure 1 - Timeline for the Legacy Afghanistan R&D Program and ASOM Contracts

Source: SIGAR analysis of Legacy and ASOM contracts 

In 2010 and 2011, ACC awarded the Legacy contracts for the purpose of establishing R&D training and 

mentoring programs, which are intended “to advance scientific and technical knowledge and apply that 

knowledge to achieve agency and national goals.”8 The Federal Acquisition Regulation (FAR) distinguishes 

between R&D contracts and contracts for supplies and services.9 In particular, “most R&D contracts are 

directed toward objectives for which the work or methods cannot be precisely described in advance.”10 As a 

result, R&D awards generally emphasize “achieving specified objectives and knowledge rather than on 

achieving predetermined end results prescribed in a statement of specific performance characteristics.”11

Furthermore, contractors are given wide latitude to conduct their work as the contract “must provide an 

environment in which the work can be pursued with reasonable flexibility and minimum administrative 

burden.”12 

In contrast, in July 2013, ACC awarded ASOM as a services contract, which is designed to ensure that the 

contractor provides definite types of services. The FAR requires these types of contracts to include 

requirements for the contracting agency to monitor and evaluate contractor activities. For example, the FAR 

states that performance work statements should enable the contracting agency to assess a contractor’s 

performance against measurable performance standards.13 

For both the Legacy Afghanistan R&D and ASOM contracts, ACC appointed ARL as the contracting officer’s 

representative (COR), meaning that ARL was authorized to perform certain administrative functions related to 

monitoring the contracts that included oversight of the contractors’ performance. Additionally, ARL designated 

CTTSO as the contracting officer’s technical representative, meaning CTTSO was the subject matter expert 

responsible for monitoring certain technical functions related to the Legacy and ASOM contracts. In January 

2013, ACC appointed CTTSO to replace ARL as the COR for the ASOM contract. ACC also designated the 

International Security Assistance Force’s Afghanistan Security–Essential Function 7: Intelligence, followed by 

the Resolute Support–Essential Function 7: Intelligence, to be the technical representative in Afghanistan, and 

 

8 FAR 35.002. 

9 Id. 

10 Id. 

11 See FAR 35.005. 

12 See FAR 35.002. 

13 See FAR 37.602-1(b)(2), “Service Contracts: Statement of Work.” 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 4 

it was responsible for monitoring day-to-day management of the Legacy and ASOM contracts throughout 

Afghanistan.14 Because the Legacy and ASOM contracts were cost-plus-fixed-fee, ACC delegated administrative 

contracting officer responsibility to DCMA, requiring it to review and approve contractor invoices.15 DCMA, in 

turn, designated DCAA as its authorized representative to assist with these reviews. 

To assist with its oversight responsibilities, CTTSO awarded contracts to RAND and ManTech International 

Corporation (ManTech). RAND’s contract required it to evaluate the Legacy R&D programs in Iraq and 

Afghanistan.16 Under this contract, RAND was to visit mentoring sites in Afghanistan; review NCC reports; 

interview Imperatis, NCC, International Security Assistance Force, and ANDSF personnel; and issue reports that 

described the essential characteristics of the programs, the environments in which they were implemented, 

and summarize program results. ManTech provided CTTSO with technical support and additional assistance 

overseeing the Legacy and ASOM programs. ManTech was responsible for conducting site visits to Imperatis’s 

headquarters in London and ANDSF sites in Afghanistan where NCC trained and mentored ANDSF personnel. 

ManTech also interviewed NCC trainers and mentors, and Afghan and Resolute Support leadership. 

IMPACT OF THE LEGACY AND ASOM PROGRAMS IS UNKNOWN BECAUSE OF 

LACK OF USEFUL PERFORMANCE METRICS 

The $314.4 million Legacy R&D program did not have adequate performance metrics or defined outcomes, 

which affected our ability to determine the program’s effectiveness. Additionally, although the ASOM program 

had some performance metrics, those relied in part on data provided by contractors. DOD hired RAND and 

ManTech to assist with program oversight and produce evaluations of the Legacy and ASOM programs; 

however, those evaluations, which also relied in part on data from contractors, showed mixed results. CTTSO 

stated that a number of factors undermined the likelihood of the programs’ success, such as the difficulty in 

finding program candidates from the ANDSF, which contributed to the mixed results of the Legacy and ASOM 

programs. Furthermore, each regional command within Afghanistan assigned one military officer to provide 

additional program oversight, and CTTSO noted, “Some were more effective than others.”17 Finally, based on 

our analysis of the Legacy and ASOM programs’ results, we determined that the programs did not fully achieve 

their stated objectives. 

Before 2012, the Legacy Contracts Did Not Contain Metrics to Evaluate Program 

Success and the ASOM Contract Contained Limited Metrics 

The Legacy R&D contracts required Imperatis to create a methodology plan that described the equipment, 

personnel, and schedule necessary to carry out the planned program, as well as performance metrics to 

evaluate program effectiveness. In fact, the performance work statement for Legacy Kabul required Imperatis 

to deliver a methodology plan that included “a detailed description of testing and metrics/data collection 

required to assess and demonstrate the Legacy-K[abul] methodology” but it did not detail what the testing and 

 

14 The International Security Assistance Force was the predecessor to the ongoing Resolute Support mission. The 

International Security Force Afghanistan Security–Essential Function 7: Intelligence became the Resolute Support Train, 

Advise, and Assist–Essential Function 7: Intelligence on January 1, 2015. 

15 FAR 16.306 describes a cost-plus-fixed-fee contract as “a cost-reimbursement contract that provides for payment to the 

contractor of a negotiated fee that is fixed at the inception of the contract . . . This contract type permits contracting for 

efforts that might otherwise present too great a risk to contractors, but it provides the contractor only a minimum incentive 

to control costs.” 

16 RAND is a federally funded R&D center sponsored by the Office of the Secretary of Defense, Joint Staff, Unified 

Combatant Commands, the Navy, the Marine Corps, defense agencies, and defense intelligence community. 

17 The Resolute Support mission has divided itself into five geographic regional commands in Afghanistan. These are Train 

Advise Assist Commands West, North, South, East, and Capital. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 5 

metrics should include. Based on our review of the plans, we did not identify any performance metrics that we 

could use to evaluate the program’s effectiveness. We determined that the Legacy contracts included 

requirements for Imperatis and its subcontractor, NCC, to develop training doctrine and courses; hire mentors, 

trainers, advisors, and compliance officers; and train and mentor ANDSF officials to develop the force’s human 

intelligence capabilities. The contracts also required Imperatis and NCC to submit a variety of reports, including 

weekly, monthly, and final reports on a timely basis. Furthermore, the quality assurance surveillance plan for 

Legacy Kabul, a document that “is put in place to provide [g]overnment surveillance oversight of the 

[c]ontractor’s quality control efforts to assure that they are timely, effective, and are delivering the results 

specified in the contract or task order” stated that the government would only monitor the timely submission of 

monthly reports, methodology plan, training methodology, and the final report. However, the FAR allows for 

significant flexibility when determining what kind of oversight and assessment is appropriate for R&D awards, 

and stresses that contracts should be written so that work can be pursued with reasonable flexibility and 

minimum administrative burden.18 We found that ACC issued the Legacy R&D contracts without clear 

performance metrics and did not establish defined outcomes that would have allowed the results Imperatis 

and NCC reported to be measured or validated. As a result, it is difficult to measure the success of each Legacy 

contract before 2012, when ACC established requirements for Imperatis and NCC to measure the capabilities 

and independence of ANDSF intelligence sites. 

In 2012, ACC introducted the Professsional Standards Transition Model (PSTM) through a modification to the 

to Legacy East and the award of the ASOM contract.19 However, we found that even with this added 

requirement and the regular contractor reports, clear metrics were not available to assess the overall success 

of each contract. For example, under the Legacy program, the methodology plans did not include metrics to 

determine the number of (1) courses to be taken by each Afghan National Army and Afghan National Police 

intelligence officer, or (2) hours contractor personnel should mentor each ANDSF intelligence officer. 

Additionally, we reviewed the “Detailed Description of Testing and Metrics” section of the Legacy final report 

and found that although it had lists of the equipment used, staffing locations, course development dates, and 

course drafting guidelines used under the Legacy R&D contracts, it did not discuss metrics or contractor 

performance. 

Finally, while the ASOM training standard operating procedures had proposed requirements for an ANSDF 

trainee to become a certified instructor or trainer, not all ANDSF participants took the courses required to 

become an advisor or trainer. CTTSO noted that although Imperatis and NCC could recommend standards for 

ANDSF personnel to be considered intelligence officers, the Afghan government was responsible for 

establishing the final requirements and approving the courses, which did not occur until the end of the ASOM 

program. 

In addition to a lack of metrics for measuring program performance, apart from a requirement for the timely 

submission of various reports, CTTSO’s attempts to assess contractor performance relied, in part, on data from 

the contractor, such as NCC’s self-assessments created by its mentors and compliance officers. We reviewed 

the self-assessments and found that NCC deemed the Legacy and ASOM programs successful, but provided 

few specifics to support those decisions. Further, through our review of NCC’s status reports, we determined 

that those reports focused on administrative data, with little information describing the programs’ progress 

toward building a sustainable human intelligence capability. For example, NCC’s November 2012 monthly 

status report for the Legacy East program had the billeting status of contractor employees, training records, 

and narrative descriptions of program events, but did not mention how these details related to program 

performance. 

 

18 FAR Part 35. 

19 The PSTM is an accountability framework designed to measure the modified Legacy East and ASOM programs’ 

performance. The PSTM used inspections to measure intelligence sites in six categories related to the mentoring and 

training provided to the ANDSF.
SIGAR 17-57-AR/Legacy and ASOM Programs Page 6 

In addition to the contractor-provided data, CTTSO also relied on contractually required status meetings, 

monthly COR assessments based on checklists (with no explanatory narrative), the up-to-date status of 

contractor invoices, oversight from one military officer in each regional command, which CTTSO noted “some 

were more effective than others,” and reports from RAND and ManTech to assess program performance. RAND 

and ManTech compiled their reports using information obtained from site visits, interviews, and their analysis 

of Legacy and ASOM data. However, we reviewed the reports and metrics CTTSO, RAND, and NCC provided and 

found them to appear to be subjective. 

For example, the monthly COR assessments of the Legacy East and ASOM programs used a sliding scale—from 

exceptional to unsatisfactory—to rate the contractor’s work. These documents had little or no discussion of why 

the COR gave a specific rating or what the COR’s baseline was for assessing progress. Furthermore, based on 

NCC data, we found that the PSTM also relied on subjective assessments of progress. Specifically, the aspect 

ratio score NCC used in the PSTM to judge whether an intelligence site where mentoring had been occurring 

was ready for transition to the ANDSF relied on six different factors ranked on a subjective scale that could vary 

between raters. For example, for one category, a ranking for “people - ability” was broadly defined as “some 

evidence of ability displayed.” 

According to contract documents, the expanded Legacy East and ASOM programs were supposed to continue 

developing the intelligence capacity model and the capacity of intelligence officers at sites throughout 

Afghanistan, readying them for transfer to the Afghan government. The contracts required NCC to prepare 

standard operating procedures for training, training plans, and course materials; create testing and metrics to 

determine the capability and independence of mentoring sites; set the minimum requirements for Afghan 

intelligence personnel to be considered as instructors or trainers; and submit weekly and monthly status 

reports. Starting in mid-2012, NCC measured the success of the Legacy program, and later ASOM, using the 

RAND reports, PSTM data, and the number of sites transferred to the MOD and MOI. NCC evaluated each 

intelligence site to determine its capability for operating independently of ASOM mentors. While multiple 

officers in Afghanistan recommended the continuation and expansion of the Legacy program, as discussed in 

more detail below, after reviewing the metrics NCC used to evaluate Legacy East and ASOM, we found they did 

not provide enough evidence to determine definitively whether the programs were successful. 

RAND’s Performance Evaluation of the Legacy Program Showed Mixed Results 

RAND’s program assessments identified positives and negatives about the Legacy program.20 In 2012, RAND 

noted that the program’s model warranted some basis for cautious optimism.21 However, in 2014, RAND 

issued a lessons learned report that focused on formulating guidelines for planning and executing future 

Legacy-like programs. In this report, RAND stated that the Legacy R&D program was not properly planned prior 

to its implementation and that the successful implementation of any future Legacy-like program required a 

seven-step comprehensive, incremental, and iterative planning process. In this same report, RAND expressed 

the need to customize the Legacy program to fit Afghanistan’s culture and capabilities, ensure Afghan 

sustainment of the program, and the need to assess program success accurately. This report emphasized the 

need for regular and systematic evaluations to (1) learn what works in local circumstances, (2) eliminate 

ineffective processes or mentors, and (3) determine where to expend resources to achieve a worthwhile return 

 

20 RAND’s assessments described the essential characteristics of the programs, the environments in which they were 

implemented, and summarized program results. 

21 RAND National Defense Research Institute, Lessons Learned from Legacy-A: Building Capacity for Local Intelligence in 

Afghanistan’s Helmand Province, September 2012. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 7 

on investment. Lastly, the report concluded that performing proper assessments and appropriate advance 

adaptation prior to launching future initiatives would prevent costly missteps.22 

In 2015, in its final Legacy assessment, RAND reported mixed results for the program’s overall success and 

expressed concerns related to program implementation and assessment. For example, RAND stated that the 

development of an integrated intelligence system had not yet occurred in Afghanistan and that concerns 

remained about the program’s viability. The RAND report concluded that the “goal has not been fully achieved. 

This is not to say that significant progress toward that goal has not occurred. It has.”23 RAND based its 

conclusion on NCC’s data, the Afghan government’s creation of a unified format for intelligence reports, and an 

increased consumption of intelligence products by Afghan forces. However, the report also noted that the 

complex nature of the program limited RAND’s ability to make a definitive assessment of the program’s 

success. 

Analyses of Legacy and ASOM Records Show That the Programs Did Not Achieve All 

Program Objectives 

ACC awarded the Legacy and ASOM contracts to develop an intelligence capacity-building model that the MOD 

and MOI approved. Under ASOM, the model proposed train-the-trainer and instructor certifications that 

required ANDSF officers to complete a series of courses that Imperatis and NCC designed. As previously 

discussed, NCC developed the courses and proposed the requirements for an ANDSF trainee to be considered 

a certified instructor or trainer; however, the MOD and MOI were responsible for making the final decision 

about what courses were mandatory, which the ministries failed to do until the end of ASOM in February 2016. 

Because Imperatis and NCC did not retain complete training records, we assessed the extent to which the 

records they did maintain documented whether individual ANDSF officers completed the required number of 

courses to be considered an intelligence instructor or trainer as a possible indicator of the Legacy program’s 

success.24 Although we found that 70 of 71 Afghan National Army officers completed the four courses required 

to be an instructor, we also found that not all intelligence trainers and instructors met the minimal 

requirements.25 For example: 

 Ten of 24 police intelligence student trainers completed all nine courses required to be a MOI trainer, 

while the 14 remaining trainers completed between one and eight of the required courses. 

 None of the four Afghan National Army student trainers completed all six courses required to be a 

MOD trainer. Although all the trainers completed at least three of the required courses, none of them 

completed more than four of the courses. 

 Five of six Afghan National Army student instructors did not complete any of the four courses required 

to be a MOD instructor, and one student instructor completed only one required course. 

As part of the Legacy R&D program and the ASOM program, NCC created 29 separate training courses, 

covering a variety of intelligence topics. Based on our review of the course records, we determined that the 

Legacy R&D program held 372 total classes, which 3,421 students attended, and ASOM held 428 classes, 

which 3,645 students attended. However, according to the Resolute Support ASOM technical monitor and NCC 

 

22 RAND National Defense Research Institute, Comprehensive Lessons Learned from Legacy Capacity-Building Programs 

and Related Initiatives, July 2014. 

23 RAND National Defense Research Institute, An Enduring Legacy? Building Local Intelligence Gathering Capacities in Iraq, 

Afghanistan, and Beyond, March 2015.

24 Under ASOM, NCC added comprehensive train-the-trainer and instructor elements to the curriculum to enable the ANDSF 

to take over responsibility for teaching human intelligence skills. An Afghan trainer delivers an entire section of Afghan 

National Army or Police human intelligence courses, while an instructor delivers a subset of courses focusing on the initial 

operating capability of human intelligence. 

25 NCC told us that DynCorp International certified the other instructor prior to the Legacy and ASOM programs. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 8 

officials, due to the difficulties measuring the intelligence capacity of each individual, NCC tried a new 

measurement of program success—aspect rating scores—for the expanded Legacy East and ASOM programs.26

The aspect rating score rated the capabilities and independence of MOD and MOI intelligence training sites on 

a scale from 0 to 5, and was supposed to measure the ability, proficiency, efficiency, and institutionalization of 

each site. A score above 4 meant that a site was ready to be fully transferred to the MOD or MOI, and a score 

of 3.5 meant that a site could be transferred, but its capabilities carried a risk of being reversible. According to 

NCC’s October 2014 Post Site Transition Report, if an intelligence training site was not rated as being able to 

be fully transferred to the Afghan government when the site was vacated due to the withdrawal of coalition 

forces, the site was issued a capability handover plan, which provided guidance in the absence of an on-site 

mentoring team. Finally, according to this same report, other sites were closed when inspections found that 

MOD and MOI personnel could not attain the necessary competence for transition within the lifetime of the 

Legacy East or ASOM programs. 

In total, only 59 of the 125 (47 percent) Legacy and ASOM intelligence training and mentoring sites were rated 

as being ready for full transfer to the Afghan government. The remaining 66 were transferred with varying 

levels of capability or not transitioned. Starting with the modified Legacy East and ASOM contracts, NCC used 

aspect rating scores to measure program success. In August 2013, NCC reported on 103 out of 111 MOD and 

MOI intelligence sites that were to be transferred to the ministries. After reviewing NCC’s inspection results, we 

found that: 

 Forty-seven MOD and MOI intelligence sites had aspect rating scores above 3.5 and were fully 

transferred, or transferred with risk of losing capabilities, to the ministries. 

 Forty-six MOD and MOI intelligence sites had aspect rating scores below 3.5, but were turned over to 

the ministries with capability handover plans that provided guidance in the absence of an on-site 

mentoring team. 

 NCC assessed 10 MOI intelligence sites as not capable of self-sustainment. After DOD directed NCC’s 

mentoring teams to leave the sites due to troop withdrawals, intelligence capability at each site 

remained underdeveloped. 

 NCC did not assess eight intelligence sites. 

By July 31, 2014, NCC had stopped mentoring under ASOM at all 111 MOD and MOI intelligence sites. As of 

August 2014, NCC reported 58 intelligence sites—18 MOD sites and 40 MOI sites—were successfully 

transferred to the ANDSF, and 42 intelligence sites—21 MOD sites and 21 MOI sites—were transferred with 

capability handover plans that included guidance in the absence of an on-site mentoring team. NCC did not 

report the other 11 intelligence sites as part of the transition success, but noted that sites were closed if NCC 

determined that they could not attain the necessary level of competence for transfer within ASOM’s expected 

lifetime. 

Under the ASOM program, NCC performed additional inspections to determine whether there were any 

changes to the aspect ratio scores since the conclusion of Legacy East. NCC found almost no changes in the 

ratings for the sites that were fully transferred to the MOD or MOI, or turned over with capability handover 

plans. For example: 

 In July 2014, NCC inspected four sites, found that the rating scores were similar for the three sites 

that were transferred as fully capable, and concluded that each site was still considered capable. NCC 

determined that the fourth site, which was turned over with a capability handover plan, was still not 

capable enough to be considered fully transferred. 

 In August 2015, NCC reported on 27 post-transition inspections that occurred more than 1 year after 

its mentoring teams left and found that the 6 MOD and 18 MOI intelligence sites that had been 

 

26 Before ACC expanded the Legacy East contract, regular status and compliance reports were used to evaluate program 

success. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 9 

transferred as fully capable continued to have similar rating scores. NCC found that the three 

remaining MOD sites, which were turned over with capability handover plans, had similar rating scores 

and could not be considered fully transferred. 

Although it had stopped mentoring at the 111 MOD and MOI intelligence sites by the end of July 2014, NCC 

continued mentoring at 14 General Command of Police Special and Investigative Surveillance sites until 

February 2016, when the ASOM program ended. NCC reported that one of these sites was fully transferred and 

the other 13 sites were transferred with capability handover plans between December 2014 and the end of 

the ASOM program. CTTSO noted that one of the causes for this low rate of transition was that these sites did 

not receive the full ASOM program due to insufficient time remaining on the contract to provide the full 

training. 

Although the aspect rating scores indicate some success in transitioning MOD and MOI intelligence sites, DOD 

and the Afghan government continued to report that there is a need for coalition assistance and the ANSDF is 

only partially capable with respect to its intelligence operations. In response to a draft of this report, DOD 

stated that these assessments addressed the entirety of the Afghan intelligence capability, as opposed to only 

the human intelligence capability that Legacy and ASOM sought to develop. However, they are illustrative of the 

need for continued coalition assistance to be capable and self-sufficient in intelligence operations. For 

example, in its semiannual reports to Congress from 2011 to 2014 on Progress toward Security and Stability 

in Afghanistan, produced while Legacy and ASOM were under way, DOD found that the MOD and MOI 

intelligence units required coalition assistance to accomplish their missions.27 Similarly, in our January 2016 

Quarterly Report to Congress, we stated that based on Afghan assessments in 2015, the “MOD and MOI 

intelligence capabilities were rated as high as “partially capable,” but none were rated as fully operational.28

DOD’s June 2016 report on Enhancing Security and Stability in Afghanistan, which followed the conclusion of 

Legacy and ASOM, noted that persistent capability gaps in the Afghan security forces’ intelligence collection 

and dissemination, along with other gaps, “have hampered more rapid improvement in their ability to maintain 

security and stability.”29 This same report also noted that the Afghan National Army is making progress in 

intelligence collection and analysis, but is still developing its ability to conduct intelligence-driven operations 

and that the MOI is “progressing in their intelligence capabilities, however, there is much room for 

improvement.”30 Although these reports focus on the intelligence operations as a whole and not solely on 

human intelligence, which was the focus of the Legacy and ASOM contracts, there was no indication of 

improvement in overall intelligence operations as a result of the Legacy and ASOM contracts. 

ACC AWARDED THE ASOM CONTRACT TO NCC AFTER CONDUCTING AN 

ASSESSMENT OF ITS ACCOUNTING SYSTEM, WHICH WAS THEN FOUND TO 

HAVE SIGNIFCANT DEFICIENCES 

In July 2013, ACC awarded NCC the ASOM cost-plus-fixed-fee contract. The FAR states that a cost-plus-fixed-fee 

contract cannot be awarded to a contractor that does not have an accounting system that can determine costs 

applicable to the contract.31 Similarly, the Defense Federal Acquisition Regulation Supplement requires that 

 

27 DOD, Progress toward Security and Stability in Afghanistan, 1230-31 Reports to the Congress, October 2011 through 

April 2014. These reports included DOD and Afghan government assessments of the ANDSF intelligence capability. 

28 SIGAR, Quarterly Report to the Congress, January 30, 2016. This report included MOD and MOI intelligence capability 

assessments for the third and fourth quarters of 2015. 

29 DOD, Enhancing Security and Stability in Afghanistan, June 2016. 

30 DOD, Enhancing Security and Stability in Afghanistan, June 2016. 

31 FAR 16.306(c); FAR 16.301-3(a)(3). 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 10 

contractors on cost-reimbursement contracts maintain an “acceptable accounting system,” which shall 

provide, among other things, timekeeping and labor distribution systems.32

The contract award documents indicate that ACC and CTTSO selected NCC for the ASOM contract based on 

NCC’s technical understanding, qualified employees, and reasonable costs, but there was no mention of its 

accounting system. According to ACC, prior to the award in July 2013, NCC’s accounting system was found by 

ACC to be adequate for determining applicable contract costs by a “very simplistic” pre-award accounting 

system survey. In its comments on a draft of this report, ACC clarified that the two-page form used for this 

purpose was simplistic in nature and was not referring to the accounting system survey conducted. ACC added 

that by three independent accounting firms reviewed NCC’s cost accounting system prior to the contract 

award. All three found the accounting system to be adequate but recommended that it be subject to audit 

following the award of the ASOM contract. 

DCAA conducted the recommended post-contract award audit of NCC’s accounting system and determined 

that the system was not adequate for accumulating and billing costs for U.S. government contracts. Based on 

the audit results, in 2014 DCMA disapproved NCC’s accounting system.33 DCMA’s final determination letter 

informed NCC “that current contracts modified to contain or future contracts that contain [Defense Federal 

Acquisition Regulation Supplement] clause 252.242-7005 . . . could be subject to withholding” due to 

significant deficiencies in NCC’s accounting system. However, we were not provided with evidence showing 

that the contracting officer sought to modify the ASOM contract to include Defense Federal Acquisition 

Regulation Supplement clause 252.242-7005, which would have permitted the government to withhold 

payments to the contractor. Moreover, even though DCMA’s final determination letter indicated that NCC did 

submit a corrective action plan intended to resolve its accounting system deficiencies, we were not provided 

with any evidence showing that the corrective plan was actually implemented by NCC or verified by the 

government. Instead, DCMA appears to have simply continued approving payments to NCC for billed services 

until the contract ended in February 2016. 

ARL AND CTTSO DID NOT PROPERLY ADMINISTER THE LEGACY R&D 

CONTRACTS AND QUESTIONS REMAIN ABOUT CONTRACT COSTS 

ARL and CTTSO did not wait for each of the Legacy contracts to end before starting the next contract. Because 

each Legacy contract was an R&D contract with the goal of building technical knowledge, awarding the Legacy 

Afghanistan, South, Kabul, and East contracts sequentially, and after the prior contract concluded, would have 

allowed ACC and CTTSO to use the full knowledge gained from each R&D contract to improve the next one. In 

addition, ACC’s use of precontract clauses allowed Imperatis to begin work on the Legacy program in 

Afghanistan before the contracts were awarded.34 Further, we have concerns related to costs Imperatis 

charged for training courses that were required by the contracts but did not take place. 

Final Assessments of the First Three Legacy Contracts Were Not Used to Improve 

Subsequent Contracts 

ARL and CTTSO did not evaluate the outcomes of each Legacy R&D contract before ACC awarded the next R&D 

contract, nor did they obtain the final Legacy R&D program results until after ACC awarded the ASOM contract. 

 

32 Defense Federal Acquisition Regulation Supplement 242.7503; Defense Federal Acquisition Regulation Supplement 

252.242-7006--Accounting System Administration. 

33 DCMA, Final Determination—Business System Disapproved, DCMAI-GE, December 23, 2014. 

34 FAR 31.205-32 defines “precontract costs” as “costs incurred before the effective date of the contract directly pursuant 

to the negotiation and in anticipation of the contract award when such incurrence is necessary to comply with the proposed 

contract delivery schedule.” 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 11 

In response to a draft of this report, DOD stated that the programs were expanded at the request of DOD 

officers in Afghanistan, and to allow for better management and flexibility of the mentors. However, the 

objective of each Legacy contract, as noted in the statement of work, was to build upon the work of the prior 

Legacy contracts. Each Legacy R&D contract required a final assessment and report, and the expectation was 

that the results and the lessons learned identified in the report would be used to improve the next Legacy 

project. However, that did not occur because ACC awarded the first Legacy contracts—Legacy Afghanistan, 

Kabul, and South—within 6 months of each other, and they had overlapping periods of performance. Further, in 

May 2012, ACC modified the Legacy East contract to combine it with Legacy Afghanistan, Kabul, and South, 

and closed the three contracts prior to each contract’s final assessment. 

CTTSO stated that its office and ARL did learn lessons throughout the Legacy programs and incorporated them 

into the later Legacy contracts. However, CTTSO did not cite any examples of those lessons learned. CTTSO 

also stated that consolidating the Legacy contracts under the expanded Legacy East contract allowed for better 

management and more flexibility of the Legacy R&D program. This decision raises questions as to why the 

contracts were issued separately in the first place, and why the contracts were not re-competed earlier as a 

services contract with more specific deliverables and more robust oversight provisions. By combining all of the 

Legacy projects into Legacy East, ARL and CTTSO further delayed the codification of any of the lessons learned 

from the programs until after Legacy East was completed, which was more than 3 years after the first Legacy 

project began. Furthermore, NCC issued its evaluation of the full Legacy program in September 2013, about 2 

months after ACC awarded the ASOM contract. As a result, it is unlikely that ARL and ACC fully incorporated any 

of the lessons learned from the Legacy program into the ASOM contract. 

Imperatis Was Allowed to Incur Costs on the Legacy Contracts Before ACC Awarded 

Them, and SIGAR and DCAA Have Questioned Other Contract Costs 

Based on our review of available billings, minutes from ARL and CTTSO meetings, and NCC’s final Legacy 

report, we found that NCC deployed staff and incurred costs under the Legacy Afghanistan, Kabul, and South 

contracts either before ACC awarded the contracts or before the approved start date for charging contract 

costs. The Legacy Kabul, South, and East contracts contained clauses that allowed Imperatis to begin charging 

costs to the contracts on a specified date, prior to the actual award date. The FAR states that precontract costs 

“are allowable to the extent that they would have been allowable if incurred after the date of the contract.”35

ACC did not incorporate an advance agreement for precontract costs into the Legacy Afghanistan contract, but 

it did incorporate such an agreement into the contracts for Legacy Kabul, South, and East, allowing Imperatis 

to incur costs for these contracts prior to their award dates.36 Specifically, we found: 

 Under the Legacy Afghanistan contract, which was the first in Afghanistan, Imperatis deployed training 

and mentoring teams on January 9, 2010, 4 months before the contract was awarded on April 9, 

2010, even though the contract did not authorize precontract costs. According to ACC, it modified the 

Legacy Iraq task order on June 9, 2009, for Imperatis to begin adapting the Legacy Iraq methodology 

for use in Afghanistan. Imperatis used about $7.7 million in funding from the Legacy Iraq task order to 

pay for this effort until ACC awarded the Legacy Afghanistan contract.37 

 

35 FAR 31.205-32. 

36 ACC awarded the Legacy Kabul and Legacy South contracts on September 28, 2010. However, based on a contract 

clause authorizing precontract costs contained in each base contract, both contracts allowed Imperatis to start incurring 

costs on July 27, 2010. ACC awarded the Legacy East contract on October 21, 2011, but the precontract clause in the 

contract only authorized precontract costs incurred after the date of the award, which appears to have rendered the 

precontract clause practically useless and the government not responsible for paying precontract costs incurred prior to the 

date of the award. 

37 DCAA conducted an audit of these incurred costs, but the results of its audit cannot be released publicly because of 

DCAA policies. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 12 

 Under Legacy Kabul, Imperatis deployed mentoring teams to four locations within Afghanistan in May 

2010, even though, according to its base contract, it was not authorized to charge costs to the 

contract until July 27, 2010. DCAA conducted an incurred cost audit that included these costs, but the 

results cannot be released publicly because of DCAA policies that restrict release for government use. 

 Under Legacy South, Imperatis again deployed mentoring teams to Afghanistan prior to the base 

contract’s precontract cost authorization date of July 27, 2010. As with Legacy Kabul, DCAA 

conducted an incurred cost audit that included these costs, but the results cannot be released publicly 

because of DCAA policies that restrict release for government use. 

As noted above, precontract costs are “allowable to the extent that they would have been allowable if incurred 

after the date of the contract.”38 However, our financial audit and subsequent review of Imperatis’s monthly 

billings, along with a DCAA financial audit, identified questionable costs. As noted in our April 2015 financial 

audit report on the Legacy East contract, despite being responsible for reviewing NCC’s invoiced costs, 

Imperatis did not have the supporting documentation for NCC’s costs, preventing our auditors from performing 

a complete review of subcontractor invoice costs, leading us to question more than $134 million in 

unsupported costs.39 We also found that Imperatis billed on average more than $1.8 million per month under 

the Legacy Afghanistan contract for the 10-month period from March 2011 to December 2011, even though 

the training courses it was supposed to conduct were canceled in February 2011. In comparison, the average 

monthly billings for the 5 months prior to the training courses being canceled were less than $180,000. The 

cancellations were due to operational commitments of the ANDSF officers and an inadequate number of 

acceptable students. SIGAR continues to review these billing anomalies. 

DCAA conducted an incurred cost audit of NCC's subcontractor invoices and supporting documentation for 

invoiced amounts from August 1, 2007, through December 31, 2013. During its audit, the agency questioned 

$51 million in costs incurred under both the Legacy R&D contracts in Afghanistan and Iraq.40 DCMA and DCAA 

were required to oversee and review the Legacy costs while the contracts were active. However, according to 

ACC, Imperatis failed to monitor and evaluate its subcontractor costs and, therefore, misrepresented some 

costs as allowable and allocable when it submitted invoices for payment. This resulted in DOD paying for costs 

the U.S. government was not legally responsible to pay, thereby increasing those contracts’ costs. 

CONCLUSION 

Although the Legacy and ASOM programs provided some assistance to the ANDSF, it is almost impossible to 

assess whether this assistance was worth the $457.7 million spent on these programs. This uncertainty is due 

primarily to a lack of performance metrics and defined outcomes for the programs, and to the government’s 

attempts to evaluate the programs by relying heavily on subjective reports provided by the contractors that 

were being paid to implement the programs. The lack of performance metrics and heavy reliance on contractor 

data appear to be connected to ACC’s decision to implement the Legacy programs through $314.4 million 

worth of R&D contracts that did not hold the implementing contractor, Imperatis, and its subcontractor, NCC, 

accountable by using specific, useful measures for program success. 

 

38 FAR 31.205-32. 

39 SIGAR, Department of the Army’s Legacy East Project: Jorge Scientific Corporation’s Lack of Supporting Documentation 

Results in about $135 Million in Questionable Project Costs, SIGAR 15-43-FA, April 21, 2015. After the Legacy program 

ended, DCAA conducted an incurred cost audit of NCC’s subcontractor invoices and supporting documentation for invoiced 

amounts from August 1, 2007, through December 31, 2013. The results of this audit cannot be released publicly because 

of DCAA policies. 

40 DCAA, Independent Audit Report on New Century Consulting Ltd.’s Proposed Amounts on Unsettled Flexibly Priced 

Contracts for Fiscal Years 2008, 2009, 2010, 2011, 2012, 2012, and 2013, 2191-2015T10160001 through 2191-

2013T10100003, August 31, 2016. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 13 

ACC also undermined its own ability to improve the Legacy and ASOM programs by awarding new Legacy 

implementation contracts without first taking the time to identify and incorporate the lessons learned from the 

prior contracts. ACC then awarded the $203.3 million ASOM contract to NCC prior to receiving the final 

assessment of the Legacy program, which meant that it denied itself the opportunity to fully assess NCC’s 

performance as a subcontractor to Imperatis. Instead, ACC decided to award the ASOM implementation 

contract to NCC after conducting only an assessment of its accounting system. 

After the award, NCC’s accounting system was found by DCAA and DCMA to have significant deficiencies that 

warranted its disapproval. However, instead of modifying the ASOM contract so the government could withhold 

funds until NCC had improved its accounting system, ACC appears to have elected to continue payments. 

Additionally, DOD has not provided evidence showing that NCC implemented a proposed corrective action plan 

or that it otherwise corrected the accounting system deficiencies that cast doubt on the accuracy of the costs it 

charged to U.S. taxpayers under the ASOM contract and as a subcontractor to Imperatis under the Legacy 

contracts. As noted above, our previous financial audit questioned more than $130 million in subcontract 

costs associated with the Legacy implementation contract. 

Lastly, although some MOD and MOI intelligence sites were successfully transferred to the Afghan ministries as 

part of the Legacy and ASOM programs, the fact remains the ANDSF still relies heavily on U.S. assistance to 

carry out its military intelligence mission. This suggests that at least the Legacy program did not accomplish its 

broader objective of developing a sustainable human intelligence capability and doctrine for source operations 

within the Afghan intelligence services. 

RECOMMENDATIONS 

To ensure that future contracts are awarded and monitored in accordance with established DOD and FAR 

guidance, SIGAR recommends that the Secretary of Defense: 

1. Review ACC’s, ARL’s, and CTTSO’s award and oversight of the Legacy and ASOM contracts to identify 

remedies to provide better oversight controls and performance measurements for future R&D 

contracts. 

2. Review ongoing ANSDF intelligence training and mentoring contracts, and incorporate into them 

requirements to enable the measurement and verification of contractor performance and contract 

outcomes, including training and mentoring results, and impacts on the ANDSF’s human intelligence 

capability. 

AGENCY COMMENTS 

We provided a draft of this report to DOD for comment. The Office of the Under Secretary of Defense for Policy 

(OUSD-P) provided informal comments, in which it concurred with both of our recommendations. Additionally, 

both OUSD-P and ACC–Aberdeen Proving Ground provided technical comments, which we incorporated into the 

report, as appropriate. ACC–Aberdeen Proving Ground‘s comments are reproduced in appendix II. We will 

publish OUSD-P’s formal comments in a separate product when we receive them. 

 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 14 

APPENDIX I - SCOPE AND METHODOLOGY 

This report examines the Department of Defense’s (DOD) efforts to develop the Afghan National Defense and 

Security Forces’ (ANDSF) intelligence capabilities through the Legacy research and development (R&D) training 

and mentoring program and the Afghanistan Source Operations Management (ASOM) training and mentoring 

program. The objectives of this audit were to determine the extent to which (1) Imperatis and New Century 

Consulting Limited (NCC) successfully performed the tasks required by the contracts and developed the ANDSF 

intelligence capability; (2) Army Contracting Command (ACC) awarded the Legacy and ASOM contracts in 

accordance with federal and DOD regulations; and (3) the Army Research Laboratory (ARL) and the Combatting 

Terrorism Technical Support Office (CTTSO) properly monitored contract performance and the Defense 

Contract Management Agency (DCMA) monitored contract costs. 

To determine the extent to which Imperatis and NCC successfully performed the tasks required by the 

contracts and developed the ANDSF intelligence capability, we reviewed training documents, course records, 

final reports, RAND National Defense Research Institute (RAND) assessment reports on the Legacy R&D 

program, and the Legacy and ASOM contracts. To determine whether NCC could demonstrate whether the 

programs increased capability of the ANDSF, we reviewed NCC compliance and performance reports; DOD 

reports on Afghanistan’s security and stability; ANDSF quarterly capability assessment data; performance 

requirements in the Legacy and ASOM contracts, methodologies, and performance work statements; RAND 

assessment reports; and ANDSF manning levels. We also interviewed NCC and Resolute Support officials. 

To determine the extent to which ACC awarded the Legacy and ASOM contracts in accordance with federal and 

DOD regulations, we reviewed the Federal Acquisition Regulation (FAR), ARL’s R&D broad agency 

announcement, the National Defense Authorization Acts of 2010 and 2011,41 the Consolidated Appropriations 

Act of 2012,42 DOD’s Afghanistan Resources Oversight Council Business Rules, and the Legacy and ASOM 

contracts. In addition, to determine whether ARL awarded the Legacy contracts to Imperatis on a competitive 

basis, we reviewed the broad agency announcement to determine whether the contract was awarded based on 

a peer and scientific review, as required by the FAR. We also reviewed the limits that the National Defense 

Authorization Acts and the Consolidated Appropriations Act placed on contract awards for initial research 

solicitations and use of the Afghanistan Security Forces Fund. In addition, we interviewed ARL, CTTSO, 

Resolute Support, and NCC officials, and sought evidence of approval for any exceptions for how ACC awarded 

the Legacy and ASOM contracts. 

To determine the extent to which the ARL and CTTSO properly monitored contract performance and DCMA 

monitored contract costs, we reviewed the FAR, ARL’s broad agency announcement, DOD’s contracting 

officer’s representative (COR) handbook,43 the U.S. Government Accountability Office’s Standards of Internal 

Controls,44 the January 2009 U.S. Counterinsurgency Guide,45 the National Defense Authorization Act of 2010, 

the Legacy and ASOM performance work statements, RAND assessment reports on the Legacy R&D program, 

NCC’s self-assessments, COR monthly assessments, and the Legacy and ASOM contracts. Additionally, we 

reviewed a list of paid Legacy and ASOM invoices, Legacy and Afghanistan Resources Oversight Committee 

 

41 National Defense Authorization Act for Fiscal Year 2010, Pub. L. No. 111-84 and Ike Skelton National Defense 

Authorization Act for Fiscal Year 2011, Pub. L. No. 111-383. 

42 Consolidated Appropriations Act, 2012, Pub. L. No. 112-74. 

43 Director, Defense Procurement and Acquisition Policy, Office of the Undersecretary Defense (Acquisition Technology & 

Logistics), Contracting Officer’s Representative Handbook. 

44 U.S. Government Accountability Office, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1, 

November 1, 1999. 

45 U.S. Government Counterinsurgency Guide, January 2009. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 15 

meeting minutes, and Imperatis’s and NCC’s Legacy and ASOM progress and final reports. We also interviewed 

officials from ARL, CTTSO, Defense Contract Audit Agency, Imperatis, and NCC. 

We did not rely on computer-processed data for the purpose of the audit objectives. We assessed internal 

controls to determine the extent to which DOD had procedures in place to administer and oversee the Legacy 

R&D and ASOM contracts. The results of our assessment are included in the body of the report. 

We conducted our audit work in Washington, D.C., and in Kabul, Afghanistan, from August 2015 through July 

2017 in accordance with generally accepted government auditing standards. Those standards require that we 

plan and perform the audit to obtain sufficient and appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a 

reasonable basis for our findings and conclusions based on our audit objectives. SIGAR performed this audit 

under the authority of Public Law No. 110-181, as amended, and the Inspector General Act of 1978, as 

amended. 

 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 16 

APPENDIX II - COMMENTS FROM THE ARMY CONTRACTING COMMAND–

ABERDEEN PROVING GROUND 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 17 

SIGAR Comment 1 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 18 

SIGAR Comment 2 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 19 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 20 

SIGAR’s Response to Army Contracting Command (ACC) Aberdeen Proving Ground’s Comments 

SIGAR Comment 1. In its comments, ACC characterized the accounting system survey that ACC conducted on 

New Century Consulting Inc.’s (NCC) accounting system as “very simplistic.” Regardless of the “unique ACCAPG [ACC–Aberdeen Proving Ground] process for completing the evaluation and survey” or that “NCC’s cost 

accounting system was reviewed by three (3) different independent accounting firms prior to the contract 

award,” the audit firms that conducted the pre-award reviews did not identify the significant issues with NCC’s 

accounting system that were found during the post-award audit. As noted in our report, the Defense Contract 

Audit Agency (DCAA) conducted a post-contract award audit of NCC’s accounting system and determined that 

the system was not adequate for accumulating and billing costs for U.S. government contracts. Based on 

DCAA’s results, in 2014, the Defense Contract Management Agency disapproved NCC’s accounting system 

SIGAR Comment 2. Although ACC insists that it “does not generate contracting requirements, control the timing 

of contracting requirements, make decisions regarding the continuation of external programs, or determining 

the benchmarks for programmatic success,” the command is responsible for performing “all necessary actions 

for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interest of 

the United States in its contractual relationship, in accordance with the FAR [Federal Acquisition 

Regulation].”46 As such, it was ACC’s responsibility to ensure, in conjunction with the Combatting Terrorism 

Technical Support Office, that both the Legacy and ASOM contracts included requirements that would have 

enabled the Department of Defense to measure the programs’ success and address program issues, as 

appropriate. 

 

 

46 See FAR 1.602-2. 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 21 

APPENDIX III - ACKNOWLEDGEMENTS 

Chris Borgeson, Senior Program Manager 

Steven Haughton, Senior Inspection Manager/Auditor-in-Charge 

Benjamin Mayer, Program Analyst 

 
SIGAR 17-57-AR/Legacy and ASOM Programs Page 22 

This performance audit was conducted 

under project code SIGAR-111A. 
Obtaining Copies of SIGAR 

Reports and Testimonies 

To Report Fraud, Waste, and 

Abuse in Afghanistan 

Reconstruction Programs 

Public Affairs 

SIGAR’s Mission 

The mission of the Special Inspector General for Afghanistan 

Reconstruction (SIGAR) is to enhance oversight of programs for the 

reconstruction of Afghanistan by conducting independent and 

objective audits, inspections, and investigations on the use of 

taxpayer dollars and related funds. SIGAR works to provide accurate 

and balanced information, evaluations, analysis, and 

recommendations to help the U.S. Congress, U.S. agencies, and 

other decision-makers to make informed oversight, policy, and 

funding decisions to: 

 improve effectiveness of the overall reconstruction 

strategy and its component programs; 

 improve management and accountability over funds 

administered by U.S. and Afghan agencies and their 

contractors; 

 improve contracting and contract management 

processes; 

 prevent fraud, waste, and abuse; and 

 advance U.S. interests in reconstructing Afghanistan. 

To obtain copies of SIGAR documents at no cost, go to SIGAR’s Web 

site (www.sigar.mil). SIGAR posts all publicly released reports, 

testimonies, and correspondence on its Web site. 

To help prevent fraud, waste, and abuse by reporting allegations of 

fraud, waste, abuse, mismanagement, and reprisal, contact SIGAR’s 

hotline: 

 Web: www.sigar.mil/fraud 

 Email: sigar.pentagon.inv.mbx.hotline@mail.mil 

 Phone Afghanistan: +93 (0) 700-10-7300 

 Phone DSN Afghanistan: 318-237-3912 ext. 7303 

 Phone International: +1-866-329-8893 

 Phone DSN International: 312-664-0378 

 U.S. fax: +1-703-601-4065 

Public Affairs Officer 

 Phone: 703-545-5974 

 Email: sigar.pentagon.ccr.mbx.public-affairs@mail.mil 

 Mail: SIGAR Public Affairs 

2530 Crystal Drive 

Arlington, VA 22202