SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative

OFFICE OF THE SPECIAL INSPECTOR GENERAL FOR AFGHANISTAN RECONSTRUCTION

Progress Made Toward Increased Stability under 

USAID’s Afghanistan Stabilization Initiative-East 

Program but Transition to Long Term Development 

Efforts Not Yet Achieved

June 29, 2012
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative

SIGAR

Special Inspector General for Afghanistan Reconstruction

SIGAR Audit-12-11 June 2012

Progress Made Toward Increased Stability under USAID’s 

Afghanistan Stabilization Initiative-East Program but Transition 

to Long Term Development Efforts Not Yet Achieved

What SIGAR Reviewed

In June 2009, the U.S. Agency for International Development’s (USAID) Office of Transition Initiatives (OTI) awarded a 3 year, 

$151 million task order to Development Alternatives, Inc., (DAI) to implement the Afghanistan Stabilization Initiative-East (ASIEast) program in Afghanistan. ASI-East focuses on awarding grant activities to local implementers in 10 districts distributed across 

four provinces in eastern Afghanistan. A key objective of ASI-East is to support the “hold” phase of the U.S. Counterinsurgency 

(COIN) strategy, which consists of “clear, hold, and build” phases designed to permanently stabilize targeted districts and 

communities. After task order award, OTI asked DAI to develop a more strategic approach to stabilization programming designed to 

target specific sources of instability at the district level. DAI began to implement this new approach, the District Stability Framework 

(DSF), in February 2010. In February 2012, OTI awarded a new 3-year, $161.5 million follow-on task order to Creative Associates 

International, Inc., to continue ASI programming efforts in eastern and southern Afghanistan under a single implementing partner.

This report assesses to what extent (1) expended ASI-East costs were allowable, allocable, and reasonable; (2) performance oversight, 

monitoring, and evaluation systems have been implemented; and (3) progress has been made toward transitioning districts to the 

“build” phase of the COIN strategy. To accomplish these objectives, we interviewed Afghanistan-based staff from USAID OTI, 

USAID’s Stabilization Unit, DAI, and third-party monitoring staff hired through DAI to provide independent monitoring and 

evaluation of the program. We also interviewed USAID OTI staff based in Washington, D.C. We reviewed the ASI-East task order, 

related procurement documents, and a sample of 20 grants. We reviewed a range of performance management documents, data, and 

systems. We reviewed, analyzed, and tested select DAI billing and timekeeping policies and procedures and incurred costs to 

determine if program costs were allowable, allocable, and reasonable. We conducted work in Kabul, Afghanistan, and Washington, 

D.C., from May 2011 to June 2012 in accordance with generally accepted government auditing standards.

What SIGAR Found

SIGAR found that although incurred costs under the ASI-East task order were generally allowable, allocable, and reasonable, certain 

cost-related issues need to be addressed. These issues included program spending that was marked by high operating costs, more than

$590,000 in questionable program costs, and a limited number of timekeeping and billing deficiencies, which increased the risk of 

inappropriate charges.

DAI implemented a range of program oversight, monitoring, and evaluation systems; however, final program results remain to be 

determined, and certain administrative issues relating to program oversight need to be addressed in the follow-on task order award. 

USAID’s efforts to evaluate ASI-East’s final program results will be critical to assisting with short and long-term decisions regarding 

whether and how extensively the DSF methodology should be implemented in stabilization efforts, such as USAID’s Stabilization in

Key Areas program. Program monitoring and evaluation centered on the three-tier system called for by the DSF methodology. 

Preliminary results indicate that ASI-East activities have been successfully implemented and that the impact of this program on 

stability at the district level has been positive, but overall stability remains poor across ASI-East’s 10 programming districts based on 

seven leading indicators of stability developed by DAI and its monitoring and evaluation subcontractor.

Despite nearly 3 years of program efforts, none of ASI-East’s target districts have transitioned to the “build” phase of the COIN 

strategy, which is designed to solidify the gains during the “hold” phase of COIN operations. OTI has only recently drafted districtlevel disengagement criteria. An exit strategy for OTI programming in Afghanistan remains to be developed under the follow-on task 

order for ASI. These efforts will need to be integrated with planned improvements and evaluations of the DSF methodology.

What SIGAR Recommends

SIGAR recommends that USAID OTI (1) review the balance between project and operating costs resulting from the application of the 

DSF methodology to ASI-East programming decisions, (2) address identified timekeeping and billing deficiencies, (3) review the 

questioned costs identified by SIGAR, (4) include outcome indicators under the DSF, (5) produce a lessons learned summary of ASIEast’s implementation of the DSF, and (6) develop approved district-level disengagement criteria and a related exit strategy for OTI 

programming in Afghanistan. When preparing the final report, SIGAR considered comments from USAID. USAID generally 

concurred with the recommendations and noted the actions it has taken or will take to address SIGAR’s recommendations. Based on 

USAID’s comments, SIGAR revised recommendations four and five to also address the Stabilization Unit Chief. Through its 

Measuring Impact of Stabilization Initiatives program, USAID’s Stabilization Unit in Kabul is responsible for monitoring and

evaluating USAID’’s stabilization efforts, including USAID’s implementation of the DSF.

For more information contact: SIGAR Public Affairs at (703) 545-5974 or sigar.pentagon.ccr.mbx.public-affairs@mail.mil
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page i

OFFICE OF THE SPECIAL INSPECTOR GENERAL FOR AFGHANISTAN RECONSTRUCTION

June 29, 2012

The Honorable Hillary Rodham Clinton

Secretary of State

The Honorable Leon Panetta

Secretary of Defense

The Honorable Ryan C. Crocker

U.S. Ambassador to Afghanistan

General John R. Allen

Commander, U.S. Forces – Afghanistan, and 

Commander, International Security Assistance Force

Dr. Rajiv Shah

Administrator, U.S. Agency for International Development

Dr. S. Ken Yamashita

USAID Mission Director for Afghanistan

Mr. Robert Jenkins

Director, Bureau for Democracy, Conflict, and Humanitarian 

Assistance/Office of Transition Initiatives (DCHA/OTI)

This report discusses the results of the Office of the Special Inspector General for Afghanistan 

Reconstruction’s (SIGAR) audit of the U.S. Agency for International Development’s (USAID) task order 

with Development Alternatives, Inc., to implement the Afghanistan Stabilization Initiative-East program. 

This report includes six recommendations to USAID that address a range of issues including questioned 

costs, programming and evaluation methodology, and transition planning.

When preparing the final report, we considered comments from USAID’s Office of Transition Initiatives. 

These comments are reproduced in appendix III. SIGAR conducted this performance audit under the 

authority of Public Law No. 110-181, as amended, the Inspector General Act of 1978, and the Inspector 

General Act of 2008.

Steven J Trent

Acting Special Inspector General

for Afghanistan Reconstruction
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page ii

TABLE OF CONTENTS

Background...................................................................................................................................................2

Program Spending Marked By High Operating Expenses and Certain 

Questionable Costs.................................................................................................................................4

Program Oversight, Monitoring, and Evaluation Systems Implemented but 

Certain Administrative Requirements not Met.......................................................................................7

OTI Lacks Approved Disengagement Criteria and an Overall program Exit 

Strategy to Move Districts from the Hold to Build Phase of the COIN 

Strategy.................................................................................................................................................11

Conclusions.................................................................................................................................................13

Recommendations.......................................................................................................................................13

Comments...................................................................................................................................................14

Appendix I: Scope and Methodology ........................................................................................................16

Appendix II: Preliminary Monitoring and Evaluation Results...................................................................18

Appendix III: Comments from the U.S. Agency for International Development......................................20

TABLES

Table 1: Administrative and Management Issues Relating to Program Oversight.......................................8

Table I: DAI Monitoring and Evaluation Preliminary Results...................................................................18

Table II: Altai Consulting Monitoring and Evaluation Preliminary Results ..............................................19

FIGURES 

Figure 1: DSF System Flowchart..................................................................................................................3

Figure 2: ASI-East Funding and Distribution (as of October 2011)............................................................4
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page iii

ABBREVIATIONS AND ACRONYMS

ASI Afghanistan Stabilization Initiative

COIN Counterinsurgency

COTR Contracting Officer Technical Representative

DAI Development Alternatives, Inc.

DCHA Bureau for Democracy, Conflict, and Humanitarian Assistance

DSF District Stability Framework

FAR Federal Acquisition Regulation

GIRoA Government of the Islamic Republic of Afghanistan

IQC Indefinite Quantity Contract 

MISTI Measuring Impact of Stabilization Initiatives

OTI Office of Transition Initiatives

SIGAR Special Inspector General for Afghanistan Reconstruction 

SWIFT Support Which Implements Fast Transition 

TAMIS Technical Administrative Management Information System

USAID U.S. Agency for International Development
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 1

Progress Made Toward Increased Stability under USAID’s Afghanistan 

Stabilization Initiative–East Program but Transition to Long Term 

Development Efforts Not Yet Achieved

A key objective of the Afghanistan Stabilization Initiative-East (ASI-East) is to support the coalition

Counterinsurgency (COIN) strategy, which includes “clear, hold, and build” phases designed to 

permanently stabilize target districts and communities in areas previously controlled by insurgents.1 The 

ASI-East program focuses on implementing small-scale community development and media grants in ten 

districts distributed across four provinces in eastern Afghanistan during the “hold” phase of the strategy,

which occurs before long-term development efforts are initiated in the “build” phase.2 ASI-East projects 

are intended to build allegiance and confidence between local communities and the Government of the 

Islamic Republic of Afghanistan (GIRoA) in target regions and districts. The intended outcome is to 

achieve immediate employment generation, improve the community’s infrastructure, and increase access 

to public services with the overall goal of improving community and government capacity and public 

support for GIRoA.

USAID’s Bureau for Democracy, Conflict, and Humanitarian Assistance/Office of Transition Initiatives 

(DCHA/OTI) implements the ASI program in Afghanistan. OTI works on the ground in countries to 

provide fast, flexible, short-term assistance targeted at key political transition and stabilization needs.

OTI works closely with USAID regional bureaus and missions and with other U.S. counterparts to 

identify programs that complement other assistance efforts and lay a foundation for longer-term 

development. In June 2009, OTI awarded a task order to Development Alternatives, Inc. (DAI) under 

USAID’s global Support Which Implements Fast Transition III (SWIFT III) indefinite quantity contract 

(IQC). The ASI-East program is a 3-year task order with a program ceiling of $151 million. After task 

order award, OTI asked DAI to develop a more strategic approach to stabilization programming designed 

to target specific sources of instability at the district level. DAI began to implement this new approach, 

the District Stability Framework (DSF), in February 2010. The DSF approach represents a shared 

terminology and a unified assessment process for identifying and resolving sources of instability in target 

areas.

This report assesses to what extent (1) expended ASI-East costs were allowable, allocable, and 

reasonable; (2) performance oversight, monitoring, and evaluation systems have been implemented; and 

(3) progress has been made toward transitioning districts to the “build” phase of the COIN strategy. 

To accomplish these objectives, we interviewed staff in Afghanistan from USAID OTI, USAID’s 

Stabilization Unit, DAI, and third-party contractor staff hired by DAI to provide independent monitoring 

and evaluation of the program. We also interviewed USAID OTI staff in Washington, D.C. including the 

 1

The COIN strategy includes working closely with Afghan counterparts to rid target areas of insurgents (“clear” 

phase), provide humanitarian relief and initial government services after it is cleared (“hold” phase), and applying 

coordinated civil-military development efforts to reinforce security gains ( “build” phase). The ASI-E program aims 

to support the transition from “hold” to “build” by working with government officials and through traditional 

community leadership structures to identify sources of instability and implement projects that reinforce security 

gains made by coalition forces. These projects lay the groundwork for long-term USAID development projects. 2 Throughout this report, we refer to 10 districts when discussing ASI-East program activity. This number of 

districts was accurate as of October 2011 and these districts do represent the core of ASI-East programming efforts. 

Through an ongoing assessment process, however, OTI made the decision to exit and enter districts at multiple 

points in the program. As of May 2012, the ASI-East program had active projects in 15 districts in Regional 

Command-East and 3 districts in Regional Command-South. 
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 2

cognizant contracting officer and contracting officer’s technical representative (COTR) for the ASI-East 

task order. We reviewed the ASI-East task order and related procurement documents including the 

SWIFT III IQC. We reviewed a range of performance management documents, data, and systems 

including DAI’s annual and quarterly performance reports, activity tracking systems, and a three-tier 

monitoring and evaluation system and related reports and studies. We reviewed, analyzed, and performed 

tests of select DAI timekeeping and billing policies and procedures and incurred costs to determine if 

program costs were allowable, allocable, and reasonable per Federal Acquisition Regulation (FAR)

principles.

3 We reviewed DAI’s grant making and oversight procedures for a sample of 20 completed 

and closed grants awarded under the task order. We also attended the 3-day DSF training course 

delivered by the COIN academy with support from trainers provided by OTI to gain a deeper 

understanding of the DSF as a new stabilization programming tool. We conducted our work in Kabul, 

Afghanistan, and Washington, D.C., from May 2011 to June 2012 in accordance with generally accepted 

government auditing standards.

BACKGROUND

The ASI-East task order required DAI to meet a phased program schedule covering pre-deployment/start 

up, full implementation, and project close out. DAI project close out efforts have not yet begun because

OTI recently extended ASI-East’s performance period by 3 months to September 2012. This extension 

was designed to allow a more orderly transition to a follow-on task order recently awarded by OTI. The 

intent of the follow-on task order is to continue the ASI-East and parallel programming efforts in ASISouth under a single task order implemented by one implementing partner. The new task order was 

awarded to Creative Associates International, Inc., in February 2012, with a ceiling price of 

$161.5 million. The new task order has an effective start date of March 2012 and an estimated 

completion date of February 2015. 

With regard to performance expectations, the ASI-East task order has two objectives: (1) create 

conditions that build confidence between communities and GIRoA through improvement of the economic 

and social environment in the region, and (2) increase public access and awareness to information about 

GIRoA’s social, economic, and political activities and policies in Afghanistan. By design, the task order 

requires DAI to be responsive to changes in program strategy by expanding, decreasing, or changing its 

program activities with as little as a few days notice. Consequently, these two objectives did not have 

associated performance goals and targets because the task order was viewed by OTI as a tactical 

programming tool and program direction might need to change on very short notice. 

At the time of award, DAI used the military Tactical Conflict Assessment and Planning Framework tool 

to identify and select stabilization projects for the ASI-East program. The framework sought the 

perceptions of local nationals through the use of a survey to determine specific sources of instability. 

Under this approach, DAI awarded 49 initial grants under the ASI-East program totaling $1.3 million. In 

December 2010, through the ASI-East task order, OTI implemented DSF to identify specific sources of 

instability and program resources accordingly. Through October 2011, DAI awarded an additional 412

grants using the DSF methodology at a total additional cost of about $21 million.

 3

Costs are allowable if they are determined to be allocable and reasonable, among other factors. A cost is allocable 

to a Federal award if it is treated consistently with other costs incurred for the same purpose in like circumstances, 

amongst other factors. A cost is considered reasonable if in its nature or amount it does not exceed that which would 

be incurred by a prudent person under the circumstances prevailing at the time the decision was made to incur the 

cost.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 3

DSF Provides a Uniform Approach for Designing and Evaluating Stability Operations

Programs

After the task order award, OTI determined that it wanted DAI to implement the ASI-East program under 

a new methodology that departed from the relatively rudimentary Tactical Conflict Assessment and 

Planning Framework tool and moved toward a more rigorous programming and monitoring and 

evaluation methodology. OTI’s DSF methodology did not follow the “burn-rate” approach to program 

spending, which emphasizes spending funds as quickly as feasible. In contrast, the DSF programming 

model takes a more strategic view toward spending, which DAI’s Chief of Party described as a “small and 

smart” approach to stabilization programming. Both OTI and DAI agreed on the need for this new 

programming methodology and, in coordination with COIN Training Academy, developed the DSF in 

early 2010 under the ASI-East program as an integrated civilian-military methodology for identifying, 

implementing, and measuring the impact of stabilization activities. 

DSF provides civilians, military, and GIRoA officials with a shared terminology and a unified assessment 

process for gauging the stability picture across a district or area, identifying local sources of instability, 

and developing coherent and tailored programs and interventions to address them. DSF emphasizes 

assessing results and modifying and adapting program activity based on empirical data. The DSF system 

requires a three-tier monitoring and evaluation system, which measures output, impact, and overall 

stability conditions at the district level. DSF implementation efforts have included developing several 

iterations of the methodology and training more than 1,200 civilian and military personnel. 

Figure 1 shows the basic elements of the DSF methodology and the four-phase process, which starts with 

situational awareness and concludes with monitoring and evaluation efforts with a feedback loop back to 

the first phase in the process. The DSF is implemented at the district-level by a civilian and military 

working group charged with designing and implementing a set of projects to address specific sources of 

instability that are aligned with stabilization programming goals.

Figure 1: DSF System Flowchart

Collect and 

Analyze 

Situational 

Awareness 

Data

Cultural 

Matrix

Local 

Perceptions

Factors 

Matrix

Sources of 

Instability 

Analysis

Tactical 

Stability 

Matrix 

Tactical 

Stability 

Matrix 

Activity 

Design 

Worksheets

Monitoring 

and 

Evaluation 

Matrix

Stability 

Indicators

DSF Roadmap

SITUATIONAL 

AWARENESS ANALYSIS DESIGN

MONITORING & 

EVALUATION

Sync 

Matrix

Source: USAID OTI
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 4

Grant Activity and Funding

ASI-East program resources are largely distributed through a “grants under contract” approach, which 

relies on DAI as the contractor to award, administer, assess, and close-out grants to local community 

implementers. As of October 16, 2011, DAI awarded 461 grants at a total cost of about $21.2 million in 

ten districts distributed across Nangarhar, Kunar, Wardak, and Paktika provinces in eastern Afghanistan.

These grants cover a wide-range of projects, such as improvements in agriculture and irrigation;

assessments of community and district availability of services; media outreach efforts that showcase 

GIRoA’s involvement in providing needed services; and a variety of infrastructure projects, including

repairing and building new roads, retaining walls, flood ditches, and sanitation facilities. Figure 2

provides a breakout of ASI-East activity funding and the number of grants awarded through October 

2011.

Figure 2: ASI-East Funding and Distribution (as of October 2011)

Source: SIGAR analysis of DAI provided grants data.

Note: Shura/jirga is an Afghan meeting of a decision making body (typically tribal/village elders). In the context of the ASIEast program, these meetings were convened to discuss grant-funded activities. Scribe activities were used to gather and 

record certain demographic information.

PROGRAM SPENDING MARKED BY HIGH OPERATING EXPENSES AND

CERTAIN QUESTIONABLE COSTS

While DAI did not exceed the $151 million cost ceiling for the ASI-East task order, program expenditures 

largely covered operating expenses, as opposed to activity level spending. DAI officials noted that 

relatively low activity spending levels represented a “small and smart” approach to stabilization, which 

OTI officials endorsed based on the application of the DSF methodology to ASI-East programming 

decisions. SIGAR identified over $590,000 in questionable program costs that will need to be reviewed 

by USAID to determine whether these costs were allowable, allocable, and reasonable. Finally, SIGAR 

noted a limited number of procedural and compliance deficiencies associated with two of DAI’s key 

business systems.

Agriculture 

and irrigation 

$3,400,008 

(64 activities)

Assessments 

$2,725,605 

(40 activities)

Call-in line 

$247,171

(5 activities)

Event planning 

$681,001 

(47 activities)

Infrastructure 

$7,294,469 

Media (89 activities)

$1,229,443

(49 activities)

Procurement 

$162,866 

(13 activities)

Scribe

$130,158

(10 activities)

Shura/Jirga

$606,828 

(55 activities)

Technical 

assistance

$1,840,337

(6 activities)

Training

 $2,864,351

(83 activities)
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 5

ASI-East Implementation Marked by Low Activity Spending and High Operating Costs

SIGAR found that DAI had spent only 25 percent of expended funds on project activities through October

2011, with the balance of funds spent on operating expenses. This spending pattern was not consistent 

with the implied spending ratios in the task order, as stipulated at the time of award. Specifically, the task 

order stipulated that up to $83 million could be spent on program activities and up to $68 million could be 

spent on operating costs.4 Therefore, the implied spending ratio was 55 percent on program activities and 

45 percent on operating costs. 

DAI representatives noted that the ASI-East program has been implemented under a “small and smart” 

approach to stabilization based on the application of the DSF methodology. One result of the application

of this “small and smart” approach to programming is that the ratio of project costs to operating costs is 

not in line with the implied spending ratio in the task order. DAI officials noted that the “small and 

smart” approach to programming was approved by OTI officials because it represents the logical outcome 

of applying the DSF to programming decisions. DAI officials also noted that this approach contrasts with 

the “burn rate” approach typically used by other assistance programs. They also noted that unlike many 

other USAID programs which often involve large capital expenditures for infrastructure development, 

ASI-East’s use of small grants to local implementers tended to depress program spending levels. 

DAI officials noted that administering and overseeing these small grants requires the same level of effort 

and operating expenses as administering and overseeing large dollar contracts and assistance instruments. 

DAI also stated the required operating expenditures needed to establish and operate the ASI-East program 

were not excessive or unreasonable. Specific operating costs included office and living space rental in 

Kabul and locations across Afghanistan, security costs, and the large number of personnel needed to 

implement and oversee the hundreds of grants awarded under the program. 

DAI Charged More Than $590,000 in Questionable Costs to the ASI-East Program

We generally found that the ASI-East incurred costs were allowable, allocable, and reasonable with the 

following exceptions: 

• OTI originally contracted directly with a contractor (QED) to provide independent monitoring 

and evaluation services for the ASI-East program. QED subcontracted with Altai Consulting to 

provide these services. The contract between OTI and QED expired in July 2011. To provide 

follow-on monitoring and evaluation services, OTI requested that DAI subcontract with Altai 

Consulting to perform additional cycles of monitoring and evaluation in three districts, at a total 

cost of over $500,000. OTI staff developed a conflict of interest mitigation plan to address 

questions of independence and oversight raised by this arrangement. The plan called for OTI to

provide oversight of the subcontract and directly receive all subcontract deliverables. DAI’s role 

in administering the subcontract was essentially limited to reviewing and approving invoices for 

payment. In response to the draft of this report, OTI’s Contracting Officer provided additional 

information regarding the basis for making a determination that the over $500,000 was allowable. 

The response, however, did not address the subcontractor oversight. The mitigation plan that was 

implemented by OTI between DAI and Altai limited DAI’s ability to provide adequate 

subcontractor oversight, as required by FAR 42.202(e)(2) . DAI’s role in administering the 

subcontract was essentially limited to reviewing and approving invoices for payment. Therefore, 

we continue to question whether the cost of over $500,000 was allowable. 

• We found that the $3,400 cost of a household effects demobilization shipment that was related to 

a DAI Iraq project was incorrectly charged to the ASI-East program. Per the FAR, such costs are 

unallocable.

 4

Dollar figures based on a revised budget approved under modification 4, approved July 27, 2010, to the task order.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 6

• DAI maintained full comprehensive and war insurance on purchased armored security vehicle 

while they were in the United States waiting for Afghan customs approval for importation. 

Property damage insurance alone likely would have provided sufficient coverage and resulted in 

an estimated cost savings of $50,000 if DAI had gone with this lower cost option. These 

additional costs appear to be unreasonable and therefore unallowable per FAR requirements. 

• While waiting for its purchased armored vehicles to arrive in Afghanistan, DAI leased armored 

vehicles, which were covered with comprehensive and war insurance per the terms of the rental 

agreement. During that time, DAI traded in old leased vehicles for newer ones. When this 

occurred, DAI added the new vehicles to its insurance policy; however, it did not remove the 

leased armored vehicles no longer being used. As a result, DAI billed $38,500 in 2011 for 

insurance costs on vehicles that were not being utilized by the project; these additional costs 

appear to be unreasonable and therefore unallowable per FAR requirements.

Relatively Minor Deficiencies in DAI’s Time Keeping and Billing Related Processes Noted

We reviewed, analyzed, and performed limited testing on controls of DAI’s time keeping and billing 

systems. We followed the General Accounting Office Standards for Internal Control in the Federal 

Government, which states that physical controls must be in place to secure and safeguard vulnerable 

assets at risk of loss or unauthorized use.

5 Generally, DAI’s time keeping and billing policies and 

procedures functioned properly, with a few minor exceptions:

• With regard to DAI Afghan staff timesheets, we observed multiple occurrences where individuals 

entered time for other individuals on physical timesheets displayed in the lobby of DAI’s 

headquarters in Kabul.

• DAI supervisors sign Afghan timesheets on a monthly basis, as opposed to the bi-weekly reviews 

conducted of expatriate staff timesheets. 

• We noted several instances where DAI did not initially furnish complete supporting 

documentation for paid invoices. For example, travel invoices did not have the proper contracting 

officer approval letter attached, procurement invoices did not have all bidding documentation 

attached, and a brokerage fee had no attached invoice. DAI was able to furnish all supporting 

documentation once requested. DAI noted that while it does not always keep all supporting 

documentation attached to invoices, supporting documentation is available in a separate location.

SIGAR questions this arrangement, which does not provide adequate assurance that OTI 

contracting staff have sufficient supporting information available when reviewing and approving 

invoice claims.

• When leasing items, contractors generally perform a buy versus lease cost analysis to ensure that 

the government receives the best value for the money. This cost analysis is not only used for 

contractor purchases, but also for subcontracter purchases. DAI’s security subcontractor, 

Edinburgh International, leased several items including computers and printers, body armor, night 

vision goggles, and soft skin vehicles without conducting a buy versus lease analysis. In certain 

cases it would have been cheaper to procure leased items. For example, SIGAR estimates that 

$20,000 could have been saved by purchasing computers and printers directly compared to 

leasing the same equipment. According to DAI, Edinburgh International did not conduct buy 

versus lease cost analyses for the items discussed above.

 5

U.S. General Accounting Office, Standards for Internal Control in the Federal Government, GAO-AIMD-00-

21.3.1 (Washington, D.C.: Nov. 1999). See also Development Alternatives, Inc.: 2010 Financial Statement and 

Audit Results (Ernst&Young, May 5, 2011). DAI’s external auditors’ opinions of its financial statements for 2009 

and 2010 considered DAI’s internal controls when performing the financial audits.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 7

PROGRAM OVERSIGHT, MONITORING, AND EVALUATION SYSTEMS 

IMPLEMENTED BUT CERTAIN ADMINISTRATIVE REQUIREMENTS NOT MET

DAI and OTI implemented program oversight and monitoring and evaluation systems through a variety of 

means, including a “grants under contract” tool managed by DAI that was adequately implemented;

however, certain administrative requirements of the ASI-East task order related to project oversight were 

not fully met.6 Program monitoring and evaluation centered on the three-tier evaluation system called for 

by the DSF methodology. Developing this three-tier monitoring and evaluation system took 16 months 

and final output, impact, and overall stability results are not expected until sometime in 2012. 

Preliminary results, however, show significant and positive results at the project level while instability 

remains a problem across ASI-East’s ten program districts in general. USAID’s efforts to evaluate final 

ASI-East’s program results will be critical to informing short and long-term decisions regarding whether 

and how extensively this new methodology should be implemented in future stabilization efforts. 

Although we noted significant levels of program oversight, monitoring, and evaluation, we also identified

a limited number of administrative issues relating to program oversight.

DAI and OTI Provided Program Oversight

DAI and OTI staff provided oversight of ASI-East program activities in ten districts across the Regional 

Command East platform through a variety of means. One key component of provided oversight is the

“grants under contract” tool that is managed by 25 DAI expatriate staff overseeing 244 DAI Afghan 

employees based in Kabul, Jalalabad, and ten District Development Teams. This process requires that a 

series of award, implementation, and close-out procedures be followed for each grant. Specific 

requirements include a document checklist, which identifies required grant documents, procurement 

documents, financial documents, and reporting and evaluation documents. In addition, financial action 

status, OTI database inputs, and approval signatures from DAI and OTI representatives are required for 

each grant. To assess whether DAI and OTI exercised adequate controls and oversight, we reviewed a 

judgmental sample of 20 completed and closed grants awarded by DAI, as of December 2011, and found 

controls to be adequate for each activity. Specifically, we found that all activity folders contained the 

information required by DAI for documenting the grant award, implementation, and close-out process.

Located in Washington, D.C., OTI’s COTR is responsible for overseeing the ASI-East program and 

approving the payment of all project invoices. The COTR’s oversight efforts are supplemented by an 

extensive network of OTI staff in Afghanistan, which includes a Country Representative based at the 

embassy in Kabul and a Field Operations and Project Manager and Deputy Regional Representative 

located in Jalalabad.

7 These individuals provide additional oversight by working directly with DAI staff 

in Kabul and Jalalabad and with Regional Command East representatives. All OTI staff have access to 

DAI-prepared weekly, annual, and quarterly performance reports, which typically include program 

description, country situation, program activity highlights, successes and challenges, activity summary, 

program appraisal, and DSF activity status.

OTI also provided oversight of the ASI-East program by conducting periodic program performance 

reviews and strategic review sessions. These reviews are designed to provide a review of task order 

progress and key challenges by staff not directly associated with the program. OTI conducted program 

performance reviews of ASI-East’s activities in November 2010 and October 2011. Key findings from 

these two reviews include the following:

 6

The ASI-East task order outlines the need for DAI in coordination with USAID/OTI to institute a (1) a monitoring 

and evaluation system; (2) weekly, quarterly, and annual performance reports; (3) participation in strategic review 

sessions; and (4) the maintenance of the OTI activity database as the backbone of the performance management 

system.

7

As of October 2011, seven OTI staff members were located in Kabul, two were located at the regional commandEast platform in Bagram Air Force base, and six others were stationed at other Coalition Forces forward operating 

bases in the districts in which OTI works.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 8

• The USAID Kabul Mission should maintain stabilization programming capacity, especially in the 

East and South regions, throughout transition over the next 3 years, despite engagement 

constraints. 

• OTI and DAI need to prepare exit criteria and transition strategies to help move districts from the 

“hold” to the “build” phase of the COIN strategy. 

• OTI should develop a new task order under the SWIFT III IQC with a single implementing 

partner to consolidate ASI-East and ASI-South.

8

To review and adjust program activities, OTI and DAI also conducted bi-annual strategic review sessions. 

The most recent session was conducted in Kabul in October 2011 and was widely attended by OTI and 

DAI staff from Washington, D.C., and Kabul and Jalalabad, as well as district field personnel. 

Participants discussed program strategy and monitoring and evaluation efforts, including DAI’s first 

“Lessons Learned” report.9 This report analyzed 90 program activities that were closed and completed in 

order to identify best practices to apply to future activities. 

Certain Program Oversight Elements Not Fully Implemented

SIGAR found that certain administrative and management requirements relating to program oversight had 

not been fully met, as shown in table 1.

Table 1: Administrative and Management Issues Relating to Program Oversight

Issue Description

Data validity concerns The IQC and task order state that DAI should maintain and use the USAID/OTI Activity 

Database as the “backbone” to generate, develop, manage, monitor, evaluate, and report 

on program activities. This database has several fields that need to be populated with 

information that captures the status of the program activity. Per an understanding with OTI 

officials, DAI used its Technical Administrative Management Information System (TAMIS) 

to track program activities. Information from this system is being modified on a weekly 

basis in order to provide a “data dump” into the USAID/OTI Activity Database. We found 

that OTI accepts the data from DAI’s TAMIS system without conducting periodic data 

reliability checks. OTI officials noted that they monitor the data uploaded into the OTI 

Activity Database; however, we were told that this monitoring is performed in Washington, 

D.C. and not locally in Kabul, where better checks on data reliability could be performed. 

SIGAR reviewed data for similar periods in the OTI Activity Database and TAMIS and 

found discrepancies that overstated TAMIS reported activity. While these discrepancies 

were ultimately explained by OTI officials, we believe data validation should occur at the 

field level to ensure consistent reporting in both systems. 

Fraud reporting We noted three instances of fraud not reported to USAID officials in a timely manner as 

required by USAID policy and FAR 11,12. Fraud-related costs, per DAI’s disclosed 

practices and USAID guidance, are to be set aside in an unallowable account so that these 

costs are not billed to and reimbursed by the U.S. government. We noted that DAI 

identified three instances of fraud between December 12, 2010, and April 19, 2011, that 

totaled approximately $1,000. In two instances, the fraud was discovered before the 

expense was reimbursed to the employee and captured in DAI’s financial software. For 

the final instance, the fraudulent amount was processed and captured in DAI’s financial 

software. After the fraudulent amount was discovered, DAI asserts that the amount was 

charged to a non-billable account and was not billed to USAID. However, we have not 

been able to confirm whether the unallowable cost was actually charged to a non-billable 

account. DAI has since changed its policy with regards to reporting fraud based on 

SIGAR’s observations. DAI will now report all instances of fraud, regardless of amount or 

if it was billed to USAID, to comply with USAID policies and FAR requirements.

 8

In February 2012, OTI awarded a task order to consolidate East and South program activity under a single task 

order to Creative Associates International.

9

DAI formally transmitted these lessons learned in its report: DAI, Monitoring & Evaluating ASI-East: Key Results 

from Output Monitoring and Overall Stability Monitoring, October 23, 2011.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 9

Issue Description

Subcontract oversight DAI’s oversight of its subcontract with its security contractor’s, Edinburgh International, is 

limited. The Defense Contract Audit Agency’s audit manual notes that prime contractor 

oversight of subcontractors should 1) include subcontract award, (2) include technical and 

financial performance monitoring, (3) ensure that indirect rate proposals and annual rate 

adjustments are submitted on a timely basis (incurred cost proposal), and (4) ensure that 

payment to the subcontractor for the work accomplished was in accordance with the 

subcontract terms and based on allowability, allocability and reasonableness principles. 

DAI did not fully follow this guidance based on our review of documentation and interviews 

with DAI staff. DAI disputes this characterization, noting that requirements one and two 

have been met through various contracting and security officials meeting with Edinburgh 

International officials and reviewing Edinburgh International contract requirements and 

billings. DAI asserts that it has addressed the third oversight requirement by negotiating 

Edinburgh International’s indirect rate at the beginning of the contract and will adjust the 

indirect rate at the end of the contract. DAI asserts that Edinburgh International has an 

adequate billing system that ensures costs billed are allowable, allocable and reasonable. 

Additionally, DAI reviews Edinburgh International’s billings to ensure their accuracy. 

Despite these cited instances of provided oversight, we found that DAI is not providing 

adequate oversight as required by the task order because we found no evidence that it is

performing yearly reviews of Edinburgh International’s overhead rates to determine their 

reasonableness. In addition, DAI has not been able to provide evidence that it reviewed 

Edinburgh International’s key business systems to ensure that process and internal 

controls are adequate for determining that the costs Edinburgh International is incurring 

and billing are allowable, allocable, and reasonable.

Source: SIGAR analysis.

Monitoring and Evaluation Systems Provide Initial Insights on Program Results and DSF 

Implementation 

Beginning with the April 2010, arrival of its current monitoring and evaluation director, DAI sought to 

develop a three-tier monitoring and evaluation system based on DSF requirements to assess, among other 

items, ASI’s effect on the overarching strategic goal of bolstering the capacity and legitimacy of the 

Afghan government at the community level. OTI contracted separately with a third-party provider, QED,

which then subcontracted with Altai Consulting, in June 2010, for an independent assessment of program 

outcomes and impact using a different assessment methodology. Preliminary monitoring and evaluation 

results as of December 2011 from both efforts show a range of documented outputs and encouraging 

impact at the program level, while overall stability was judged as being low across ASI-East’s ten 

programming districts. When DAI and Altai Consulting’s evaluation efforts are completed later in 2012, 

OTI and DAI believe they will have the information they need to evaluate ASI’s overall impact and the 

relative success of DSF as a stability programming model. Appendix II contains further details on the 

DSF and Altai Consulting monitoring and evaluation methodologies and the preliminary results of these 

efforts.

DSF and Altai Consulting Monitoring and Evaluation Methodologies

The DSF monitoring and evaluation methodology focuses on measuring results at three levels: output, 

impact, and overall stability.10 DAI’s monitoring and evaluation unit designed and implemented a system 

that follows the DSF methodology. DAI’s monitoring and evaluation staff collected and analyzed data 

for program output and impact measurement and contracted with RSI Consulting to conduct data 

 10Evaluation activities normally focus on outputs, outcomes, and impacts. As noted in our report, the DSF 

evaluation methodology does not currently address outcomes. Outputs are defined as measurable outputs from 

program activities such as miles of drainage ditches built or hours of media broadcast each week. Outcomes are 

defined as the observed effects of the outputs on the beneficiaries of the program such as improved agriculture 

yields or improved perceptions of the Afghan government. Impact is defined as the degree to which the observed 

outcome(s) can be attributed to the funded activities.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 10

collection and analysis efforts for measuring ASI’s contributions to overall stability. DAI data collection 

efforts for all three levels will be concluded before the task order ends in September 2012.

Issues and questions addressed by DAI using the DSF model at each level include the following:

• Level 1/output: Have scheduled activities been completed and successfully implemented? Are 

external factors affecting the implementation of scheduled activites?

• Level 2/impact: Is the intended impact or change being seen in the environment? Do observed 

changes in the environment represent progress toward stated objectives and a diminishment of the 

root causes of instability? How are external factors influencing or causing any observed changes?

• Level 3/overall stability: Are the collective efforts (ASI-East and any other programming efforts) 

to address the sources of instability in a given district leading to improved stability as measured 

by a standard set of seven indicators? Does trend data over time indicate that the security 

situation is getting better or deteriorating in a given district?

Altai Consulting monitoring and evaluation consists of four cycles that are scheduled to conclude in June 

2012. The first cycle focused on monitoring program outputs. Cycles 2 through 4 will address 

monitoring, impact, and overall stability questions in two districts for cycle 2 and three districts for cycles 

3 and 4.

Reports Show Mixed Results on Output and Overall Stability 

DAI issued its first formal report on level 1/output results in September 2011.11 Issued as a “lessons 

learned” report, the document presents DAI’s analysis of 90 completed program activities which found 

that these activities had achieved their intended purposes. The report also highlighted success stories that

should help guide future programming decisions. The report focused on how to improve (1) activity

implementation through community buy-in, physical presence of GIRoA officials, and publicizing 

projects; and (2) activity operations through adopting the Afghan First Initiative,

12 vetting suppliers and 

trainers, providing performance feedback, and ensuring that cash payments for work are timely. 

In October 2011, RSI Consulting provided DAI with a final report on level 3/overall stability results

demonstrating that overall stability in all ten ASI-East districts is relatively low based on the key 

indicators of stability.13 Specifically, the low levels of stability are attributed to security issues within 

districts, a lack of confidence in the local and national government, and underlying cultural differences 

between locals and outsiders.

Altai Consulting independent monitoring and evaluation efforts have generally confirmed the preliminary 

results revealed by DAI’s early monitoring and evaluation efforts.

DAI’s First Impact Assessment Released in January 2012

In January 2012, DAI released its first assessment of the impact ASI-East programming had on nine 

sources of instability in four districts.

14 The goals of the assessment were to identify stability-related 

changes in the selected districts over the last 18 months, to examine the factors that explain those 

changes, and to understand the effectiveness of different programming strategies used in each district. 

 11DAI, Lessons Learned Report: Afghanistan Stabilization Initiative-East, September 19, 2011.

12Since March 2006, U.S. military and civilian agencies operating in Afghanistan have taken steps to ensure that a 

greater number of contracts are awarded to Afghan companies. Over time, these efforts have become collectively 

known as the Afghan First Initiative.

13RSI Consulting, Overall Stability Assessment in 10 Districts in Afghanistan, October 2011.

14USAID published DAI’s assessment in the following document, USAID, Preliminary Impact Assessment in Four 

Districts in Eastern Afghanistan: Khogyani, Marawara, Saydabad, and Urgun, January 16, 2012.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 11

This assessment concluded that ASI-East programming was found to have a positive impact on 5 of the 

9 sources of instability. In one case, ASI-East programming was found to have reduced the source of 

instability. In three cases, the data gathered could not establish a positive or negative effect. The report 

noted that because data was collected in the middle of programming, results of this assessment are not 

comprehensive or predictive of the final impact of ASI-East. Although preliminary, these findings offer 

feedback for ASI-East as it refines its strategy in the final phase of programming and for follow-on 

application of the DSF methodology.15 

The assessment concluded that DSF is an effective and versatile methodology that allowed it to identify 

changes in the environment and adjust ASI-East programming decisions accordingly. The assessment 

also provided recommendations for improved implementation of the DSF including (1) beginning 

program activities after an area has reached the “clear” stage of the COIN strategy, (2) increasing

coordination with the military is needed so programming can start immediately after military operations 

cease, (3) increasing community involvement by GIRoA, and (4) performing community assessments 

before programming begins.

Project Outcomes Not Included in DSF Monitoring and Evaluation Methodology

We did not identify methodological flaws or problems with how DAI and Altai Consulting are 

implementing their monitoring and evaluation efforts. However, we did note that the DSF methodology 

does not include measuring outcomes. Measuring project outcomes would help implementers gain 

feedback on how projects are received by the beneficiaries and provide a leading indicator for level 2

impact analysis. Adding project outcomes to level 1 analysis on outputs will allow for earlier feedback 

and project adjustment to obtain better results.

We also noted that level 3 analysis on overall stability was first reported 28 months into the task order. 

While the analysis provided valuable information, it would have been better if it had been performed at 

the beginning of the task order or when stabilization efforts begin in new districts. This would have 

provided baseline information for subsequent results analysis. This is a key lesson learned for all future 

stabilization programs using the DSF methodology.

OTI LACKS APPROVED DISENGAGEMENT CRITERIA AND AN OVERALL 

PROGRAM EXIT STRATEGY TO MOVE DISTRICTS FROM THE HOLD TO BUILD 

PHASE OF THE COIN STRATEGY

Despite nearly 3 years of program activity, none of DAI’s target districts have transitioned to the “build”

phase of the COIN strategy.16 SIGAR found that approved disengagement criteria for individual districts 

and an overall transition strategy for the program remain to be developed. OTI Afghanistan has drafted a 

set of disengagement criteria for districts that could provide a clear basis for deciding to terminate “hold” 

programming in a given district; however, the draft criteria lack any target indicators or benchmarks to 

provide the basis for such a decision. These target indicators or benchmarks would likely be tied to the 

output and impact measures which DAI and OTI’s monitoring and evaluation contractor are pursuing. 

Under the provisions of the SWIFT III IQC, OTI is to develop an exit strategy for each OTI program 

 15The report noted other methodology limitations. The most significant limitation was the lack of baseline 

information, which made it difficult to compare the situation before the ASI-East program began addressing the 

sources of instability. In addition, the assessment was conducted during the summer, a period when violence 

typically increases, which may have influenced respondents’ assessments of stability in their districts. Finally, the 

ASI-East program was only one of several stabilization programs in the operational districts, other actors may 

influence the stability objectives ASI-East is designed to achieve. This presented a challenge to attributing observed 

changes to the ASI-East program.

16In responding to our draft report, USAID noted that districts in Wardak and Kunar provinces, where ASI-East 

programming took place, were recently identified as ready for transfer to GIRoA (see app. III).
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 12

within 18 months of the program start-up, including the ASI-East program. OTI has not yet developed an 

exit strategy to guide its programming efforts, which must be integrated with U.S. government and 

mission-wide transition planning efforts for Afghanistan. However, OTI officials noted that such a 

transition strategy would be developed in connection with the 3-year extension of the ASI program that 

began in February 2012.

OTI Lacks Approved Disengagement Criteria and a Program Exit Strategy

OTI programs are designed to be short, targeted, and flexible. In accordance with its SWIFT III IQC, 

OTI generally plans to exit a country program within 2 years of initiating activity. In Afghanistan, OTI 

expected that the 3 year ASI effort would lead to at least some districts transitioning from the “hold” to 

the “build” phase of the COIN strategy. All OTI programs are required to have an exit strategy developed 

within 18 months of program start-up. Both OTI led program performance reviews of ASI-East program 

noted the need for OTI to prepare exit criteria and exit strategies to help move districts from the “hold” to 

the “build” phase of the COIN strategy. This issue was also identified as an area of concern by the 

USAID Office of the Inspector General in its November 2011 report on the implementation of the ASI 

program in southern Afghanistan.

17 The report noted that the mission does not have a transition plan and 

recommended that USAID Afghanistan identify the districts in which it intends to implement posttransition development projects, develop transition plans for districts in which it plans to implement 

future projects, and close-out programs for districts where it will not implement new projects.

On October 21, 2011, OTI representatives in Afghanistan developed a draft set of disengagement criteria 

in response to recommendations from the two program performance reviews conducted by OTI and 

specifically included criteria under which ASI-East may discontinue stability programming in a district. 

However, the draft lacks target indicators or benchmarks that could provide the basis for applying the 

broad criteria described in the draft criteria. These target indicators or benchmarks should be tied to the 

impact and overall stability measures which DAI and Altai Consulting are currently developing.

The OTI official who drafted the disengagement criteria was subsequently re-assigned; however, OTI’s 

current Country Representative noted the document has not yet been vetted or discussed with Mission or 

military colleagues at post. OTI officials noted that these draft exit criteria will be finalized in connection 

with the new ASI task order.

OTI officials noted that an 18-month district exit strategy has not been developed and that no other 

transition plans have been put in place by the embassy. OTI officials noted that an exit strategy will be 

developed under the follow-on task order to ASI-East that will combine OTI stability programming 

efforts for both the South and East under a single task order, which was awarded in March 2012, for a 

3 year period. The development of this exit strategy will need to be consistent with any final 

disengagement criteria OTI develops and carefully coordinated with ongoing civilian and military 

transition planning efforts. 

The issue of disengagement criteria and exit strategy are linked to current and future implementation of 

the DSF methodology. Responsibility for further development and evaluation of the DSF has been 

transferred from OTI to USAID Afghanistan’s Stabilization Unit, which will now provide DSF training 

and future assessments of and enhancements to the methodology. A Stabilization Unit official noted that

the Stabilization Unit intends to include a review of the DSF under USAID’s Measuring Impact of 

Stabilization Initiatives (MISTI) program. MISTI is a $15-million, three-year program that began in 

March 2012, which will provide third-party impact assessment and evaluation services for the 

Stabilization Unit. As part of this effort, the contractor will be expected to evaluate the implementation of 

the DSF and draw lessons learned.

 17USAID Office of Inspector General, Audit of USAID/Afghanistan’s Afghanistan Stabilization Initiative for the 

Southern Region, Audit Report No. F-306-12-001-P, November 13, 2011.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 13

CONCLUSIONS

As a key building block of the U.S. COIN strategy, stabilization programs such as ASI must succeed if 

the United States is to achieve its short and long-term military and economic support drawdown goals. 

As the pilot implementation of the DSF methodology, the ASI-East program provides a critical 

opportunity to extract lessons learned for application to follow-on stabilization awards which will use the 

DSF methodology. Key challenges facing the implementation of ASI-East’s stabilization efforts and 

related follow-on efforts include (1) relatively low project spending levels and high operating costs, 

(2) certain procedural deficiencies with key DAI business systems and certain questioned costs, (3) the 

implementation and assessment of the DSF programming methodology, and (4) the lack of approved 

district-level disengagement criteria and a country-wide exit strategy for OTI’s programming efforts. 

Additionally, it is noted that the ASI-East program has not yet transitioned any target districts from the 

“hold” phase to the “build” phase of the COIN strategy.

RECOMMENDATIONS

To help ensure that the ASI-East program is implemented within the terms of OTI’s IQC and the ASIEast task order and applicable cost principles, SIGAR recommends that USAID/DCHA/OTI in 

Washington, D.C., in conjunction with OTI Kabul mission staff: 

1. Review the ASI-East program spending patterns to determine if the current spending levels 

represent a reasonable balance between program spending versus operating costs. 

2. Instruct the contractor to address and correct the identified timekeeping and billing internal 

control weakness and procedural shortcomings.

To ensure that ASI-East program costs are allowable, allocable, and reasonable, we recommend that the 

Contracting Officer in Washington, D.C.:

3. Review the more than $590,000 in questioned costs outlined below and recover these funds as 

deemed appropriate: 

a. The $500,000 in potentially unallowable costs associated with DAI’s sub-contract award to 

Altai Consulting for monitoring and evaluation services.

b. The $3,400 in unallowable shipping costs associated with a different DAI contract.

c. The $50,000 in wasteful spending on insurance cost for non-operational armored vehicles.

d. The $38,500 in wasteful spending on insurance for leased armored vehicles.

To improve ASI-East program performance oversight, monitoring, and evaluation systems and assess the 

overall utility and success of the DSF programming model, SIGAR recommends that USAID/DCHA/OTI 

in Washington, D.C., in conjunction with OTI Kabul mission staff and the Stabilization Unit Chief, as 

appropriate:

4. Incorporate an assessment of activity outcome results into level 1/output measurement of the DSF 

monitoring and evaluation methodology and perform level 3/overall stability monitoring and 

evaluation when entering new districts.

5. Provide input to the Stabilization Unit Chief to produce an interim and final lessons learned 

summary of the ASI-East program’s implementation of the DSF to be considered by senior 

USAID, military, and State Department officials as future decisions are made regarding the 

application of the DSF methodology to other stabilization programs.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 14

To help ensure future progress toward transitioning districts to the “build” phase of the COIN strategy, 

SIGAR recommends that USAID’s Bureau for Democracy, Conflict, and Humanitarian Assistance/Office 

of Transition Initiatives in Washington, D.C., in conjunction with OTI Kabul mission staff:

6. Finalize draft disengagement criteria to facilitate programming decisions at the district level and 

develop an exit strategy to guide overall OTI programming decisions at the country level.

COMMENTS

In commenting on a draft of this report, USAID/OTI generally concurred with the recommendations and 

noted the steps that it has taken or will take to address them. USAID also provided some general and 

technical comments and clarifications, particularly regarding the role of ASI-East in the larger COIN 

strategy, which we incorporated into the final report, as appropriate. USAID’s comments and our 

response to these comments are reproduced in appendix III.

In response to recommendation one, USAID/OTI concurred with the recommendation and stated that 

reviews have taken place during the course of the program and will continue for the remainder of the 

program. OTI provided clarification of the terminology we used in the report and recommendation 

regarding our characterization of programming spending levels. In response, we incorporated the 

distinction between program costs versus operational costs, as opposed to program costs versus overhead 

costs.

Regarding the second recommendation, USAID/OAA requested clarification of the guidance that was 

used. We provided the guidance that we used in SIGAR comment six in appendix III.

In response to recommendation three, USAID/OAA concurred with the recommendation and has taken or 

will take steps to recover any money that is determined unallowable. USAID specifically stated that 

(1) the $500,000 in questioned contract costs was allowable and no recovery action was required; (2) the 

$3,400 in questioned shipping costs has already been recovered; (3) the $50,000 in questioned insurance 

costs is currently being reviewed by the contracting officer; and (4) the $38,500 in questioned insurance 

costs is currently being reviewed by DAI and any overpayment adjustments made by the insurance carrier 

will be recovered.

In response to recommendation four, USAID/OTI concurred that level 3 (overall stability) analysis should 

be performed when entering a new district, but did not concur that outcomes should be added to level 1 

analysis because they do not have ownership of the DSF. In addition, USAID/OTI stated that the ASI 

program has more robust analysis at the level 1 (output) stage, which provides some level of outcome 

monitoring, and is looking at options to most effectively measure outputs, outcomes, impact, and overall 

stability in future programming and further noted that MISTI will facilitate this process. We agree that 

OTI no longer has ownership of the DSF and updated our recommendation to also include the 

Stabilization Unit, given the collaborative relationship between the DSF coordinator and the Stabilization 

Unit.

In response to recommendation five, USAID/OTI concurred that lessons learned from the DSF should be 

shared with other stakeholders and implementers. USAID/OTI did not concur with the recommendation 

to develop a lessons learned DSF summary, stating it was a duplicative effort under the Stabilization 

Unit’s ongoing efforts. We believe that OTI remains the appropriate office to produce a comprehensive 

DSF lessons learned study, given its role and mission. However, we updated our recommendation to 

include the Stabilization Unit, given its ongoing effort with the MISTI program to compile lessons 

learned on all programs.

In response to recommendation six, USAID/OTI concurred that a disengagement strategy at the district 

level and an exit strategy at the country level should be developed. USAID/OTI noted that it has begun to 

draft district disengagement criteria and will finalize the criteria under a new OTI program. OTI stated 
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 15

that it would ensure that the disengagement criteria are finalized prior to May 2013, allowing for an 

informed assessment of when OTI should disengage from a particular district/area of operation. 

Decisions to disengage will take into account whether OTI can guide or support follow on programming 

and the Administrator’s Stabilization Guidance and will include discussions pertaining to GIRoA 

transition plans for provinces. Additionally, USAID will work with the Embassy mission to determine an 

appropriate country level disengagement strategy.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 16

APPENDIX I: SCOPE AND METHODOLOGY

This report provides the results of the Office of the Special Inspector General for Afghanistan 

Reconstruction’s audit of USAID’s Afghanistan Stabilization Initiative-East program to implement smallscale community development and media activities to local implementers in target districts for the 

36 month period from June 26, 2009 through June 25, 2012. This report assesses to what extent 

(1) expended ASI-East costs were allowable, allocable, and reasonable; (2) performance oversight, 

monitoring, and evaluation systems have been implemented; and (3) progress has been made toward 

transitioning districts to the “build” phase of the COIN strategy. We reviewed documents covering the 

period June 2009 through June 2012.

To determine if incurred project costs were allowable, allocable, and reasonable, we examined invoices, 

evaluated timekeeping and billing policies and procedures related to internal controls, and had our 

forensic team analyze DAI’s financial data. With the invoice review, we judgmentally selected 

377 invoices from 6 other direct cost expense types for testing. For each invoice, we determined whether 

adequate support documentation existed and required approvals were obtained. In addition, to access the 

reliability of computer-processed data, we (1) interviewed officials to discuss the reliability of the data; 

(2) conducted testing and checked for missing data, erroneous or incomplete entries, and duplicates; and 

(3) compared system generated data with documents in the project files. For the timekeeping and billing 

systems, we interviewed DAI staff in Kabul, Afghanistan to gain an understanding of how the policies 

and procedures worked, obtained policy and procedure manuals, and performed walk through testing. We 

compared the results and noted any discrepancies. We also reviewed the reasonableness of DAI’s 

application of indirect cost rates and fixed fees. Additionally, we obtained and reviewed DAI’s external 

auditors’ management report for the two most recent audits performed and noted any internal control 

weaknesses identified. We investigated any anomalies to determine if the costs we properly recorded.

To determine whether performance oversight, monitoring and evaluation systems had been implemented,

we examined the task order to determine the reporting requirements. We interviewed USAID officials in 

Kabul, Afghanistan and the contracting officer in Washington, DC to determine the reports that were 

being sent and received and compared them to the reporting requirements. Additionally, we interviewed 

the contracting officer and COTR in Washington, DC to determine how they provide oversight to the 

program. We examined 20 judgmentally selected activity folders to determine if they had all the 

necessary reporting documentation, the proper USAID signatures noting the activity was complete and 

documentation of completion. We also examined DAI’s performance management system to determine if 

it met the task order requirements and if it was performing adequately. We examined USAID’s third 

party evaluator and the reports produced to determine what oversight and performance reporting was 

being provided. Based on the third party evaluator’s reports, we determined what program challenges 

were being encountered and how USAID and DAI were addressing the challenges.

To determine if progress has been made toward transitioning districts to the “build” phase of the COIN 

strategy, we examined the task order to identify any transition requirements and compared it to work 

performed by DAI. We also interviewed USAID and DAI officials to note if any districts covered by the 

program had transitioned into the next phase. Finally, we analyzed the DAI 90 activities completed as of 

September 2011.

We reviewed the financial data, including invoice and payment data, provided by USAID and DAI and 

determined that the data provided was sufficiently reliable for the purposes of our audit. We also assessed 

the adequacy of internal controls over DAI’s timekeeping and billing systems and procedures through 

tests of selected transactions, interviews and physical observations. 

We conducted work in Kabul, Afghanistan, and Washington, D.C., from May 2011 to June 2012, in 

accordance with generally accepted government auditing standards. These standards require that we plan 

and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe the evidence obtained provides a 
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 17

reasonable basis for our findings and conclusions based on our audit objectives. The audit was conducted 

by the Office of the Special Inspector General for Afghanistan Reconstruction under the authority of 

Public Law No. 110‐181, as amended, the Inspector General Act of 1978, and the Inspector General 

Reform Act of 2008.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 18

APPENDIX II: PRELIMINARY MONITORING AND EVALUATION RESULTS

Tables I and II outline DAI and Altai Consulting monitoring and evaluation systems and preliminary 

results through January 2012. 

Table I: DAI Monitoring and Evaluation Preliminary Results

Level Description Preliminary Results

Level 1

(program outputs)

DAI collects output information for all 

awarded activities in its its TAMIS 

database and generates Final 

Monitoring Reports for each completed 

activity. A DAI monitoring and 

evaluation staff member prepares a 

lessons learned report using data from 

the TAMIS system for a sample of 90 

completed activities.

After each activity is completed, DAI 

produces a Final Monitoring Report, which 

is used to collect data at the close of each 

activity. This report provides detailed 

information about the outputs and activities, 

the implementation process, challenges 

encountered and lessons learned, the 

grantee’s performance, and project 

stakeholders’ assessments of the activity’s

overall performance. 

In September 2011, DAI completed its first 

lessons learned report that analyzed 90

closed activities. This report found that, 

overall, activities are successful at 

achieving intended purposes. The report 

also highlighted methods of implementation 

that were successful and should be used in 

the future to ensure that other activities

achieve their goals and are successful.

Level 2

(program impact)

DAI started an assessment of the 

impact of clusters of activities against 

specific sources of instability through 

case studies. These case studies 

examine the outcomes of projects on 

district areas. The impact assessment 

is entirely qualitative and is gathering 

data from a range of stakeholders 

including program staff, district teams, 

residents of the district, and 

stakeholders in stability working groups 

and provincial reconstruction teams.

A DAI monitoring and evaluation staff 

member managed the research and 

production of the case studies which 

are designed to note observable 

changes in stability and identify which 

program strategies have and have not 

succeeded.

DAI released the preliminary results of its 

impact assessment in four districts in 

January 2012. In 5 of 9 cases, ASI-East 

programming was found to be a positive 

factor. In one case, ASI-East programming 

was found to be a mitigating factor. In 3

cases, the data could not establish a 

positive or negative effect.

Additionally, the assessment found that the 

DSF methodology allowed for identifying

sources of instability and adjusting 

programming to meet address them. The 

assessment recommended (1) that ASIEast programming should consider the 

COIN status of districts and restrict high 

visibility projects until districts are moved 

from the”shape” phase, (2) more 

coordination with the military so that

projects follow military operations for better 

effectiveness, and (3) increased community 

involvement of government officials.

Level 3 

(overall progress toward

stability)

DAI, in concert with OTI, developed 

seven district-level stability indicators 

within the DSF and selected methods 

for their measurement. The purpose is 

not to measure direct impact of DAI 

projects, but rather to identify overall 

stability in the areas in which DAI 

operates. To standardize data 

collection, DAI awarded a subcontract 

to an Afghan research firm, RSI 

Consulting, to collect data on the 7

indicators across 10 districts. 

In October 2011, RSI Consulting provided 

DAI with a final report for ten districts 

indicating that overall stability in all ten 

districts is negative based on the key 

indicators of stability.Specifically, the 

decline in stability is attributed to security 

issues within districts, a lack of confidence 

in the local and national government and 

underlying cultural differences between 

locals and outsiders.

Source: SIGAR Analysis of DAI M&E Documents
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 19

Table II: Altai Consulting Monitoring and Evaluation Preliminary Results

Category Description Preliminary Results

Cycle 1 

(monitoring)

OTI contracted with QED, which

subcontracted with Altai Consulting, to 

perform third party monitoring of DAI’s 

projects in a new innovative way. 

In October 2010, Altai issued an 

inception report which summarized a 

plan to develop methodological 

guidelines for M&E. Altai primarily 

relied upon physical project visits, 

perception surveys and case study 

reviews of completed projects.

In March 2011, Altai released its first cycle 

report that examined two districts in the 

East. This report found that across all 

districts, ASI projects generally addressed 

the grievances of the community, mostly 

addressed the priority problems of the 

community, built governance capacity, and 

were being implemented according to their 

designed plan.

Cycle 2 

(monitoring/impact/overall 

stability)

In cycle 2, Altai had started an 

assessment of the impact of clusters of 

activities against specific sources of 

instability through case studies. These 

case studies examine the outcomes of 

projects on district areas. The impact 

assessment is entirely qualitative and is 

gathering data from a range of 

stakeholders including program staff, 

district teams, residents of the district, 

and stakeholders in stability working 

groups and provincial reconstruction 

teams.

In July 2011, Altai released its cycle 2 

report for the same two districts examined 

previously. The report found that overall 

projects were generally received well from 

the communities. The report provided 

insights into community perceptions, 

provided lessons learned, and noted that 

projects need a better association with the 

Afghan government for increased 

credibility.

Cycles 3 and 4 

(monitoring/impact/overall 

stability)

Cycles 3 and 4 will apply the same 

methodology used in cycle 2, with 

further development of tools.

Reports planned in two districts (plus one

new district and five new villages) in 

January/February 2012 and May/June 

2012.

Source: SIGAR Analysis of Altai Consulting monitoring and evaluation documents.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 20

APPENDIX III: COMMENTS FROM THE U.S. AGENCY FOR INTERNATIONAL 

DEVELOPMENT

See SIGAR 

comment 1.

See SIGAR 

comment 2.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 21

See SIGAR 

comment 3.

See SIGAR 

comment 4.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 22
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 23

See SIGAR 

comment 5.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 24

See SIGAR 

comment 6.

See SIGAR 

comment 7.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 25

See SIGAR 

comment 8.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 26

See SIGAR 

comment 8.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 27

See SIGAR 

comment 9.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 28
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 29
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 30

See SIGAR 

comment 10.

See SIGAR 

comment 11.

See SIGAR 

comment 12.

See SIGAR 

comment 13.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 31
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 32

The following are SIGAR’s responses to USAID /DCHA/OTI’s letter dated June 6, 2012:

1. We replaced the word “security” with “stability” in the report title.

2. Our draft report acknowledged that programming under the ASI-East project pertained to short 

term stability programming and is meant as a precursor to longer-term development programming.

3. We updated the report to include the reference to Kunar and Wardak districts in footnote 16.

4. We provided additional clarification of ASI-East’s role in the overall COIN strategy in footnote 1.

5. SIGAR updated the terminology in the report from overhead to operational expenses. However, 

we segregated programming costs from all other operational costs to highlight the amount of 

expenditures that went directly toward activity level spending and the amount that went toward

supporting those activities. As noted in our report, the ASI-East task order, at the time of award, 

stipulated an implied spending ratio of 55 percent on program activities and 45 percent on 

operating costs. Therefore, we do not believe it is invalid to segregate these costs to highlight the 

spending patterns.

6. As part of our review of DAI and its financial reporting to USAID, we reviewed DAI’s 

timekeeping and billing systems to ensure that they were designed and functioning properly and 

effectively with respect to preventing, detecting, and correcting internal control deficiencies. 

Based on General Accounting Office Standards for Internal Controls in the Federal Government, 

dated November 1999, we identified minor deficiencies in DAI’s internal control that we believed 

warranted USAID’s attention. Taking corrective actions now will help mitigate systemic or more 

widespread occurrences of these or other deficiencies. The following lays out the guidance we 

used to address the recommendation regarding the time keeping and billing internal control 

weaknesses and procedural shortcomings.

a. SIGAR believes that increasing the frequency of formal supervisory review from monthly to 

semi-monthly of Afghan timecards would help mitigate the potential of fraudulent entry. 

Increasing supervisory review of timecards is based on best practices and is consistent with 

the procedures used for ex-patriot staff. 

b. Having complete and available supporting documentation attached to paid invoices is 

required by FAR 52.215-2. Doing so helps facilitate the timeliness and accuracy of invoice 

retrieval for questions regarding claims, internal or external audits, or other procedural 

requests.

c. SIGAR maintains that DAI should have performed a cost analysis, once security was in place, 

to determine the reasonableness of the cost to lease the items. This practice is consistent with 

FAR 31.201-3, “Determining Reasonableness.” A cost analysis would have aided DAI in 

determining whether the items it leased would have been cheaper to purchase over the course 

of the project. 

7. Based on M/OAA’s response that it had taken actions to determine that the $500,000 questioned 

costs are allowable, we requested the analysis that the DCHA/OTI Contracting Officer used to 

make this determination. M/OAA provided its response to our request on June 20, 2012, which 

stated that it believed the cost to be allowable based on FAR clause 52.216-7, Allowable Cost and 

Payment and FAR 31.201-2, Determining Allowability. In his response, the Contracting Officer 

noted that (1) the costs were reasonable and in-line with other M&E studies conducted by Altai 

and others in Afghanistan; (2) the costs were allocable to the task order and; (3) the terms of the 

task order and IQC were followed with regard to subcontracting. We do not dispute the 

Contracting Officer’s characterization of the reasonableness of the costs; however, the response 

does not address the lack of subcontractor oversight, as discussed on page 5 of the report. The 
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 33

mitigation plan that was implemented by OTI between DAI and Altai limited DAI’s ability to 

provide adequate subcontractor oversight as required by FAR 42.202(e)(2). DAI’s role in 

administering the subcontract was essentially limited to reviewing and approving invoices for 

payment, and did not allow for normal oversight responsibilities. Therefore, we continue to 

question whether the over $500,000 was allowable.

8. We recognize that OTI no longer has ownership of the DSF and that it is being used by several 

different entities. As such, we updated Recommendation 4 to also include the Stabilization Unit

Chief, so that OTI and the Stabilization Unit may work together, along with any other 

stakeholders, to amend the DSF. As noted in our report, we continue to believe that adding 

project outcomes to level 1 analysis allows for programming strategy to adapt faster based on 

continuous feedback instead of waiting for level 2 impact analysis to be performed. 

9. We recognize that the Stabilization Unit has an ongoing effort with the MISTI program, which is 

in its beginning stages, to compile lessons learned on all programs. However, according to a 

USAID Stabilization Unit official, the MISTI task order would be amended to remove the ASIEast program from the Stabilization Unit’s evaluation of stabilization programs and the 

Stabilization Unit and OTI would rely on an external contractor, instead of its own internal 

assessment, to provide feedback on the ASI-East program. Therefore, we maintain that OTI 

remains the appropriate office to produce a comprehensive DSF lessons learned study. The DSF 

became operational under OTI and remained the primary ASI-East programming tool for over 2

years while training over one thousand civilian and military personnel. Given the Stabilization 

Unit’s role, since February 2012, in the ASI-East program, we revised recommendation five to 

also include the Stabilization Unit Chief. 

10. The reference to OMB Circular A-122 was removed.

11. We believe the notion of the burn rate approach provides important context in light of the method 

used to program the first 49 grant activities by December 2009 for ASI-East under the military 

Tactical Conflict Assessment and Planning Framework. The shift toward the DSF approach to 

programming, as opposed to the burn rate approach, is a more tactical approach and was a 

significant undertaking on USAID’s part to steer away from the use of burn rate methods where 

programs were slow to start during their implementation phase.

12. We updated the report to reflect “Grants Under Contract” as a tool.

13. The DCAA manual provides guidance to audit agencies—in this case, to determine if a contractor 

performed adequate oversight over a subcontractor. FAR 42.202(e)(2) requires that prime 

contractors provide oversight over their subcontractors.
SIGAR Audit-12-11 Contractor Performance and Oversight/Stabilization Initiative Page 34

(This report was conducted under the audit project code SIGAR-047A).
SIGAR’s Mission The mission of the Special Inspector General for 

Afghanistan Reconstruction (SIGAR) is to enhance 

oversight of programs for the reconstruction of 

Afghanistan by conducting independent and objective 

audits, inspections, and investigations on the use of 

taxpayer dollars and related funds. SIGAR works to 

provide accurate and balanced information, evaluations, 

analysis, and recommendations to help the U.S. Congress, 

U.S. agencies, and other decision-makers to make 

informed oversight, policy, and funding decisions to

• improve effectiveness of the overall reconstruction 

strategy and its component programs;

• improve management and accountability over funds 

administered by U.S. and Afghan agencies and their 

contractors;

• improve contracting and contract management 

processes;

• prevent fraud, waste, and abuse; and

• advance U.S. interests in reconstructing 

Afghanistan.

Obtaining Copies of SIGAR 

Reports and Testimonies

To obtain copies of SIGAR documents at no cost, go to 

SIGAR’s web site (www.sigar.mil). SIGAR posts all 

publically released reports, testimonies, and 

correspondence on its web site.

To Report Fraud, Waste, and 

Abuse in Afghanistan 

Reconstruction Programs

To help prevent fraud, waste, and abuse by reporting 

allegations of fraud, waste, abuse, mismanagement, and 

reprisal contact SIGAR’s hotline:

• Web: www.sigar.mil/fraud

• Email: sigar.pentagon.inv.mbx.hotline@mail.mil

• Phone Afghanistan: +93 (0) 700-10-7300

• Phone DSN Afghanistan 318-237-3912 ext. 7303

• Phone International: +1-866-329-8893

• Phone DSN International: 312-664-0378

• U.S. fax: +1-703-601-4065

Public Affairs Public Affairs Officer

Phone: 703-545-5974

Email: sigar.pentagon.ccr.mbx.public-affairs@mail.mil

Mail: SIGAR Public Affairs

2530 Crystal Drive

Arlington, VA 22202