APOJ 16-4 

22 January 2016

Measuring the Immeasurable: Assessing the Effectiveness of Engineering Civic Assistance Projects

By Maj. Orlando Craig

The United States and its allies have employed a different approach in this struggle for the hearts and 

 minds of men. The Western strategy has been to assist the underdeveloped countries to maintain 

their political independence ... and to improve their standard of living. Civic action has been a significant 

element of this assistance.

 Lt. Col. Neil B. Mills, USMC, An Analysis of Civic Action in Selected Underdeveloped 

 Countries

U.S. foreign policy in the modern era continues to struggle with how to best use the military instrument of 

national power to achieve national objectives in conflicts that increasingly center among people and populations. 

This struggle requires an approach not normally associated with military operations. In his masters of military 

arts and science thesis titled, An Analysis of Civic Action in Selected Underdeveloped Countries, Lt. Col. Neil B. Mills, 

USMC, described this approach: 

The Western strategy has been to assist the underdeveloped countries to maintain their political 

independence ... and to improve their standard of living. Civic action has been a significant element of this 

assistance.1

Without context, one may mistake these remarks as a reference to recent American military strategy. Most would 

be surprised to learn that this observation was made in 1964 by a U.S. Army Command and General Staff College 

student examining civic assistance in the context of U.S. military and coalition operations in the Philippines, Malaya, Laos, Korea, and Vietnam. 

Nearly fifty years later, humanitarian and civic assistance (HCA) remains a significant part of U.S. strategy. 

Since the attacks of September 11th, use of HCA has increased exponentially as the geographic combatant commands increasingly rely on these projects as one way to shape their areas of responsibility. However, the current 

limitations of the U.S. defense budget threaten to reduce or eliminate these projects. In light of these conditions, 

and despite their touted successes, concerns exist about the effectiveness of these programs. Specifically, policy 

makers and military officers question the projects’ efficacy in achieving national security objectives as prescribed 

by U.S. law and Department of Defense (DoD) policy. 

Gaps in policy and assessment methods governing HCA produce a situation where no adequate system 

exists to measure the effectiveness of HCA projects toward meeting national security objectives. Drawing this 

conclusion required answering two secondary questions, which provide a holistic understanding of the topic and 

facilitate a comprehensive assessment. First, one must determine the objectives that HCA is used to achieve, which 

requires reviewing U.S. law, DoD policy, and studying the historical evolution of HCA. Second, one must evaluate 

current methods of assessing the effectiveness of HCA programs. This is critical as methods of assessment determine how well a course of action addresses solving a problem. 

These two questions were examined by researching the effectiveness of a particular form of HCA known 

as engineering civic assistance projects (ENCAPs). Monetarily, these account for the preponderance of HCA 

projects. Furthermore, they have an extensive history of use by the U.S. that lends to in-depth analysis. Though 

the conclusions and proposed assessment model are ENCAP specific, the concepts can be extrapolated to apply to 

other forms of HCA given their extensive use by the DoD.

APOJ 1

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

Army Press Online Journal is published by-monthly by The Army Press to provide cutting edge content on topics related to the Army 

and national defense. The views expressed belong to their authors, and do not necessarily represent the official view of the Department of the Army, the Department of Defense, or any other government institutions or agencies. APOJ 1
APOJ 2

APOJ 16-4 

22 January 2016

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

What are the Objectives of Engineer Civic Assistance Projects?

The first step in ascertaining the effectiveness of ENCAPs is to determine the objectives of the program. 

Primarily, these objectives are established by U.S. law and further refined in DoD policy. However, just looking at 

these documents merely scratches the surface. Verifying relevance of these objectives requires studying ENCAP 

objectives from their historical origins to their modern equivalents. Examining these two areas provides context 

for their use and allows assessment of the prescribed objectives’ validity. Only with valid objectives can the DoD 

develop indicators to measure the effectiveness of its projects and assistance efforts.

ENCAPs find their modern framework established by the National Defense Authorization Act of 1987, 

which amended Title 10 United States Code. This established civic assistance into law and formalized both the 

term and definition for HCA. It also required that HCA, if utilized, would be “provided in conjunction with military operations.”2

 Additionally, the law authorized the DoD to execute HCA activities if those activities promote 

“the security interests of both the United States and the country in which the activities are to be carried out; and 

the specific operational readiness skills of the members of the armed forces who participate in the activities.”3 The 

DoD further requires that HCA activities improve basic living conditions; enhance the legitimacy of the host nation; promote interoperability; generate long-term positive perceptions; and enhance security. These requirements 

form the baseline of the objectives of modern ENCAPs. However, to confirm that these objectives are pertinent 

and the ones that require assessment, one needs to scrutinize the development of the concept of HCA since the 

Civil War. 

The history of ENCAPs divides into three distinct periods: the Civil War through World War I, the interwar period through World War II, and post-World War II. While each period saw changes in the scale and use 

of ENCAPs in keeping with evolving U.S. strategic interests, their overall objectives remained consistent. Lt. Col. 

Robert L. Bullard, who served in the Philippines in the early 1900s, described these efforts as “all means, short of 

actual war, used by the dominating power in the operation of bringing back to a state of peace and order the inhabitants of a district lately in hostilities.”4

This first period, between 1860 and 1920, saw a dramatic expansion of the U.S. in terms of both territory 

and power due to Civil War Reconstruction, the Indian Wars, and operations in newly acquired territories. As a 

result, the U.S. military gained significant experience in small-scale, contingency operations that would serve as 

the foundation of HCA doctrine. An example of such efforts occurred in Cuba in the late 1900s. During his tenure 

as military governor of Cuba, MG John R. Brooke “maintained law and order, gave wartime refugees emergency 

assistance, enforced new sanitation codes, and built roads, sewers, and schools.”5 Similar such actions occurred in 

the Philippines, Panama, Puerto Rico, and numerous other countries. At the onset of World War I, the military 

lost much of its vast experience and forgot the lessons of these operations. The U.S. military did not emphasize 

HCA concepts again until the inter-war period, albeit, haphazardly. 

The second period of ENCAPs occurred during the inter-war period from the late 1910s through 1945. 

Both the U.S. Army and the U.S Marines recognized the need to be able to conduct operations focusing on stability. The Marines, because of their small size and their ability to deploy rapidly, garnered this experience through 

operations in conjunction with the Department of State. Often the Marines would deploy in areas along the coasts 

of foreign countries to protect American lives, property, and interests. This task frequently required working with 

the respective country on projects, such as the construction of roads and bridges, as part of larger efforts to establish order. Similarly, the Army developed preliminary doctrine based on comparable small-scale operations. 

This, combined with operational war planning, drove the development of civic assistance concepts for 

the Army, especially as the specter of war loomed in both Europe and the Pacific. In 1940, students at the U.S. 

Army War College authored a study titled, “The Administration of Civil Affairs in Occupied Alien Territory,” that 

served as the initial Army field manual on the topic. As World War II progressed, the crucible of the world’s most 

cataclysmic war created a fertile ground for the development of HCA. Though primarily focused on 
 APOJ 16-4 

22 January 2016

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

reconstruction operations, the U.S. developed organizations and theories that gave rise to the concepts of security 

cooperation and civic assistance that initiated the third period of ENCAP development.

The third evolution of ENCAPs began in earnest at the dawn of the Cold War in the 1950s, with the U.S. 

government focused on countering the communist influence around the globe. This timeframe saw the continued 

use of ENCAPs as a mechanism to achieve national interests. Armed Forces Assistance to Korea was the first such 

major program. Together, Korean and American military forces conducted combined operations to construct 

schools, clinics, markets, and other projects to “help raise [South Korea] from the depths of poverty, giving a people 

strength to resist the nearby communist plague which feeds on adversity.”6

Following this program’s success, the United States expanded HCA activities throughout the world. This 

included Latin America, and, in particular, Southeast Asia. During the latter portion of the Vietnam War, the 

United States implemented the Civil Operations and Revolutionary Support Program, which combined both civilian and military civic assistance efforts. However, as the Vietnam War grew increasingly unpopular amongst the 

U.S. population and its policy makers, the experience was lost despite its considerable success, and civic assistance 

became a victim of the war.7

In the immediate aftermath of the Vietnam War, there was apprehension regarding the use of the military 

in a nation-building role, thus, civic assistance fell from the limelight as an element of U.S. policy. It was not until 

the 1980s that the U.S. slowly re-elevated HCA as a viable option for regional engagement. The geographic combatant commands continued to expand civic assistance programs within their areas of responsibility, especially in 

Africa and the Pacific. Today, combatant commanders continue to use HCA as a strategic tool to influence and 

shape their respective operational environment utilizing many of the same principles and concepts developed in 

the earliest days of ENCAP execution. 

Understanding the requirements of ENCAPs as prescribed by U.S. law, and further established by DoD 

policy, is critical to assessing their effectiveness. Though terms have evolved over time, the DoD utilized ENCAPs 

in essentially the same manner: to achieve objectives that mirror those currently outlined by U.S. law and DoD 

policy. Interestingly, these objectives remain consistent despite the lack of continuity in accumulating the lessons 

learned from previous periods. Establishing this continuity ensures that the objectives are valid thus paving the 

way for the development of indicators that the can be used to assess their effectiveness.

How Does the U.S. Military Assess Engineering Civic Assistance Projects?

Determining whether ENCAPs achieve national security objectives requires analysis of the military assessment process, to include the assessment requirements and their sources. These requirements emanate from 

four sources. Since HCA is a military operation, military doctrine is the first source and prescribes the military’s 

basic assessment process. Similar to doctrine, the Universal Joint Task List serves as the second source, identifying 

two tasks specifically related to HCA. Both of those tasks include assigned metrics to aid in the assessment process. U.S. law and DoD policy are the third sources of assessment requirements. Finally, the geographic combatant 

commanders have the latitude to implement any theater specific evaluation criteria to assess the programs’ effectiveness. Understanding these sources and requirements reveals gaps in the assessment process. 

The primary factor that affects the assessment process stems from gaps in the DoD policy, leading to 

inconsistency in ENCAP evaluation. Specifically, the DoD only requires that HCA activities be assessed twice 

at most. The first assessment occurs within the first thirty days after project completion. This evaluation is the 

after-action review for the project. The second required assessment occurs one year after project completion. 

However, DoD Instruction 2205.02: Humanitarian and Civic Assistance (HCA) Activities, only mandates this second 

assessment if it is considered cost effective. If the combatant command decides that conducting a one-year assessment is feasible, it is at their discretion to decide how many and which ENCAPs, or other HCA activities, to assess. 

As a result, there is no incentive to conduct a one-year assessment. Given today’s fiscal uncertainty, there is

APOJ 3
APOJ 4

 APOJ 16-4 

22 January 2016

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

potential that this could curtail the limited assessments that already occur. Thus, the conditions exist to exasperate 

the information rift in regards to ENCAPs’ effectiveness. 

This leads to the second policy gap; there is no forcing function to capture baseline data, and, therefore, 

no effective indicators through which to measure change. This creates a gap for evaluation, as progress cannot be 

assessed if the original inputs and conditions are unknown. This shortfall widens when someone not familiar with 

the project--that can occur for a multitude of reasons--is required to make an evaluation. Thus, the focus of assessment become quantitative, that is, focused on measures of performance, as opposed to measures of effectiveness, 

which assesses a change in the behavior of the overall system.8

Together these factors affect the DoD’s ability to determine the level to which a particular ENCAP 

achieved particular objectives. The Government Accountability Office conducted a least two studies on this topic 

in the recent past. One study, carried out in 2012, found that the DoD could not directly link HCA activities, including ENCAPs, to specific effects or objectives. This was similar to the results of a study conducted in the 1990s. 

Assessing ENCAPs requires a comprehensive approach that evaluates all elements of an ENCAP throughout the 

course of the project cycle. 

A Proposed Model for Assessing the Effectiveness of Engineering Civic Action Projects.

Development of a comprehensive model for assessing the effectiveness of ENCAPs must focus on improving two phases of the entire process. First, and most importantly, baseline data needs to be collected from all areas 

that are being evaluated. Without baseline data, there is truly no impartial way to evaluate effectiveness. Second, 

this data collection needs to expand beyond DoD Humanitarian Assistance and combatant command objectives 

to include three other areas. 

First, the assessment must include an evaluation of engineer specific qualities of the project. Second, assessments should measure project efficiency in terms of overall costs. Finally, any assessment must include specific U.S. 

training objectives to provide an accurate appraisal of the project’s training value. All of these elements together 

will provide a holistic assessment of the ENCAP.

Assessing progress in any of these areas requires the development of indicators that allow measurement 

of progress toward desired outcomes, and that are crucial to the entire assessment process. Once the indicators 

are in place, the project manager’s collection of baseline data on the indicators is essential. This data provides the 

starting point for measuring change due to the particular project. Without this initial data, the entire assessment 

becomes purely subjective and based solely on the assessors’ judgment and perception. Collecting baseline data is 

not restricted to desired outcomes; it applies to engineer specific qualities of the project as well.

ENCAPs provide a lasting, visual symbol of U.S. cooperation with the respective partner nation. Therefore, 

assessment of the quality of the engineer specific tasks and outputs is necessary over the long-term to ensure project quality, sustainability, and ergonomics. Since the vast majority of ENCAPs consist of construction or renovation of structures, infrastructure assessment is the logical model assessors can use to evaluate structures over the 

long-term. First, the Overseas Humanitarian Shared Information System should capture project as-built drawings 

as attachments to provide baseline data regarding final construction. The American Institute of Architects defines 

as-built drawings as “drawings ... prepared by the contractor [that] show, in red ink, on-site changes to the original construction documents.”9

 Those conducting assessments in the future could reference the as-built drawings, 

if necessary, during or after the infrastructure assessment. The SWEAT/IR Book published by the United States 

Army Engineer School provides a very basic methodology for evaluating the critical components of infrastructure 

that does not require a trained engineer to execute. When performed over time, this would facilitate changes to 

facility plans and components to provide a final product that would enhance the long-term positive perception of 

the U.S. government, a key requirement of both law and policy. To better assess the overall cost of an ENCAP, the 

assessment should incorporate other funding sources used throughout the project cycle. 
APOJ 5

 APOJ 16-4 

22 January 2016

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

Broadly speaking, these costs delineate into three additional categories beyond HCA appropriation: transportation, U.S. Soldier sustainment, and host nation support. The sources of funding vary for these categories based on 

the type of exercise and project. However, incorporating these costs will aid in providing context to the true value 

of the ENCAP as it relates to assessment. This is relevant to maximizing the effects of ENCAPs given today’s fiscal 

uncertainties.

Finally, comprehensive evaluation of ENCAPs requires U.S. military units to define their specific training 

objectives for their unit when selected to participate in an ENCAP. Once specified, units need to assess their proficiency before, and after ENCAP execution. These results, when compiled, would provide specific data on specific 

mission essential tasks for the DoD to quantify ENCAP’s effects on the operational readiness of U.S. forces beyond 

just a qualitative assessment. Thus, described graphically:

Figure 1: Proposed DoD ENCAP Assessment Model. Source: Created by author. 

This model begins to address these shortcomings across four areas. First, indicators regarding the effectiveness of ENCAPs in regards to outcomes need to continue to be refined. Second, assessment needs to include 

a framework by which to evaluate the specific engineer components of the project. Third, all the costs associated 

with an ENCAP need to be captured to aid in the cost-benefit analysis. Finally, the assessment process needs to 

evaluate the training value received by U.S. military personnel through the construction of the ENCAP. This model is by no means complete. To aid further development requires discussing recommended changes to policy and 

areas warranting further research.

The primary change to policy should occur within the language of DoD Instruction 2205.02 Humanitarian 

and Civic Assistance Activities. It should direct the combatant commands to conduct a long-term assessment of a 

portion of their ENCAPs. If this change is made, these assessments should be incorporated into a searchable database to facilitate rapid and precise analysis of lessons gleaned from ENCAPs across the entire DoD.

Given the recommended policy change, several topics warrant further research. First, given the relative 

similarity of the types of ENCAPs, development of a standardized list of indicators would ease the burden of the 

assessment process. A solid base of indicators exists within relevant literature. The Hamlet Evaluation System10, 

Developing a Prototype Handbook for Monitoring and Evaluating Department of Defense Humanitarian Assistance 

Projects, Assessing the Value of U.S. Army International Activities, the universal joint task list, and extensive civilian 

sources11 contain a wide variety of indicators for deeper analysis.

Furthermore, the DoD should expand research beyond just that of ENCAPs. HCA includes a variety 
APOJ 6

 APOJ 16-4 

22 January 2016

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

of medical, veterinary, and dental activities that have the same requirements for assessment. Examining effectiveness in regards to those specific topics would further enhance the DoD’s HCA efforts. However, this evaluation 

needs to be balanced and, therefore, mandating the long-term assessment of every project is unsustainable. Therefore, researching what HCA projects to assess, and how often to assess them, would be of great utility in today’s 

fiscally constrained environment.

Conclusions

As the U.S winds down the conflicts that have defined it for over a decade, combatant commands will 

gradually assume a role where they posture and prepare for the next conflict. Combatant commanders will continue to use HCA projects, including ENCAPs, to shape and influence their areas of responsibility. However, the 

lack of long-term assessment prevents the DoD from assessing the extent of ENCAPs’ effectiveness. Given widespread employment of ENCAPs across the combatant commands, addressing this gap is necessary. The proposed 

model could then be adapted to apply to other HCA efforts as well. This would enable the combatant commands 

to maximize the effects of HCA programs in a complex and fiscally constrained environment. 

This article proposes a model that forms an initial approach to address the existing evaluation gap. However, the model only serves as a guide for the evaluation of ENCAPs based on the factors discovered during research. 

Ultimately, as is the case with all military operations, no one model is a complete solution. As Lt. Col. Neil B. Mills 

noted in the conclusion of his thesis, “the most important factor of all, and the one upon which all ... guidelines are 

dependent, is the [military] officer himself. ... It is for his consideration and use that they are offered.”12

Major Orlando Craig, U.S. Army is currently a Plans Officer for 95th Civil Affairs Brigade at Fort Bragg, NC. Prior to assuming company command, he was assigned as an Operations Officer in the Plans and Operations Cell of the Deputy Chief of 

Staff, Engineering, Headquarters, U.S. Army Pacific at Fort Shafter, HI overseeing the planning, resourcing and execution of the 

engineering civic assistance projects and teams in Mongolia, Thailand and Palau. MAJ Craig took command of 643d Engineer 

Company in May 2011. During command, the company executed over twenty engineering civic assistance projects in seven 

different countries and territories that included Thailand, Mongolia, Philippines and Palau. For their efforts, the company was 

awarded the2011 LTG Emerson C. Itschner Award, recognizing them as the best engineer company in the Army. He holds a 

B.S. from Norwich University, M.S. from Missouri Institute of Science and Technology, and an M.M.A.S. from the U.S. Army 

Command and General Staff College.

NOTES

1. Mills, Neil B., “An Analysis of Civic Action in Selected Undeveloped Countries,” (Master’s thesis, U.S. Army Command and General Staff College, 1964): 142, accessed 20 November 2014, www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA365714.

2. Humanitarian and Civic Assistance Provided in Conjunction with Military Operations, US Code 10 (as amended 

2006), § 401, 216 accessed 21 March 2015, http://www.gpo.gov/fdsys/pkg/USCODE-2013-title10/pdf/USCODE2013-title10-subtitleA-partI-chap5-sec153.pdf.

3. Ibid.
APOJ 7

 APOJ 16-4 

22 January 2016

FORT LEAVENWORTH, KANSAS

T H E AR M Y PRES S

4. Bullard, Robert, “Military Pacification,” Journal of the Military Service Institution of the United States 46 ( January–February 1910): 1-24, accessed 8 May 2015, http://babel.hathitrust.org/cgi/pt?id=njp.32101049985763;view=1up;seq=9.

5. Birtle, Andrew J., U.S. Army Counterinsurgency and Contingency Operations Doctrine 1860-1941 (Washington, DC 

Center for Military History, 1998), 3.

6. The Big Picture, episode 201, “Armed Forces Assistance to Korea,” produced by The U.S. Army Signal Corps Photographic Center, accessed 22 March 2015, https://www.youtube.com/watch?v=npBi7CDI-4g.

7. In an attempt to create unity of effort in the battle against the Viet Cong, the U.S. government integrated both civilian and military components under one, combined chain of command. Though not entirely free of shortcomings, the 

program was successful enough that by 1970, “the [Viet Cong] insurgency had largely been defeated, making it possible to hold elections in over 10,000 hamlets and 2,000 villages throughout South Vietnam.” For more on the program, 

see Jeremy P. White, Civil Affairs in Vietnam (Washington, DC: Center for Strategic and International Studies, 2009), 9, 

accessed 21 March 2015, http://csis.org/files/media/csis/pubs/090130_vietnam_study.pdf.

8. Chairman, Joint Chiefs of Staff, Joint Publication ( JP) 3-29, Foreign Humanitarian Assistance states that measures of 

performance “measure task performance ... [and] are generally quantitative, but also can apply qualitative attributes 

to task accomplishment,” while measures of effectiveness “assess changes in system behavior, capability, or [operational 

environment]. Therefore, in the case of a school, a measure of performance may be the number of students in attendance. A measure of effectiveness may be the change in reading levels amongst the students.

9. American Institute of Architects, “Terminology: As-Built Drawings, Record Drawings, Measured Drawings,” AIA Best 

Practices ( June 2007): 1, accessed 13 May 2015, http://www.aia.org/aiaucmp/groups/secure/documents/document/

aiap026835.pdf.

10. During the Vietnam War, American Advisors, as a part of the Civil Operations and Revolutionary Support Program, 

utilized the Hamlet Evaluation System to assess the civic assistance efforts. It went through various iterations but consisted of six categories with appropriate questions concerning conditions subordinate to those categories. Relevant to 

ENCAPs, the categories included Administrative and Political Activities, Economic Development, and Security. Each of 

these categories and its subordinate conditions was evaluated on a five-point, Likert-style Scale.

11. Obviously, there are civilian humanitarian organizations that are very adept at evaluating humanitarian assistance 

efforts as it one of their core functions. Some pertinent sources include, Evaluation of Humanitarian Action: Pilot Guide 

published by the Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP), 

Learning through Evaluation with Accountability and Planning: World Vision’s Approach to Design, Monitoring and Evaluation published by World Vision, and the Humanitarian Charter and Minimum Standards in Humanitarian Response 

published by The Sphere Project.

12. Lt. Colonel Neil B. Mills, “An Analysis of Civic Action in Selected Underdeveloped Countries” (Master’s thesis, U.S. 

Army Command and General Staff College, 1964), 159, accessed 20 November 2014, http:// www.dtic.mil/cgi-bin/

GetTRDoc?AD=ADA365714.