WARFARE CENTERS 

DAM NECK
January 2015
Integration & Interoperability | 1

Leading the Charge in Surface Warfare Mission Engineering

Naval Surface Warfare Center, Dahlgren Division (NSWCDD) leads 

the technical integration and interoperability (I&I) of highly complex 

systems of systems that make up our nation’s Surface Warfare capabilities. 

A highly motivated workforce of scientists, engineers, and technicians 

routinely orchestrate the full set of mission kill chains that make up 

surface warfare, each of which is composed of hundreds of components 

across tens of systems fielded potentially on multiple sea-based platforms. 

This challenge would seem daunting if it were not for our strong legacy in 

mathematics, the traditional engineering disciplines, information sciences, 

and most notably, a holistic systems engineering approach to capability 

development and sustainment. Envisioning a new capability that exploits 

a technological development from the laboratory is only the first step of 

a highly disciplined engineering process. Integrating that new capability 

with the existing capability to safely and effectively expand the technological advantage of our warfighters at sea is also a critical step. Our mission 

engineers and architects are committed to maintaining a mission focus 

throughout the development of the myriad of piece parts, to the final test, 

certification, and delivery of an integrated whole.

This edition of the Leading Edge features articles describing NSWCDD’s 

work in pioneering integrated solutions for the Surface Navy as part of the 

overall Navy I&I Enterprise. This edition is structured to highlight our 

specific contribution to the two primary areas of focus, Developmental I&I 

and Operational I&I. You will see that we have parlayed our traditional 

systems engineering disciplines that develop highly integrated and interoperable shipboard combat and weapon systems with an expanded view to 

the overall warfighting mission at the Naval and Joint Force levels. By 

aligning more closely with the operational community, the technicians and 

the tacticians are now working much more closely together to a common 

mission goal. A much greater focus on “Technical-to-Tactical” is evolving 

across the Navy Enterprise, and we at NSWCDD are quickly ramping up to 

answer the call for Surface Warfare.

I invite you to explore the Integration and Interoperability edition of Leading Edge and learn about how we are implementing these game-changing 

concepts into our mission engineering. I am proud to be Commander of 

one of the Navy’s premier research and development facilities for integration and interoperability and am confident that NSWCDD will continue its 

legacy as the leader in Warfare Systems Development and Integration.

At NSWC Dahlgren, we turn ships into WARSHIPS.

Introduction

Captain Brian R. Durant

Commanding Officer

Naval Surface Warfare Center

Dahlgren Division
WARFARE CENTERS 

DAM NECK

2 | LEADING EDGE January 2015

Dr. James D. Moreland, Jr.

Chief Engineer

Naval Surface Warfare Center 

Dahlgren Division

Neil T. Baron

Distinguished Scientist for 

Combat Control

Naval Sea Systems Command

Integration & Interoperability 

Edition · January 2015

Subject-Matter Leads U.S. Northern Command

Admiral Bill Gortney, Commander, United States Fleet Forces Command

Office of the Assistance Secretary of the Navy (Research, Development & Acquisition)

Mary Lacey, Deputy Assistant Secretary of the Navy for Research, Development, 

Test & Evaluation 

Naval Surface Warfare Center, Dahlgren Division (NSWCDD)

Captain Brian R. Durant, Commanding Officer

Dennis M. McLaughlin, Technical Director

Janice R. Miller, Corporate Communications Director (Acting)

Margie Stevens, Production Coordinator

Clement Bryant, Cover Design

Renee Parker, Editor

Brandy Plucker, Layout Design & Graphic Artist

Merritt Hepinstall, Graphic Artist

Trey Hamlet, Graphic Artist/3D Modeling

Bill Tremper, Graphic Artist

Chris Markwith, Graphic Artist

Darren Barnes, Author

Jonathan Brown, Author 

Dave Clawson, Author

Mary Ann Cummings, Author

Stephen Haug, Author

Linda Learn, Author

Gregory Little, Author

Sarah Lloyd, Author

Greg McHone, Author

William Ormsby, Author

Joseph Pack, Author

Peggy Rogers, Author

Sara Wallace, Author

Joel Washington, Author

Keith Wheeler, Author

Michael Zemore, Author

Tony Zilic, Author

Combat Direction Systems Activity (CDSA) Dam Neck

Commander Matthew Carroll, Strike Force Interoperabililty Officer

Leading Edge magazine—produced by the Naval Surface Warfare Center, Dahlgren Division, 

Dahlgren, Virginia—is an official, authorized publication of the Naval Warfare Center Enterprise. 

The purpose of the publication is to showcase technical excellence across the Warfare Center 

Enterprise and promote a broader awareness of the breadth and depth of knowledge and support 

available to the Navy and the Department of Defense.

NSWCDD-MP-15-00096 01/15

Approved for public release; distribution unlimited.
Integration & Interoperability | 3

Table of Contents

OVERVIEW

Mission Engineering

Integration and Interoperability (I&I)......................................... 4

By Dr. James D. Moreland, Jr.

DEVELOPMENTAL

INTRODUCTION

Maintaining Technical and Operational Cohesiveness 

Across Mission Areas...................................................................... 15

By Mary Lacey

Integrated Capability Framework:

Building a Metadata Structure for Diverse Products......... 18

By Stephen R. Haug

System of Systems Engineering Guidance ............................25

By William F. Ormsby

Integrated Mission Baselines:

Using Fleet-Owned, System Command Serviced 

Architectures to Improve System Integration and 

Interoperability................................................................................36

By Joseph Pack

Capability Acquisition Management:

Modifying the Acquisition Process to Emphasize I&I.........49

By Joel Washington

I&I From a Cost Perspective.........................................................56

By Jonathan Brown and Sarah Lloyd

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based

on Mission Effectiveness..............................................................65

By Greg McHone

The Science of Integration “What Lurks in the Darkness 

of the Interstitial Space?” ............................................................. 76

By Neil Baron

Capability Evolution Documents:

Managing R&D and S&T Investments to Improve

Warfighting Capabilities...............................................................84

By Anthony Zilic

Integration and Interoperability Fratricide Safety..............90

By Peggy L. Rogers and Michael G. Zemore

OPERATIONAL

INTRODUCTION

Strengthening the Technical-to-Tactical Link to Ensure 

Fleet Readiness................................................................................97

By Admiral Bill Gortney

Warfare Capability Baseline:

Assessing Operational Gaps.....................................................100

By David A. Clawson, Sara E. Wallace, 

Gregory D. Little, and Keith Wheeler

Closing the Gaps...Capability Solutions Management....106

By Linda Learn

Technical-to-Tactical Interfaces:

Importance of Consistent Interface between 

Warfighters and Scientists/Engineers................................... 110

By Greg McHone and Dr. James D. Moreland, Jr.

Exploring Solutions Across Doctrine, Organization, 

Training, Materiel, Leadership and Education, 

Personnel and Facilities (DOTMLPF) Using 

Orchestrated Simulation through Modeling (OSM)......... 117

By Sara Wallace and Mary Ann Cummings

Mission Level Test and Experimentation:

Drive Mission Level Performance During Early 

Development .................................................................................126

By Neil Baron, Darren Barnes, 

and Dr. James D. Moreland, Jr.

Strike Force Interoperability Officer (SFIO) Program........134

By Commander Matthew Carroll, USN
4 | LEADING EDGE January 2015

Table of Contents

4 | LEADING EDGE January 2015

Mission Engineering

Integration and Interoperability (I&I)

By Dr. James D. Moreland, Jr.

Dr. James D. Moreland, Jr.

Chief Engineer

Naval Surface Warfare Center

Dahlgren Division
Integration & Interoperability | 5

Table of Contents

Mission Engineering Integration and Interoperability (I&I)

The complex and now highly integrated machines 

of naval warfare continue to evolve enabling higher 

precision, more effective power projection and safer 

defensive postures for the Fleet. The interconnectedness of our own social fabric is finding its way into our 

ships, airplanes, submarines, networks, tanks, and the 

very weapons they deliver. This paper identifies the 

critical events that have occurred as a direct result of 

the Chief of Naval Operation’s (CNO) Integration and 

Interoperability (I&I) Summit on 9 December 2010 to 

include the development of the Vice Chief of Naval 

Operations’ (VCNO) I&I Activity which is driven 

by a formal charter. The Naval Systems Commands 

(SYSCOMs) are charged to place an increased emphasis on assessing the I&I of warfare systems to support current and future readiness for critical mission 

threads. The assessment of naval technologies, systems 

and/or capabilities requires a system-of-systems (SoS) 

approach to analyze the impact of making these naval 

investments across the diverse domains of surface, 

undersea, air, land, and networks as well as maritime 

coalition force integration. These assessments are 

executed following a systematic, quantifiable, and 

iterative approach referred to as Mission Engineering 

which combines the structure of Systems Engineering 

and the tactical insights of operational planning. The 

findings are captured in “effects/kill chains” to clearly 

identify operational needs based on the way we plan 

to fight through mission threads captured in our 

Combatant Command’s Operational Plans (OPLANs) 

and Contingency Plans (CONPLANs). Mission Engineering emphasizes capability-based assessments to 

produce integrated warfighting capabilities that can 

be translated into specific programmatic guidance 

for strategic programs. The technical baselines developed will interject warfare system details into the 

Planning, Programming, Budgeting, and Execution 

(PPBE) System and Naval Capability Development 

Process (NCDP) to drive the Program Objective 

Memorandum (POM) for today’s readiness and the 

future capabilities of our naval force. The overall 
Table of Contents

6 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

objective of this I&I Activity is to provide responsive, 

credible analyses and engineering to inform decision 

makers of the results, insights, and alternatives of 

organic naval warfare capabilities for the integration 

of these capabilities within joint warfighting campaigns enabling more effective civilian and military 

leadership decisions.

To achieve the naval enterprise goal of having its 

major systems interoperate to create warfighting capability requires an accountable governance structure 

with mission-level processes and tools. Naval material 

acquisition traditionally delivers piece parts of warfighting capability (e.g., radars, weapons, command 

and control systems) and the platforms that deliver 

these piece parts to the fight (e.g., ships, submarines, 

airplanes, tanks, etc.). In today’s acquisition process, 

these individual programs are matured independently 

with the resulting negative effect of integrating SoS 

when delivered to the Fleet. Developments and purchase contracts are many times sub-optimized for 

programmatic business and production purposes. 

The time between the initial vision of an integrated 

warfighting capability and the realization of all the 

necessary piece parts coming from various program 

offices can span many years or even decades. An 

additional long-term mission focus activity must be 

appended to the acquisition process to assure the integrated warfighting capability originally envisioned 

is actually delivered. A mission-based emphasis is 

required to focus across the entire developmental 

timeline from describing, integrating, testing, and 

finally delivering warfighting effects at best value 

across representative operational environments. Since 

these developments can take a long time to be realized, 

there is a critical need for factually informed, datadriven mission thread assessments from OPLANs and 

CONPLANs to continually describe and reconfirm 

warfighting gaps. 

Mission architectures serve as the bridge for 

tactical-to-technical understanding by describing 

warfighting capabilities in a functional context 

and, most importantly, to set detailed force-level 

engineering requirements. These requirements need 

to be actively tracked through decomposition into 

Programs of Record (PoRs), validated, and certified at 

Fleet delivery. In addition, technical reference designs 

are needed for the major interface implementations, 

and mission-based testing is required to provide 

independent operational assessments of mission 

capabilities across systems. These elements are the 

enablers to support decision-making and governance 

for warfighting effects, ensuring the naval systems 

are integrated and interoperate to create the needed 

warfighting capabilities to effectively execute mission 

threads. While it is absolutely necessary to define 

the effects/kill chains and to govern these products, 

there also needs to be government control and 

accountability of major weapon system interfaces 

across the warfighting domains of surface, undersea, 

air, land, and networks. Specifically, defined technical 

reference designs delineating how interoperability 

needs to be achieved across diverse domains is 

crucial for mission success as measured by desired 

effects criteria.

Definitions

The following definitions provide contextual 

information for critical terms used in this article and 

throughout this edition of Leading Edge magazine.

Integration: The composition of a capability 

by designing and assembling elements in a way 

that allows them to work together to achieve an 

intended purpose.

Integrated Capability Technical Baseline: An 

architecture that consists of a list of functions and 

some indication of their interfaces or Information 

Exchange Requirements and interactions with 

each other and with functions located outside 

the system’s boundary. Establishing this baseline 

facilitates a shared understanding across multiple products, organizations, and disciplines with 

respect to mission needs.

Integration Readiness Level (IRL): A systematic 

measurement of the interfacing of compatible 

interactions for various technologies (components) and the consistent comparison of the maturity between integration points.
Integration & Interoperability | 7

Table of Contents

Mission Engineering Integration and Interoperability (I&I)

Interoperability: The ability of two or more systems or components to exchange information 

and to effectively use the information that has 

been exchanged.

Interstitial Space: The interstitial space consists 

of the logical information characterizing the relationships between system integration. It is where 

the interface requirements and characteristics 

between elements are exercised. Characterization 

of the interstitial space enables insight into higher 

level system behaviors.

Mission Engineering (ME): Planning, analyzing, 

organizing, and integrating current and emerging 

operational concepts for the purpose of evolving the end-to-end operational architecture and 

capability attributes, across the Doctrine, Organization, Training, Materiel, Leadership and Education, Personnel and Facilities (DOTMLPF) spectrum, including anticipated Blue Force (BLUFOR) 

and Opposition Force (OPFOR) behaviors, that 

are needed to inform the communities of interest 

involved in fulfilling mission needs statements.

Systems Engineering (SE): An interdisciplinary 

approach and means to enable the realization of 

successful systems. This structured, quantifiable, 

iterative and transparent development approach 

integrates all the disciplines and specialty groups 

into a team effort that proceeds from system conception through disposal covering the complete 

life cycle of a system.

Purpose

The I&I Activity is a concerted approach to develop 

recommendations for functional end-to-end accountability for I&I sequential activities that incorporates 

warfighting capabilities assessment; gap and solution 

recommendations across the DOTMLPF spectrum; 

adjudication, resourcing and acquisition of solutions; 

successful Fleet implementation; validation of solutions; and a workable governance. This new approach 

incorporates a mission focus on integrated capability 

development into the traditional SE “V” as depicted 

in Figure 1.

Figure 1. Mission Engineering Within the Systems Engineering “V” Model
Table of Contents

8 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Background

On 9 December 2010, the CNO hosted a summit on 

I&I. During the summit, the concept of an approach 

to I&I was briefed that would provide for sequential 

activities as a proactive means to: identify shortfalls 

in current capabilities (Warfare Capability Baseline); 

develop comprehensive solution recommendations 

to identified operational gaps (Capability Solution 

Management); and process the results within the 

Department of the Navy (DON) for approval, execution, and implementation in the Fleet. The critical 

outcomes of this summit are identified below:

Integration and Interoperability 

Summit Outcomes

• Sharpen our focus on creating Integrated Warfighting 

Capabilities

• Develop/Institute formal governance approach for 

creating Integrated Warfighting Capabilities

• Promote awareness of existing capabilities in our 

Warfare Centers (people and facilities) to create 

Integrated Warfighting Capabilities

• Obtain agreement to adjust some Acquisition 

processes to ensure interoperability requirements are 

emphasized

An update on the progress of developing an effective approach to I&I was briefed on 20 April 2011 

during the CNO Executive Group (CEG-V) meeting 

that included a proposed sequence of critical steps 

and the associated roles, responsibilities, accountability, and products for each step. Commander United 

States Fleet Forces Command (CUSFFC) and VCNO 

directed a war-game to assess the functionality of the 

I&I approach presented at the CEG-V, with a specific 

focus on organizational alignments, accountability, 

and capability to pass results between critical steps. 

Leadership direction was to evaluate the ability to 

achieve executable results within the framework of 

existing processes. The war-game was conducted 13-16 

June 2011 and consisted of three and a half days of 

mission analysis and course of action (COA) comparisons that revealed findings and recommendations on 

how to institutionalize I&I activities and influence 

positive behavior change within the naval enterprise. 

The war-game objectives were set as shown below:

Integration and Interoperability 

Wargame Objectives

• Identify and align supporting processes and expertise 

required to address effects/kill chain capability 

shortfalls

• Provide for development of notional DOTMLPF 

integrated capability recommendations and actions to 

address shortfalls

• Demonstrate a methodology to effectively address 

the products from each step of I&I activity, leveraging 

the available USN processes for capability 

development and implementation

• Define the concept of a warfare mission area owner 

Over 40 war-game participants were drawn from 

across the naval enterprise providing expertise in 

Systems Engineering, testing, operations, acquisition, and governmental processes. Where possible, 

the I&I approach leverages existing DON processes 

(e.g., Joint Capability Integration and Development 

System (JCIDS), Capability Based Assessments (CBA) 

process, NCDP, Programming, Planning, Budgeting 

and Execution (PPBE) System and the Analysis of 

Alternatives (AoA) process). Specific focus areas were 

accountability and expertise within each step, and 

product wholeness of each deliverable. For the purposes of the war-game, all I&I organizations executed 

their respective roles within existing resources, recognizing that full execution of I&I within DON may 

require individual organizations to realign resources 

and re-prioritize current efforts.

Executing the I&I Activity 

The information and products developed under 

the I&I Activity provide naval leadership with current 

capabilities and future requirements, thus equipping 

decision makers with the information necessary to 

better prioritize limited resources. This naval leadership charge focused on conducting end-to-end mission thread assessments of critical warfare mission 

areas and to rectifying the identified operational gaps. 
Sensor

Observer

Strike

Platform

Tactical

Fires C2

Operational

Fires C2

Mission Area/ ROC

(Capability)

Person/Role

(Performer)

Platform

(Performer)

Resource/ Information

Mission

(Scenario)

Task (Activity)

System

Function

(Activity)

System

(Performer)

TTP/

Doctrine

(Rule)

Measure

Condition

Desired

Effect

Mission Area/

ROC

(Capability)

Defined By Set Of Achieves Specified By

Enables

Performs

Resides On

Exchanges

Uses

Resides On

Exchanges

Consumes/produces

Consumes/produces

Allocated To

Implemented By

Performed Under

Performed IAW Specified

Constrained By

Constrained By

Performed IAW Specified

Composed Of The Time

Ordered Sequence Of

ICF defines critical data elements,

relationships, taxonomies, and

implementation guidance to 

address I&I Use-Cases

MISSION TECHNICAL BASELINES

Mission

Areas,

Capabilities

Level 1

Tasks

Scenario,

Mission

Thread

IER,

Functional

Reqts

Enabling Non Kinetic

 AMW MW SUW C3 INT EXW NCO NV FHP LOG

 AW STW ASW MOB IO MOS NSW FSO STS CON

ROC/POE,

JCAs

ICF data model includes canonical

DoDAF relationships for SoS

traceability to mission

SYSTEM/MISSION TECHNICAL BASELINES

Functional

Capabilities

Node

Connectivity

System to Node

Mapping

Standards

Views

INTEGRATED

CAPABILITY

TECHNICAL

BASELINE

Kinetic

ASW-1

Prepare/

Configure

ASW-2

Search

ASW-3

Search

ASW-4

Track

ASW-5

Classify

ASW-6

Engage

ASW-7

Assess

ASW-8

Defend

ASW-9

Post Mission

Task

Std 1 Std 2

CONOPs

Node A Node B

Information

Element

System

Function 1

(SF1)

Activity 1 Activity 2 System

Function 2

(SF2)

NODE A NODE B System 2

SV-4

System 1

SV-4

SF1 SF2

Comm

Link

S1

S2

SYSTEM/MISSION ALIGNMENT

Integration & Interoperability | 9

Table of Contents

Mission Engineering Integration and Interoperability (I&I)

The recommended fixes would be drawn from the 

entire DOTMLPF spectrum to consider all possible solution sets. These solution sets must then be 

synchronized across the individual elements of the 

spectrum to conduct complete and rigorous integrated 

warfighting capability transformation. 

The I&I Activity execution plan takes into account 

a holistic approach to include operational, developmental, and conceptual execution states. Operational 

I&I focuses on near-term investments to execute quick 

returns to the naval force and serve as the foundation for future interoperability enhancements. Developmental I&I emphasizes mid-term investments to 

build on the “as-is” foundation and strives to produce 

a highly interoperable naval force by 2025. Finally, 

Conceptual I&I drives long-term research and development (R&D) investments to help guide the science 

and technology (S&T) efforts necessary to design, 

create and implement a future naval force network 

that is fully interoperable with the joint force.

In order to solve complex integration and interoperability challenges early in the systems engineering 

life cycle, it has become apparent that these issues 

need to be addressed during experimentation and 

test on the left-side of the Systems Engineering “V” 

Model. This requires the ability to execute mission 

threads in a representative operational environment. 

Since the integration of new and legacy systems is 

required, a live/virtual environment must be developed to incorporate modeling and simulation (M&S), 

Hardware-in-the-Loop (HWIL) and virtual testbeds 

to include virtual worlds. Agent-Based M&S techniques and frameworks has become a critical research 

and development area for investigating the behaviors 

between systems, which provides insights on emergent 

behaviors not characterized in requirements. Likewise, virtual worlds and virtual testbeds have been 

developed to provide a representative environment for 

Figure 2. Integrated Capability Framework - Meta Data Structure
Mission Technical Baseline (MTB)

Aligned to ROC/POE mission areas —

Defines the mission “what”

Integrated Warfighting

Capability Package (IWCP)

Full set of acquisition data required

to execute the modifications to all

assets (platforms, networks,...)

required to comply with ICTB

Integrated Capability Technical 

Baseline (ICTB)

Brings Principal Warfare Commander

assets together to perform the mission 

(Kill Chain) then adds technical depth 

to enable IWCP development

 IWCP

Search

P-3/8A

DDG

LCS

SSN

Other

Detect

P-3/8A

DDG

LCS

SSN

Track

P-3/8A

DDG

LCS

SSN

Classify/ID

P-3/8A

DDG

LCS

MH-60R

SSN

Engage

P-3/8A

DDG

LCS

MH-60R

SSN

Assess

P-3/8A

DDG

LCS

MH-60R

SSN

Defend

P-3/8A

DDG

MH-60R

Building ASWC 

Constructive

DDG

Construction Helo Virtual

LCS Live Torpedo

Construction

Link 16, Satcom, Hawklink

SDREN/

JMETC

Table of Contents

10 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

both training and the development of mission-level 

requirements. This ability to evaluate I&I during 

early engineering development better prepares the 

acquisition community for success during final operational test, which has now been transformed into 

mission-based test.

A foundational element of working across many 

mission-based products is a structured metadata 

structure to relate and store crucial information for 

easy retrieval in a tool agnostic way. The I&I team 

has developed an Integrated Capability Framework 

(ICF) to define cross-domain relationships as shown 

in Figure 2. This framework develops the contextual 

relationships across the naval enterprise, which allows 

for an effective working relationship and better understanding of roles and responsibilities when working 

between technical disciplines and tactical tribes.

This metadata structure needs to support the 

alignment of crucial technical baseline elements of 

the I&I Activity. These technical products define the 

mission requirements and engineering development 

details across programmatic boundaries through the 

appropriate architecture products with an emphasis 

on effective mission success. The goal is to define 

the required SoS linkages to provide an integrated 

warfighting capability as illustrated in Figure 3. These 

products serve as the technical reference documents 

to drive synchronization.

Operational I&I

This aspect of I&I addresses the near-term factors associated with current warfighting capability 

gaps. The process provides critical decision-making 

information through a feedback loop to the acquisition process on capability delivered according to the 

methodology in Figure 4.

The assessment of naval technologies, systems and/

or capabilities requires a system-of-systems (SoS) 

approach to analyze the impact of making these naval 

investments across the diverse domains of surface, 

undersea, air, land, and networks as well as maritime coalition force integration. This assessment is 

Figure 3. Mission Level Technical Reference Products
DOT_LPF

Warfare Area Owner Mission Area Advocacy

Processing Resourcing Decisions

Warfare Capability

Baseline (WCB)

Capability Solution

Management CSM

Integrated Capability

Packages

Materiel Solution

Development

DOTMPLF Solution

Implementation

ICP

Resource ($)

Solutions Test & Validate Solution Closes Gap

OPNAV/ASN (RDA)

Are ICP recommendations

completely within OPNAV

Resource Sponsor span 

of control?

VCNO FFC/CPF (ICP Signed)

DNS

(Gate

Keeper)

No

Yes

RDA

In Scope

N8

 Not in Scope

Gate & 

Review

N1

N4

N2/6

High 8

Brief to

N8 or

N2/W6

N8 or

N2/N6

Review

NCB/

R38 0-7

A

B

C

D

R3B

COTF/Warfare

Center

Fleet Collaborative 

Team (FCT)

WCOE Lead

WCB

Report

Capability

Management Team

SYSCOM Lead

Mission Area

Kill Chain Assessment

Mission Area

“Effects/Kill Chains”

Selection

Key Deliverable:

Prioritized Mission

Areas and 

“Effects/Kill Chains”

Capability Gap

Identification

Key Deliverable:

‘Effects/Kill Chains”

Capability Gap Report

Capability Gap Closure

Recommendations

Key Deliverable:

Integrated Capability

Package (ICP)

Prioritized Capability Gap

DOTMLPF Solution Set

Key Deliverable:

Resource ($)

Solution

Informed Investment

Decisions

Assessment

Agenda

OPNAV/ASN(RDA)

Resourcing

Decisions

DOTMLPF

Solution

Development

Kill Chain

Assessment

Integration & Interoperability | 11

Table of Contents

Mission Engineering Integration and Interoperability (I&I)

accomplished through the development of effects/kill 

chains to illuminate capability advantages and disadvantages of the alternatives; consider joint operational 

plans; examine sufficient feasible alternatives; characterize key assumptions, variables, and sensitivities; 

as well as assess technology risk and maturity. Figure 

5 shows an example of a notional Air Warfare (AW) 

effects/kill chain which consists of eight mission tasks 

(plus three sub-tasks) and six C3 nodes as illustrated 

by the rectangles.

The second phase of this process identifies proposed DOTMLPF solution sets to fix the degraded/

broken effects/kill chains according to Fleet priorities. The Warfare Centers of Excellence serve as the 

lead for these solution sets, which are documented in 

an Integrated Capability Package (ICP). The subject 

matter experts across the DOTMLPF spectrum collaborate on solutions considering important trade offs 

across that spectrum as well as the synchronization 

of modifications across the spectrum elements.

An example of this collaboration is the Naval 

Integrated Fire Control Counter Air (NIFC-CA) project, which was officially recognized as a joint venture 

in 2002. This system-of-systems engineering effort 

extends the Naval Theater Air and Missile Defense 

battlespace to the maximum kinematic range of our 

weapons. The capability focuses on targets beyond the 

detection range of the shooter, including Engage-onRemote (EoR) and Over-the-Horizon (OTH) targets. 

Formal scoping and structure were required based on 

detailed examinations using effects/kill chains and 

operational test data to determine operational needs 

for Fleet leadership. A critical governance element of 

this formal integrated warfighting capability structure 

involved the decision to direct Program Executive 

Office – Integrated Warfare Systems (PEO IWS) to 

establish a NIFC-CA Systems Engineering and Integration Project Office to integrate across the elemental 

programs in support of the development and acquisition of a NIFC-CA capability. This instantiation of 

a formal NIFC-CA project began with the critical 

elements identified in Operational I&I, namely the 

Figure 4. Integration and Interoperability Activity Methodology
AW-1

Prepare

Configure 

AW-2

Search

AW-3

Detect

AW-4

Track

AW-5

ID

AW-8

Defend

AW-6

Engage

AW-7

Assess

AW-6a

Launch

AW-6b

Control

AW-6c

Weapon

C3-1

Initialize

Plan 

C3-2

Process

Display 

C3-3

Decide:

Commit 

C3-4

Declare:

Hostile 

C3-5

Decide:

Engage 

C3-6

Decide:

Assess 

Platform 1

Decision

Maker 1 on

Platform X 

Decision

Maker 2 on

Platform Y 

Platform 2 

AW tasks & scores are above this line.

C3 tasks and scores are below this line.

Primary kill-chain path 

Alternative

kill chain path

C3 task name & ID number 

C3 scoring node for

task C3-5 

C3 data link scoring line 

AW task name

& ID number 

AW scoring node for

task AW-6b 

Table of Contents

12 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

determination of facts-based operational gaps and 

recommended solution sets across the DOTMLPF 

spectrum and warfighting domains. 

Developmental I&I

In today’s environment, most technical decisions 

driving the POM are based on the advertised performance of a system as a prediction of what should be 

versus data-driven measurements obtained during 

operational evaluations representing ground truth of 

what is. These operational evaluations can provide a 

higher level integrated Technology Readiness Level 

(TRL) of the system in context with the real environment and therefore the I&I of multiple systems 

to achieve a warfighting capability. What has been 

missing is the inclusion of the Integration Readiness 

Level (IRL) into the modulus where the interaction 

between systems is evaluated to recognize the activity resident in the interstitial space between systems 

where SoS behaviors are realized. This bridge between 

tactical and technical operations provides additional 

technical depth to existing Front End Assessment 

(FEA) products to ensure accuracy in determining 

capability gaps and therefore acquisition decisions 

on what needs to be procured for Fleet readiness. The 

overall objective is to produce a data-informed Warfighting Capability Plan as part of the PPBE System 

to eliminate financial waste, increase competition, 

and procure more relevant products.

Another critical element toward improving the 

acquisition process is to provide good linkages 

between the left and right sides of the SE “V” and 

prepare for increased technical emphasis on mission 

requirements. The scientists and engineers engaged 

in the I&I Activity are currently taking the lessons 

learned in evaluating today’s independent systems on 

the right side of the SE “V” (refer to Figure 1) to better 

inform the left side of the SE “V”. This is accomplished 

Figure 5. Notional Air Warfare Effects/Kill Chain
Integration & Interoperability | 13

Table of Contents

Mission Engineering Integration and Interoperability (I&I)

through DASN RDT&E initiatives by modifying the 

Systems Engineering Technical Reviews (SETRs) and 

acquisition Gate Review Requirements to identify 

problems early in the development process and therefore drive toward better success in the production 

of I&I systems while gaining more pre-Milestone B 

trade space. In addition, the SoS Guidebook is being 

updated to incorporate mission-level SE operational 

objectives and designated technical authorities. These 

efforts are also providing the naval workforce with 

more experience in understanding ME and the production of SoS capabilities versus myopic views of 

individual systems, which may or may not affect 

the overall capability. A major part of this education has come from the development of the Mission 

Level Assessment and Evaluation (MLA&E) process, 

which is an engineering approach to an iterative Fleet/

Acquisition capabilities-based methodology by which 

operational needs are defined through warfightervalidated mission architectures that influence Joint/

Fleet rehearsals, exercises and experiments.

Continuing with the NIFC-CA example thread, 

Developmental I&I elements play a crucial role as 

NIFC-CA executes as a capabilities-based acquisition 

project, levying minimal requirements onto the component systems while deriving SoS capability from 

the federation of these independent systems. Multiple 

programs must be tightly synchronized across both 

actual development and future acquisition strategies. 

In today’s environment, this approach has matured 

into the execution of three effects/kill chains called 

From the Air (FTA), From the Sea (FTS), and From the 

Land (FTL) with the challenge of keeping all in sync 

on a daily basis. The NIFC-CA project exploits capabilities inherent in existing systems, optimizes current 

and emerging technologies in component system 

upgrades, integrates them and performs effects/kill 

chain tests, therein forming an interoperable SoS to 

maximize future air defense capabilities. Early in 

the effort, the emphasis was to support system definition and architecture development, performance 

prediction, performance assessment, system test and 

risk reduction efforts, system analysis, modeling and 

simulation, and capability demonstrations in order to 

drive the acquisition decisions in the PPBE system. 

This project also facilitated the development of the 

concept of operations with the warfighter to maximize 

effectiveness when deployed with the Fleet.

Conceptual I&I

Conceptual I&I concentrates on the longer range 

objectives to establish government-controlled trade 

space to enable more affordable and flexible weapons 

systems. The naval environment is implementing 

Open Architecture (OA) as an overarching strategy 

to acquire, upgrade and maintain weapons systems 

using an evolutionary approach to achieve commonality. Conceptual I&I takes into account acquisition 

law, program delivery schedules, and supportability 

from both a financial and personnel resources perspective. This convergence of business practices and 

technical design agility is producing modular systems 

with greater success in the I&I of complex systems. A 

challenge and significant part of this future direction 

involves the linkage between new and legacy components/interfaces to facilitate rapid development and 

effective I&I between systems.

In order to reduce the total ownership cost (TOC) 

of weapons systems, it is essential to design and 

develop systems based on cross-domain solutions 

with a move away from platform-centric capabilities. Sharing data and building modular systems is 

essential to leverage across programs (buy once and 

reuse) with the ultimate objective to reduce cycle time 

for transitioning new capabilities to the operational 

environment. The success of the Open System Architecture (OSA) depends heavily on an innovative and 

well-educated workforce. The naval enterprise must 

produce a workforce that is well versed in identifying 

and managing cross-domain and life cycle dependencies, understanding and responding to adverse 

vendor behaviors, ensuring that competition yields the 

desired results, and incorporating OSA best practices 

as an integral part of program management.

Force level security and safety are being investigated as part of Conceptual I&I under the auspices 

of the Naval Ordnance Safety & Security Activity 

(NOSSA), incorporating and utilizing the Weapon 
NEAR-TERM I&I

• Addresses current warfighting

capability gaps, provides 

feedback loop to acquisition 

process on capability delivered

LONG-TERM I&I

• Longer range objective to

establish Governmentcontrolled trade space to

enable more affordable and

flexible weapons systems

MID-TERM I&I

• Improvements across acquisition

system to deliver fully integrated and 

interoperable warfighting capability

Gate Reviews

Systems Engineering

Technical Review (SETR)

Resource Requirements

Review Board (R3B)

Mission Level Assessment

and Evaluation (MLA&E)

SoS Engineering Guidebook

Integration Readiness Levels (IRL)

SYSCOM Technical Authority

Mission Technical Baseline

Capabilities-Based T&E Continuum (LVC)

Warfare Capability Baseline/

Integrated Capability Package

Governed by

VCNOI&I Charter

Rapid

Response

supporting 

deploying

CSG/ESG

Sensor/Platform

Interface & Engineering

Standardization

Mission Level

Experimentation & Test

Warfare Analysis

Network Enabled Weapon

Controller Interface Module

Government Lead

Systems Integrator

Table of Contents

14 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

System Explosives Safety Review Board (WSESRB) 

and its current processes. This is essential to ensure 

that the resultant distributed effects/kill chains will 

perform effectively and safely across all SoS component systems. Key to Conceptual I&I are discovering 

and documenting how best to review and assess the 

I&I characteristics of weaponized systems to understand safety risks; identify hazards and causal factors; assess monitors and mitigations; assess test and 

validation; and issue I&I safety findings or actions. 

According to systems engineering best practices, 

common taxonomy, processes, tools, and objective 

quality evidence shall be injected into naval policies 

and guidance, into SETR criteria and the Probability 

of Program Success (PoPs)/Gate Review process and 

metrics.

The critical areas of investigation for the I&I Activity are illustrated in Figure 6.

These areas of investigation were addressed in a 

SYSCOM Mission Level Technical Authority Summit 

held on 9 December 2013 to synchronize processes 

and approaches for the execution of these complex 

initiatives across the SYSCOM communities. This 

event provided reviews and in-depth discussions on 

the critical elements of the I&I Activity and set the 

path forward.

Recurring actions sponsored by the Program Executive Offices (PEOs) are driving change in the way 

we do business based on this facts-based approach 

concentrating on end-to-end execution of mission 

threads. Discussions have focused on the coordination of the Capability Phasing Plan (CPP), Surface 

Warfare Technical Requirements Group (SWTRG), 

I&I Activity, and Warfare Improvement Program 

(WIP) processes. As part of this coordination, detailed 

discussions are occurring with the Major Program 

Managers (MPMs) to discuss the execution of the 

I&I process and the critical output products driving 

the POM.

As demonstrated within the NIFC-CA project, 

successful management of acknowledged SoS systems engineering projects requires reaching across 

organizational boundaries to establish a capability-based endgame. NIFC-CA is charged with bringing 

Figure 6. Intergration and Interoperability Activity Critical Areas of Investigation
Integration & Interoperability | 15

Table of Contents

Mission Engineering Integration and Interoperability (I&I)

together independent major defense acquisition programs (MDAP) as component systems of an integrated warfighting capability. The NIFC-CA DODAF 

architectures, in the form of Integrated Capability 

Technical Baselines, have proven to be powerful tools 

for capturing the functionality, communications, and 

essential information of this acknowledged SoS. These 

products serve as the authoritative source of information to capture conceptual systems engineering 

ideas and guide tasking to form actual engineering 

products. NIFC-CA has implemented the key systems 

engineering and software engineering techniques, 

including modularity, open architecture, abstraction, 

and information hiding which are critical to Conceptual I&I activities. These techniques were applied 

during the functional allocation and distribution 

process resulting in a system that is far more extensible, allowing dramatic evolution and innovation 

in the future.

Conclusion

An increased emphasis has been placed on accessing the I&I of warfare mission capabilities through 

the effective and efficient integration of current and 

future systems. These assessments are being accomplished through the development of effects/kill chains 

to identify I&I issues between the critical warfighting 

systems across mission areas. This is being accomplished through the assessment of effects/kill chains 

for specific tactical situations (TACSITs) threads 

within joint Operational Plans. The future vision of 

this mission-level analysis capability is to interject 

warfare system technical details into the major acquisition processes to drive the POM for future naval 

integrated warfighting capabilities. 

It is absolutely necessary to define operational gaps 

through effects/kill chains and to govern these, but 

also needed is government control and accountability 

of major weapon system interfaces and integrated 

capability reference baselines. The naval environment’s annual acquisition budget is founded on a 

“commodity” philosophy and is driven by a fast-paced 

environment where rapid decisions must be made 

without adequate time to determine the second-order 

impacts (interoperability, non-synergy of material 

procurements, operational capability imbalances, etc.) 

the acquisition decisions within the POM. Although 

the commodities are continually manipulated, rippling effects may aggregate consequences upon warfighting capabilities that are exercised by the Fleet at 

the force-level and therefore require investigation. The 

goal is to maintain technical and operational cohesiveness across mission areas in a fiscally constrained 

environment while increasing the overall capability 

for the warfighter. Adding a structured mission focus 

to the acquisition process will serve the I&I needs of 

integrated warfighting well. We don’t acquire commodities the way we fight, but we MUST engineer for 

the way we fight. 
Table of Contents

16 | LEADING EDGE January 2015

Introduction

To ensure the future technological superiority of our Fleet and Force, 

it is critical that prudent Department of the Navy (DON) Research, 

Development, Test, and Evaluation (RDT&E) investments provide combat 

effectiveness, affordability, and improved reliability and maintainability in 

our current and future weapon systems. With increasing fiscal pressure, it is 

imperative that the DON balances tactical and strategic investments across 

all accounts, ensures RDT&E investments target the correct warfighter 

missions, and expeditiously transition technologies and capabilities to Fleet 

and Force operators. The basic concepts of Integration and Interoperability 

(I&I) prompt us to look across the effects/kill chains to see how systems 

really work together in an operationally relevant environment. Through 

the rigor of experiments, testing, data-driven assessments, and review, the 

DON is looking for game-changer innovations that effectively integrate 

technology and systems to provide affordable solutions for our Sailors and 

Marines.

The Navy is aggressively pursuing I&I with the goal of maintaining technical and operational cohesiveness across mission areas while increasing 

the overall integrated warfighting capability for the warfighter. Front-end 

assessments based on operational evaluations that include the I&I of 

multiple systems ensure accuracy in determining capability gaps and lead to 

better acquisition decisions. A data-informed Warfighting Capability Plan 

is not limited to material solutions, but posits solutions across the Doctrine, 

Organization, Training, Materiel, Leadership and Education, Personnel, 

Facilities and Policy (DOTMLPF-P) spectrum. This approach couples 

rigorous analyses, coordination, and collaboration to produce holistic views 

across domains and functionalities. We are also modifying the Systems 

Engineering processes, particularly Technical Reviews and Gate Reviews, 

to identify disconnects early in the development process and thereby drive 

better success into the production of integrated and interoperable systems.

Even more critical than the processes and tools we put in place to enable 

systems integration and interoperability are our people. I&I analysis is 

largely an inherently governmental function and to truly understand 

the technical dimension of the military problem, our workforce must be 

engaged in technical work. For that reason, in concert with OPNAV, the 

Assistant Secretary of the Navy for Research Development and Acquisition 

(ASN(RDA)) is relying on our technical workforce—civilian and military—to lead and conduct I&I analyses. We are investing particularly in our 

Warfare and Systems Centers to do hands-on work to ensure the capability 

and capacity the DON needs are available. The I&I activity is shedding light 

on the technical requirements and organizational governance that must be 

satisfied to successfully deliver system of systems capability. 

Ms. Mary Lacey

Deputy Assistant Secretary of the Navy

Research, Development, 

Test & Evaluation
Table of Contents

Integration & Interoperability | 17

Maintaining Technical and Operational Cohesiveness 

Across Mission Areas

DEVELOPMENTAL

INTEGRATION & INTEROPERABILITY
Mi

(C

Person/Ro

(Performe

Platform

(Performer) 

System

(Performer

Resides On

Uses

Resides On

18 | LEADING EDGE January 2015

Table of Contents

Integrated Capability Framework:

Building a Metadata Structure for Diverse Products

By Stephen R. Haug

There are many producers and users of architecture data in the Integration and Interoperability (I&I) 

community, including operational mission planning, 

testing, training, acquisition and development, capability analysis and definition, cost analysis, and many 

more. No matter the purpose, the architecture created must include sufficient detail to meet the needs 

of the intended users; it must be created to enable 

communication and collaboration between the various users of the architecture, including the intended 

users and the unknown future users. This is especially 

important given the increasingly collaborative and 

net-centric nature of Navy and Joint missions. Missions performed by the Navy are rarely performed 

by a single ship or aircraft; rather, they are usually 

performed by operators using multiple systems on 

multiple platforms simultaneously.

There are many frameworks1 with different data 

models and taxonomies (languages) that attempt 

to enable architecture definition, but most fail to 

address all issues associated with defining architectures. Many times, frameworks use terms that are 

not understood by all users or the frameworks 

focus on one user at the expense of others. Each user 

has unique needs, but in order to integrate capabilities, the user processes must occur in collaboration 

across Navy operational, acquisition, test, training, 

and engineering stakeholders. To solve these issues, 

a cross System Command (SYSCOM) team created 

the Integrated Capability Framework (ICF)2 that 

leverages existing frameworks and is intended to be 

the common, overarching framework that drives and 

ties together each of the user processes.
Mission Area/

ROC

(Capability)

Person/Role

(Performer)

Platform

(Performer)

Resource/

Information

Mission

(Scenario)

Task

(Activity)

System

Function

(Activity)

System

(Performer)

TTP/

Doctrine

(Rule)

Measure

Condition

Desired

Effect

Defined By Set Of Achieves Specified By

Enables

Performs

Resides On

Exchanges

Uses

Resides On

Exchanges

Consumes/produces

Consumes/produces

Allocated To

Implemented By

Performed Under

Performed IAW Specified

Constrained By

Constrained By

Performed IAW Specified

Composed Of The Time

Ordered Sequence Of

ssion Area/

ROC 

Capability)

ole

r)

Resource/ Information

Mission

(Scenario)

Task (Activity)

System

Function

(Activity)

r)

TTP/ Doctrine

(Rule)

Measure

Condition

Desired

Effect

Defined By Set Of Achieves Specified By

Enables

Performs

Exchanges

Exchanges

Consumes/produces

Consumes/produces

Allocated To

Implemented By

Performed Under

Performed IAW Specified

Constrained By

Constrained By

Performed IAW Specified

Composed Of The Time

Ordered Sequence Of

Integration & Interoperability | 19

Table of Contents

ICF Creation

The first step in creating the ICF was to understand 

the needs of the various users. The ICF team identified 

a set of expected users of the ICF (both as architecture 

creators and users of ICF compliant architectures) 

and interviewed representatives from those positions 

and communities. After analyzing interview results, 

as well as other input from the I&I community, a few 

common themes emerged:

• Architectures must be built with consistent 

taxonomies.

• Architecture data must be from authoritative 

sources.

• Architectures must be based on missions as 

understood by the operational Navy.

• Architectures must contain a minimum set 

of data and views to be useful in defining a 

mission-based architecture made of people, 

platforms, and systems.

• System and mission architectures must be 

able to be mapped to each other.

Simultaneously, the ICF team evaluated the existing frameworks and taxonomies, as well as the Navy 

and DoD policy and guidance that constrained the 

ICF solution. One constraint identified was that all 

architectures created in the DoD are required to be 

Department of Defense Architecture Framework 

(DODAF) compliant. Similarly, the Department of 

the Navy (DON) Chief Information Officer (CIO) has 

issued an Architecture Development Guide (ADG) 

that constrains how DODAF architectures are created.
Mission Area/

ROC

(Capability)

Person/Role

(Performer)

Platform

(Performer)

Resource/

Information

Mission

(Scenario)

Task

(Activity)

System

Function

(Activity)

System

(Performer)

TTP/

Doctrine

(Rule)

Measure

Condition

Desired

Effect

Mission Area/

ROC

(Capability)

Defined By Set Of Achieves Specified By

Enables

Performs

Resides On

Exchanges

Uses

Resides On

Exchanges

Consumes/Produces

Consumes/Produces

Allocated To

Implemented By

Performed Under

Performed IAW Specified

Constrained By

Constrained By

Performed IAW Specified

Composed Of The Time

Ordered Sequence Of

Table of Contents

20 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

ICF Solution

The first issue addressed by the ICF was the 

common taxonomy. Without a common language, 

including the types of architecture data, the relationships between the data types, and the authoritative 

source of definitions of that data, architectures cannot 

be commonly understood or used by all parties. The 

DODAF Meta Model (DM2), which defines a number 

of types of data and relationships in architectures, was 

evaluated to determine the minimum set needed to 

fully define a mission-based architecture, including 

a determination that no additional relationships are 

needed. The resulting data model is shown in Figure 1. 

In the diagram, a commonly understood term is the 

first item in each data type block, and (as needed) the 

DODAF term is provided in parentheses. A definition 

for each data type is provided in Table 1.

In addition to defining the minimum data set 

needed for fully defined mission architecture, the 

ICF includes authoritative sources for the terms and 

definitions for each type. Without that common definition, architectures cannot be easily understood by 

all parties. As an example, consider the term used 

for a sweetened, carbonated beverage. According to 

the Harvard Dialect Survey3, more than 12% of the 

country uses the term “coke” to refer to all soft drinks, 

but a person in the northeast U.S. (where “soda” is 

the much more common term) may not understand 

an architecture that used “coke” to refer to anything 

but Coca-Cola®.

Similarly, without a common data model and taxonomy, architectures cannot easily be used in combination with other architectures. Using the same 

example, if an architecture was created for a soft-drink 

dispensing system that was to provide six dispensers 

of “coke,” and an architecture was created for a restaurant that needed a “Coke” dispenser, putting the two 

architectures together may not result in the desired 

effect. Even worse, if the restaurant architecture called 

for a “soda machine,” the “coke dispenser” may not 

even be considered.

To identify the appropriate authoritative sources, 

the ICF team evaluated existing taxonomies for each 

type of data to identify an authoritative source, taking 

Figure 1. Integrated Capability Framework Data Model
DATA ELEMENT DESCRIPTION TAXONOMY/SOURCE

Mission Area/Required 

Operational Capability 

(ROC) (Capability) 

The abilities needed to execute 

specified courses of action, 

organized into the specific segments 

of the overall Navy mission.

Mission Areas and Required Operational 

Capability/Projected Operational 

Environment Statements (Office of the 

Chief of Naval Operations Instruction 

(OPNAVINST) C3501.2K)

Task (Activity) Work, not specific to a single 

organization, weapon system or 

individual that transforms inputs 

into outputs or changes their state. 

(DODAF)

Navy Tactical Tasks (NTAs) as assigned to 

platforms/systems/organizations in Navy 

Mission Essential Task Lists. Note that 

detailed scenarios may require extending 

the leaf level activities defined in the NTAs.

Tactics, Techniques, 

and Procedures (TTP)/

Doctrine (Rules)

Fundamental principles by which the 

military forces or elements thereof 

guide their actions in support of 

national objectives. (JP 1-02) 

The constraints placed on any Task 

or System Function in performance 

of a mission scenario.

Operational Plans (OPLANs), Navy TTP

System Function 

(Activity)

Work assigned to a specific system 

that transforms inputs to outputs or 

changes the system state. (DODAF)

Joint Common System Function List 

(JCSFL)

Integration & Interoperability | 21

Table of Contents

Integrated Capability Framework: 

Building a Metadata Structure for Diverse Products

into account policy and guidance. An important 

factor in each choice was the appropriate “owner” 

of the taxonomy. For example, Tasks (Activities), 

are performed by operational Navy personnel – the 

enlisted and officer Sailors out in the Fleet. As such, 

it is appropriate to use a taxonomy in use in the operational Navy, the Universal Naval Task List (UNTL), 

rather than one made by acquisition personnel. A 

sample of the resulting ICF taxonomy is provided 

in Table 1.

Once the minimum data set and the authoritative sources were defined, the ICF team defined a 

minimum set of products or views into the data. 

Each product is built from a subset of data in the ICF 

Data Model, and from the authoritative sources. In 

addition to the prerequisite Overview and Summary 

Information (AV-1) that defines the scope and purpose 

of the architecture, the ICF identifies two major sets 

of products – the Mission Technical Baselines (MTB) 

and the System/Platform Technical Baselines (S/PTB). 

The list of products is shown in Table 2, including a 

description and the DODAF terminology.

MTB products are based on a Tactical Situation 

(TACSIT) definition of the mission to be performed, 

which includes key details such as the desired effect 

of the mission, the required capabilities, the tactics 

and rules of engagement (ROE), the tasks to be performed and the measures of performance (MOPs) for 

each task in support of the desired effect. Also in the 

TACSIT are the information that must be exchanged, 

and the system functions needed in support of the 

operational tasks. It is especially important to capture 

the ROE and the tactics as the Fleet expects to execute 

the mission, since these can greatly affect ordering of 

tasks and who performs them; the ability of the force 

to meet the desired effect; and even which systems are 

critical in support of the mission. As an example, if 

only command nodes are permitted to declare a target 

hostile, then the communications systems and networks on all platforms become critical systems for the 

mission even for a self-defense engagement, a detail 

that could be easily overlooked without the ROE.

The S/PTB products define the following: the platforms and systems that are expected to be part of the 

Table 1. Data Element Descriptions and Authoritative Source
ICF MISSION TECHNICAL BASELINE PRODUCTS

1. Concept of Operations (CONOPs)/OV-1/TACSIT

2. Required Capabilities/CV-2, CV-4, CV-5, and CV-6

3. Organizational Relationships Chart/OV-4

4. Mission Thread/OV-6a and OV-6c

5. Information Exchange Requirements (IERs)/OV-3

6. Functional Requirements/SV-5a

ICF MISSION TECHNICAL BASELINE PRODUCTS

1. Functional Capabilities/SV-4 and SV-7

2. System to Node Mapping/SV-1

3. System/Node Connectivity View/SV-2 and SV-6

4. System/Interface Standards/StdV-1

Table of Contents

22 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

mission; the functionality each must provide and 

the associated MOPs; the platforms on which each 

system resides; and the communications between 

each system and platform.

In order to fully define and evaluate the mission 

and identify potential interoperability issues, the 

systems and platforms in the architecture should 

be specific whenever possible. Instead of generically 

defining a carrier, helicopter, or radar, the specific 

hull number, aircraft type, and systems should be 

specified. Otherwise, details like different data link 

implementations or different radar performances can 

be overlooked, and the analysis of mission capability or interoperability of the systems would then be 

incomplete.

Identifying the owner of the views, is just as necessary as identifying the data and views needed. The 

view owner has the most understanding of the data 

in the view and is empowered to make the decisions 

about the views. For example, the operational view 

products in a mission-based architecture define the 

mission from the perspective of the warfighter, including the desired effect, tactics, environment, chain of 

command, tasks to be performed, etc. In order to be 

representative of actual Navy operation, this data must 

all be owned and defined by the Fleet, e.g., organizations like Fleet Forces Command (FFC), Pacific Fleet 

(PACFLT), the Operational Test and Evaluation Force 

(OPTEVFOR), and the Warfare Centers of Excellence 

(WCOEs). Similarly, system view products must all be 

owned by the program managers who are responsible 

for producing the platforms and systems.

The final step in creating the ICF compliant mission 

architectures is to combine the MTB with the several 

S/PTBs to create an Integrated Capability Technical 

Baseline (ICTB). Creating the MTB and S/PTBs using 

the data model and authoritative sources, is significantly easier than when using individually defined 

terms. As an example, since the MTB defines the 

required system functions using the Joint Common 

System Function List (JCSFL), and the S/PBT products 

define available system functions using the same list, 

matching needed functionality to provided functionality is possible. Work will still be needed to fully 

integrate the products, of course. For example, an 

MTB that says it needs a system to “Acquire and Track 

Targets” (JCSFL number 2.1.39) will require more 

analysis to determine what kind of target and how 

well it must be tracked (e.g., an E2 Hawkeye, designed 

for air/surface tracking and command and control, 

will likely be of little use in detecting submarines). 

However, having all sensor systems that are capable 

of acquiring and tracking targets as defined in the 

JCSFL use that term and definition enables discovery 

Table 2. ICF MTB and S/PTB Products
Mission

Thread

Ship Specific Mission Tasks 

Ship Specific

Combat

System 

Ship Specific System Threads

Mission 

CONOPS/TACSIT

Platform Specific 

Mission Tasks

Platform Specific 

Combat System

Platform Specific 

System Threads

Mission

Effectiveness

(MOEs)

MOPs for all

elements of 

Mission Thread

Ship, Aircraft, and

System Technical

and Performance

Requirements

Integration & Interoperability | 23

Table of Contents

Integrated Capability Framework: 

Building a Metadata Structure for Diverse Products

of systems that can perform the function – something 

that is much more difficult if each architecture uses 

different terms. Figure 2 depicts an ICTB with linkages between the products.

Having a completed ICTB then enables the definition and analysis for the operational, testing, training, 

and acquisition and development communities that 

are not easily performed otherwise. For example, a 

simple analysis can determine if physical communication paths exist among the S/PTBs to enable the 

operational information exchanges identified in the 

MTB; more detailed analysis can identify if the communication bandwidth is sufficient, and if there are 

redundant paths for the data. This analysis can be 

used to identify additional system requirements to 

the acquisition community or the need to change the 

information plans or available platforms/systems for 

the operational community to complete the mission. 

Further analysis can begin to identify I&I issues 

such as how specific platforms and systems will be 

used, how specific missions enable analysis to determine where functionality is duplicated in the force; 

or where the same data is transferred via more than 

one communication path, which can lead to interoperability issues.

The completed ICTB also enables a more complex 

analysis. As illustrated in Figure 2, each system on 

each platform has performance requirements defined 

for the functionality performed. Each system function 

supports an operational task that has a measure of 

performance supported by both the system function 

and the operator, and each operational task measure 

is linked to the desired effect of the mission being 

performed. Having this complete linkage allows the 

Fleet mission planner to evaluate the ability of the 

force to achieve the desired effect and to define the 

Figure 2. Example ICTB Products and Linkages
Table of Contents

24 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

operator performance level requirements for each supporting system. Having a completed mission thread 

with those same measures of performance allows the 

training community to develop training curricula, 

and the testing community to develop test scenarios 

to measure system performance and operator proficiency. Similarly, the ICTB allows analysis of the 

systems, platforms, and sailors to enable definition 

of gaps linked to Fleet need, as well as defining new 

DOTMLPF solutions, including prioritization and 

definition of new and modified systems.

Conclusion

Using the ICF for creating architectures enables 

all users to more effectively and efficiently perform 

their tasks and improves collaboration. From the 

Fleet perspective, use of the ICF enables more complete definition of warfighter need and more effective 

collaboration with the acquisition community. As the 

MTB owners, the Fleet will be able to more easily and 

completely communicate to the acquisition community how they intend to execute the missions, input 

that is critical for all types of analysis. The acquisition community is already required to generate 

DODAF-compliant architecture products. Using a 

mission-based approach in compliance with the ICF 

and Fleet-owned mission architectures help ensure 

that the architectures are useful for all communities, with little to no extra cost. Definition of system 

requirements, analysis of systems, and system test all 

benefit from having relevant operational scenarios 

to help ensure completeness and correctness. The 

ability of the Fleet to use ICTBs will enable definition of training requirements and curricula, and 

evaluation of operator proficiency. Finally, use of the 

ICTBs will enable complete, accurate analysis of mission capability, identification of potential interoperability issues, and DOTMLPF solutions to gaps in 

mission capability. 

References

1. “Framework” definition: “A logical structure for classifying and 

organizing complex information”, Federal Enterprise Architecture 

Framework.

2. “Integrated Capability Framework Operational Concept” Document, 

Version 2.0, 30 September 2013.

3. Vaux, Bert, and Scott Golder, “The Harvard Dialect Survey”, Harvard 

University Linguistics Department, 2003.
Integration & Interoperability | 25

Table of Contents

Every day, all around the globe, U.S. 

Navy officers and sailors make decisions 

to accomplish their assigned missions.

The decisions are influenced by the 

capabilities possessed by the units 

assigned to the mission. As defined in 

the Chairman of the Joint Chiefs of 

Staff Instruction (CJCSI 3500.01G; 15 

March, 2012) a capability is:

“The ability to achieve a desired 

effect under specified standards and 

conditions through a combination 

of means and ways across doctrine, 

organization, training, materiel, 

leadership and education, personnel, 

facilities and policy (DOTMLPF-P) 

to perform a set of tasks to execute a 

specified course of action.”

This multi-faceted capability construct has served as a model for Naval 

Operations and materiel procurement 

for many years. However, as attrition 

warfare has been replaced by network 

centric operations, including cyber 

warfare, the complexity of engineering 

effective weapon systems has grown. 

Engineering guidance has evolved to 

handle the increased complexity. For 

example, the highly compartmented 

system engineering disciplines used 

System of Systems Engineering Guidance

By William F. Ormsby
Table of Contents

26 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

to develop the Submarine-Launched Ballistic Missile 

(SLBM) in support of the Strategic Triad Concept 

are obsolete for systems used in more modern and 

collaborative combat scenarios. System of Systems 

(SoS) engineering guidance responds to: the deficiencies in traditional system engineering practices; 

the resulting Fleet’s dissatisfaction with strike-force 

interoperability; and the crucial need for integrated 

sensor, weapon, and command and control (C2) systems that support coordinated decision-making in 

dynamic threat environments.

The Deputy Assistant Secretary of the Navy 

(DASN) for Research, Development, Test, and Evaluation (RDT&E) Chief Systems Engineer (CHSENG) 

is the naval technical authority within the acquisition structure for ensuring compliance with overall 

Department of the Navy (DON) Enterprise Architecture (EA) policy along with integration and interoperability of current and future DON acquisition programs. The DASN has the responsibility to provide 

guidance for SoS engineering processes to translate 

mission operational requirements into measurable 

and testable mission performance requirements. The 

office of the Assistant Secretary of the Navy Research, 

Development and Acquisition (ASN (RD&A)) Naval 

System of Systems Engineering Guidebook Version 

3.0, was developed to aid Program Managers and 

System of Systems Engineers in the performance of 

their duties. It is intended to support the Naval acquisition community, particularly Program Executive 

Offices (PEOs), Systems Engineering Integrated Product Teams (SEIPTs), and Mission Area Systems Engineers in implementing capability-based acquisition 

in accordance with SECNAVINST 5000.02.

This article describes the latest version of the SoS 

Engineering Guidebook including recent updates 

made by a national technical team of subject matter 

experts composed of Naval Sea Systems Command 

(NAVSEA), Naval Air Systems Command (NAVAIR), 

and Space and Naval Warfare Systems Command 

(SPAWAR) personnel. The guidebook promotes a 

shared understanding among engineers developing 

various warfare systems, of effects desired by operators 

who employ those collective systems in accordance 

with a chosen course of action. This concept of “starting with the end in mind,” a common mission model, 

leads to fewer inconsistencies in system designs and 

reduced costs associated with integration of systems 

conceived to deliver mission enhancements. Excerpts 

from the guidebook are highlighted in this article to 

stimulate greater efficacy in acquisition and improved 

operational readiness in support of National Security.

Integrated Capability Framework (ICF) – 

Justification for and Nature of Changes

The SoS Engineering Guidebook V3.0 substantiates 

its content and value by describing Carrier Group 

issues first identified in 1998. It was determined that 

target track information being passed between platforms resulted in track ambiguity (1 threat aircraft 
8900W 8855W 8850

5230

1177

0000

1156

1162 1177

7616

0176

7612

1156

1162

8900W

1 Aircraft

3 Tracks

3 ID’s

AEGIS

BASELINES

5.3.6.1

ACDS BLOCK1

OP EVAL

HUE CITY

VICKSBURG

Mission Area/

ROC

(Capability)

Person/Role

(Performer)

Platform

(Performer)

Resource/

Information

Mission

(Scenario)

Task

(Activity)

System

Function

(Activity)

System

(Performer)

TTP/

Doctrine

(Rule)

Measure

Condition

Desired

Effect

Mission Area/

ROC

(Capability)

Defined By Set Of Achieves Specified By

Enables

Performs

Resides On

Exchanges

Uses

Resides On

Exchanges

Consumes/produces

Consumes/produces

Allocated To

Implemented By

Performed Under

Performed IAW Specified

Constrained By

Constrained By

Performed IAW Specified

Composed Of The Time

Ordered Sequence Of

Integration & Interoperability | 27

Table of Contents

System of Systems Engineering Guidance

with 3 different tracks and 3 different IDs) which 

negatively impacted Battlegroup interoperability (see 

Figure 1). As a result, initial policy was set in place to 

ensure Warfare Systems are fully developed, mature, 

reliable, and have completed interoperability testing in a systems of systems environment prior to 

deployment. The current guidebook recognizes that 

no amount of end item testing will make a system 

more interoperable, and that modern mission modeling techniques can be used to preclude system designs 

that would lead to similar interoperability issues. 

In late 2010, the Chief of Naval Operations (CNO) 

hosted a summit on Integration and Interoperability 

(I&I) where he was briefed on a concept for improving 

acquisition practices that would resolve many issues 

that were degrading Fleet readiness. Subsequently, 

actions were taken to convene a Multi-SYSCOM 

team to define and validate an Integrated Capability 

Framework (ICF) that formalizes a core set of data 

elements, relationships, and taxonomies, as shown in 

Figure 2, that improve efficacy of integrating systems 

into Fleet missions. The ICF includes: Department of 

Defense Architecture Framework (DODAF) models 

to distill the critical context required to decompose 

mission-level requirements into SoS and component system functional and performance requirements; implementation guidance and associated roles 

and responsibilities for interaction with the Fleet 

to authoritatively capture Fleet needs; references to 

authoritative sources, data taxonomies, and common 

standards for capturing the complex relationship 

Figure 1. Battlegroup Interoperability Deficiencies

Figure 2. Integrated Capability Framework Data Model
Table of Contents

28 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Figure 3. Universal High-Level Operational Concept Graphic (Operational View OV-1)
Integration & Interoperability | 29

Table of Contents

System of Systems Engineering Guidance

between the ICF products, across platforms, systems, 

and components. 

Improving warfighting capability in a measureable way requires identification of all types of data 

that contribute to the measure. The ICF Data Model 

includes the relationships and dependencies between 

data types and can be shared with various stakeholders. Cursory review of the ICF Data model reveals 

SoS engineering disciplines are less about the system 

and more about the data related to the system, either 

directly or indirectly. The number of relationships 

among these twelve data elements, each of which 

consists of multiple items, exposes the complexity of 

SoS engineering. That is, decomposition of system 

behaviors (i.e., functions and timing) requires contextual details of all eleven other data elements. Modern 

tools help manage the complexity, and the SoS guidebook describes methods for modularization that allow 

data collection that results in a mission model with 

maximum cohesion and minimal coupling. The ICF 

Mission Model development focuses on:

• Describing the mission in operational 

language through development of Concept 

of Operations (CONOPS) and Tactical 

Situation (TACSIT) description documents;

• Describing the capabilities required to 

accomplish the mission;

• Modeling the activities, resource exchanges, 

and system functions that are used to 

accomplish the mission.

A historical example of mission decomposition provides a basis for understanding updates to SoS engineering guidance. Figure 3 is a universal high-level 

operational concept graphic (Operational View, OV-1). 

Such a graphic, together with a textual description 

of a concept for operational mission performance, is 

often considered the initial step in mission modeling. 

In the 1950s, the mission was to keep the Soviet Union 

at bay. The CONOPS for this mission required the 

United States to have thousands of nuclear warheads 

and three ways to deliver them on target. The three 

legs of the nuclear “triad” included: heavy bombers, 

Submarine-Launched Ballistic Missiles (SLBM), and 
Table of Contents

30 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

land-based intercontinental ballistic missiles. Three 

technologies “saved the Navy’s bacon.”1 Miniaturized 

warheads, solid propellants, and very high accuracy 

gyros made it possible for the Navy to start the Polaris 

SLBM Program. Policy in the 1950s established fire 

control (FC) of the SLBM capability as an inherently 

government function, and the SLBM FC system has 

continued to be developed by government scientists 

and engineers at Naval Surface Warfare Center Dahlgren Division (NSWCDD) ever since. 

Over the years, the Polaris Program evolved to the 

Poseidon Program and then to the Trident Program, 

each with greater targeting accuracy requirements. 

SLBM engineering disciplines were thus influenced by:

• Geodesy, that branch of applied mathematics 

which determines the shape, curvature, and 

dimensions of the earth;

• Geopotential, the potential of the earth’s 

gravity field;

• Satellite Geodesy, the use of artificial satellite 

techniques to measure the distance between 

launch locations and target locations and 

the gravitational forces on ballistic missiles 

during flight. 

Within this context, the platforms and systems represented in Figure 3 that are relevant to SLBM become 

apparent and performance measures are understood. 

An expectation is created for a description of an information exchange (Department of Defense Architecture Framework (DODAF), OV-3), together with 

the technical standard profile (DODAF, TV-1), that 

codifies the conditions that govern ballistic missile 

trajectories, bounds the maneuvers of submarines 

relative to ballistic missile targets as a function of 

launch location, and influences the designs of fire 

control and guidance systems. 

To characterize the mission, one must fully understand and characterize the environment upon which 

the mission will be exercised. Continuing our SLBM 

example, Figure 4 illustrates the TV-1: the World 

Geodetic System (WGS84), “a model of the world” 

that resulted from years of data collection. It shows 

that the earth is not perfectly spherical; rather certain 

locations (blue) are below the reference ellipsoid, and 

other locations (orange) are above. The undulations for 

given launch and target locations are easily calculated 

and enable achievement of required accuracy. 

Today we are dealing with new threats and with 

inter-dependent missions. The advances of U.S. 

adversaries, such as mobile missile launchers; the 

Navy transformation to network-centric operations, 

including use of unmanned air vehicles (UAVs); and 

multi-mission requirements have created capability dependencies that require greater collaboration 

with the Fleet. More iteration between the ICF data 

elements is anticipated to discover the dominant 

parameters of missions. The success of the SLBM 

Program confirms the return on investment in a 

commitment to “modeling the world” and the need 

for collecting data over long periods. Operating on 

what we simply believe to be true (spherical earth) 

can hinder the development of the required precision for the machine operation to achieve mission 

success. Future data collection will be conducted in 

accordance with capability-based modeling practices 

that describe dependencies and enable predictable 

mission performance across kill-chains. 

The nature of changes described in the SoS Engineering Guidebook fulfills the Navy I&I Charter to 

provide more informed input into the existing Planning, Programming, Budgeting and Execution, and 

Joint Capability Integration and Development System 

processes. The changes leverage existing end-to-end 

effects/kill chain gap analyses efforts and requires collaboration among operational and acquisition subject 

matter experts. The guidance enables a disciplined 

assessment of I&I gaps at the mission level and development of holistic (i.e., DOTMLPF) recommendations to inform investment decisions. The changes 

mandate development and use of authoritative mission models to ensure the system requirements are 

complete, consistent, correct, and testable. Version 3 

of the guidebook also describes activities to govern 

the evolution of mission models to converge the 

designs and implementations of all systems across 

an effects chain. 
NGA - GeoSciences Division

EGM2008 2.5 Minute Geoid Heights

-107

66

Meters (m)

-180o -150o -120o -90o -60o -30o 0o 30o 60o 90o 120o 150o 180o

-180o -150o -120o -90o -60o -30o 0o 30o 60o 90o 120o 150o 180o

-30o

-60o

0o

30o

60o

-30o

-60o

0o

30o

60o

Toponymic information is based on the Geographic Names Database, containing official standard names approved by the United States Board on Geographic Names and 

maintained by the National Geospatial-Intelligence Agency. More information is available at the Products and Services link at www.nga.mil. The National Geospatial-Intelligence 

Agency name, initial, and seal are protected by 10 United States Code Section 425.

Integration & Interoperability | 31

Table of Contents

System of Systems Engineering Guidance

One basic tool to help characterize parameters 

within the ICF is the use of Modeling and Simulation 

(M&S). Models and simulations should continue to 

be used as needed to support requirements definition, 

analyze the Force Package configuration and design, 

and mitigate identified risks through engineering 

analyses. M&S supports the assessment of functional 

and performance characteristics, integration and 

interoperability, network throughput and bandwidth, 

supportability, and Human System Integration (HSI) 

issues such as maintainability, usability, operability, 

and safety. Moreover, M&S provides the ability to 

predict SoS performance as specified before system 

design and testing. 

Managing the System of Systems Acquisition

The guidebook describes the desired, iterative 

nature of systems development noting that the verification feedback loop at each level of the system life 

cycle is the mechanism for providing development 

control. Iteration of the design reduces the risk of 

developing a system that does not meet the needs 

of the user. As stated in the Defense Acquisition 

Guidebook, “The ultimate purpose of the System 

Engineering (SE) processes is to provide a framework 

that allows the SE team to efficiently and effectively 

deliver a capability to satisfy a validated operational 

need.” Claiming that current derivation of system 

functionality is done at the discretion of the individual Program Managers, the guidebook promotes 

significant use of venues that take system engineers 

into Fleet operations, where DOTMLPF capabilities 

and limitations are experienced first-hand. 

The collaboration on mission-based test design and 

Fleet Exercise and Experimentation events (FLEX) are 

highlighted as advancements in mission engineering. 

These venues are fleet-led and fleet-driven, and they 

establish fact-based assessments of baseline capability 

Figure 4. EGM2008 2.5 Minute Geoid Heights2
MISSION

Left of SE “V”

Future Systems

Middle of SE “V”

System Development

Right of SE “V”

Operational Systems

INTEGRATED CAPABILITY FRAMEWORK

MISSION

PLATFORM

SYSTEM

COMPONENTS

PLATFORM

SYSTEM

COMPONENTS

Warfare

Capabililty

Baseline

Capability

Solution

Management

Mission Level

Assessment

and Evaluation

Capability 

Acquisition 

Management

Table of Contents

32 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

and fully vetted evaluations of concepts of future 

capabilities. Figure 5 portrays a mission engineering 

life cycle and shows how I&I activities exploit an ICF 

to provide continuity of Fleet insight throughout the 

life cycle of all systems in an effects chain.

While a naval governance process for a mission-level system of systems has not yet been established, the guidebook prescribes business processes 

that would govern the execution of those practices to 

ensure integrity in the acquisition of horizontally integrated capabilities. It defines the attributes of a governance process that will provide mission engineers 

the top-down authority to exercise the disciplines 

described in the guidebook. The governance is based 

on the premise that PEOs, PMs, and system engineers 

recognize the dependencies among the many systems 

employed in a kill-chain to achieve a desired operational effect. Furthermore, the program manager’s 

number one responsibility, “to deliver systems to 

satisfy validated warfighting requirements at optimal 

life-cycle cost,”3 is best achieved through governance 

described here. The roles of organizations identified 

in Figure 6 are defined in the SoS Engineering Guidebook to achieve the goals of the Navy Integration 

and Interoperability Charter, which was signed and 

promulgated by the Vice-CNO on December 19, 2012.

The Navy I&I Charter endorses a business process 

for the organizations to interact from a mission area 

effects/kill chain perspective, to develop integrated 

solution recommendations, to inform investment 

decisions, and to verify all system dependencies have 

been implemented. Figure 7 illustrates the temporal 

aspects of an effective I&I battle rhythm that follows the traditional DoD two-year Program Objective Memorandum (POM) cycle with overlap from 

cycle to cycle. 

The information required from mission engineers 

to support effective I&I governance becomes clear 

during the mission decomposition activities. Figure 8 

illustrates such activities at the various organizational 

levels (vertical) and in the integrated capability phasing timeframes (horizontal). 

Figure 5. Mission Engineering Life Cycle
Integration & Interoperability | 33

Table of Contents

System of Systems Engineering Guidance

Notes:

1. Coordinate with other MATAs as required

2. May act as MMA chair

3. In some capability analysis Operational Agencies fill sponsor role (i.e.: COTF, Numbered Fleets, MOCS, NWDC)

4. May be MSDT lead If desired

Mission Products

Support, Guidance, and Governance

Program (System) Products

• Guidance, Policy

 and Standards

• SETR/MS Review

• System

 Architectures

• Compliance

• Mission Area

Management

• Maintain Navy

Enterprise

Architecture

Repository

(NEAR)

• Configuration

Management

• Commonality

(NEARG)

• Style Guide

Sponsor Layer 3

Mission Modeling

Authority (MMA)

Working Group

I & I

Direction(s) ASN RDA DON CIO

Policy and 

Governance

CND N(s)

Mission Area

Technical

Authority (MATA)

 1, 2

PEO

Mission Model 

Authority 

Chair 4

Mission 

Agent/

Architect

Program

Authority(s)

S&T

Representative(s)

Fleet 

Collaboration

Team (FCT)

Material 

Solution

Development

Team (MSDT)

Tech 

Authority(s)

ASN Rep

 3

Program 

Agent(s)

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr

WCB CSM

WCB CSM

POM XX:

POM XX:

POM XX:

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr

I&I Assessment Agenda:

Identifies Kill Chains for 

Assessment

Warfare Capability

Baseline (WCB):

Assesses Weapon

Target pairings for gaps;

delivers WCB Report

Capability Solutions

Management (CSM):

Creates and delivers

Integrated Capability

Package (ICP)

Warfare Improvement

Program (WIP)):

Considers the ICPs for

inclusion into the

integrated Prioritized

Capability Lists (IPCLs)

ICP/IPCL Processing:

The ICP/IPCL enters

Navy Requirements

Acquisition Processes

Figure 6. DON Architecture Organization

Figure 7. Notional I&I Battle Rhythm
Table of Contents

34 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Examples of past activities are described below to 

illustrate the data collection activities highlighted in 

Figure 8, and the use of data to support acquisition 

decisions.

Develop Baseline System Architecture and “To-Be” 

System Phasing Options: In the early 1980’s, 

NSWCDD scientists and engineers used the WBS72/84 model, which was developed for SLBM, in 

work with the Naval Research Laboratory (NRL) 

in developing optional constellations of the Global 

Positioning System (GPS). The GPS architecture, 

i.e., the number of satellites; the number of orbital planes; the inclination of the orbital planes; 

and the angular separation of satellites in each 

plane, was evaluated to ensure reliability and availability of the system. While the first GPS Block I 

satellite was launched in 1989, GPS modernization continues to meet growing military, civil, and 

commercial performance and accuracy needs. 

The use of GPS-enabled capabilities is critical 

to modern combat operations, especially those 

employing UAVs.

Sponsor

Operational

and Shore

Establishment 

Identify Mission

Area Architectures

required

Approve Cost,

Schedule and

Performance

(CSP) Data

Initiate, and Provide Governance of,

Mission Modeling Processes

Approve Mission Models to

Inform investment Decisions

Mission Model

Authority (MMA) 

Identify Agencies

to Support Mission

Modeling

Administrate

MMAWG

Activities and

Approve Models

Approve

Distribution 

of Mission

Models

Adjudicate and

Distribute

Updates for

“To-Be” Mission

Model

Validate Mission

Models and

Recommend

Approval

Inform

Mission Model

Stakeholders

MMA Working

Group (MMAWG) 

Identify Mission

Agent/Architect

and Convene

MMAWG

Identify

“To-Be” Mission

Model Data

Requirements

Develop and

Recommend

“To-Be” Mission

Models

Document

Mission Model and

Archive Data

Recommend

Mission Analysis

Reports

Mission Agent/

Architect

Research and

Reference

Mission Data 

for MMAWG

Establish

Desired Effect

and DOTMLPF

Dependencies

Characterize

Mission and

Research/Develop

Mission Models

Analyze Potential

Mission Solutions

and System

Dependencies

Maintain and

Use Mission

Model to Analyze

Readiness

Draft Mission

Analysis Reports

Define Mission

Model Summary

and CSP

Objectives

Program

Authorities 

Identify Program

and Attend

MMAWG

Identify System

Model Summary

an CSP Data

Approve Baseline

System Architecture

and Initiate

“To-Be” System

Evolutions

Negotiate

“To-Be” System

Architecture

Alignment to

Mission Model

Validate

System Models

and Recommend

for Use

Inform

System Model

Stakeholders

Program Agents

Research and

Reference System

Model Data

Identify System

Assumptions

and CSP

Dependencies

Develop

Baseline System

Architecture and

“To-Be” System

Phasing Options

Integrate Concept

for System

Performance

into DOTMLPF

Mission Model

Support Mission

Model Analysis

to Improve

Readiness

Contribute to

Mission Analysis

Stakeholders

SYSCOM, FCT,

S&T, I&I Tech

Authority 

Identify Agents/

Architects and

Attend MMAWG

Research and

Reference

Mission Model

Data

Integrate Concepts

for Effects-Chain

Performance into

Mission Model

Define Criteria

for Integrated 

Capability

Acquisition

Management

Support Mission

Model Analysis

to Improve

Readiness

Contribute to

Mission Analysis

Reports

Planning and

Organization Phase Development Phrase Approval and Configuration

Management Phase

Figure 8. Organizational Interactions Required for SoS Engineering Governance
Integration & Interoperability | 35

Table of Contents

System of Systems Engineering Guidance

Validate System Models and Recommend for Use: 

In the mid 90’s the Navy was adrift waiting for 

entrance criteria to be met prior to an SLBM readiness assessment event. Policy held that “demonstration and shakeout” (DASO) tests must be 

preceded by fire control simulations and guidance system simulations that produced the same 

predicted outcome. Until current mission engineers at NSWCDD were consulted, the cause for 

different M&S predictions was unknown. Mission 

engineering data was used to reveal discrepancies 

between guidance system design specifications 

and source code implementation. Once the source 

code implementations were represented in FC 

models, the simulations produced DASO predictions that met entrance criteria and the readiness 

assessment commenced. 

Identify “To-Be” Mission Model Data Requirements: In the summer of 2000, the National Missile 

Defense (NMD) Joint Program Office (JPO) was 

scheduled to report to Congress with a “Deployment Readiness Review.” A primary question 

to be answered, which was posed by Army and 

Air Force operational test agencies, was, “are we 

deploying bulldozers to the right location (for placing anti-ballistic missiles (ABMs))?” Engineers at 

NSWCDD who had access to requirement specifications for all five NMD components validated 

the physical architecture of the NMD system of 

systems in three steps. First, they partnered with 

civilian Army and Air Force test engineers to get 

the “as-is” ground-based test data and in-flight 

test data that characterized baseline limitations. 

Second, they used specifications for “to-be” NMD 

performance requirements to develop credible 

M&S of the specified SoS. Third, they collected 

measures of effectiveness from simulations that 

exercised new information exchanges to employ 

NMD capabilities against stressing scenarios. 

The historical anecdotes provided above are unique 

in detail, but serve to convey the broader benefits of 

mission modeling and simulation. However, there is 

a recognized deficiency in number and comprehensiveness of mission architectures. The cause, from the 

perspective of this author, is that DODAF architecture development has been “for compliance” and “in 

isolation.” As a result, comprehensive, coherent, and 

accurate mission models are not available to simulate 

and predict outcomes of alternative Courses of Action 

(COAs) for modern combat operations.

With promulgation of the Navy I&I charter—and 

like the SLBM Fire Control function—integration 

of individual contractor-delivered systems is now 

recognized as an inherent government function. As 

such, Multi-SYSCOM mission engineers will collaborate with the ultimate purpose of delivering more 

interoperable capabilities. The SoS Engineering 

Guidebook V3.0 provides a vital tool for managing 

acquisition activities to develop and use common 

mission models, across multiple PEOs, to evolve operational readiness. 

References

1. Dr. Russell H. Lyddane, “Dahlgren – Times of Crisis”, June 1977.

2. http://earth-info.nga.mil/GandG/wgs84/gravitymod/egm2008/

egm08_wgs84.html

3. SECNAVINST 5400.15C, 13 September 2007.
TERNAL NON-TECHNICAL KNOWLEDGE

TECHNICAL

KNOWLEDG

E

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

All Doctrine vs

elevant Doctrine

ven Conditions

Current Training

and NMETs

Platform Manning

Including Workload

Observed/Experienced

Mission Performance

Gaps (Multi-Theater)

Unsponsored/Implied

Stakeholders

Platform Readiness

Table of Contents

36 | LEADING EDGE January 2015

Integrated Mission Baselines:

Using Fleet-Owned, System Command Serviced Architectures to 

Improve System Integration and Interoperability

By Joseph Pack
Integration & Interoperability | 37

Table of Contents

Integrated Mission Baselines: 

Using Fleet-Owned, System Command Serviced Architectures to Improve System Integration and Interoperability

Current acquisitions require the use of architectures 

during both the design and integration phases of development in accordance with a series of Department of 

Defense (DoD) and civilian publications that regulate 

system development and engineering best practices.

Within specific development communities, such as 

communication and networking, architectures are 

used far more effectively because of their obvious 

benefits in managing diverse resources and data types 

across complex enterprises. However, for many Navy 

weapons, sensors, and combat systems, as well as 

other future technologies, architectures have been 

used more frequently to maintain compliance with 

current acquisition regulations than to aid in predicting and mitigating future interoperability problems 

that risk degrading Fleet readiness or preventing an 

emerging technology from improving Fleet readiness 

to its full potential.

The objective of Navy integration and interoperability (I&I), as articulated in the Vice Chief of 

Naval Operation’s-endorsed I&I charter, is to improve 

existing acquisition practices by establishing a Navy 

organization that better works across acquisition 

communities and program offices. This is accomplished in part by encouraging data management 

and exchange throughout organizations that have 

traditionally experienced stovepipe behavior. Architectures, being proven effective for integrating diverse 

data types and relationships, can be used to realize 

the I&I objective of improved data sharing.

This article will examine the application of developing mission reference architectures that focus on 

capability and operationally driven models. The application of such models ranges from simplifying collaboration between the acquisition community and the 

Fleet, to providing validated mission context toward 

systems and modeling development.

System Architecture and Mission 

Architecture Integration Using Integrated 

Mission Baselines (IMBs)

Traditional use of architectures has focused on 

system, system-of-systems, and software architecture, 

with emphasis on the material aspects of a given 

enterprise or on the satisfaction of interfacing and 

resource exchange requirements. Mission architectures are fundamentally different from these forms of 

architecture in their focus on the Fleet’s desired effect 

and the operations required not only to achieve the 

desired effect but also on a mission’s doctrine, training, and conditional constraints. These differences 

are more clearly understood by observing different 

architecture types articulated in past and present 

Department of Defense Architecture Framework 

(DODAF) specifications.

An “integrated architecture” is said to be a product where all architectural viewpoints and models 

uniquely define and consistently interpret data. Architecture is said to be integrated when it meets the 

following criteria:

• The architectural objects common to more 

than one view are identical or linked by 

underlying data relationships. 

• All objects that have relationships across 

views are linked by underlying data 

relationships.

For many, if not most system architectures, these 

definitions sufficiently describe the level of integration 

built into their products, and for good reason. System 

architectures exist to adequately describe an individual system’s full functionality, physical configuration, 

and employment. This implies a level of independence 

from a larger enterprise that makes anything more 

than an integrated architecture of limited value to 

component engineers and systems analysts.

However, an integrated architecture does not physically exist in absence of a larger enterprise. Both 

physical configuration and employment strategies 

for a given system are informed by the larger Navy 

enterprise architecture. This larger architecture is 

referred to as a “federated architecture,” defined in 

DODAF v1.5 as a product that “allows the architecture user a means to examine the enterprise from 

all aspects of the Doctrine, Organization, Training, 

Materiel, Leadership and Education, Personnel and 

Facilities (DOTMLPF) concept.” This implies that 

even though a system architecture may exist at the 
Services Operational Activity A

Operational Activity BOperational Activity COperational Activity DOperational Activity EOperational Activity FOperational Activity GOperational Activity HOperational Activity IOperational Activity JOperational Activity KOperational Activity LOperational Activity MOperational Activity NOperational Activity OOperational Activity POperational Activity QOperational Activity ROperational Activity SOperational Activity T

Service 1 X X X

Service 2 X X X

Service 3 X X X

Service 4 X X X X X X

Service 5 X X X

Service 6 X X X X X

Service 7 X X X

Service 8 X X X X X X X

Capability 1 Capability 2 Capability 3 Capability 4 Capability 5

DISR Service Area Service DISR Standard and

Source Document

Information-Processing Standards Higher Order Languages

Software Life-Cycle Process

Geospatial Data Interchange

Motion Imagery Data Interchange - Video

Distributed-Object Computing

Information-Transfer Standards Data Flow Network

Command and Control Information (C2I) Network

Physical Layer

Network Interface

Layer Management

File Transfer Standards

Remote Terminal Standards

Network Time Synchronization Standards

Web Services Standards

Connectionless Data Transfer

Transport Services Standards

Information Modeling, Metadata, and

Information Exchange Standards

Activity Modeling

Data Modeling

Object-Oriented Modeling

Human Computer Interface Mandates

Information Security / Information

Infrastructure Standards

Password Security

Application Software Entity Security Standards

Virtual Private Network Service

Intrusion Detection Service

Human-Computer Interface Security Standards

Flow 1

Activity 2

Services Operational Activity A

Operational Activity BOperational Activity COperational Activity DOperational Activity EOperational Activity FOperational Activity GOperational Activity HOperational Activity IOperational Activity JOperational Activity KOperational Activity LOperational Activity MOperational Activity NOperational Activity OOperational Activity POperational Activity QOperational Activity ROperational Activity SOperational Activity T

Service 1 X X X

Service 2 X X X

Service 3 X X X

Service 4 X X X X X X

Service 5 X X X

Service 6 X X X X X

Service 7 X X X

Service 8 X X X X X X X

Capability 1 Capability 2 Capability 3 Capability 4 Capability 5

DISR Service Area Service DISR Standard and

Source Document

Information-Processing Standards Higher Order Languages

Software Life-Cycle Process

Geospatial Data Interchange

Motion Imagery Data Interchange - Video

Distributed-Object Computing

Information-Transfer Standards Data Flow Network

Command and Control Information (C2I) Network

Physical Layer

Network Interface

Layer Management

File Transfer Standards

Remote Terminal Standards

Network Time Synchronization Standards

Web Services Standards

Connectionless Data Transfer

Transport Services Standards

Information Modeling, Metadata, and

Information Exchange Standards

Activity Modeling

Data Modeling

Object-Oriented Modeling

Human Computer Interface Mandates

Information Security / Information

Infrastructure Standards

Password Security

Application Software Entity Security Standards

Virtual Private Network Service

Intrusion Detection Service

Human-Computer Interface Security Standards

INTEGRATED ARCHITECTURE

 An architecture where architecture data

elements are uniquely identified and 

consistently used across all products 

and views within the architecture.

FEDERATED ARCHITECTURE

 Provides a framework for enterprise

architecture development, maintenance, and

use that aligns, locates, and links disparate

architectures via information exchange

standards (i.e., taxonomies).

EA Reference Model Taxonomy

WMA* BMA* DIMA* EIEMA*

is part of is equal to

Service Agency COCOM

Subordinate architecture mapped to MA-level by C/S/As

Operational

Activity X

SV-5c

OV-5

TV-1

Service 1

Service 3

Service 2

Service 4

External System 1

Data Flow 1

Data Flow 2

Data

Flow 3

External Service 1 Data Flow 4 External Portal 1

Data

Flow 5

Data

Flow 6

Data

Flow 7

Data

Flow 8

Data Flow 9

Service 1

Service 3

Service 2

Service 4

External System 1

Data Flow 1

Data Flow 2

Data

Flow 3

External Service 1 Data Flow 4 External Portal 1

Data

Flow 5

Data

Flow 6

Data

Flow 7

Data

Flow 8

Data Flow 9

SV-4b

Service

Functionality Y

Service Z

A1 Activity 1

A2 Activity 2

Flow 2

Flow 3

Flow 4 A3 Activity 3

Managed by MA Authority

* WMA — warfighting mission area, BMA — business mission area, DIMA — DoD portoin of the intelligence mission area, 

EIEMA — enterprise information environment mission area

Table of Contents

38 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

integrated architecture level, it is dependent on the 

information contained in a larger federated architecture. For systems developed for strategic and tactical 

applications, a larger federated architecture is represented through mission area architectures owned by 

senior uniformed leadership and the numbered Fleets 

as depicted in Figure 1.

While system architectures contain the entire functionality of a given system and a description of its use, 

in context of its design-to-platform or system-of-systems, mission architectures contain the partial functionality of multiple systems required to achieve a 

given desired effect or mission outcome; both products 

rely on information from the other. Mission architecture informed from the design-to-functionality 

and performance from the system architecture, and 

the system architecture must adhere to the strategic 

and tactical employment schemas articulated in the 

mission architecture, as illustrated in Figure 2.

This relationship identifies the requirements for 

systems engineers to support mechanisms that allow 

for both the information publishing and subscribing 

to some form of federated mission architecture. One 

such mechanism is the IMB.

Definition

An IMB is a mission-based reference architecture 

that models the capabilities and activities required to 

achieve a desired effect for a particular mission and 

set of environments or conditions. Similar to reference architecture in software engineering practices, 

an IMB is used as a template to begin production of 

a fully federated architecture using platforms and 

systems that span multiple program offices, platforms, and DoD services. By serving as a reference 

across multiple organizations, an IMB serves as a 

mechanism whereby disparate system architectures 

originating from different organizations may converge 

on a common understanding of how a mission is 

executed by the Fleet.

Architecturally, the IMB is heavily reliant on Capability Viewpoints (CVs) and Operational Viewpoints 

Figure 1. Integrated vs Federated Architecture as Modeled in DODAF V1.5
Federated Knowledge Consumed by

Architecture for Selected Mission

Relevant Knowledge 

from System 

Architecture ‘B’

FLEET-OWNED CONSTRUCTS

Navy

Federated

Knowledge

ACQUISITION COMMUNITY-OWNED CONSTRUCTS

Capabilities Doctrine/

Conditions Missions DesiredEffects

Navy

Tasks

Platforms/

Personnel

Capabilities

for Mission

Activities for

Capabilities

Personnel/

Platforms for

Activities

Mission

Architecture

Systems/

Components

Requirements/

Measures

Interface

Knowledge

Configuration

and Use

Program

Office ‘A’

Knowledge

Component

Functionality

HSI

Networking/

Comms

Employment

Integrated

(System)

Architecture

Program

Office ‘A’

Knowledge

Systems/

Components

Requirements/

Measures

Interface

Knowledge

Configuration

and Use

Program

Office ‘A’

Knowledge

Component

Functionality

HSI

Networking/

Comms

Employment

Integrated

(System)

Architecture

Program

Office ‘B’

Knowledge

Systems/

Components

Requirements/

Measures

Interface

Knowledge

Configuration

and Use

Program

Office ‘A’

Knowledge

Component

Functionality

HSI

Networking/

Comms

Employment

Integrated

(System)

Architecture

Program

Office ‘C’

Knowledge

Systems for

Activities

Relevant Knowledge from

System Architecture ‘A’

Relevant Knowledge from

System Architecture ‘C’

Integration & Interoperability | 39

Table of Contents

Integrated Mission Baselines: 

Using Fleet-Owned, System Command Serviced Architectures to Improve System Integration and Interoperability

(OV). Selected viewpoints focus on defining the mission to be executed; the complimentary capabilities 

required to complete the mission; activities required 

of personnel, platforms, and organizations for each 

activity; and doctrinal constraints placed on mission 

execution by virtue of relevant conditions (e.g., environmental, geopolitical) and locations.

The role of an IMB is to serve as the Fleet-owned 

and endorsed mission reference for system employment and enterprise readiness evaluation. The significance of Fleet ownership is nontrivial. Current 

mission architectures, if produced at all, are the result 

of data collection efforts on the part of system commands (SYSCOMs) and program offices. When each 

program office is responsible for its own mission 

architectural development, the potential exists for 

multiple interpretations of the same mission, as illustrated in Figure 3. The material risk to the Navy is the 

deployment of multiple systems into the Fleet built on 

non-standard battlefield interpretations, resulting in 

inconsistent or unpredictable mission performance. 

More often than not, this inconsistency or unpredictable mission performance results in a degraded Fleet 

readiness until sufficient interoperability mitigation 

strategies can be developed, vetted, and implemented. 

The use of Fleet-owned IMBs enforces a level of consistency among program offices that mitigates this 

risk, as illustrated in Figure 4.

Structure and Use-Case

Throughout fiscal year 2013, an I&I-funded effort 

sought to define, construct, and investigate IMB 

Figure 2. Fleet and Acquisition Community Construct Data Ownership
FLEET-OWNED CONSTRUCTS

Navy Scenarios

(OPLAN)

ACQUISITION COMMUNITYOWNED CONSTRUCTS

Navy Activities

(UNTL)

System Functions

(CSFL)

Navy Capabilities

(ROC/POE)

Difficult to maintain 

consistency between

programs, organizations,

services, etc.

Navy Needs

System

Architecture

1.1

System

Architecture

1.2

System

Architecture

1.3

System

Architecture

2.1

System

Architecture

2.2

System

Architecture

2.3

PEO 1 PEO 2 PEO 3

System

Architecture

3.1

System

Architecture

3.2

System

Architecture

3.3

System 1.1System 1.2System 1.3System 2.1System 2.2System 2.3System 3.1System 3.2System 3.3

Navy Systems

Program

Office

Architecture

1

Program

Office

Architecture

2

Program

Office

Architecture

3

FLEET-OWNED CONSTRUCTS

Navy Scenarios

(OPLAN)

ACQUISITION COMMUNITYOWNED CONSTRUCTS

Mission

Area

Architecture

2

Mission

Area

Architecture

1 Mission

Area

Architecture

3

Mission

Area

Architecture

4

Mission

Area

Architecture

5

Navy Activities

(UNTL)

System Functions

(CSFL)

Navy Capabilities

(ROC/POE)

Fleet-Owned IMBs remove the risk that each

program office operate under a proprietary

understanding of a given mission.

Navy Needs

System

Architecture

1.1

System

Architecture

1.2

System

Architecture

1.3

System

Architecture

2.1

System

Architecture

2.2

System

Architecture

2.3

PEO 1 PEO 2 PEO 3

System

Architecture

3.1

System

Architecture

3.2

System

Architecture

3.3

System 1.1System 1.2System 1.3System 2.1System 2.2System 2.3System 3.1System 3.2System 3.3

Navy Systems

Table of Contents

40 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Figure 3. Communicating Data Needs Absent a Common Data Management Environment

Figure 4. Communicating Data Needs with a Common Data Management Environment
Integration & Interoperability | 41

Table of Contents

Integrated Mission Baselines: 

Using Fleet-Owned, System Command Serviced Architectures to Improve System Integration and Interoperability

usage. The Naval Surface Warfare Center, 

Dahlgren Division (NSWCDD), took the lead 

by developing a surface warfare IMB, focused 

on missions typical of Littoral Combat Ships. 

Architecturally, the IMB consisted of the All 

Viewpoint (AV), CV, and OV products in 

Table 1.

The selection of IMB viewpoints focuses 

on accurately capturing data elements from 

the Integrated Capability Framework Data 

Model that sufficiently and distinctly identifies 

what the Fleet is trying to accomplish, how it 

is trying to accomplish it, and which agencies, operators or performers are required to 

accomplish it. Figure 5 depicts these elements.

The architectural approach taken to create 

the products in the surface warfare IMB followed a methodology developed for the sole 

purpose of modeling Navy missions called 

the Capability-Based Modeling Methodology 

(CBMM).

Fleet-First Architectural Methodology

The importance of using CBMM instead 

of other common methodologies is due to 

its focus on Fleet capabilities. CBMM was 

developed based on the theory that most Fleet 

stakeholders can be categorized into one of 

three domains: Fleet Capability, Operational, 

and Technical (see Figure 6). The relationships 

between these domains aid in identifying necessary authorities for sources of data as well 

as potential consumers.

The CBMM primarily focuses on the Fleet’s 

desired effect; without this, the mission would 

not exist. Therefore, it is the Fleet’s desired effect 

that is often the most well-defined part of a mission, 

especially since the method of mission execution 

and performance of relevant systems may be highly 

variable on external factors not fully realized before 

the mission is executed. 

For this reason, the CBMM encourages initially 

establishing a capability taxonomy to accurately 

characterize the nature of the Fleet’s desired effects. 

Following the capability taxonomy, architecturally 

called a CV-2, architects can begin identifying dependencies between capabilities (CV-4) and activities 

required for successful achievement of a capability 

(CV-6). Only after capabilities have been properly 

characterized and mapped can OV and System Viewpoint (SV) creation begin.

VIEWPOINT ACRONYM PURPOSE

Overview and Summary 

Information 

AV-1 Provides a brief overview of 

information contained within 

the product.

Integrated Dictionary AV-2 Architecture data repository 

with definitions of all 

terms used in all products.

Capability Taxonomy CV-2 A hierarchy of capabilities 

which specifies all the 

capabilities that are 

referenced throughout 

one or more Architectural 

Descriptions.

Capability 

Dependencies

CV-4 The dependencies between 

planned capabilities and the 

definition of logical groupings 

of capabilities.

Capability to 

Operational 

Activity Mapping

CV-6 This view describes the 

mapping between the 

capabilities required and the 

activities that enable those 

capabilities.

High-Level Operational 

Concept

OV-1 High-level graphical/textual 

description of operational 

concept.

Operational Node 

Connectivity 

Descriptions

OV-2 Information and resources 

exchanged between 

nodes and the relevant 

attributes of that exchange.

Organizational 

Relationships Chart

OV-4 Organizational, role or 

other relationships among 

organizations.

Operational Activity 

Taxonomy

OV-5a Taxonomy of all activities 

included in the given 

architecture - useful for 

identifying the operational 

scope of the architecture.

Operational Activity 

Mapping

OV-5b Mapping of relationships 

between tasks that helps 

to identify the required 

sequence of events that may 

take place given a known 

initial set of conditions.

Table 1. Integrated Mission Baseline Core Models
Person/Role

(Performer)

Platform

(Performer)

Resource/

Information

Mission

(Scenario)

Task

(Activity)

System

Function

(Activity)

System

(Performer)

TTP/

Doctrine

(Rule)

Measure

Condition

Desired

Effect

Defined By Set Of Achieves Specified By

Enables

Performs

Resides On

Exchanges

Uses

Resides On

Exchanges

Consumes/produces

Consumes/produces

Allocated To

Implemented By

Performed Under

Performed IAW Specified

Constrained By

Constrained By

Performed IAW Specified

Composed Of The Time

Ordered Sequence Of

IMB Elements

LEGEND

IMB Partial 

Elements

TACSIT Specific 

Elements

Mission Area/

ROC

(Capability)

FLEET

CAPABILITY

DOMAIN

What capability does 

the Fleet endeavor 

to achieve?

OPERATIONAL

DOMAIN

What actions must

operators and platform 

take?

TECHNICAL

DOMAIN

What system must be

deployed and what

functionality must

they possess?

Developed to Achieve

Requires Performance Of

Enables Achievement Of

Evaluated Based On

Enables Performance Of

Requires Specified Function Of

Table of Contents

42 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Figure 5. Integrated Capability Framework Data Model Highlighting Integrated Mission Baseline Data Elements

Figure 6. Capability-Based Modeling Methodology Acquisition Triad
Integration & Interoperability | 43

Table of Contents

Integrated Mission Baselines: 

Using Fleet-Owned, System Command Serviced Architectures to Improve System Integration and Interoperability

This approach forces architects to focus on the 

Fleet first rather than a particular system or systems. 

It puts Fleet-owned data, such as doctrine and current rules of engagement, in front of an engineer’s 

understanding or a program office’s interpretation 

of the missions of interest. Because of this Fleet-first 

approach and because of the genesis in AV and CV 

products, the CBMM serves as an efficient method of 

architectural development for a mission baseline-level 

reference architecture.

Impacts to Architecture Development

As with most models, modeling languages, and 

engineering tools, architectures are only useful if 

designed for the correct purpose. Currently, many 

program offices, SYSCOMs, and Fleet organizations 

use architectures for communication, collaboration, 

and compliance. These purposes, while important 

for the daily function of all Navy organizations, 

do not lend themselves to achieving goals set forth 

in the DODAF regarding integrated and federated 

architectures.

Improvements to Current Architecture 

Development

Architectures are currently required by both the 

Joint Capability Integration and Development System’s acquisition process and the latest iteration of 

DoD Instruction 5000.02E, “Operation of the Defense 

Acquisition System.” Though required, the proper 

generation, use, and governance over architectures has 

traditionally been ambiguous. Programs interested 

in acquiring a new system, making modifications to 

an existing system, or attempting to transition a new 

technology, use mission architectures to map system 

functionality into Navy operations using DODAF 

viewpoints. As mentioned earlier, mission architectures represent a form of federated architecture under 

the DODAF construct and, as such, require the architects to understand not only their specific system focus 

but also systems used throughout the entirety of the 

mission as well as how operations are conducted by 

the uniformed organizations of interest.

Under the previous, non-I&I-informed system, it 

was up to the individual program offices or field components to develop, verify, validate, and exploit their 

mission architectures. For SYSCOMs, this required 

architects to compile data from numerous internal and 

external sources. Some sources of information were 

known and others were combined through known 

associations and professional contacts. Even today, 

system data tends to be better understood, as are 

sponsors, direct users or stakeholders identified by the 

sponsor, as well as initial performance requirements 

and capability analyses. The SYSCOM-level engineer 

may not understand without proper integration with 

Fleet components the latest tactics, doctrine, training 

(including relevant mission essential tasks), personnel 

or manning, platform readiness and availability, and 

other relevant geopolitical information critical to the 

successful employment of a new system or technology.

To the individual architect, this may not seem like 

any more of a burden on one party than another (see 

Figure 7). However, there are numerous programs—

all reaching out to the same (or similar) organizations— asking for common data for each independent 

architectural effort, as shown in Figure 8. The result is 

a risk of overburdening data requests on relevant Fleet 

components and redundant manpower investments 

by the Navy. There is also a risk that two independent 

programs attempting to address a common mission 

area or environment may base their architectures 

on data originating from disparate sources, which 

often results in different solutions or inefficiently 

integrated products into a common mission area. 

Figure 8 illustrates how multiple architectural efforts, 

when accessing common data sources, can create 

unnecessary complexity and additional task loading 

on Fleet organizations.

If architects were to use an IMB established and 

maintained by the new I&I organization and owned 

by the Fleet, the Fleet would not be overburdened 

by redundant requests for data. Common use of 

single data sets also allows for more consistent data 

gathering and exchange among architects spanning 

across different and distinct Navy programs, as shown 

in Figure 9.
EXTERNAL NON-TECHNICAL KNOWLEDGE

TECHNICAL

KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

All Doctrine vs

Relevant Doctrine

Given Conditions

Current Training

and NMETs

Platform Manning

Including Workload

Observed/Experienced

Mission Performance

Gaps (Multi-Theater)

Unsponsored/Implied

Stakeholders

Platform Readiness

Table of Contents

44 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Figure 7. Data Sources Consumed and Managed by the Mission Architect
ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

All Doctrine vs

Relevant Doctrine

Given Conditions

Current Training

and NMETs

Platform Manning

Including Workload

Observed/Experienced

Mission Performance

Gaps (Multi-Theater)

Unsponsored/Implied

Stakeholders

Platform Readiness

Redundant data collection, overtasking of non-SYSCOM

personnel, potential for incongruent data collected from

disparate sources for a common mission, and additional

potential costs in lost opportunity for synergy

between architects and data collectors.

NWDC/WCDE

CSFTL, CSFTP

Strike Groups,

DESRONs,

LCSRONs

Other Units,

Joint Forces,

Coalition Forces

Numbered Fleets, 

COCOMs

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

ARCHITECT

Own System

Design

Performance

and Interface

Requirements

Sponsors and

Sponsored

Stakeholders

System

Capabilities

and

Limitations

External

System

(System of 

Systems) 

Designs

TECHNICAL KNOWLEDGE

NON-LOCAL

LOCAL CORPORATE KNOWLEDGE

All Doctrine vs

Relevant Doctrine

Given Conditions

Current Training

and NMETs

Platform Manning

Including Workload

Observed/Experienced

Mission Performance

Gaps (Multi-Theater)

Unsponsored/Implied

Stakeholders

Platform Readiness

Utilizing IMBs create less work

for the program-level architects.

NWDC,

WCDE

CSFTL, 

CSFTP

Strike Groups,

DESRONs,

LCSRONs

Other Units,

Joint Forces,

Coalition Forces

Numbered 

Fleets, 

COCOMs

IMBs are developed and

managed by I&I activities.

Key to the IMB is the Fleet’s ownership 

of the data.This structure allows Fleet 

components easiercontrol over how their 

data is ditributed, updated and utilized.

I&I Database Containing Integrated

Mission Baselines (IMBs) for Reference

and Solution Architectures for Reuse.

SUW IMBs

AW IMBs

ASW IMBs

ISR IMBs

Other IMBs

Integration & Interoperability | 45

Table of Contents

Integrated Mission Baselines: 

Using Fleet-Owned, System Command Serviced Architectures to Improve System Integration and Interoperability

Figure 8. Complexity Caused by Independently Functioning Architecture Efforts

Figure 9. Simplification of Data Collection and Management Using an Integrated Mission Baseline
Fiscal Year FY14 FY17

Investments for Project

Completion ($)

FY15 FY16

% of New/Non-Validated

CV-6 Modules

Development

AccreditationValidationTotal Cost

Development

AccrValTotal Cost

Savings

Development

AccrVal

Savings

Development

AccrValTotal Cost

Savings

100% New 70% New 55% New 45% New

30% 

Validated 45% 

Validated

55% 

Validated

Total Cost

Table of Contents

46 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Implications to Increased Fidelity and 

Consistency of Modeling and Simulation 

(M&S)

Following an IMB construct benefits an organization not only through managing data gathering 

among architects but also in managing and coordinating data gathering among those in the acquisition 

community who need readily accessible data external 

to their corporate organization. A prime example of 

this type of data sharing occurs during investigations and development of M&S strategies as well as 

each tool’s verification and validation process. For a 

modeling tool to maintain integrity with the mission 

and to correlate with data produced by other tools, 

it must access data external to the technical community and owned by the Fleet, such as doctrine, 

tactics, and physical limitations of the deployed force. 

To the individual M&S team, it may seem viable to 

simply contact the Fleet components relevant to a 

given modeling effort. However, when viewed from 

the Fleet component perspective, a single engineering 

or acquisition organization may have multiple M&S 

teams contacting a single Fleet component office for 

common sets of data. This not only risks damaging 

the Fleet’s perception of that engineering organization 

but also reduces the Fleet’s willingness to work with 

the engineering community.

These organizational risks can be alleviated by 

implementing IMBs into the data sharing and gathering process, which will provide Fleet components a 

single technical point of contact from which they may 

publish and subscribe data. This also alleviates the 

responsibility of each M&S team to contact multiple 

Fleet components, allowing them instead to simply 

consult a single IMB reference product that contains 

vetted and validated Fleet information.

Effort Cost Savings

Typically, as project complexity decreases, so does 

estimated cost. As previously noted, complexities in 

organizing architecture and M&S data needs can 

be greatly simplified through the use of IMBs. This 

represents significant potential for reductions in labor 

spent developing, maintaining, and exploiting contacts and releases architectures and M&S developers 

for more prudent tasking.

Future fiscal year efforts may also experience 

cost saving by way of architecture reuse, as shown in 

Figure 10. By leveraging the CBMM architectural 

methodology, IMB products are inherently compartmentalized by capability. Suppose a mission architecture contains capabilities for conducting intelligence, 

surveillance, and reconnaissance (ISR) operations 

and capabilities for conducting small caliber gunfire 

Figure 10. Notional Demonstration of Cost Savings Over Time 

Using Modularized Rework from Past Architectures
Table of Contents

Integration & Interoperability | 47

Integrated Mission Baselines: 

Using Fleet-Owned, System Command Serviced Architectures to Improve System Integration and Interoperability

engagements. During the next fiscal year, suppose the 

same organization wants to produce a similar mission 

architecture but instead, looks at engagements with 

directed energy weapons. Both the former and latter 

architectures would use the same ISR capabilities, 

thereby preventing the latter architectural effort from 

investing in redundant capabilities. 

Using an activity-based architectural approach 

would require a revalidation of the activity flow 

throughout the architecture and an object-based 

approach would require revalidation of platforms 

since key engagement systems were changed. However, because the CBMM organizes everything by 

capability, the architecture contains an inherent 

modularity. The IMB, composed using the CBMM 

approach, enjoys the same modularity and, therefore, 

may experience significant cost savings as more IMBs 

are produced.

Additional cost savings may be realized through 

the effective and efficient consolidating of data gathering and consistent use of known information. In a 

new age of austerity, where funding for data collection during live at-sea events and opportunities for 

costly tests are diminishing, engineers must maintain 

clear and open lines of data gathering and sharing 

to ensure programmatic limitations do not degrade 

the mission competence used to produce systems for 

warfighters. Using an IMB to coordinate data across 

development teams and programs mitigates risks 

created by inconsistent and insufficient data collection 

and interpretation, which in turn leads to unexpected 

expenses associated with data reconciliation.

Conclusion: 

IMB is Architecture Used Properly

Models are developed for a purpose; an architecture’s purpose being the management of data in 

accordance with a data model. The purpose of an IMB 

is to coordinate data gathering and sharing and to 

maintain open data availability across an enterprise. 

Use of an IMB allows integrated system architectures 

to connect into the larger federated Navy enterprise, 

which supports the goals of DODAF and the Global 

Information Grid Architecture. By integrating system 

architectures into Fleet-owned and -validated mission architectures, program offices may predict I&I 

problems before investing heavily in manufacturing 

and prototyping. The IMB serves as the vehicle that 

allows program offices to consume that Fleet-owned 

data necessary for such I&I analysis.

IMBs also serve the important purpose of reducing 

cross-organizational complexity. The “spirit of I&I” 

has been one of collaboration and open data exchange. 

Without some mechanism to act as a catalyst for the 

tenets I&I endeavors to establish, changing the current 

means of architectural and system development will 

be difficult. By building, supporting, and encouraging, 

if not enforcing, the use of IMB-like architectures, 

that catalyst exists and enables many of the parallel 

efforts articulated in the I&I charter.

By integrating efficient data management through 

use of IMBs, the broader Navy community is better 

enabled to avoid costly problems with system interoperability and ensure efficient material and non-material solution integration to create a high level of 

readiness throughout the Fleet. 
Table of Contents

48 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY
Capability Solution Management

Finding solutions to gaps 2

Capability Acquisition Management

Managing the production of solutions 3

Alignment of I&I Processes

Identifying stakeholders and best practices 4

1 Integrated Capability Framework

Overarching I&I task structure

Integration & Interoperability | 49

Table of Contents

Capability Acquisition Management:

Modifying the Acquisition Process to Emphasize I&I

By Joel Washington

Maintaining alignment of the myriad of program acquisition activities 

spanning multiple program offices, and often across different Program 

Executive Offices (PEOs), to realize a complete mission thread of warfighting 

capability, is a daunting endeavor. Success means managing and maintaining alignment of specific interdependent system attributes of the mission 

thread during system development through what has traditionally been 

an asynchronous set of technical and programmatic activities. This is one 

of the four critical core areas the Naval Surface Warfare Center, Dahlgren 

Division (NSWCDD), Integration and Interoperability (I&I) Mission 

Engineering team, sponsored by the Deputy Assistant Secretary of the 

Navy Research, Development, Test and Evaluation has had to address:

This article describes the Capability Acquisition Management (CAM) 

Integrated Product Team (IPT) tasks, which are aimed at fusing force-level 

I&I considerations throughout the system acquisition decision process by 

modifying Systems Engineering Technical Review (SETR), Gate Review, 

and Probability of Program Success (PoPS) criteria.
Table of Contents

50 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Background

I&I represents synchronization of a series of deliberate steps to provide an end-to-end capability view 

from an “effects chain” perspective. Where possible, 

the I&I approach leverages existing Department of 

Navy processes (e.g., Joint Capabilities Integration 

Development System, Capability-Based Acquisition, 

Office of the Chief of Naval Operation’s (OPNAV) 

N81 Warfighting Capability Plan, and the acquisition 

analysis of alternatives (AoA). I&I also provides a disciplined Doctrine, Organization, Training, Materiel, 

Leadership and Education, Personnel, Facilities and 

Interoperability (DOTMLPF-I) analysis to develop 

issues based on empirical data, facilitate solution 

recommendations to achieve effects chain wholeness, 

and align I&I recommendations to Navy leadership’s 

informed budget and acquisition/investment decisions. The overarching goal is to deliver strategic 

readiness by ensuring the Fleet is capable of successfully executing the most important effects chains 

that, together, form the Navy’s core contribution to 

Combatant Commander Operation Plans.

I&I contributors consist of system commands 

(SYSCOMs), warfare centers, OPNAV, Commander 

Operational Test and Evaluation Force (COTF), and 

the Fleet. The Fleet provides mission area advocacy 

from establishing the I&I demand signal through processing approved solutions. The COTF provides the 

framework and focal point for expertise to accomplish 

selected effects chain assessments. SYSCOMs provide 

the technical rigor to accomplish I&I materiel issue 

assessments. OPNAV and the Assistant Secretary of 

the Navy (Research, Development, and Acquisition) 

[ASN(RDA)] provide the expertise and options to 

leverage the full range of processing and implementation options for I&I solutions as illustrated in Figure 1.

The CAM IPT was specifically tasked to recommend policy changes to support mission engineering, 

incorporate rigor into mission engineering aspects of 

the acquisition process, and to mitigate I&I issues. 

Candidate policies for change were found in the Technical Review Manual, Naval Sea Systems (NAVSEA) 

instructions, Naval Systems Engineering Technical 

Review Handbook, and OPNAV instructions. Emphasis was placed on the criteria required to support 

mission thread and kill chain execution.

In looking at potential policy changes, the CAM 

IPT addressed the full end-to-end traceability of mission capability requirements in specific domains by 

examining:

• Data

• Interfaces

• Decomposition/allocation of end-to-end 

technical requirements

• Dependencies

• Commonality

Any proposed policy changes would be vetted 

against and help ensure compliance with overarching 

I&I acquisition policy statements in SECNAVISNT 

5000.2E, “Department of the Navy Implementation 

and Operation of the Defense Acquisition System and 

the Joint Capabilities Integration and Development 

System,” 1 September 2011:

PMs shall ensure the (I&I) of all operations, 

functions, system interfaces, data, software based 

services, distributed decision-making systems, 

human processing capabilities, situational awareness 

systems, and other systems to reflect the requirement 

for all system elements: hardware, software, facilities, 

sustainment infrastructure, personnel and data. 

Interoperability shall be addressed by including 

system of systems (SoS) or federation of systems 

(FoS) considerations ....(6.1.6 Interoperability and 

Integration)

The CAM IPT settled on modifying three interrelated target areas that would then be codified through 

proposed policy changes: specifically, the SETR process for technical monitoring capability development; 

the 2-Pass, 6-Gate acquisition management schema 

for decision control measures; and finally the PoPS 

dashboard for monitoring program health assessment 

and reporting. We will explore the CAP IPTs efforts 

in each of these three areas in this article.
WARFIGHTING

CAPABILITLY

- POM/Governance

- R3B

- N00X Wholeness

- N81 

Capability/Capacity

- Near/Mid Focus

- Operational view

- Baseline

- TACSIT

- DOTMLPF

- Dashboard

- CMT

- Force Level/ SoSSE

- Capability 

Competence

- DOTMLPF (Limited)

- System Performance

- Force Level/SoS 

influence

- Capability Gate 

Review

WARFIGHTING

CAPABILITLY

ASN

OPNAV

SYSCOM/

WARFARE

CTR

FLEET,

WCOE,

COTF

Integration & Interoperability | 51

Table of Contents

Capability Acquisition Management: 

Modifying the Acquisition Process to Emphasize I&I

Figure 1. Warfighting Capability
Systems Engineering Technical Review Timing

Acquisition

Milestone/

Phases

Products

Acquisition

Gate

Reviews

Technical

Reviews

PHASE I A PHASE II PHASE III B PHASE IV C IOC FOC

Pass 1 Pass 2

ITR ASR SRR-I SRR-II SFR PDR IBR PDR1 CDR TRR FRR OTRR SVR/

FCA/PRR

Product

Baseline

Product

Baseline

System

Functional

Baseline

System

Specification

Preferred

System

Concept

Techincal

Baseline

Allocated

Baseline

Technology Readiness Assessment

Software Related Reviews

Technical Reviews

Program Reviews

Acquisition Review F=Gates

1 PDR Closure

ASR _ Alternative System Review ISR _ In-Service Review PRR _ Production Readiness Review

CDR _ Critical Design Review ITR _ Initial Technical Review SFR _ System Functional Review

FCA _ Functional Configuration Audit OTRR _ Operational Test Readiness Review SRR _ System Requirements Review

FRR_ Flight Readiness Review PCA _ Physical Configuration Audit SSR _ Software Specification Review

IBR _ Integrated Baseline Review PDR _ Preliminary Design Review SVR _ System Verification Review

IRR _ Integration Readiness Review TRR _ Test Readiness Review

TRA SSR IRR TRA

PCA ISR

Material Solution

Analysis (MSA)

Material

Development

Decision

System

Requirement &

Technology

Development

System

Architecture &

Technology

Demonstration

Engineering & Manufacturing Development Production &

Deployment

Operations &

Support

LRIP/IOT&E Sustainment &

Disposal

FRP Decision Review

Service Approved

CDD

Prototypes

1 2 3 4 5 6

Table of Contents

52 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Systems Engineering Technical Review

The first area the CAM IPT focused on was the 

SETR process. The SETR process is an integral element 

of the acquisition process and life cycle management, 

as traditionally depicted in Figure 2. Technical reviews 

coincide with and support key acquisition milestone 

decisions and gate reviews in the acquisition process, 

and provide an independent assessment of emerging 

designs against plans, processes, standards and specifications, and key knowledge points in the development process. The SETR process is integral to naval 

systems engineering and is consistent with existing 

and emerging commercial standards.

Although the SETR process consists of several 

technical reviews, depicted as red triangles in Figure 

2, the CAM IPT assessed I&I-related criteria and 

targeted three specific reviews: System Requirements 

Review (SRR); System Functional Review (SFR); and 

Preliminary Design Review (PDR). Each review is 

described below.

SRR The objective of the SRR is to ensure that 

operational requirements have been successfully represented by system requirements; the 

system design has the potential of meeting 

these system requirements when acceptably 

balanced with cost, schedule, and technical 

risk; and the plans for moving ahead with 

system development are reasonable and of 

acceptable risk.

SFR The objective of the SFR is to review and 

approve the system’s technical description, 

including its system requirements and architecture, and to establish the system’s functional baseline.

PDR The primary objective of a system or subsystem PDR is to review and approve the system 

architecture as defined by the allocation of 

requirements to the subsystems and configuration items, thereby creating the allocated 

baseline.

Figure 2. Systems Engineering Technical Review Process
FIRST PASS SECOND GATE

OPNAV Chaired ASN(RDA) Chaired

Gate 1 Initial Capabilities Document Approval Gate 4 System Design Specification Approval

Gate 2 AoA Approval Gate 5 Request for Proposal Release

Gate 3 Capabilities Development Document/

Concept of Operations Approval

Gate 6 In-Progress Review, Capabilities 

Production Document Approval

Integration & Interoperability | 53

Table of Contents

Capability Acquisition Management: 

Modifying the Acquisition Process to Emphasize I&I

Mission I&I SETR Event Tier Development

The CAM IPT started with a list of criteria that had 

been previously developed by an earlier SYSCOM I&I 

team. A tier structure was used to help analyze and 

determine I&I entry and exit criteria, leading to the 

traceability and evaluation of the system of systems 

throughout the life cycle. The SETR evaluation criteria 

was developed and documented for inclusion into the 

SETR handbook. The tier structure for organizing 

SETR criteria consisted of five tiers:

Tier 1 SETR event

Tier 2 Categories: mission engineering, systems 

engineering, test and evaluation, IA, open 

architecture

Tier 3 Subcategories of Tier 2, such as capability 

gaps analysis

Tier 4 Evaluation criteria

Tier 5 Detailed criteria designed to help evaluate 

Tier 4 criteria

The tier structure numbers are used to identify 

the categories and the actual I&I evaluation criteria.

2-Pass, 6-Gate Process

The second area the CAM IPT addressed was 

acquisition milestone management and milestone 

decision-making policies and processes. The Navy 

has mandated that all programs use the 2-pass, 6-gate 

process for program evaluations. The goal is to ensure 

programs are adequately assessed during the system 

engineering development process. As depicted in 

Table 1, the gate review process ensures alignment 

between service-generated capability requirements 

and acquisition, and improves senior leadership decision-making through better understanding of risks 

and costs throughout a program’s entire development.

Specifically, the CAM IPT reviewed Gate 3 and 

Gate 6 templates for inclusion of I&I activities and 

actions. This included rewriting Gates 3 and 6 entry 

and exit criteria, based on the understanding of 

SoS engineering. For Gate 3, the CAM IPT recommended providing test and evaluation strategies for 

developing end-to-end test plans. Early operational 

assessments, prior to developmental testing, were 

also recommended to reduce integration risk and 

SoS testing. For Gate 6, the CAM IPT recommended 

the use of mission threads for improving system and 

platform operational effectiveness, including how 

mission threads can be used to assess DOTMLPF 

material solutions.

Incorporating these changes to Gates 3 and 6 will 

provide the rigor necessary to ensure the program 

under review is fully reporting I&I characteristics 

and is capable of transitioning to the next phase with 

minimal risk.

Probability of Program Success (PoPs)

The Program Health Assessment Guidance, PoPS, 

assesses the health of an acquisition decision and the 

likelihood of implementation and associated impacts 

to capabilities by measuring a program’s real ability 

to accomplish its objectives and effectively illustrate 

Table 1. 2-Pass, 6-Gate
Program

Resources

External

Influencers

Program

Planning/Execution

Parameter Status

PoPS

Program

Requirements

Scope Evolution

CONOPS

Budget

Manning

Acquisition

Management

Industry/Company

Assessment

Cost Estimating

Contract

Execution

Government Program

Office Performance

Test and

Evaluation

Technical Maturity

Sustainment

Software

Fit in DoD &

Service Visions

Program Advocacy

Interdependencies

Congress

COCOM

OSD

Joint Staff

Department of the Navy

International Partners

CNO/CMC 

(Joint Programs)

Table of Contents

54 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

that status to decision makers. For Gates 3 and 6, 

the CAM IPT reviewed the associated PoPS criteria 

for inclusion of I&I acquisition requirements. When 

holistically implemented, PoPS produces an interactive dashboard (see Figure 3) that displays a program’s 

current “health status,” and enables leadership to 

identify issues, determine mitigation strategies, and 

allocate resources accordingly across the program.

The PoPS family tree is intended to provide a holistic look at a program and provide leadership an easily 

interpreted (red-yellow-green) consistent assessment 

of current program state. Of particular interest to the 

CAM IPT was the linkage, shown in Figure 4, among 

the SETR technical reviews, the acquisition 2-Pass, 

6-Gate events, and the continual PoPS measures and 

program status. The proposed SETR I&I criteria for 

SRRs, SFRs, and PDRs supports the I&I requirements at acquisition Gates 3 and 6. This information 

is reported through PoPS, which provides a holistic 

response to realizing capability acquisition management for mission-level I&I.

If criteria questions reveal issues during any of 

the targeted reviews, the program manager or senior 

leader can assess the program’s health based on program requirements, resources, program planning and 

execution, and external influencers.

Recommended Policy Changes to 

Support I&I

The CAM IPT identified four specific policy-rich 

documents at various levels of the Navy’s acquisition 

and development organizational structure to achieve 

coverage of mission-level I&I requirements at organization depth:

• OPNAVINST 9070.2a, “Signature Control 

Policy for Ships and Crafts of the U.S. Navy”

• CFFCINST/CPFINST 4720.3b, 

“C5ISR Modernization Policy”

• NAVSEAINST 5000.9/MARCORSYSCOM 

Order 5400.5/SPAWARINST 5000.1/

NAVAIRINST 5000.24, Naval Systems 

Engineering Technical Review Handbook

• Technical Review Manual Version 2.0, 

PEO IWS and NAVSEA 05H

The recommended update to these instructions 

ensures that integration and interoperability are considered across Navy systems of systems, and provide 

evaluation criteria for SETRs and design reviews.

Figure 3. Program Health Assessment Dashboard
SETR Criteria

Program

Resources

External

Influencers

Program

Planning/Execution

Parameter Status

Program

Requirements

Scope Evolution

CONOPS

Budget

Manning

Acquisition

Management

Industry/Company

Assessment

Cost Estimating

Test and

Evalution

Technical Maturity

Sustainment

Software

Contract Evalution

Government Program

Office Performance

Fit in Visions

Program Advocacy

Interdependencies

Naval PoPS

Gate Review

GATE 3 (CDD/CONOPS) REVIEW

PROGRAM NAME ACAT: _________

Joint Designation: ________

Program Manager Name: PM Code

Requirement Officer Name: RS Code

Briefer Name: Code

Review Date MM/DD/YY

CLASSIFICATION (A)

Advisor (Name) Code CLASSIFICATION (A)

Integration & Interoperability | 55

Table of Contents

Capability Acquisition Management: 

Modifying the Acquisition Process to Emphasize I&I

Summary

The goal of the CAM IPT was to address 

I&I issues and reduce the programmatic risk 

to enabling mission-level requirements. To do 

this, the CAM IPT developed new SETR criteria questions that addressed I&I across mission 

and platform systems of systems, proposed 

entry and exit criteria for the 2-Pass, 6-Gate 

review process, and recognized the value of 

having I&I issues visible in the PoPS dashboard. Codification of the new SETR criteria 

was accomplished by providing recommended 

additions to current policy documents. 

Figure 4. PSETR/Gate Reviews and PoPS Linkage
Table of Contents

56 | LEADING EDGE January 2015

In today’s budget climate, how can the Department 

of Defense (DoD) ensure they are still developing and 

procuring the equipment that our warfighters need?

By focusing on specific mission areas and looking 

across the Doctrine, Organization, Training, Materiel, 

Leadership and Education, Personnel and Facilities 

(DOTMLPF) spectrum for solutions to address mission capability gaps, Integration and Interoperability (I&I) is uniquely suited to find the “knee in the 

curve” and maximize the return on investment. A 

key component to this optimization concept is the 

ability to accurately predict costs across a spectrum 

of mission solutions. An accurate prediction of costs 

enables decision makers to efficiently utilize their 

resources to attain the maximum performance for the 

minimal cost, identify potential trade-offs, and identify areas for cost reduction and/or avoidance. This 

article discusses the role of cost estimating in the I&I 

process, including the challenges of mission-level cost 

estimates. It also provides an evaluation of current 

cost estimating techniques that may be applicable to 

mission-level analysis. Finally, it explores promising 

areas of research that may mitigate some of the current 

cost estimating challenges, such as the potential for 

using existing I&I artifacts, technical metrics generated during the I&I process, and Integrated Readiness 

Level (IRL) as metrics for cost assessments. The integration of cost analysis into the I&I process combines 

research and development with operational support 

and readiness in order to provide cutting edge cost 

support to the warfighter.

I&I From a Cost Perspective

By Jonathan Brown and Sarah Lloyd
Warfare Area Owner

Mission Area Advocacy

Processing

Resourcing Decisions

WCB

Assessment

WPN-TGT Pairings;

Effects Chains

CSM

Integrated Capability

Packages

Materiel Solution

Development Team

SYSCOM Lead

Materiel Solution

Development

DOTMPLF Solution

Production/

Acquisition

ICP

Resource ($) Test & Validate Solution Closes Gap Solution

OPNAV/ASN (RDA)

Are ICP recommendations

completely within OPNAV

Resource Sponsor span 

of control?

VCNO FFC/CPF (ICP 

Signed)

DNS

(Gate

Keeper)

No

Yes

RDA

In Scope

N8

 Not in Scope

Gate & 

Review

N1

N4

N2/6

High 8

Brief to

N8 or

N2/W6

N8 or

N2/N6

Review

NCB/

R38 0-7

A

B

C

D

R3B

COTF/Warfare

Center

Fleet Collaborative 

Team (FCT)

WCOE Lead

This is where the 

SYSCOMS contribute

WCB

Report

SCHEDULE COST

TECHNICAL

Integration & Interoperability | 57

Table of Contents

I&I From a Cost Perspective

Where is cost involved in I&I?

Figure 1 depicts the current I&I process. As previously discussed in this edition of Leading Edge, 

the Integrated Capability Package (ICP) prioritizes 

the set of DOTMLPF solutions proposed to cover 

the identified capability gap or set of gaps. Cost estimates are required for trades studies and analysis of 

alternatives as well as for each proposed solution in 

the set. Having estimates of the required funding for 

each solution allows decision makers to make more 

informed decisions. As illustrated in Figure 2, tradeoffs between cost, technical, and schedule are required 

when faced with limited resources. 

Challenges of I&I Cost Analysis

Cost Growth

While there are many different ways to measure 

the extent and nature of cost growth, there are multiple possible explanations and potential solutions for 

improvement. Often, technology development and 

acquisition programs are launched with too little focus 

on system and mission-level requirements needed to 

develop realistic cost estimates. Program managers 

may have optimistic schedules and assumptions, such 

as stable requirements, with corresponding low initial 

cost estimates that are appealing to funding sources. 

Realistically, however, significant requirements volatility is common during the early acquisition phase, 

which in turn causes significant cost growth. When 

attention isn’t given to possible changes and growth in 

requirements, budget overruns occur and programs 

Figure 1. Schematic of the I&I Process1

Figure 2. Trade-off between Cost, Technical, and Schedule
Team SMEs

SME Network

Table of Contents

58 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

risk being canceled. In order to prevent programs 

from being canceled, one must acknowledge and 

plan for the likelihood that requirements will grow 

and change. Canceled programs consume resources 

with little or no return on investment; moreover, 

they represent lost opportunities for viable programs 

that may also fail due to lack of funding. The result 

is a replicating program failure effect. The longer the 

program continues before cancellation, the greater 

the effect.

Requirements and Data 

Squeaky wheels get the grease, and dedicated, tenacious program managers are more likely to receive 

program funding. Cost analysts rely on these system-specific champions to help overcome obstacles to 

prioritizing the cost team’s technical or data requests. 

Costs cannot be determined for requirements that 

are not defined in technical terms or do not indicate 

the level of effort required. Therefore, cost analysts 

depend upon obtaining data from the engineering and 

technical teams to ascertain what costs will be affected 

by the various alternatives. Most important to the 

I&I efforts is to obtain candid technical assessments 

of impact to other systems to determine the causal 

effects on the cost. A strong program manager will set 

priorities that ensure the engineering and technical 

teams are thorough and responsive to the needs of 

the cost analyst’s data requests.

Program Focused

Generally, cost estimates, similar to most other 

systems engineering tasks, are program focused. 

Cost estimating within the context of the I&I process requires a paradigm shift from program focused 

estimates to mission focused estimates in order to 

accurately predict costs across a spectrum of mission 

solutions. When implemented, this type of analysis is 

similar to a large-scale Analysis of Alternatives and 

requires the cost analyst to have breadth vs. depth 

of analysis. Typically, cost analysis requires some 

level of specialization. Successful cost analysts often 

focus on one specific group of systems or platforms 

or one area of cost analysis. Examples of specializations include combat systems, ships, submarines, 

rotary wing aircraft, etc. Often, the specific system 

or platform is large; therefore the cost analyst gains 

a breadth of experience in a multitude of subsystems 

that exist within this area of specialization. Similar 

to the I&I technical subject matter experts, I&I cost 

analysts need to have not only a broad range of experience but also a large network of experts to leverage 

when required. 

Subject Matter Experts 

Expert opinion, or subject matter expert (SME) 

input, is important to cost estimating by bringing a 

wealth of experience and knowledge to the estimating process. Many times, experts help to identify 

analogous systems or provide data on labor hours or 

testing requirements to build, operate, and maintain 

a system. At the very least, an expert can provide his 

or her opinion on cost drivers, functional form of a 

regression and engineering rules-of-thumb on which 

the cost analyst can consider in the estimate. When 

working across multiple platforms, systems, and capabilities, it is not possible to have a dedicated SME for 

each potential solution on the team as compared to 

a typical system acquisition cost IPT with dedicated 

resources. A dedicated team with a robust network 

of SMEs is needed to have access to the required 

technical information, similar to a technical warrant 

holder pyramid as depicted in Figure 3.

Opportunities for I&I Cost Analysis

Despite multiple challenges presented by integrating cost analysis into the I&I process, many 

opportunities exist. I&I related research will set the 

Figure 3. I&I SMEs Require Large SME Network for Support
Integration & Interoperability | 59

Table of Contents

I&I From a Cost Perspective

conditions for providing more accurate cost estimates 

early in the acquisition process (pre-Milestone and 

Milestone A). Mission-level analysis reduces uncertainty, provides better metrics and allows for a more 

accurate cost model; hence, reducing the overall cost 

and increasing the probability of program success. 

Current cost estimating practices include providing 

cost estimates to program management throughout 

each phase of the acquisition life cycle. Cost estimates 

are often required at the earliest stages of a program’s 

development, i.e., Milestone A or earlier, when the 

acquisition strategy and requirements are loosely 

defined. This yields rough order of magnitude cost 

studies, surrounded by a high degree of uncertainty 

upon which important program decisions are made. 

Furthermore, sometimes program-specific analysis 

focuses on a single platform and a single system. This 

may lead to a product based on a limited sight picture 

rather than providing the customer with a holistic, 

system-of-systems life-cycle analysis that is important 

to providing the decision maker with various program 

options (both technical and cost). This program-specific analysis increases the risk that problems will 

arise when the system or capability must interact 

and interoperate with other systems to accomplish 

its mission. Interoperability problems are becoming 

a focus area and are expensive to fix once the system 

or capability has been developed. An example of this 

is the recent Accelerated Mid-Term Interoperability 

Improvement Project (AMIIP) that was created to 

address Fleet concerns about force-level interoperability; specifically to address consistent correlation 

between Cooperative Engagement Capability (CEC) 

and Tactical Data Link (TDL) tracks.2 Mission-level 

analysis helps to avoid these interoperability issues 

and escalating costs by addressing system interactions required to accomplish the mission early, at 

the concept phase, in an acquisition program, thus 

reducing potentially expensive interoperability “fixes” 

in the future.

Another acquisition problem currently facing the 

Navy is the development of technologies or capabilities with no clear insertion point into the Fleet. 

These “orphaned” programs have already sunk the 

cost of development but provide no direct benefit 

to the warfighter because they are never integrated 

into a platform. More likely, they are installed onto a 

platform but not integrated into the combat system. 

The overall I&I process would reduce orphaned programs by either identifying a specific mission gap 

each capability will be addressing or, if no mission 

need or no better solution is identified, the program 

will not be recommended.

The final opportunity to consider relates to the 

technical data available for cost estimation. The I&I 

process puts an emphasis on upfront mission architecture. This architecture process produces metrics 

that have the potential to be used for cost analysis. 

The emphasis on upfront mission architecture in the 

I&I process ensures that the metrics are received early 
Initiation and research

Your audience, what you

are estimating, and why

you are estimating it are

of the utmost importance

Assessment

Cost assessment steps 

are iterative and can be

accomplished in varying 

order or concurrently

Analysis

The confidence in the point or 

range of the estimate is crucial 

to the decision maker

Presentation

Documentation and

presentation make or

break a cost estimating

decision outcome

Analysis, presentation, and updating the estimate steps

can lead to repeating previous assessment steps.

Source: GAO.

Define the

estimate’s

purpose

Develop the

estimating

plan

Define

the

program

Determine

the

estimating

structure

Identify

ground

rules and

assumptions

Develop the point

estimate and compare

it to an independent

cost estimate

Obtain

the

data

Conduct

sensitivity

Conduct a

risk and

uncertainly

analysis

Document

the 

estimate

Present

estimate to

management

for approval

Update the

estimate to

reflect actual

cost/charges

Table of Contents

60 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

and often. This topic will be expanded on later in this 

article, but some technical metrics that potentially 

influence cost are information exchange messages 

and Technology Readiness Levels (TRLs).

Evaluation of Current Cost Process 

Suitability to I&I Cost Analysis

The following section summarizes the current cost 

analysis process and identifies the suitability to its 

applicability to the I&I process. Figure 4 represents the 

Government Accountability Office’s (GAO) cost estimating process. Often, cost analysts apply a tailored 

version of the GAO’s Cost Estimating Process that is 

most applicable to the program and associated tasks.

Define the Estimate and Develop the Estimating Plan

One of the most critical first steps is to define the 

scope of the cost estimating effort and to set the due 

date for the completed cost estimate. Upfront planning will still be required for any cost estimate, I&I 

or otherwise. 

Define the Program

I&I cost estimates will require the analyst to define 

the program. This will be more difficult for an I&I 

program because instead of a single program, system 

or capability, the I&I solutions will likely be a range 

of options to address the given mission. For traditional acquisition programs, this defining information 

comes from a Cost Analysis Requirements Description (CARD) or other written documentation from 

the program office. When the CARD is unavailable 

or does not include enough information, the cost 

analyst and technical team must work together to 

identify constraints and reasonable assumptions that 

will define the effort. For I&I, similar information is 

still required; however, there will no longer be a single 

source document or program office. Furthermore, it 

is unlikely that the depth of information required in a 

CARD will be available for each option. Similar to, but 

broader than an AoA, a prioritized list of information 

required for an I&I cost estimate must be provided 

by a network of program offices and SMEs. The list 

of information required could be far more extensive 

than other early estimation efforts.

Determine the Estimating Structure

An I&I cost estimate will still require the development of a Work Breakdown Structure and identification of an estimating method for each. This part of the 

process will likely be similar to other cost estimates. 

Identify Ground Rules & Assumptions (GR&A)

While the actual ground Rules and Assumptions 

(GR&A) developed for an I&I cost estimate may vary 

from a traditional cost estimate, the process to identify 

them will be similar.

Figure 4. The Cost Estimating Process3
Estimating by

ANALOGY

Estimating by

PARAMETRICS

Estimating by

ENGINEERING

Estimating by

ACTUAL COSTS

What is it?

When used?

How is it done?

PROS

CONS

Single value from

single data point 

Measure of trends

across programs 

Detailed build-up of

Lab, Mtl & OH $ 

An extrapolation of

current program cost 

Early in Program

Pre-Milestone A & 

Milestone A 

Milestone B Late in Program 

Milestone C & LRIP

LRIP

Full Rate Production

Adjust analogous

system cost or create

cost factor 

Apply statistical

methods to cost of 2

or more systems

(i.e., Develop a CER) 

Estimate at lowest 

cost level & sum

costs by WBS 

Use trend from your

current contract to

estimate your final

system $ 

Fast, inexpensive

Easy to change

Based on >1 data

point =>less risky

Can measure error 

Easy for what-ifs

More details enables

better visibility into

cost drivers 

Most costs are known

CAIG prefers over

other methods 

Based on single

historical data point

+> risky!

Trends to be more

subjective 

Constrained by

amount & quality of

data (GIGO) 

Statistics can be

misleading 

Labor Intensive

Slow, Expensive

Can lose sight of 

“big picture”

Usually too late to

use actual costs to

adjust or build budget

Not a 1:1 correlation

of prototype-toproduction costs 

Integration & Interoperability | 61

Table of Contents

I&I From a Cost Perspective

Obtain The Data

Data gathering is the most difficult task in the cost 

estimating process. Data collected falls into two basic 

categories: data from previous programs used as a 

basis for cost relationships and technical data for the 

current program used to project future costs from the 

historical actuals. For a typical cost estimate, the analyst must identify SMEs, analogous systems, previous 

cost estimates on the topic, relevant databases, Cost 

Analysis Requirements Description, and pertinent 

program and engineering data from the program 

office. As mentioned above, for I&I cost estimates, this 

process is difficult since there are multiple platforms 

and systems involved to address a specific mission. 

However, there are potential new sources of technical 

data available from the I&I mission architecture to 

help address this complexity. 

Develop the Point Estimate

There are four accepted methods by which 

cost estimates are generated: analogy, parametric, 

engineering, and actuals. Table 1 summarizes the 

four methods.

Analogy

Analogy entails using a single comparison value 

from a system with similar characteristics. This 

method is most often used early in the acquisition 

phase when there is little or no data available on the 

new system. While it is relatively quick and easy to 

use compared to with other methods, the results may 

be generalized and subjective. 

Parametrics

The parametric approach uses statistical means 

to measure trends across multiple programs. This 

method requires a large amount of data; however, 

the outcome yields measurable and tractable results. 

Engineering Estimate

An engineering estimate is a detailed, bottom-up 

approach which is usually used later in the acquisition 

process. Gathering the data required can be a slow 

process that is labor intensive but costs are estimated 

at a high level of detail which will provide better 

insight into cost drivers. 

Table 1. Cost Estimating Methodologies4
Table of Contents

62 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Actual Costs

Actual costs or “actuals” are an extrapolation from 

the current data of the costs that have been incurred. 

These costs are usually the best and most precise costs; 

however, often the data becomes available too late in 

the acquisition cycle to be useful.

While the actual analogies or technical metrics 

used in a parametric estimate may differ from a traditional cost estimate to an I&I cost estimate, there are 

no expected variances to the four acceptable methods. 

Given the information required to use an engineering 

buildup or actual cost, employing these methods 

will not likely be frequently used in the I&I process. 

The fact that the I&I analysis is conducted early in 

the acquisition life cycle will necessitate a larger use 

of analogies and parametrics. However, this is not 

absolute. For example, if an existing technology is 

proposed as a solution in an ICP, the actual costs of 

that item would then be used in the I&I cost estimate. 

Final Steps in the Estimating Process

The final steps, shown in Figure 4, of uncertainty 

analysis, presentation to management, and updating 

the estimate will require no anticipated changes from 

the traditional process. However, there is a subtlety 

to be emphasized. A typical cost estimate for a program or system usually resides under a single System 

Command (SYSCOM) and program office. There may 

be additional stakeholders, but the SYSCOM and 

program office are responsible for providing the final 

review of the cost (SYSCOM Cost Organization) and 

technical (program office) assumptions and methodologies. By contrast, an I&I cost estimate could 

include multiple programs or systems that potentially cross multiple SYSCOMs (NAVSEA, NAVAIR, 

MARCORSYSCOM, etc.). Who would provide the 

final review and be responsible for the validity of an 

estimate that crosses multiple SYSCOMs and program 

offices? Some options for answering this question 

could include designating a lead SYSCOM for each 

ICP. The ICP lead SYSCOM would be responsible for 

approving the entire package. An alternate approach 

would be to require individual SYSCOM sign off on 

the respective technical area of a single ICP. The final 

process is yet to be determined, but responsibility 

of the I&I cost and technical review will need to be 

established.

Cutting Edge Research and Promising Topics

Cost estimates for I&I products present several 

challenges to the cost estimator. These challenges also 

open up new cost research areas to help mitigate or 

overcome these challenges. The following topic areas 

are of particular interest.

Messages, Software Estimating, And Analysis 

Modeling (SEAM)

A 2009 study conducted at NSWCDD, Software 

Estimating and Analysis Modeling (SEAM), investigated current software size estimating processes 

involving Software Lines of Code (SLOC), recommended improvements, and suggested alternatives. 

One promising alternative technical metric suggested 

was the measurement of messages. Messages are the 

packets of information sent from one software component to another. In the research, a preliminary 

statistical relationship was established between the 

number of messages and the number of SLOC. While 

this technical metric for cost measurement requires 

more research before utilized for cost analysis, it is 

very promising from an I&I perspective. SLOC is the 

primary software sizing metric for cost analysis, but 

as mentioned above, is not likely to be available early 

on during ICP development. Because of the emphasis 

on mission architecture, message counts are likely to 

be available for I&I analysis. If the preliminary results 

remain stable under further research, this new metric 

will provide a link between I&I mission architectures, 

SLOC and cost. To capitalize on this research would 

require the I&I ICP development process to include 

sufficient time to conduct the architecture development at a low enough level to be useful to the cost 

estimate.

Readiness Levels

Previous GAO assessments have correlated low 

technology maturity with programmatic problems; 

programs that began development with immature 

technologies saw a 32% cost growth and 20 month 
Risks/Unknowns

Product

Requirement

TRL 1 2 3

4 5

6 7

8 9

HIGH RISK

for product launch

LOW RISK

for product launch

Integration & Interoperability | 63

Table of Contents

I&I From a Cost Perspective

schedule growth.5 There is a growing demand for 

cost analysis during the early phases of the acquisition cycle when the level of technology maturity is 

low. Therefore, a promising area of future research, 

and one that serves to provide significant dividends 

in the future, includes relating cost to TRL, or any 

of the various readiness levels, in the face of limited 

data (see Figure 5).

“Maturing new technology before it is included on 

a product is perhaps the most important determinant 

of the success of the eventual product—or weapon 

system.” The GAO (1999) also encouraged the use 

of “a disciplined and knowledge based approach of 

assessing technology maturity, such as TRLs, DoDwide (p. 7).”6

Types of Readiness Levels

There are multiple types of readiness levels that 

may be relatable to cost. Technology Readiness Level 

(TRL) is probably the most commonly known readiness level because it is used as the DoD standard 

evaluation system to categorize hardware, software 

or system concepts. 

TRL is simply a measure of an individual technology and it does not account for readiness for use 

in the system context. The definition of TRL varies 

across the Department of Defense, Department of 

Homeland Security, Department of Energy, National 

Aeronautical Space Administration (NASA), and 

European Space Agency. Integration Readiness Level 

(IRL) is the status of the connections between the 

technologies. System Readiness Level (SRL) is the 

system-level appraisal of development maturity and 

is a function of IRL and SRL. Manufacturing readiness levels (MRLs) are quantitative measures used to 

assess the maturity of a given technology, component 

or system from a manufacturing perspective. Finally 

Cost Readiness Levels (CRLs), as used by NASA, 

are designed to communicate the quality of the cost 

product and its fitness for use.

Application to Cost

In theory, acquisition program managers generally prefer to seek technologies at TRL 6 or higher; 

however, in practice, this is not always possible and 

consequently less mature technologies are selected for 

development. Cost analysis and support are requested 

by program managers throughout all phases of the 

acquisition timeline. Therefore, to increase the accuracy of cost estimates early in the acquisition cycle and 

reduce the variance of the total cost estimate; further 

research should be conducted where cost intersects 

Figure 5. Programmatic Risk as a Function of Technology Readiness Level7
Table of Contents

64 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

with TRL, IRL, SRL, MRL and/or CRL. Research that 

statistically describes the relationship between cost 

and the various readiness levels will enable cost analysts to reduce the variance of the total cost estimate 

and provide program managers with more precise 

cost figures upon which to base decisions. 

Extension of Current Work

Limited studies, research, and databases exist on 

relating cost-to-readiness levels.8 The TRL calculator, 

developed by the U.S. Air Force, provides a snapshot 

of maturity at a given point in time. The technology 

program management model, developed by the U.S. 

Army, is used to help with technology transition for 

program managers. In order to characterize readiness levels in terms of cost, further research must 

be conducted, and the notion that cost is a function 

of maturity and risk, transition time (duration) and 

acquisition phase must be tested.

Conclusion

Moving to an I&I based acquisition process has 

many advantages to the warfighter. The increased 

focus on missions allows decision makers and 

resource sponsors to focus limited resources on targeted improvements or acquisitions. Accurate cost 

estimates are the keys that unlock successful trade 

offs. Cost analysis from the I&I perspective presents unique challenges and opportunities. Most of 

the challenges arise from the breadth of platforms, 

systems or capabilities that may be considered for 

a particular mission. This necessitates a network of 

technical SMEs and cost analysts to cover the requisite 

knowledge and skills. The opportunity related to mission-focused cost analysis is the ability to efficiently 

characterize alternatives in order to attain the optimal 

benefit for the least cost. Again, this requires accurate, 

defendable cost estimates. Overcoming these challenges while producing accurate estimates will require 

some ingenuity and cost research. Fortunately, there 

are several promising areas of research, including 

messages and readiness levels, awaiting interest and 

resources. 

References

1. Learn, “I&I Overview”, March 2013. 

2. Bill Williford, Director, Integrated Combat Systems, “Maritime 

Crossroads: Strategies for Action”, www.navsea.navy.mil/Media/

SAS2013/2.%20PEO%20IWS.pdf, April 2013. 

3. http://www.gao.gov/new.items/d093sp.pdf 

4. https://dap.dau.mil/acquipedia/Pages/ArticleDetails.

aspx?aid=8e8f5bf3-f517-4e98-8c07-80bb8670a830 

5. www.dodmrl.com/Gordon_NDIA.ppt 

6. GAO, “Best practices: Better management of technology development 

can improve weapon systems outcome”, 1999, (GAO/NSIAD-99- 162). 

7. GAO, 1999, p. 24 

8. Studies related to cost and readiness levels: El-Khoury and Kenley, 

2012; Lee & Thomas, 2003; Hoy & Hudak, 1994; and Dubos & Saleh, 

2008.
Integration & Interoperability | 65

Table of Contents

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based on 

Mission Effectiveness

By Greg McHone

Mission Level Assessment and Evaluation (MLA&E) is a Mission Engineering approach 

to an iterative fleet/acquisition capabilities-based methodology, by which operational 

needs are defined through warfighter-validated mission architectures that influence 

joint and fleet rehearsals, exercises, and experiments. MLA&E’s warfighter integration intensive methodology evolves the Chief of Naval Operation’s (CNO’s) 

Integration and Interoperability (I&I) coordination with the fleet to one that is 

centered on warfighting capability and the technical-to-tactical excellence principle. Embedding MLA&E engineers and architects with the warfighter creates 

conditions for a philosophical change in evaluation, experimentation, and assessment that enables mission engineers to participate in the planning, execution, and 

analysis of fleet events to provide convincing evidence of warfighting capability 

through fleet-validated mission architectures. These validated operational-based 

architectures are then applied to existing programs and evolving technology to 

improve warfighting readiness. Under MLA&E, mission engineering starts with 

the desired warfighting capability, then scopes the platforms, systems, and performers necessary to achieve the desired warfighting capability. 

Above:

Valiant Shield 2014

Joint and fleet forces 

rehearsed during a joint 

maritime exercise to 

achieve a desired 

warfighting capability.
• Requires Critical Measures:

 - Quantitative and qualitative measures of performance or effectiveness

 - Tied to factors that impact Mission Success

• Requires Appropriate Scoring Criteria for each Critical Measure:

 - Used to assess Full Capability, Limited Capability, or Insufficient Capability

 - May vary by TACSIT and potentially by individual platform/system

MLA&E EASEL CRITERIA

Platforms support of the mission

task will be scored against Critcal

Measures A, B, and C, D using

approciate Scoring Criteria for each

to achieve mission success

Mission

Task

Mission

Success Mission

Task

Mission

Task

Starting with Mission

Success, criteria flows

from “Right to Left” and

breaking traditional

program paradigm

• No “hits” on mission

 essential unit (MEU), i.e.,

 Mission kill on every

 target in the raid

• No friends engaged

Platform 1

Platform 2

Mission Success

Factor 2

Critical Measure B

Mission Success

Factor 1

Critical Measure A

Mission Success

Factor 4

Critical Measure D

Mission Success

Factor 3

Critical Measure C

Table of Contents

66 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Definition

The Vice Chief of Naval Operations (VCNO) 

established MLA&E activity under the I&I Charter, 

19 DEC 2012. Scope, products, and resourcing of 

MLA&E were not yet defined in the charter, as compared to other prominent I&I activities, i.e., Warfare 

Capability Baseline (WCB) (weapon to target pairing 

kill-chain analysis) and Capability Solution Management (proposed solutions to gaps identified in killchain analysis). Since the early establishment of the 

Chief of Naval Operation’s I&I initiatives, MLA&E’s 

scope, approach, and deliverables have been vetted 

and refined through various fleet applications. The 

current version of the Integrated Capability Framework Operational Concept Document (ICF/OCD), 

dated 30 SEP 2013, defines MLA&E as understanding 

and quantifying how well a mission capability is currently achieved, and exploring, across the Doctrine, 

Organization, Training, Materiel, Leadership and 

Education, Personnel and Facilities (DOTMLPF) 

spectrum, the concepts for measurable mission 

improvement; enabling the insertion of I&I stakeholders into Fleet Exercises and Experimentation 

venues; and applying the ICF/OCD to capture data 

needed to evolve mission performance.

MLA&E consists of applying two related activities with different applications to fleet rehearsals 

and exercises. The “assessment” aspect of MLA&E 

identifies “baseline” warfighting capability through 

development of “as is” mission architectures derived 

from fielded systems. “Evaluation” is the application 

of MLA&E to fleet experiments to capture future 

Figure 1. MLA&E Criteria
Integration & Interoperability | 67

Table of Contents

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based on Mission Effectiveness

“to be” mission architectures for non-materiel and 

materiel modifications. Through collaborative Fleet/

Engineering planning and execution processes, 

both assessment and evaluation applications of 

MLA&E have direct implications for near- to longterm holistic capability solutions that lead to improved 

warfighting readiness.

The MLA&E approach considers the larger I&I 

process, as promulgated in the VCNO’s I&I Charter, to 

embed the ICF/OCD data model application with fleet 

operations and realize the potential of MLA&E with 

Warfare Capability Baseline’s kill-chain assessments 

and Capability Solutions Management’s Integrated 

Capability Package development for proposed materiel 

and non-materiel solutions. MLA&E fleet-endorsed 

mission architectures represent the Deputy Assistant 

Secretary of the Navy (DASN) Research, Development, Testing and Evaluation (RDT&E) efforts to 

understand warfighter requirements for acquisition 

solutions that are available to the WCB, as well as 

address gaps identified in Warfighting Capability 

Assessments (WCA) that focus engineering and architectural resources for proposed integrated materiel 

and non-materiel solutions. Warfighting capability 

is represented by “Mission Success” in Figure 1.

The DASN RDT&E resourced the initial MLA&E 

effort where mission engineers from all Navy Systems 

Commands (SYSCOMs) were to validate the ICF/

OCD by applying the model to a major fleet exercise. 

The Fleet Experimentation Exercises and training 

category is one of eighteen uses defined by the ICF/

OCD. Once ICF/OCD was applied, MLA&E activity 

was to provide feedback on the data model’s utility 

for capturing fleet operations. To determine the best 

fleet operations candidate, MLA&E team reviewed 

a list of major exercises that provided the necessary 

mission area opportunities to apply ICF/OCD. Based 

upon concurrence from Commander, Pacific Fleet 

(PACFLT) Warfighting Assessment and Readiness 

(WAR), Valiant Shield 2012 was the major fleet venue 

selected for initial MLA&E application of the ICF/

OCD. The discussion that follows uses Valiant Shield 

2012 as a representative use case.

Valiant Shield 2012

Valiant Shield is a recurring blue-red operational 

and fleet exercise leveraging joint participation of 

across service assets to counter a notional theater 

threat to a strategic ally in the Pacific theater of operations. The primary mission area focus of Valiant 

Shield 2012 revolved around Anti-Subsurface Warfare 

(ASW). However, from previous Valiant Shield exercises, several capability gaps in the Anti-Surface Warfare (ASuW) domain had been identified. Through 

coordination with PACFLT WAR staff, the MLA&E 

team began executing towards modeling ASuW operations. With PACFLT WAR, and given capability gaps 

identified during previous Valiant Shield exercises, the 

MLA&E team scoped their modeling effort down to 

the subject of Surface Action Group (SAG) operations. 

Validating the ICF/OCD by applying it to a 

major fleet exercise was the principal I&I objective 

of MLA&E for Valiant Shield 2012. To accomplish 

this objective, the team examined the utility of the 

ICF data model to support fleet analysis, assessment, 

and requirements definition of mission performance. 

Additional objectives of MLA&E for Valiant Shield 

were to capture mission area architecture by identifying data attributes that establish system performance 

requirements and constraints, develop compliant 

architecture for capability integration, and establish 

fleet user interface views to graphically represent 

Valiant Shield 2012 is an integrated joint training exercise that offers the 

opportunity to integrate Navy, Air Force and Marine Corps forces at sea. 

(U.S. Navy photo by Mass Communication Specialist 3rd Class 

William Pittman/Released)
Table of Contents

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

68 | LEADING EDGE January 2015

an applied ICF concept. The final objective was to 

support institutionalization of the CNO’s I&I mission assessment processes by establishing MLA&E 

application to fleet exercises and experiments in order 

to explore DOTMLPF solutions that meet near- and 

long-term operational challenges. Embedding I&I 

mission engineers with fleet users to identify and articulate operational needs for Research, Development, 

and Acquisition was necessary to capture MLA&E 

impact—Integrate fleet readiness with system development and performance. 

Applying MLA&E during Valiant Shield 2012 provided a full spectrum of data collection activities in 

compliance with the ICF data model for architecture 

development. ICF specifies the following data elements 

and their relationships: 

• Authoritative warfighting capability data 

elements, relationships, and taxonomies 

• Canonical viewpoints and models to capture 

mission and system/platform capability data 

• Reference to authoritative guidance, 

standards, and sources for information

• Configuration management guidance

• Common standards for capturing and 

sharing framework

• System/Mission Alignment Model 

(relationships, mappings)

In addition, the MLA&E activities exercised new 

roles and responsibilities of organizations that collaborate to construct the mission model and system/

platform configuration baselines for Valiant Shield. 

The MLA&E’s capabilities-based approaches proved 

that opposing forces capabilities are essential in determining warfighting capability and were therefore 

integrated into the Valiant Shield architecture via 

friendly force’s ability to defend and offensively engage. 

Opposing forces’ conceivable capabilities would desire 

to achieve similar capabilities to those of the U.S. Navy, 

at least at a high level of abstraction such that tasks 

from the Universal Navy Task List (UNTL) may apply 

to both opposing forces and friendly forces. Application of ICF to MLA&E by operational and acquisition 
Integration & Interoperability | 69

Table of Contents

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based on Mission Effectiveness

stakeholders produced a foundation on which to build and evolve 

ME disciplines for improving fleet 

readiness. For example, Valiant 

Shield forces executed the sinking 

of a target ship (SINKEX) during 

2012 exercise. Under traditional 

methodologies, the architecture 

would contain weapon, platform, 

targeting sensor, and target for 

the SINKEX. During this event, 

MLA&E produced architecture 

that identified the interdependencies of all participating platforms, systems, performers, and 

authoritative doctrine necessary 

to achieve the desired effect: sinking of the target ship. Figure 2 is a 

photograph of weapon impact on 

target ship.

Capturing fleet user perspective and determining mission 

area owner guiding principles 

were necessary to efficiently and 

effectively translate mission area requirements into 

acquisition and engineering processes. Under this 

principle, PACFLT WAR served as the mission area 

owner and provided governance over the operational 

data used for architectural development. 

Mission Area Owner

Establishment of a “Mission Area Owner” is the 

keystone to MLA&E’s ability to obtain fleet validated 

and advocated mission threads that drive capability 

based acquisition:

“Fleet ownership of mission from definition, 

through acquisition, to execution with ability to 

assess and define mission needs to acquisition 

community in order to ensure systems will meet 

full mission needs or understand and agree to 

limitations.” 

(PACFLT WAR: Valiant Shield 2012 Final Planning Conference)

Fundamental to capturing a fleet Mission Model in 

the ICF is the establishment of the guiding principles 

for mission area application. This ensures fleet and 

acquisition objectives and tasks stay true to agreed 

expectations by following clear and concise guiding 

principles to performing mission-level assessment 

processes for the application of the ICF data model to 

mission areas. The following are representative guiding principles of MLA&E agreed upon with PACFLT 

WAR in applying I&I activities to capture fleet user 

requirements:

• Establish continuity of ICF/OCD across near, 

mid, and long-term to scope acquisition 

decisions and DOTMLPF solutions.

• Provide convincing evidence to show 

operational benefit and impact.

• Establish rigorous process to model 

operational environment that provides 

consistency and is responsive to change.

Figure 2. Sink Exercise (SINKEX)
Table of Contents

70 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

• Ensure operational requirements are 

governed and managed by the fleet and 

understood by the acquisition community as 

a user.

• Define interaction with fleet and acquisition 

stakeholders, consisting of periodic reviews 

to ensure needs and expectations are 

understood.

• Allow mission architects to manage the fleet 

validated operational data and programs of 

record systems data for development of a 

common mission area-based architecture.

As the Mission Area Owner for Valiant Shield 2012 

MLA&E, PACFLT WAR served as the Fleet Representative to the Mission Technical Baseline Authority. 

The Fleet Representative is the overall lead for Mission Technical Baseline development and validation, 

responsible for representing fleet requirements and 

interests, and to ensure the Mission Technical Baseline 

accurately reflects or adequately models operational 

reality (DOTMLPF). Mission Technical Baseline will 

be discussed in detail later in this article. 

With a defined objective to generate a relationship between the fleet’s desired effects and the system 

functions that are necessary to achieve those effects, 

development of a capability-based model was required 

to produce mission area architectures. 

Capabilities-Based Modeling Methodology

The MLA&E approach to applying the ICF/OCD 

to produce mission architectures is captured through 

a capabilities-based modeling methodology. Capabilities Based Modeling Methodology enables the 

operational, research, and acquisition communities to 

interpret and decompose the Commander’s guidance 

with consistency using the ICF data model that leads 

to efficient and effective mission architecture development. The MLA&E team established and followed a 

Capabilities Based Modeling Methodology to develop 

architectural products that model fleet operations 

starting at the fleet’s desired effects. Capabilities Based 

Modeling Methodology focuses on capabilities rather 

than individual point solutions or specific mission 

threads; the architecture can be used as a source of 

mission data and activity relationships for a multitude 

of different mission threads. Figure 3 graphically 

demonstrates the roadmap followed to create fleet 

validated mission threads in a reusable format for 

engineering and system design.

The objective of Capabilities Based Modeling 

Methodology is to generate a relationship between 

the Fleet’s desired effects and the system functions 

performed by material solutions generated by the 

acquisition community. Through this relationship, 

the fleet can clearly articulate the current state of 

deployed material solutions, their current assessment 

of system performance, and their current and future 

system needs as compared to particular desired effects 

and capabilities.

To maximize Fleet relationships and minimize 

impact to operational staff battle rhythm, Capabilities 

Based Modeling Methodology processes are integrated with existing fleet planning, assessment and 

analysis processes. Leveraging the efficiencies of existing fleet processes enables a more effective review of 

MLA&E products by operational planners. Returning 

to the Valiant Shield example, to ensure efficiencies 

optimize I&I impact, the MLA&E team integrated 

with exercise warfare and assessment syndicates 

to identify stakeholders, products and attributes. An 

additional tenant to this approach was to capture 

the existing PACFLT data collection process and compare it with I&I data, supplementing as needed to 

complete the architecture. 

Establishing a common language between the fleet 

and mission engineers is essential for operational 

architecture (data) development. This operational/

acquisition taxonomy was realized through significant 

warfighter integration during Valiant Shield 2012. 

The data required to develop Department of Defense 

Architecture Framework (DODAF) Data Model 2 

(DM2)-compliant views is comparable to the data 

needed by fleet readiness planners to perform analysis, 

complete assessments, and generate requirements. The 

variation between fleet and acquisition stakeholders 

is the way the data is organized and viewed based on 
MLA&E CAPABILITIES BASED ROADMAP

Summary Document

Combines commander’s 

guidance & mission statement 

with objectives, score, goals, etc.

Describe the Scenario

Provide a graphic and narrative

that explains fleet exercise.

Capability Taxonomy

Identify Required Operating

Capabilities (ROC) focused on.

Mission Capability Dependencies

Identify relationship between

multi-mission capabilities.

Capability-to-Activity

Identify doctrinal measure that

support a given capability.

Mission Threads

Warfighting capability defined and

Fleet validated in a format useable

by engineering for system design.

Captures operational relationships

to achieve desired effect as 

mapped into exercise planning.

AV-1

CV-1

CV-2

CV-4 CV-6

CV-5

SV-5

SV-4

SV-7

SV-1

SV-2

SV-3

OV-2

OV-3

OV-4

OV-6

SV-10

OV-1

OV-5

AV-2

StdV-1

StdV-2

Diagram shows an iterative Warfighter/Engineering model

where mission engineering influences operational exercises 

to provide convincing evidence of combat readiness.

EXTERNAL MODEL

MISSION THREAD

Integration & Interoperability | 71

Table of Contents

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based on Mission Effectiveness

unique user requirements. For example, Operational 

Views (OV), such as an OV-6a and OV-6c, would be 

viewed by fleet users as a “mission thread,” that is, a 

snapshot in time and uniquely tailored to their data 

requirements. Timelines may include initial conditions that impact how each activity is performed or 

how well the overall mission thread is able to satisfy 

the overall desired effect. 

Desired effects are inherently linked to a capability. Under capability-based methodology, MLA&E 

began by identifying the purpose, objectives, scope, 

hypotheses, and desired effects of the Fleet and then 

derived from them the capabilities required to achieve 

mission success. Through a collaborative fleet/mission 

engineer effort, the AV-1 therefore evolved into Commander’s Guidance and Intent for the I&I activities 

during Valiant Shield. Collaborative AV-1 provided a 

common reference document to influence the schedule of events planning that supports achievement of 

overall assessment and acquisition objectives through 

the development of capabilities-based architecture. As 

such, the principal product from MLA&E efforts are 

mission-based architectures created using a capabilities based modeling that reflects overall warfighting 

readiness against a defined threat. 

Mission Technical Baseline Architecture

Mission Technical Baseline (MTB) architecture 

development of mission threads has evolved to 

become the principal product for MLA&E related 

I&I activities. ICF MTB data elements and products capture operational requirements and identify 

required capabilities at the mission task and interface 

level that provides a top level Mission Area and Navy 

Tactical Tasks (NTA)-based task organization that 

provides a common operational framework for Navy 

I&I activities. This decomposition provides a mechanism to discover and define needed systems and 

Figure 3. Capabilities Based Roadmap
Table of Contents

72 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

platforms interfaces and behavior required to support 

end-to-end mission capability. MLA&E is the foundation for evolving mission engineering disciplines 

for improving warfighting readiness that provide 

the vehicle for developing mission architectures that 

support fleet and acquisition users. There are varying 

states and uses for an MTB that is determined by 

multiple stakeholders and their application of the 

data for specific products. Representative examples 

of MTB from MLA&E application include: 

• Snapshot of a specific instance assessment of 

deployed warfighting capability

• Reference that baselines applicability to 

programs and Navy-wide deployed systems

• Excursion derived from experimentation to 

influence research

Snapshot MTB of deployed mission architecture 

is developed to support fleet analysis, assessment, 

and requirements generation of existing warfighting capability. Mission architecture is derived from 

defined mission objectives and effects, and deployed 

Joint and Navy assets operating under theaterdefined doctrine to rehearse the ability to effectively 

execute operational and contingency plans. These 

mission effects are mapped up to authoritative mission capabilities, i.e., Joint Capability Areas (JCA), 

Required Operational Capabilities (ROC) and Projected Operational Environment (POE), Universal 

Naval Task List (UNTL), Navy Mission Essential Task 

List (NMETL), and down to platforms and systems 

that support achievement of those effects. Valiant 

Shield 2012 represents a specific instance of Mission Technical Baseline development that supports 

baseline assessment of deployed warfighting capability. MLA&E mission engineers provided mission 

threads developed from MTB to PACFLT WAR Data 

Analysis Working Group (DAWG) to provide convincing evidence of mission effectiveness for ASuW 

mission area. Theater-specific Mission Technical Baseline that supports assessment of warfighting capability 

is made available to the development and validation 

of “reference” architectures. Deployed and reference 

baseline MTBs are system agnostic as to how missions 

will be conducted.

Reference MTBs are “top-down” derived from 

authoritative mission capability requirements, i.e., 

JCA, ROC, UNTL, and NMETL. This differs from 

snapshot MTB in that a specific theater operational 

commander’s desired effects are the starting point and 

core of the mission thread. Reference MTB serves to 

align authoritative mission capability requirements 

with fielded systems and systems under development 

to better inform current acquisition processes and 

demonstrate applicability to improving warfighting readiness. Reference MTBs are consistent with 

approved concepts of employment used to provide 

more informed input to the Joint Capabilities Integrated Development System (JCIDS) and the Planning, 

Programming, Budgeting and Execution (PPB&E) 

processes. Offensive Anti-Surface Warfare (OASuW) 

Alternative of Analysis (AOA) used MTB derived 

from Valiant Shield snapshot mission threads to better 

understand fleet Long Range Anti-Ship Cruise Missile 

(ASCM) employment conditions that impact acquisition decisions. 

Excursions from baseline and snapshot MTBs are 

made to use authoritative “as is” architectures to drive 

experimentation. Excursion MTB captures experimentation data for “to be,” or proposed warfighting 

capability to influence research, and technology and 

doctrine maturation. Excursion MTB MLA&E processes are embedded with Fleet Exercise and Experimentation working groups to refine mission level 

performance metrics and requirements for systems/

platforms that are measurable and testable within 

a mission capability framework before entering a 

formal acquisition program. In conjunction with 

Naval Warfare Doctrine Command (NWDC) and 

PACFLT WAR, MLA&E processes are applied to ongoing fleet exercises to gain further insights and to verify 

the methodology with the fleet in order to formalize 

ICF data model application to at-sea experimentation.

The Reference MTB provides the operational 

conditions and desired warfighting capability to the 

Integrated Capability Technical Baseline (ICTB). The 

ICTB compares different architectures by allocating a 
System

Baseline

Mission

Thread

Mission

Capability

Objective

Mission

Effectiveness

(MOEs)

MOPs for all

elements of

Mission

Thread

(ISR, IO, C3,

Weapon)

Systems,

Platforms,

Transport and

Technical

Performance

ICF manages 

the complex

relationship

between mission

capability

objectives 

and system

performance

Sensor

A

Net Ops

Sys B

C3 Ops

C

Weapon

D

ISR

Delay

Net

Recording

Time

C3

Delay

Engagement

Range

• Methodology for developing DoDAF 

 architectures to manage mission data 

 with a “Fleet First’ approach

• Allows for execution of MLA&E defined 

 purpose, objectives, scope, questions and 

 Fleet desired effects

 - Aids analysis of system performance 

 impacts on overall mission execution

• Data relationships improve:

 - Defining unique fleet perspectives 

 for operational analysis

 - Perform warfare area assessment

 - Requirements definition

• Model development is driven by establishing 

 the connections between capabilities and 

 technical development.

• Mission threads developed via an OV-6c

 - Accompanying doctrine associated 

 with each task implicitly composes 

 the OV-6a

MLA&E EASEL FOR ICF/OCD FLEET APPLICATION

MLA&E impact: Integrate Fleet readiness with system deployment and performance

Integration & Interoperability | 73

Table of Contents

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based on Mission Effectiveness

set of systems (technical) that implement the reference 

architecture and execute the architecture to generate metrics that quantify the quality of the specific 

architecture. Sometimes an implementation of an 

MTB will modify workflows at both the operational 

activity sequence and functional activity sequence 

levels. Both snapshot and reference MTBs are useful 

for the development of an ICTB. Leveraging snapshot 

and reference mission architectures for Anti-Surface Warfare from Valiant Shield 2012, an ICTB was 

created to support the Littoral Combat Ship (LCS) 

Surface Module for Fast In-Shore Attack Craft (FIAC) 

mission that tied platform activities to fleet-validated 

mission thread during the Harry S. Truman Strike 

Group’s Sustainment Exercise.

Benefits of MTB expand to both fleet and SYSCOM 

organizations and work towards improving Navy 

acquisitions by producing better material solutions, 

and identifying when non-material solutions are 

factors to current or future capability gaps. Execution of the MTB allowed the MLA&E team to identify a coherent, fleet-endorsed approach to modeling 

operations that starts with capabilities and seeks to 

draw a connection between fleet desired effects and 

system performance. Participation in fleet events, as 

with Valiant Shield 2012, allowed the MLA&E’s mission engineering team to apply the ICF/OCD to an 

at-sea naval exercise and gain endorsement by Fleet 

advocates for the MLA&E process. The coordination 

between SYSCOMs and fleet organizations ensured 

that mission architecture products contain the appropriate scope and balance of activities necessary to 

accurately model end-to-end mission capabilities. 

Figure 4 depicts how system functions and required 

operational capabilities are related through an MTB 

created using a capability based model.

MLA&E is a DASN RDT&E initiative that is complementary to other I&I activities, while using a more 

Figure 4. Capabilities Based Mission Model Transition
Table of Contents

74 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

operational perspective that is more closely tied to 

fleet activities and understanding of warfighter needs. 

The DASN RDT&E’s objective is to institutionalize 

and transition MLA&E processes to various fleet and 

acquisition stakeholders. Using Valiant Shield 2014 

as the test bed, RDT&E is partnering with PACFLT 

WAR and other fleet readiness and experimentation commands, along with acquisition sponsors, 

Program Executive Offices, and SYSCOMs to assess 

MLA&E impact on overall warfighting capability. 

Participating organizations have unique objectives, 

from applying MLA&E processes to better inform and 

improve existing processes for improving warfighting 

readiness through a common data model for analysis, 

assessment, operational, and sponsored requirements 

generation, and system design considerations.

PACFLT WAR embedded the MLA&E engineers 

and architects with Valiant Shield Warfare Assessment Working Groups to assist in the generation of 

hypotheses, objectives, and assessment criteria to 

influence planning and execution of fleet events. The 

objective is to validate that Mission Technical Baseline 

architectures created during fleet assessment events 

will better inform analysis and requirements generation that leads to improved near-term warfighting 

readiness. Additionally, PACFLT WAR has endorsed 

MLA&E support for experimentation during Valiant Shield to determine how best to apply ICF/OCD 

data model to long-term assessment of warfighting 

capability.

The MLA&E team embedded with Valiant Shield 

Experimentation Working Group to leverage ASuW 

snapshot and reference Mission Technical Baselines 

for Long Range SAG Takedown excursion architecture 

development. This effort evolves MLA&E baseline 

“assessment” during Valiant Shield 2012 to “evaluation” through excursion architecture. MLA&E, an 

NWDC sponsored initiative, will serve as the model 

for linking multiple ASuW experiments during Valiant Shield. MLA&E initiative is the umbrella for 

separate Find, Fix, Finish, and Engage experiments 

for an end-to-end kill-chain experiment. NWDC is 

assessing the viability of MLA&E processes and ICF/

OCD data model for improving fleet experimentation.

Leveraging Valiant Shield 2014, a collaborative 

MLA&E and Naval Ordnance Safety and Security 

Activity (NOSSA) Weapon System Explosives Safety 

Review Board (WSESRB) team was established to 

identify safety and fratricide risks in the effective execution of fleet mission threads. This team, in support 
Integration & Interoperability | 75

Table of Contents

Mission Level Assessment and Evaluation:

Driving Fleet Exercises and Experimentation Based on Mission Effectiveness

of DASN RDT&E will document a process that will 

integrate weapons safety engineering into Mission 

Engineering and document gaps in acquisition personnel, policy, and processes. Focus of MLA&E effort 

is on integrating Mission Engineering, Systems Engineering, and Software Systems Safety Engineering into 

the VCNO’s I&I Activity and system of system reviews 

utilizing the standing WSESRB processes. Integration 

effort will discover and document how best to review 

and assess the I&I characteristics of weaponized systems to understand safety risks, identify hazards and 

causal factors, assess mitigations, assess test and validation, and issue I&I Safety Findings or Actions to 

applicable programs under WSESRB review. Focus 

will be on fleet weapon systems, combat systems, 

and the respective network interfaces associated with 

fratricide. The overall objective is to apply MLA&E to 

further enable the institution of Mission-Level Engineering and I&I efforts in support of ASN RD&A’s 

strategic goals of fleet safety.

Commander, U.S. Fleet Forces N7 (Training) 

sponsored MLA&E support of Commander, Strike 

Forces Training Atlantic (CSFTL) assessors to identify improvements in data collection and analysis 

during fleet certification events, specifically, Composite Unit Training Exercises (COMPTUEX). Fleet 

Training and Readiness processes employ a NMETL 

mapped to specific Navy mission areas (capabilities) 

in a continuous improvement process called the Navy 

Warfare Training System. The owners of NMETLs 

use feedback from exercises, operations, and other 

events to improve how they articulate requirements, 

measure performance, certify readiness, and implement improvements. Effort to date has focused on 

FIAC defense during Harry S. Truman and George 

H.W. Bush Strike Groups’ Sustainment Exercises 

(SUSTEX) and COMPTUEX. It is expected that 

MLA&E application of the ICF/OCD data model 

will lead to better defined measures of effectiveness 

that lead to advanced levels of fleet readiness.

A key acquisition stakeholder from programs of 

record leveraging MLA&E’s Warfighter Integration 

is Program Executive Office for Integrated Weapon 

Systems (PEO-IWS). In conjunction with MLA&E, 

PEO-IWS is sponsoring Mission Technical Baseline 

development during Valiant Shield to produce an 

Integrated Capability Technical Baseline for Aegis 

Capability Baselines Surface Warfare mission area. 

Other acquisition stakeholders are Naval Sea Systems 

(NAVSEA) PEO-LCS for Surface and Mine Warfare 

Modules, Naval Air Systems (NAVAIR) Program 

Office for OASuW and Tomahawk Weapons Systems, 

thereby ensuring a broad understanding of operational needs by the acquisition community.

Conclusion

DASN RDT&E resources and MLA&E’s warfighter 

integration efforts to transition technical solutions to 

tactical capabilities create conditions for a philosophical change in evaluation, experimentation, and assessment that enable mission engineers to participate in 

the planning, execution, and analysis of fleet events to 

provide convincing evidence of warfighting capability 

through fleet-validated Mission Technical Baselines. 

These validated operational-based architectures are 

then applied to doctrine, existing programs, and 

evolving technology to improve warfighting readiness. 

Validation and transition of MLA&E processes and 

products with acquisition and fleet stakeholders institutionalizes Integration and Interoperability activities 

with existing operational readiness improvement 

processes. 
Table of Contents

76 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Neil T. Baron

Distinguished Scientist for 

Combat Control

Naval Sea Systems Command

The Science of Integration

“What Lurks in the Darkness of the Interstitial Space?”

By Neil Baron

Naval warfare and the warship have evolved out of necessity, human 

intellect, and technology to equitably compare with many other large 

complex machine developments of humanity, such as vehicles for manned 

space flight and exploratory devices such as subatomic particle accelerators. 

Developments demand the highest levels of integration. Investments are 

large, time scales are long, technical complexity is at an extreme, and 

constancy of purpose is continually challenged. Although visions are 

great, success in such large, complex, and highly integrated technical 

endeavors is rare. It is not enough merely to will it to happen. Success 

demands a holistic systems focus with the right tools and techniques 

to allow the best and brightest minds to see into the darkness of the 

unknown, to envision what could be from what is, to synergize and 

to maintain dedication from the initial vision until the very different 

realization of the machine at delivery.
Integration & Interoperability | 77

Table of Contents

The Science of Integration

“What Lurks in the Darkness of the Interstitial Space?”

Integration

Navies are unique in that they represent a nation’s 

significant capital investment due to the size and complexity of the machines (ships) utilized for combat. 

The architecting and engineering of machines of 

war demand the highest technical scrutiny during 

development to integrate the known, to invent the 

unknown, and to hold the public trust during the 

lengthy acquisition process, thus assuring the warfighter and the taxpayer that the best will be delivered 

in the end. The design of the naval warship and its 

role in the highly distributed nature of modern naval 

surface warfare, like other highly complex modern 

machines of today, requires striking a delicate balance 

among the domains of people, process, hardware, and 

software that when integrated successfully makes the 

holistic system perform. A primary role of the systems 

integrator is to bring such diverse domains together 

successfully, monitor their evolutionary progress, and 

reach an optimized balance between what is usually 

conflicting demands. Such domains are not naturally 

predisposed to interact successfully without a significant amount of engineering due diligence. Hidden 

flaws and faults, or even over reliance, in any one of 

the domains can lead to overall system failure. 

The Navy laboratory structure, being able to execute the inherently governmental technical tasks, 

holds the kernel of necessary talent to technically succeed in such large scale complex systems engineering 

endeavors where flaw and failure lurk around every 

corner. The British researchers, Prencipe, Davies, and 

Hobday (2003) of the University of Sussex, address 

the many facets of successful engineering at the large 

complex system of systems level in their book, The 

Business of Systems Integration:

1

The early phases of design require the 

communication of a vision for how the system should 

operate at the highest level, and then communicating 

the many visions of the lower level components 

among those that must ultimately integrate them 

together. When the visions become artefacts, any 

communication problems or just simple errors made 

in the design become obvious once those artefacts 

are connected to each other. All of the information 

that the designers used, either implicitly or explicitly, 

in the creation of the artefact become elements 

of that artefact. Unlike humans, who might not 

recognize the implications of lack of information or 

wrong information, the artefacts interact with each 

other the moment they are connected and operated 

together. This is why systems integration is the 

ultimate point at which social misunderstandings 

become manifest.

This article explores a technique for the systems 

integrator to better define and manage the highly 

complex nature of system to system dynamics, a critical element of engineering the system of systems 

that comprise modern naval warfare. This will be 

critical if the complexity of the overall machine is 
Ship

Helo

Targets

Control

Engage

1

2

3

Interstitial Space

Network, Communications,

Protocol, Data Model,

Sensor Data, Command Messages,

Control Messages, Weapons Data,

Determinism, Securtity, etc.

Detect

Table of Contents

78 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

to be managed effectively and the desired system 

behaviors are to be realized and available on demand.

Integration: A Surface Warfare Example

Let’s illustrate the concept of integration through 

a simplified Naval Surface Warfare example. The 

radar and optical sensors of a surface combatant 

are challenged to see surface contacts over the horizon resulting in a limited operational picture for 

combat command and control. Utilizing a helicopter 

to extend the range of the surface combatant’s operational picture over the horizon is one option. The 

helicopter, if integrated with the surface combatant, 

could be used to identify, and if necessary, engage 

hostile targets at greater distances from own ship 

thus increasing effective battle space. With surface 

warfare engagement control resident on the ship and 

an airborne sensor feed and weapon on the helicopter, 

integrating the two platforms into a new expanded 

surface warfare capability is conceivable. See Figure 1. 

Integration would require the filling of the interstitial space between the two platforms to achieve the 

desired system of systems behavior and control. The 

two platforms would perform as one integrated system 

of systems to complete the mission. 

Integration: Focusing on the Interstitial 

Space

Using the term “interstitial” helps differentiate 

and focus on the dimension between two objects that 

interact from the more traditional focus on the objects 

themselves. The subtle but key shift in concentrating 

on the interstitial from the physical enables the engineer to focus on the dimension of design where the 

interaction is housed and exercised. Simply put, this 

is where integration happens. The interstitial space, 

from classical use, is loosely defined as a gap between 

spaces full of structure or matter. It is usually thought 

Figure 1. Focusing on the Interstitial Space
Integration & Interoperability | 79

Table of Contents

The Science of Integration

“What Lurks in the Darkness of the Interstitial Space?”

of with physical properties such as an area, volume 

or space between two objects. For example, between 

the wheels and the engine of a car there is a transmission, drive shaft, a differential and an axle. These 

objects fill the interstitial space between the wheels 

and the engine to mechanically enable the function 

of powered mobility. 

More generally, and even more interesting to the 

system of systems integrator, is to envision the interstitial space as simply the domain that enables key 

elements of behavior between two objects that have 

the potential to interact for some higher purpose. 

In this more general definition, the functional and 

logical interactions between geographically disparate objects can have their interstitial relationships 

filled with information-based technologies such as 

computer-based networks (both wired and wireless), 

protocols, data models, command signals, etc. In 

our surface warfare example, it is a data link and 

a set of data protocols with warfighting command 

functionality between a surface combatant and an 

off-board helicopter that realizes a ship controlled 

and helicopter executed surface warfare engagement. 

The systems engineering of complex machines has 

contributed to huge intellectual leaps in understanding the physical interaction of things from the molecular level of atomic particle physics to the celestial level 

of astrophysics. On a more practical level, systems 

engineering has lead the technical advancement of 

machines to ease the human condition. The system of 

systems integrator must equally master the non-physical interfaces between systems and platforms that 

serve to establish functional and logical relationships between devices. Synchronizing, monitoring, 

and managing the behaviors of multiple systems to 

achieve a larger goal are usually the duties of the control system. The control system seeks to establish the 

physical, functional, and logical relationships that are 

realized through the interstitial space between system 

elements so they can behave as an integrated unit. 

System engineers need insights into the functional 

and logical dynamics of the interstitial space just as 

they currently enjoy today with technical insight into 

the physical dynamics. 

Having a Model

Delving deeper into the interstitial space, some key 

models are presented below to describe the context 

around system integration and the interstitial space. 

Physical, Functional, and Logical

There are three dimensions one must fully consider 

when integrating: the physical, the functional, and 

the logical. The physical dimension of integration is 

well understood. For thousands of years, it was the 

only form of integration that took place. With the 

modern digital age of electronics, communications, 

and the embedded computer, we are now witness to 

the significance of the other two dimensions of this 

model. The functional dimension, an intellectual 

construct to describe relationships, is characterized 

through the activity, purpose or task of the system 

or subsystem and how the system works or operates to perform that purpose. Likewise, the logical 

dimension is more traditionally defined as the networking topology that defines the architecture of the 

communications between the subsystems. System 

of systems integrators interested in focusing on the 

interstitial space will spend most of their efforts on the 

functional and logical dimensions of the design. The 

two examples in Table 1 illustrate this. The first row 

is a non-military, non-system example of this model 

applied to the development of a piece of literature. 

The second example is Navy-specific.

PHYSICAL FUNCTIONAL LOGICAL

Book, Chapter, Paragraph Information, Entertainment, Expansion and 

Promotion of Thought, Reference, etc.

Theme, Story, Characters, etc

Ship, Combat System, Missile Mobility, Command & Control, Weapons Control, 

etc.

Communication Links, Protocols, 

Signal Timing, etc.

Table 1. Dimensions of System Integration and the Interstitial Space
Table of Contents

80 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Warfighting Model

Another key model we rely on to describe the 

functional relationships of warfare is the warfighting 

model sequence of Plan, Detect, Control, Engage, and 

Assess. This terminology varies within and across 

the service cultures (Army, Navy, Air Force, Marine 

Corps), but each variation attempts to accomplish 

the same functional result in a similar sequence of 

activities.

A kill chain, or more generically, a mission thread, 

is a specific example of the warfighting model utilizing specific systems to perform the desired functions 

within the context of an operational scenario. A kill 

chain can be very local e.g., a Marine with a machine 

gun engaging an enemy combatant or very global with 

a multi-service system of systems ballistic missile 

defense capability defending a nation. Since each of 

these general functions may be performed by many 

systems and subsystems, with personnel trained and 

optimized to perform against threats, the precision 

of the interplay across the kill chain (through the 

interstitial space) is critical to a successful warfighting 

outcome. The role of the integrator is to understand, 

characterize, and discretely specify the necessary relationships between the systems and people to achieve 

the desired global behavior for the overall kill chain. 

Insight into how well the integrator is achieving the 

goals of system behavior management/control is 

“hiding” in the interstitial space between the systems. 

Next, we will take a look at a tool to help the integrator see into the darkness of the interstitial space 

and bring to light, through measurement, how well 

the integration is maturing during system design, 

development, testing, and delivery.

Integration Readiness Level (IRL)

The establishment and use of “readiness levels” as a 

tool for the systems engineer to characterize the evolutionary maturity of a specific dimension of system 

design has been around for decades. Large-scale projects have found utility with the use of readiness levels 

as a way to simply convey to stakeholders where a 

specific design dimension of interest is in its evolutionary development. NASA is credited in the 1970’s 

with the creation of the Technology Readiness Level 

(TRL) as the first of a series of readiness level indicators. The Department of Defense (DoD) has since 

codified the use and utility of the TRL in acquisition 

policy and instruction across the services. Technology 

readiness, as its name implies, tends to focus on the 

physical instantiation of the system under development. As such, TRL focuses on “the box.” This works 

well for the physical dimension of integration but 

does little to provide insight into the functional and 

logical dimensions of the overall system, especially 

in a system of systems context. Since the integrator 

is equally as interested in what goes on between the 

boxes (the interstitial space) to achieve the local and 

global behaviors of the system, a new readiness level, 

the Integration Readiness Level (IRL), has been postulated to specifically focus on the interstitial space 

and work in concert with the TRL.

IRL Definition

The IRL follows the same readiness model structure as the TRL, which makes it easier for the technical 

and acquisition communities to use and understand. 

An IRL 1-9 level (1-lowest, 9-highest) numerical designation is assigned to a system at a specific point 

in time during its development to characterize the 

state of the integration that is resident in the interstitials of the system (see Figure 2). IRL levels 1-3 are 

the realization of an interface between two systems, 

much like the TRL levels 1-3 are the initial discovery 

and analysis levels for a technology. IRL levels 4-6 

are higher levels of integration and control between 

two systems, much like the TRL levels 4-6 are the 

physical realization of the technology in a laboratory 

environment. Finally, IRL and TRL levels 7-9 are the 

demonstration and validation stages in ever more 

representative operational environments.

IRL Use and Utility

The vast majority of modern system developments rarely get past IRL level 3. For stand-alone or 

self-contained systems, this may be adequate. But in 

an increasingly interconnected world, where “The 

Internet of Everything” is a term bantered about and 
TECHNOLOGY READINESS LEVELS

INTEGR

ATION READINESS LEVELS

TECHNOLOGY — ELEMENT INTEGRATION — INTERFACE

TRL IRL

9 9

8 8

7 7

6 6

5 5

4 4

3 3

2 2

1 1 DiscoverInterface Build DemonstrateDemonstrate Integrate

READINESS LEVELS

Definition Definition

Actual system “flight proven” through

successful mission operations.

Integrated is Mission Proven through

successful mission operations. 

Actual integration completed and

Mission Qualified through test

and demonstration in the system

environment. 

System prototype demonstration in The integration of technologies has

been Verified and Validated with

sufficient details to be actionable. 

System/subsystem model or

prototype demonstration in a

relevant environment. 

The integrating technologies can

Accept, Translate, and Structure

Information for its intended application. 

Component and/or breadboard

validation in relevant environment. 

There is sufficient Control between

technologies necessary to establish, 

manage, and terminate the integration. 

Component and/or breadboard

validation in laboratory environment. 

Analytical and experimental critical

function and/or characteristic proof

of concept. 

There is Compatibility (i.e., common

language) between technologies to

orderly and efficiently integrate and

interact. 

Technology concept and/or application

formulated. 

Basic principles observed and reported.

There is some level of specificity

to characterize the Interaction

(i.e., ability to influence) between

technologies through their interface.

An Interface between technologies

has been identified with sufficient

detail to allow characterization of

the relationship. 

There is sufficient detail in the

Quality and Assurance of the

integration between technologies. 

Actual system completed and

“flight qualified” through test and

demonstration. 

Integration & Interoperability | 81

Table of Contents

The Science of Integration

“What Lurks in the Darkness of the Interstitial Space?”

individual weapons may have their own Internet Protocol (IP) addresses, systems with a TRL 7 and an IRL 

3 will no longer integrate well with other systems. 

This disconnect between the physical, functional, 

and logical maturity of system development has led 

to dissatisfied stakeholders and end users. It is seen 

as a failure of system integration that has manifested 

itself as interoperability problems in the fleet.

The upsurge of the Interoperability and Integration 

(I&I) pandemic we are experiencing in the fleet today 

has largely come about due to increased warfighting 

demands, and from envisioning how very expensive 

and individually competent weapon systems could 

and should operate more easily together and in new 

ways. The aim is to provide a force multiplier (the 

synergy that comes with successful system of systems realization) with little additional effort/cost 

by drawing out latent capabilities from our current 

force structure by integrating existing systems in new 

ways. The frustration lies in not being able to easily 

realize the holistic kill chain across the system of 

systems due to I&I problems. Today’s I&I problems 

are recognizable through the lens of the low IRLs of 

the individual weapon system components we are 

attempting to integrate. Having an IRL designator 

for the interstitial space and actively analyzing and 

managing the IRL during system development is a 

first critical step to improving functional and logical 

integration and designing the mission rather than just 

delivering the boxes. 

Figure 2. Technology and Integration Readiness Levels
Garden pool stinks

Fill the pool with water and

put the fish back in

Scoop the fish

out and drain the

water

The bottom

stinks too

Dig out bottom, cart

the stinking stuff off 

in a wheel barrow

Replant the

water plants

Put fresh gravel

on the bottom

Hard days

work

Two dead

fish

Pool doesn’t

stink

Results In:

Two

Months

Later

Table of Contents

82 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Figure 3. Stinky Garden Pool Analogy
Integration & Interoperability | 83

Table of Contents

The Science of Integration

“What Lurks in the Darkness of the Interstitial Space?”

Complexity

As we presently experience the ubiquitous nature 

of information technology all around us, caution 

is warranted regarding the understanding of and 

the dealing with system complexity. Large complex 

system of systems development does not lend itself 

to oversimplified serial problem solving employed 

by more traditional engineering and program management techniques. The interrelationships among 

components and subsystems are never completely 

known for all conditions, although they can and must 

be characterized for a very specific desired response, 

based on discrete input and controlled conditions. The 

richness of response that comes from interdependency 

and integration leads to complexity and uncertainty. 

A system development program that is preoccupied with immediate goals tends to see bundles 

of many independent mini-systems instead of one 

overarching system. With a desire for simplification 

of understanding, the program decomposes design 

problems (symptoms) and the potential solutions into 

a discrete cause-and-effect modulus. Information 

overload is also a contributor. With a desire to “solve” 

the problem expeditiously, a minimum set of data is 

gathered and analyzed that can logically support a 

proposed plan of action. If the problems are oversimplified in a complex interdependent system and a serial 

symptom-solution/symptom-solution model is in 

place, unforeseen causal relationships can eventually undo the solution. This scheme becomes readily 

apparent in a simple analogy presented in Dietrich 

Dorner’s, The Logic of Failure (1996),2 where oversimplification of complex phenomena can lead to 

unintended consequences (see Figure 3).

Oversimplification tends to result in selecting 

and focusing on one variable as central (the problem), while ignoring the complex interrelationships 

among the multitude of interdependent variables in 

the system. In the above example, the contributing 

factors - pool depth, oxygen content and stratification, water circulation, bacteria type and metabolism 

- all contributed to the undesired symptom, a stinky 

garden pool. A simple, permanent, and much less 

laborious (costly) solution was available in installing 

a small recirculation pump to prevent the anaerobic 

foul smelling bacteria from forming on the bottom of 

the pool. Unfortunately, this option was never considered due to a serial symptom-solution mindset. It is 

obvious why simplification is a desired strategy for a 

quick solution, which appears to be much more efficient, avoids apparent unnecessary data gathering and 

analytical efforts, and streamlines planning. Schedule 

pressures and an overzealous desire to please, temporarily, are also contributing factors. The lead systems 

integrator of a complex system must avoid the strong 

tendency toward oversimplification when schedule 

pressures are great and technical problems are not 

well understood; he must seek out the often hidden 

root cause (stagnant water) and bring to light simple 

and much more effective (and permanent) solutions.

Conclusions

Mastering the discipline of integration requires 

taking a much harder look at the interstitial space, 

that gap between spaces full of structure and matter. 

The highly networked system of systems construct 

for modern naval warfare demands that the system 

integrator bring new light and technical discipline 

into the interstitial space to combat I&I deficiencies 

and better manage the global system behavior. One 

mechanism to gain vision into the darkness of the 

interstitial space is the use of an IRL designator to 

continually monitor the maturation of the system 

design as it relates to the key integration dimensions of 

functional and logical response within a kill chain or 

mission context. This is yet another step toward harnessing the complexity of large-scale systems design 

and shedding light on the potential design deficiencies 

that lurk in the darkness of the interstitial space. 

References

1. Prencipe, Davies, and Hobday; 2003, University of Sussex, “The 

Business of systems Integration” (pg 52).

2. Dorner, Dietrich; 1996, “The Logic of Failure” (pg 71).
Table of Contents

84 | LEADING EDGE January 2015

Capability Evolution Documents:

Managing R&D and S&T Investments to Improve

Warfighting Capabilities

By Anthony Zilic

Acquisition decisions are made in the context of a complex system of 

systems. These decisions are driven by many different factors; threats, 

capability gaps, requirements, legacy architectures, measures of effectiveness and performance, business process changes, and strategic choices.

Developers currently synthesize their plans and milestones through 

an iterated manual process until reaching stakeholder consensus that 

capability, cost, performance and schedule are appropriately balanced. 

Articulating the vision and resulting plan to senior leadership to explain 

how capability matures over time is more art than craft. Imagine if 

your car’s dashboard changed every time you turned the key, or worse, 

every time you looked down to check your speed. Providing a common 

holistic view of how drivers influence technology and how technology 

matures into capability is the challenge to managers, designers, scientists and engineers.

This article proposes the use of “Capabilities Evolution Documents” 

(CEDs) as a “common dashboard.” The CED is a data-driven, high-level 

depiction of a mission framework, enabling capabilities, warfighting 

systems and supporting technologies. By exploring the foundational 

CED elements and the relationships across capability, technology and 

threshold, the reader will better understand how these documents foster 

improved management of the Navy’s research and development and 

science and technology investments. Before exploring the elements of 

the CED, it helps to review a basic approach to Systems Engineering 

illustrated in the process known as the Systems Engineering “V.”
Operational Need

Technical Processes

• Stakeholder

Requirements

Definition

• Requirements

Analysis

• Architecture

Design

Technical Processes

• Transition

• Validation

• Verification

• Integration

• Implementation

• Decision Analysis

• Technical Planning

• Technical Assessment

Technical Management Processes

• Requirement Management

• Risk Management

• Configuration Management

• Technical Data Management

• Interface Management

Enables a Balanced Approach for Delivering Capability to the Warfighter

SYSTEMS ENGINEERING

Delivered Capability

Requirements Validated

Solution

Design Product

Decomposition

IOC/FOC

OT&E

DT&E

Realization

Integration & Interoperability | 85

Table of Contents

Capability Evolution Documents:

Managing R&D and S&T Investments to Improve Warfighting Capabilities

The Systems Engineering “V”

Systems Engineering is an interdisciplinary 

approach and means to enable the realization of successful systems. It focuses on defining customer needs 

and required functionality early in the development 

cycle, documenting requirements, and then proceeding with design synthesis and system validation 

while considering the complete Systems Engineering 

domain, including:

• Cost & Schedule

• Performance

• Manufacturing

• Test

• Training & Support

• Operations

• Disposal 

Systems Engineering integrates all the disciplines 

and specialty groups into a team effort forming a 

structured development process that proceeds from 

concept to production to operation. Systems Engineering considers both the business and the technical 

needs of all stakeholders with the goal of providing a 

quality product that meets the user needs. The International Council on Systems Engineering provides 

resources explaining this process (http://www.incose.

org/practice/whatissystemseng.aspx).

Processes have matured since the Systems Engineering discipline was recognized in the 1950’s. 

Figure 1 depicts the modern process as applied to 

design and acquisition of military systems, commonly 

known as the System Engineering “V.” 

Figure 1. Systems Engineering “V”1
Table of Contents

86 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Inputs and Processes

Processes and inputs that flow down the left side 

of the diagram represent increasingly granular and 

specific statements of the user’s needs. The technical 

processes are iteratively applied until it is possible to 

develop a detailed production design, test plans, and 

employment concepts. This activity is undertaken 

using well defined “Technical Processes” listed on 

the left side of the diagram.

Outputs and Processes

After selecting a design that best meets the requirements, tasks flow up the right side of the diagram 

until capability has been delivered to the warfighter. 

The activity progresses using well defined “Technical Processes” listed on the left side of the diagram. 

Both the top-down requirements flow and bottom-up 

capability development are governed by Technical 

Management Processes which, when properly applied, 

ensure requirements “creep” i.e., uncontrolled changes 

or continuous growth in a project’s scope, is minimized within the context of the system design.

Introduction to the Capability Evolution 

Document Concept

It is easy to imagine the complexity of developing 

the acquisition documentation package. There is 

often very little detail regarding enabling science 

and technology (S&T). A new artifact depicting 

the relationship of S&T to a warfighting need is 

required. This artifact will articulate the total system 

of warfighting need to research, acquisition and 

fielding. The Capabilities Evolution Document (CED) 

is proposed as a solution to this need.

The CED in Figure 2, is a depiction of technology 

flowing through development and integration to meet 

a warfighting need over time. It is a top-down mission 

architecture enriched by the bottom-up, data-driven 

summaries of research, technology, systems, platforms, missions, and drivers. Drivers can be strategy, 

capability gaps, requirements, technical gaps, policy 

- all of which correspond to the upper left side of the 

System Engineering “V.”

As the focus flows through Mission(s) to Capability 

to Systems/Components to Technology Advancements, the impact of today’s research is shown to 

be increasing in the far term. Reversing this flow corresponds to the right side of the System Engineering 

“V.” The relationship of solid research and/or technology development today is aligned to operational 

needs of the near, mid and far term, thus reinforcing 

research investments (Budget Activity 2 and Budget 

Activity 3 S&T projects) as part of the Systems Engineering processes.

Common taxonomy is a key factor of practical 

and usable CED’s. Properly developed, sets of CED’s 

underpinned by shared taxonomy can be analyzed 

across missions. This helps identify cases where technologies can contribute to multiple capabilities. For 

example, improved information technology has application to many DoD systems from weapons development to business office operations. Such analysis 

is possible if CED developers not only use the same 

vocabulary but also the same definitions. In later 

paragraphs, we explore techniques to enforce shared 

taxonomy across communities.

Figure 2 is an idealized CED model. In practice, 

the model is adaptable to incorporate only the layers 

needed to help “tell the story” of how user’s needs 

guide technology advancements and system development. Other themes available in the CED are the identification of missing projects or programs from the 

critical path to capability development and how newly 

available technologies can inform system design.

“New capabilities desired by national leadership 

may involve modifications to kill chains, Command 

and Control (C2) constructs, improved coordination, 

and performance. These capabilities must be realized 

through modifications to programs of record and 

integration across elements of the system that have 

their own independent programmatic momentum. 

A challenge of Systems of Systems Engineering is to 

objectively evaluate competing solutions and assess 

the technical viability of trade off options.” 2 Figure 3 

depicts how CED’s both are informed by the system 

engineering process and inform leadership of key 

relationships between capabilities and end items.
CAPABILITY PLAN COMPOSITION

Articulating Operational Needs and Aligning Technology

Developments Over the Near, Mid and Far Term

• Mission/Threat Analysis

• “State of the Market”

• Mission Capability Needs

• KPP’s

• DOTMLPF

• KPP’s to System

Requirements

• Business Opportunities for

cost and risk reduction by

cross Program collaboration

• Interface Control

• Modularity, Opponents, and

Commonality

• Rapid Certification

• Venture Capital Investing

• Make a Business Better

• Serve a Priority or Die

• Budget

• PR and POM

Uncertainty Becomes

Reduced Over Time

Low

Moderate

High

None

Low

Moderate

None

None

Low

None

None

None

Strategy

PAST NOW MID-TERM FAR-TERM

$ $ $ $

• W.F. Concept

• Capabilities

• KPP

Force Groups

• Carrier

• Expeditionary

• TBMO

• Etc.

Net Centric 

Sys Platforms

• Ship/Sub

• Air

• Land

System Install/

Deploy Plan

System Dev

Specs and Stds

R&D/S&T

Resourcing

DOTMLPH

Uncertainly

to 2008

to 2013

to 2020

2006-2008 2008-2013 2013 plus 2020

Force Focus

Capability Focus

System Design, Build, Test Focus

System Advancements

Policy Docs Policy Docs Policy Docs Policy Docs

DOTMLPF DOTMLPF DOTMLPF DOTMLPF

Kill Chains

OSM

Mission

Engineering Models Data

Capability

Evolution

Document

MISSION CAPABILITY 

Assessments

MISSION CAPABILITY 

Package

Integrated Capability 

Framework

Measure Progress

Verify

Dynamically 

Visualize

Inform

Align

Detail Improve

Generate

Generates

Inform

Improves

Utilize Mission-Based Scenarios

Cloud Chart

Integration & Interoperability | 87

Table of Contents

Capability Evolution Documents:

Managing R&D and S&T Investments to Improve Warfighting Capabilities

Figure 2. Conceptual CED Model

Figure 3. How Capability Evolution Documents Fit in System Engineering Process
Table of Contents

88 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Automation Requirements

CED generation is a multi-disciplinary participatory activity. Critical discussion among program 

managers, system engineers, and both technical and 

operational subject matter experts contributes to the 

CED. Figure 4 depicts the results of a work session 

where the interrelationships (strings) of specific developments (note cards) are worked out and understood 

by a set of participants. While effective, employment 

of this approach is not an efficient use of time in the 

long-term due to the maintenance demands of manually placing note cards and string on a war room 

wall. Also, the product is not easily reproduced or 

copied to other media types.

At a minimum, the tool must be able to track relationships and changes, and graphically display an 

entire plan for achieving an Operational Capability 

such that the relationship of maturity of capability 

and time are clearly shown. All database variables 

listed below must be included.

Functionally, such a tool needs to:

• Track capabilities, programs, and activities

• Manage linkage across programs and 

fielding process

• Provide multiple views (e.g., roll-up, 

timeline, functional, relational, etc.)

• Provide a flexible and extensible database

• Provide menu driven data entry capability 

and connection to the Integrated Digital 

Environment

• Track risk and dependencies

• Handle varying levels of classification 

• Provide network based access

Summary

The primary goal of the CED is to develop processes and tools to manage, track, and visualize the 

dependencies across time from desired warfighting 

capability, through system development, down to 

foundational S&T while considering the links between 

the following: 

• Strategy

• Mission

• Systems of Systems to perform missions

• System components

• Technological advancements and how they 

impact system performance

• Warfare concepts and capabilities

• Key performance parameters

• Force Structure such as Carrier Strike 

Groups, Expeditionary Warfare Groups and/

or Tactical Ballistic Missile Defenses Groups

• Network enabled systems and platforms and 

how the they relate

• Systems installation and deployment plans

• Full range of RDT&E (Research, 

Development, Test, and Evaluation)

• Doctrine, Organization, Training, Materiel, 

Leadership and Education, Personnel and 

Facilities (DOTMLPF) modifications and/or 

alternatives to achieving capability 

The taxonomies, relationships, and tracking 

analysis shown in Figure 5 must be included.

Achieving this goal will ensure the application of 

System of Systems and Mission Engineering methodologies results in a better alignment of future RDT&E 

with warfighting needs. 

Figure 4. Manually Constructed CED
TIME

S&T

Component

Element

Combat System

Network

Platform

Battlegroup

Operational Capabilities

Future Naval Capabilities Gaps

OPNAV Requirements

Threat

Mission Area

Integration & Interoperability | 89

Table of Contents

Capability Evolution Documents:

Managing R&D and S&T Investments to Improve Warfighting Capabilities

References

1. Defense Acquisition Guidebook, Chapter 4 -- Systems Engineering, 

Defense Acquisition University

2. Robert K. Garrett Jr., Steve Anderson, Neil T. Baron, and James D. 

Moreland Jr., Managing the Interstitials, a System of Systems Framework 

Suited for the Ballistic Missile Defense System, Journal of Systems 

Engineering, 2011

Figure 5. Taxonomies, Relationships, and Tracking Analysis Needed for CED
Table of Contents

90 | LEADING EDGE January 2015

Integration and Interoperability

Fratricide Safety

By Peggy L. Rogers and Michael G. Zemore
Integration & Interoperability | 91

Table of Contents

Integration and Interoperability Fratricide Safety

When most people hear the word safety, they often consider 

a variety of conditions that may affect public and working class 

sectors. For example, safety to some would mean product safety 

(e.g., using products obtained as consumers), occupational 

safety and health (e.g., workplace hazards and health factors), 

personal safety (e.g., avoiding accident through precautionary actions), or safety due to environmental conditions (e.g., 

extreme weather). Although these are all valid considerations 

for the term safety, this article will address safety as an engineering discipline falling within the systems engineering arena 

as applied to military systems. The definition of System Safety 

Engineering (SSE) is “an engineering discipline that employs 

specialized knowledge and skills in applying scientific and engineering principles, criteria, and techniques to identify hazards 

and then to eliminate the hazards or reduce the associated risks 

when the hazards cannot be eliminated.” 1 In other words, an SSE 

expert actively studies requirements and design proposals to 

recommend alternatives that will lower the risk of accidents for 

the life of the system. That would include accidents associated 

with designing, producing, handling, installing, testing, maintaining, operating or disposing of the system. Applying that 

engineering philosophy to military systems ensures our highly 

sophisticated combat systems, information infrastructures, 

and weapons remain safe pre, during, and post mission. This 

article will discuss the history of SSE and its transformation 

from single system or component focus to the collection of 

systems as a complex yet highly functional System of Systems 

(SoS). It will then consider the challenges of applying SSE to an 

even wider view of the mission engineering and warfighting 

environments deemed Integration and Interoperability (I&I). 

The Naval Ordnance Safety and Security Activity (NOSSA) is 

leading the charge in defining system safety engineering’s application to I&I, terming the resulting application I&I Fratricide 

Safety. NOSSA has obtained the engineering support of the 

Naval Surface Warfare Center, Dahlgren Division (NSWCDD), 

in this quest to define and apply I&I fratricide safety analysis 

techniques. There is a recognized critical need for an expansion in the application of SSE to widen the view of analytical 

assessment for systems in the real-world environments in 

which our military operate. The goal is to lower the risk to the 

warfighter and preserve valuable assets for the Department of 

Defense (DoD).
Table of Contents

92 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Background/History

The formal application of SSE principles using 

military standards began in the 1960’s, charged by 

an alarming number of accidents. One such accident 

occurred on the carrier USS Forrestal (CVA 59), a 

defining example of the need for SSE and its application to weapon systems. On 29 July, 1967, USS Forrestal had several aircraft on deck preparing for flight 

during combat operations. Aircraft, once taxied to the 

runway, underwent manual procedures to electrically 

connect wing mounted rockets for pending missions. 

The procedures were performed on the runway as 

a safeguard to ensure any potential mishap with a 

rocket ignition would fire down the runway and off 

the bow of the ship without hitting critical ship systems or other aircraft on the flight deck. There were 

two independent steps for electrical connectivity—one 

to connect the electrical umbilical to the rocket and 

a second to remove the arming pin. Once completed, 

the pilot had control of the rockets using the fire 

button in the cockpit. In the case of USS Forrestal, 

the electrical umbilical was being connected while 

the aircraft was parked and before it was positioned 

for takeoff. This decreased the preparation time for 

takeoff once the plane was on the runway. It seemed 

acceptable at the time since any rocket activation still 

required the arming pin to be removed (safety barrier) 

and a firing signal. Unfortunately, on this particular 

day during pre-flight preparations, a parked aircraft 

with the electrical umbilical connected to a Zuni 

rocket fired a rocket unintentionally as the aircraft 

transitioned from external to internal power. Since 

the aircraft was parked at the time, the rocket pointed 

towards other aircraft on the deck. Once the rocket 

fired and crossed the deck, it exploded into parked 

aircraft causing tremendous fire and explosion. As 

depicted in Figure 1, the crew of USS Forrestal worked 

to extinguish the raging fire; 134 Sailors were lost 

that day.

Figure 1. Crew Members Fighting Fires Aboard USS Forrestal, 29 July 1967
Integration & Interoperability | 93

Table of Contents

Integration and Interoperability Fratricide Safety

The obvious question was, how did the rocket fire 

given the arming pin was inserted and the firing 

button was not engaged by the pilot? In the investigation that followed, it was determined the single act of 

connecting the umbilical to the rocket while parked 

did not cause the accident. As a matter of fact, no 

single action caused the accident. With the umbilical 

connected, the arming pin maintained electrical isolation of the rocket from the fire switch in the cockpit. 

The arming pin had a long tail attached to it for ease 

in identification and removal on the flight line. Unfortunately, it was determined the arming pin tail could 

catch wind and become dislodged from the socket 

without human intervention. On that day in July 1967, 

it was windy on the flight deck and that arming pin 

dislodged inadvertently. With the umbilical connected 

and the arming pin dislodged, only the fire button in 

the cockpit prevented inadvertent firing. There was 

no pilot error that caused the rocket to release. The 

rocket actually received an electrical signal to release 

as a result of final aircraft power changeover. Once 

the pilot appropriately transitioned the plane from 

external to internal power, a power spike infiltrated 

the circuit sending the inadvertent signal to fire the 

rocket. The resulting mishap was infamous, the loss 

of life devastating, and the need for system safety 

engineering evident.

System Safety Engineering

After the incident aboard USS Forrestal and other 

shipboard safety events, safety practitioners were 

embedded within design teams to bring a unique 

perspective to the systems engineering process, providing alternative views on the consequences of design 

decisions that could prove to be hazardous to the warfighter. These practitioners implemented system safety 

analysis techniques, many of which are used for individual systems today. The safety practitioner would 

concentrate on the weapon or energetic components as 

the designs were typically single function intent with 

stand-alone modes of operation. Safety techniques 

such as fault tree analysis, event tree analysis, sneak 

circuit analysis, failure modes and effects analysis, 

and common cause failure analysis are examples of 

safety analysis techniques implemented.

If one of these techniques, such as fault tree analysis, had been applied to USS Forrestal and the Zuni 

rocket-firing event prior to the accident, it would have 

identified the conditions required for rocket ignition. 

This would have allowed engineers to assign probability numbers to each condition to characterize and 

understand the potential for mishap. This, in turn, 

would have allowed a focused effort on mitigations 

to prevent probabilistic conditions that could lead 

to mishap. In other words, this analysis technique 

would likely have discovered the potential for arming 

pin dislodgement and power surge, increasing the 

potential for inadvertent rocket ignition. The analysis 

results would have motivated the implementation of 

mitigating measures to avoid the accident. The knowledge gained from the USS Forrestal accident helped 

institutionalize SSE and its application to individual 

systems. That concentrated SSE effort and the associated requirements for safety engineering evolved as 

numerous analysis types and techniques were developed, culminating in safer weapon systems and a 

series of updates to MIL-STD-882, DoD Standard 

Practice for System Safety.

Combat System Safety Engineering

Within the last 20 years, the portability of technology and the introduction of software, firmware, 

and programmable logic for major weapon system 

functionality have significantly influenced shipboard designs. Modern systems are now developed as 

multi-functional and multi-mission Combat Systems 

(CSs), as an arrangement of interdependent systems 

that are associated or connected to provide warfighting capability. It was the complex multi-functional 

combat system approach that posed new questions 

concerning safety. Given each individual system was 

analyzed for safety, did that truly characterize the 

risks of all systems interacting as a collective combat 

system? The answer was no, and the system safety 

engineering discipline was challenged to define and 

execute safety programs that encompassed the SoS 
COMBAT SYSTEM

********** *********** *** ****** ********* ********** *** ******

********** *********** *******

********** *********** *******

* * * * * * * * *

* * * * * * * * *

********** *********** *** ****** ********* ********** *** ******

COMBAT SYSTEM

********** *********** *** ****** ********* ********** *** ******

********** *********** *******

********** *********** *******

* * * * * * * * *

* * * * * * * * *

********** *********** *** ****** ********* ********** *** ******

Table of Contents

94 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

philosophy for Navy acquisition. The system safety 

community stepped up to the challenge by developing 

analytical methods and safety risk assessment techniques for collective combat systems, with NSWCDD 

on the leading edge. The primary objective was, and 

continues to be, identifying unique hazards that exist 

within the CS context and mitigating those hazards 

to prevent death, injury, and damage. The difficulty 

in executing CS safety programs is that systems that 

make up the SoS often have individual and independent Program Managers, Principals for Safety, safety 

analyses, engineering processes, schedule drivers, 

and program urgencies. Thus, understanding and 

characterizing all systems collectively as a CS and performing the CS safety analyses within a constructive 

and collaborative engineering environment is a challenge. Meeting this challenge requires employment of 

a safety engineering approach that was developed to 

include special methods, techniques, and processes 

while leveraging support from Navy leadership and 

individual programs.

The combat system level safety analysis techniques 

extend beyond individual system level analyses and 

focus heavily on shipboard operations, operational 

training, interfacing systems, contextual data threads, 

situational awareness, and complex casualty configurations. The scope of the analysis covers all systems 

involved in situational awareness, command, and 

control of the weapons on the platform. The SSE analysis techniques applied to these scalable, dynamic, 

SoS configurations are not necessarily different in 

name from single system safety efforts, but are very 

different in context and execution. The safety engineer 

must understand the workings of any given system 

and its contribution to the collective combat system 

for consideration of hazards and mishap scenarios. 

Therefore, the safety engineer for combat systems 

must be a trained expert with contextual intellect, 

chartered to understand and identify hazards associated with the collection of systems. Figure 2 represents 

the SoS context, a complex combat system aboard an 

Aegis destroyer, delivering Anti-Submarine Warfare, 

Figure 2. USS Milius (DDG 69) with Aegis Combat System Illustration
COMBAT SYSTEM

********** *********** *** ****** ********* ********** *** ******

* * * * * * * * *

* * * * * * * * *

********** *********** *** ****** ********* ********** *** ******

COMBAT SYSTEM

********** *********** *** ****** ********* ********** *** ******

********** *********** *******

********** *********** *******

* * * * * * * * *

* * * * * * * * *

********** *********** *** ****** ********* ********** *** ******

COMBAT SYSTEM

********** *********** *** ****** ********* ********** *** ******

********** *********** *******

********** *********** *******

* * * * * * * * *

* * * * * * * * *

********** *********** *** ****** ********* ********** *** ******

Integration & Interoperability | 95

Table of Contents

Integration and Interoperability Fratricide Safety

Anti-Air Warfare, Surface Warfare, Strike, and Ballistic Missile Defense capabilities. In the figure, the 

combat system diagram illustrates the number of 

individual systems installed on the ship that interrelate to make up the combat system configuration.

Battle Group System Safety Engineering

Performing safety analyses and mitigating risks for 

a collective combat system has been an evolutionary 

step for the safe training and use of shipboard weapons. However, the fleet does not operate a collection 

of ships, often called a battle group, as isolated platforms steaming in close proximity with a hope of 

defending against the adversary. On the contrary, 

our fleet is comprised of numerous ships mixed and 

matched in combination and configuration to best 

maximize the capabilities for a given circumstance. 

For a battle group, situational awareness, track databases, and even weapon system engagements can be 

shared across multiple platforms to manage all mission objectives. In this case, the safety engineer would 

not focus on hard-wire interfaces between systems 

within a combat system configuration, but the distributive functions of situational awareness, command, 

and control across the battle group as illustrated in 

Figure 3. Much like CS safety, this introduces new 

challenges for the system safety engineering discipline 

in defining and executing safety analyses. Studies 

must focus on identifying and mitigating particular mishap risk scenarios that span across multiple 

platforms. In this case, the safety practitioner must 

be an expert in battle group configurations, operations, communications, and tactics. This safety expert 

must utilize safety artifacts and engineering analyses 

performed by the combat system safety teams as the 

functional and foundational basis for assessment, 

while combining that with unique skill and knowledge to understand how multiple configurations, 

communications and combat system operations could 

create hazards beyond what an individual ship would 

experience.

Figure 3. Four U.S. Ships with Unique Combat Systems Within a Battle Group Interacting
System

Baseline

Mission

Effectiveness

Mission

Threads

System

Baselines

Fratricide Avoidance 

in the Operating Environment

I&I FRATRICIDE SAFETY The application of System Safety Engineering methods, analyses, and criteria 

to lower the risk of operational fratricide.

Table of Contents

96 | LEADING EDGE January 2015

DEVELOPMENTAL INTEGRATION & INTEROPERABILITY

Integration and Interoperability Fratricide

As the evolution of naval warfighting capability 

continues, we are now seeing an increased focus on 

I&I. This includes the philosophical approach to engineering as it relates to SoS and battle groups, but 

extends beyond the battle group to adopt a focused 

approach to mission accomplishment and mission 

success. Utilizing this approach, every system engaged 

within the construct of the operational Navy can be 

evaluated for function, availability, operation, reliability, communication, command, and effectiveness towards the success of a mission. This approach 

ensures a continued focus on mission success, as 

opposed to early and individualized system definition that may or may not support mission success 

when deployed within the variety of combat system 

configurations. Clearly, the focus on I&I and mission 

success must include system safety. For any engineering effort associated with energy and the potential for 

injury and damage, safety and the system life cycle 

must be considered during the engineering phases. 

In this case, the safety engineering scope is centered 

on fratricide, described as the unintentional impact 

of dangerous energy to a friendly force or object.

One focus of I&I activity is to fully characterize 

mission effects by viewing individual systems and 

system performance within a defined kill chain. The 

focus ensures end-to-end execution for any and all 

of the kill chains defined. To support this activity, a 

structured approach has been developed to define 

mission models matched to supporting systems, platforms, and baselines. The alignment provides the 

kill chain assessment matched to warfare capability, 

mission threads, functional requirements, and product baselines. Although the methods, techniques, 

and instruments for I&I fratricide safety are not 

yet defined, the framework presented with the I&I 

activities provides an opportunity for system safety 

Figure 4. I&I Fratricide Safety within the Mission Engineering Construct
Integration & Interoperability | 97

Table of Contents

Integration and Interoperability Fratricide Safety

to engage in the mission engineering approach to 

add functional requirements to support I&I fratricide avoidance. Figure 4 provides an illustration of 

operational views-flowed to effects/kill chains and 

the technical baselines with an added objective of 

fratricide avoidance.

As the Navy moves forward with I&I and the inclusion of safety in the I&I fratricide avoidance initiative, 

the intent is for NOSSA, in collaboration with Deputy 

Assistant Secretary of the Navy (Research, Development, Test, and Evaluation (DASN (RDT&E)), to 

focus on integrating Mission Engineering, Systems 

Engineering, and Software Systems Safety Engineering into the I&I activity and SoS reviews utilizing the 

standing Weapon System Explosives Safety Review 

Board (WSESRB) and its current processes. The team 

will discover and document how best to review and 

assess the I&I characteristics of weaponized systems 

to understand safety risks, identify hazards and causal 

factors, assess mitigations, assess test and validations, 

and issue I&I Safety Findings or Actions to applicable programs under WSESRB review. Focus will be 

on fleet weapon systems, combat systems, and the 

respective network interfaces associated with fratricide. The team will characterize the materials and 

tools necessary to perform these tasks while providing 

recommended taxonomy, processes, tools, and objective quality evidence requirements for incorporation 

into Navy policies and guidance, Systems Engineering 

Technical Review (SETR) criteria, and Probability of 

Program Success (PoPS)/Gate Reviews. The overall 

objective is to apply Mission-Level Systems Engineering to further enable the institution of Mission-Level 

Engineering and I&I efforts in support of the Assistant 

Secretary of the Navy (Research, Development, and 

Acquisition) (ASN (RDA)) strategic goals surrounding 

safety within the fleet.

Conclusion

The I&I fratricide safety engineering initiative will 

provide valuable insight into the identification of 

safety risks that have not been characterized in the 

past. Knowledge is power, and the knowledge that is 

gained from this engineering initiative will provide 

the DoD the power to save lives, as well as provide 

for a more effective contributor to the mission of the 

warfighter. 

References

1. Military Standard 882E, Department of Defense Standard Practice 

System Safety, 11 May 2012.
Table of Contents

98 | LEADING EDGE January 2015

Introduction

The Navy must identify, correct, and work to prevent gaps between 

platforms, sensors, systems, and weapons that form kill chains. Integration 

and Interoperability (I&I) improves warfighting capability by assessing 

gaps between systems and platforms, and by providing gap closure 

recommendations across the range of Doctrine, Organization, Training, 

Materiel, Leadership and Education, Personnel and Facilities solutions. 

I&I activity also identifies barriers – breaks in the kill chain – and aligns 

responsible stakeholders across the naval enterprise to effectively resolve 

those barriers and ensure Fleet readiness. The ability to assess and obtain 

feedback from the warfighter through a tight technical-to-tactical linkage 

with scientists and engineers is foundational for this ongoing effort.

United States Fleet Forces Command contributes to the prioritization for 

the I&I assessment agenda to ensure the focus remains on the warfighter 

and commanders. The resulting I&I assessment agenda focuses on capability gaps in operational and contingency plans. This integrated approach 

connects input from Fleet operators to the technical community. It 

supports our responsibilities as the chief advocate for warfighter readiness 

production in the Fleet. I&I ensures that our policies, resources, and 

products are aligned and prioritized to deliver ready forces to the naval 

component and combatant commanders.

I encourage you to read and learn more about I&I in this edition of 

Leading Edge. Investing time and effort in I&I pays great dividends; it 

will contribute to a whole and ready Fleet that is capable of meeting the 

operational challenges of the future. 

Admiral Bill Gortney

Commander

United States Fleet Forces Command
Strengthening the Technical-to-Tactical Link

to Ensure Fleet Readiness

OPERATIONAL

INTEGRATION & INTEROPERABILITY

Table of Contents

Integration & Interoperability | 99
Table of Contents

100 | LEADING EDGE January 2015

Warfare Capability Baseline:

Assessing Operational Gaps

By David A. Clawson, Sara E. Wallace, Gregory D. Little, and Keith Wheeler

Mission engineering, including the desire to predict 

and assess the warfighting capability of the Navy, is 

nothing new. Analysts in the private and public sectors 

have supported warfighting capability assessment 

for years. Most studies have relied on subject-matter experts and the use of modeling and simulation 

(M&S) to assess system performance. While this 

approach has proven to be invaluable to the acquisition and engineering communities, experience has 

shown it does not necessarily capture integration and 

interoperability (I&I) issues that could impact the 

performance of complex systems. Manually assessing 

performance using technical expertise alone is not 

feasible for most cases and, although M&S allows 

analysts to model very complex system behavior, 

models may be wrong, may not accurately represent 

interfaces or important (possibly unknown) interactions between systems, and may not accurately reflect 

how the systems are used once deployed. Clearly, the 

identification of gaps in our warfighting capabilities 

requires that these and other issues to be considered 

in the system assessment process. This article provides an overview of the Warfare Capability Baseline 

(WCB) assessment process, which identifies current 

capability gaps using an approach primarily based 

on operational test data.
WCB Team

Lead (COMOPTEVFOR)

Deputy (COMOPTEVFOR)

Surface Group

Mission Engineer 

(NSWCDD)

Mission Integrator

(COMOPTEVFOR)

Mine Group

Mission Engineer 

(NSWCPCD)

Mission Integrator

(COMOPTEVFOR)

Undersea Group

Mission Engineer 

(NUWC)

Mission Integrator

(COMOPTEVFOR)

Air Group

Mission Engineer 

(NAWS)

Mission Integrator

(COMOPTEVFOR)

C3 Group

Mission Engineer 

(SPAWAR)

Mission Integrator

(COMOPTEVFOR)

Integration & Interoperability | 101

Table of Contents

Warfare Capability Baseline: Assessing Operational Gaps

Background

Investigation into a possible fleet issue necessitated 

the assessment of current fleet capabilities grounded 

with data collected from fleet exercises and system 

operational and developmental tests. WCB began as a 

pilot study led by Commander, Operational Test and 

Evaluation Force (COMOPTEVFOR). Its mission is to 

provide senior Navy decision makers with rigorous, 

fact-based reports of current baseline systems-of-systems warfighting capabilities “that are not available 

anywhere else in the Navy.”1

Since the pilot study, the WCB assessment process 

has been executed three times (referred to as “increments”). Each successive increment has expanded 

the scope of the assessment to include more threats 

and more warfare areas. As of this writing, approximately 150 assessments have been performed across 

six warfare areas: Ballistic Missile Defense (BMD), Air 

Warfare (AW), Surface Warfare (SUW), Anti-Submarine Warfare (ASW), Mine Warfare (MIW) and 

Electronic Warfare (EW).

The WCB assessment process is derived from the 

Naval Air Systems Command (NAVAIR)’s Horizontal Integration and Capability Assessment Process 

(HICAP). Some modification of the HICAP scoring 

criteria was necessary to address the expanded scope 

of the WCB assessment. Specifically, Threat Evaluation 

and Weapon Assignment is a critical, complex activity 

not explicitly identified in the HICAP functions.

Assessment Team

The WCB team comprises a diverse set of naval 

agencies including COMOPTEVFOR; NAVAIR; 

Naval Surface Warfare Center, Dahlgren Division 

(NSWCDD); Naval Surface Warfare Center, Panama 

City Division (NSWCPCD); Naval Undersea Warfare 

Center (NUWC); Naval Air Weapons Center (NAWS) 

China Lake; and the Space and Naval Warfare Command (SPAWAR). An organizational chart depicting 

the team hierarchy of the WCB assessment team is 

displayed in Figure 1.

COMOPTEVFOR leads the overall WCB assessment. The assessment team is divided into five groups: 

surface; mine; undersea; air; and command, control, 

and communication (C3), each led by a different warfare center.

The lead at each warfare center is referred to as a 

“Mission Engineer” (ME) and is responsible for working with MEs from other warfare centers to develop 

and review scoring criteria, provide the rationale 

behind the assessment, and provide support for the 

Figure 1. Warfare Capability Baseline Organization
Table of Contents

102 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

integrated assessment of all kill chains, including the 

creation of the final report. Each ME also forms a 

working group of “Principal Investigators” (PIs), who 

are responsible for scoring systems in their particular 

area of expertise. (e.g., the surface sensor PI will score 

the performance of surface-based sensors such as 

the SPY-1B). PIs also work with warfare center subject-matter experts and keep program offices informed 

as the assessment proceeds.

Each group is assigned a “Mission Integrator” 

(MI), who is an active military COMOPTEVFOR 

representative. MIs serve as liaisons between WCB 

technical personnel and the warfighter community. 

MIs’ roles include working with the fleet to develop 

tactical situations (TACSITs) used in the assessment, 

working with warfare centers to ensure current Navy 

doctrine and tactics are represented, and supporting 

the integrated assessment of kill chains.

Several other agencies played a role in the WCB. 

These agencies include Program Executive Offices 

that provided access to platform, sensor, weapon, 

and network test data; and Commander U.S. Fleet 

Forces Command (CFFC), who provided prioritized 

weapon target pairs for assessment as well as the 

mission context and criteria for each kill chain. The 

Warfare Centers of Excellence also provided inputs 

into developing TACSITs and provided feedback on 

WCB findings.

Assessment Process

The WCB assessment process consists of several 

steps:

• Defining and prioritizing weapon-target 

pairs of interest 

• Developing kill chains and TACSITs 

• Developing scoring criteria 

• Scoring kill chains

• Performing an integrated kill-chain 

assessment 

• Reporting assessment findings

A brief overview of each step is provided.

Defining and Prioritizing Weapon-Target Pairs

The WCB assessment process begins with a 

fleet-prioritized list of weapon-target pairs (WTPs) 

that identify a specific weapon for use against a specific target. CFFC requests each fleet to provide a prioritized list of WTPs that are of interest to them. The 

CFFC and COMOPTEVFOR use these submissions 

to determine the WTPs to examine in the next WCB 

increment. For each WTP, the WCB team attempts 

to answer the question: “Can this weapon be effectively employed to achieve the desired effect against 

this target under the circumstances described in 

this TACSIT?”

Developing Kill Chains and TACSITs

Scored kill chains are the basic tool of WCB assessments, are used to answer the above question, and 

are the foundation for WCB products. A kill chain 

consists of mission tasks or functions required to 

successfully employ a specific weapon against a specific threat and the platforms that could provide the 

required functionality (e.g., target detection could be 

done by an aircraft or a surface ship). In addition, the 

kill chain includes the major decision nodes (e.g., the 

decision to commit an aircraft to visually identify a 

tracked object) as well as the communication links 

required to transmit information between and within 

units. The specific mission tasks and C3 nodes represented in a kill chain are dependent on the mission 

area being examined and are subject to change from 

one increment to the next. In increment 3, the AW 

kill chains consisted of eight mission tasks plus three 

subtasks and six C3 nodes, as illustrated in Figure 2.

Given the number of combinations of mission 

tasks and platforms and/or performers that could 

carry out those tasks, there could be many “paths” 

through a kill chain. Many of these paths will have 

“broken” links, some will have “weak” but not broken 

links, and (hopefully) some will be comprised of 

all “strong” links. Since assessing all possible paths 

through a kill chain is not generally possible, WCB 

focuses on assessing the primary path, which represents the Navy’s preferred path, based on current 

doctrine, training, etc., and possibly one alternative 
AW-1

Prepare

Configure 

AW-2

Search

AW-3

Detect

AW-4

Track

AW-5

ID

AW-8

Defend

AW-6

Engage

AW-7

Assess

AW-6a

Launch

AW-6b

Control

AW-6c

Weapon

C3-1

Initialize

Plan 

C3-2

Process

Display 

C3-3

Decide:

Commit 

C3-4

Declare:

Hostile 

C3-5

Decide:

Engage 

C3-6

Decide:

Assess 

Platform 1

Decision

Maker 1 on

Platform X 

Decision

Maker 2 on

Platform Y 

Platform 2 

AW tasks & scores are above this line.

C3 tasks and scores are below this line.

Primary kill-chain path 

Alternative

kill-chain path

C3 task name & ID number 

C3 scoring node for

task C3-5 

C3 data link scoring line 

AW task name

& ID number 

AW scoring node for

task AW-6b 

Integration & Interoperability | 103

Table of Contents

Warfare Capability Baseline: Assessing Operational Gaps

path that may circumvent weak or broken links in the 

preferred path. In Figure 2, the primary path is indicated by the solid black line connecting the scoring 

nodes for Platform 1, and an alternative sensor path 

is indicated by the dashed black lines connecting the 

scoring nodes for Platform 2.

Kill chains are inherently tied to TACSITs, which 

are vetted through an annual TACSIT development 

effort with participation from fleet warfighters. These 

TACSITs set the baseline year and provide warfighting 

scenarios for kill chain assessments. The TACSIT contains several important factors that must be considered 

in the assessment, including the threat, the participating platforms, the operational environments, and 

expected methods of employment showing associated 

geometries and timelines throughout mission execution. In short, TACSITs provide the context within 

which kill chains will be scored.

Developing Scoring Criteria

Each kill chain function must be scored in the 

context of and constraints defined by its TACSIT. 

Scoring criteria define what the systems must do. In 

general, Green, Yellow, Red or White scores, as shown 

in Table 1, are applied for each scored platform as 

follows:

• Green: The platform provides the full level 

of performance required by the TACSIT.

• Yellow: The platform provides a partial 

or degraded level of performance in the 

constraints defined by the TACSIT. For 

instance, a Yellow score would be applied 

to performance that occurred between a 

desired and a minimum threshold, if both 

were defined. In some circumstances, the 

desired and minimum threshold are the 

same, and a Yellow score does not exist, 

i.e., performance is either Green or Red.

• Red: The platform fails to provide the 

minimum level of performance required 

by the TACSIT.

• White: Test data does not exist or is 

insufficient to accurately score system 

performance. The PI responsible for scoring 

a given system or function makes the 

Figure 2. Notional Unscored Kill Chain
Critical 

Measures

Notional Scoring 

Criteria

Notional Score 

and Rationale Overall Task Score

Launch_1 Is the correct weapon 

selected?

GREEN: No issues 

observed in testing

YELLOW

Launch_2 Is all prelaunch data 

provided to the weapon?

YELLOW: Hardware 

interface issues prevented 

launch in x of y attempts

Launch_3 Is the launcher able to fire 

all requested rounds for 

the engagement?

GREEN: No issues 

observed in testing

Table of Contents

104 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

decision as to whether there is sufficient 

data to score system performance. In some 

cases, accredited M&S is used to supplement 

test results. If the M&S supports the trend 

observed in the test data, then the function 

will be scored; otherwise, the White score 

remains in effect.

Although, as explained above, the decision to 

apply a White score is left to the discretion of the 

responsible PI, the Green, Yellow and Red scores must 

be justified and traceable to specific scoring criteria 

developed for each mission area. For the AW mission 

area, each mission task is broken down into a set 

of critical measures. These critical measures are tied to 

factors impacting mission success in a given TACSIT. 

The ME and PIs develop the scoring criteria used to 

score each of a task’s critical measures. Table 1 provides a notional set of critical measures and scoring 

criteria for the Launch subtask.

Scoring the Kill Chain

After receiving access to the test data, PIs must 

sift through the data to determine which tests are 

applicable to the mission task they are scoring, given 

the conditions specified in the TACSIT. For example, 

a PI may reject a test because the target drone is not 

sufficiently representative of the threat. The PI then 

uses the applicable test data and the appropriate scoring criteria to score each of a task’s critical measures. 

The lowest score assigned to any of a task’s critical 

measures determines the task’s overall score for a 

given platform. Notional scores and rationale are 

provided in column 3 of Table 1.

Note: This data is for illustrative purposes only and 

is not consistent with the actual WCB assessment. The 

overall task scores (e.g., Green, Yellow, Red or White) 

are applied to the scoring nodes and data links shown 

in Figure 2 to provide a top-level summary of the kill 

chain’s health.

It is important to understand that each mission task 

in a kill chain is scored independently of the tasks that 

precede it. For example, the Launch task will be scored 

as if all tasks “upstream” were scored Green, so it is 

possible to have a Green Launch task even though the 

Track task is scored Red. This is done to avoid masking 

the overall strength or weakness of a “downstream” 

link, so that multiple issues affecting the successful 

completion of a kill chain can be identified.

Performing an Integrated Kill Chain Assessment

In this step of the assessment process, all of the 

individual platform and system scores and scoring 

rationale are discussed in a working group consisting 

of the entire WCB team. The national team critiques 

the scores and justifications for every critical measure 

in every kill chain to ensure they properly reflect the 

scenario laid out in the TACSIT and accurately reflect 

current capability in the TACSIT context. The weeklong national team meeting allows the WCB team 

to work through issues in real time and finalize all 

scoring with the full consensus of the team across 

all mission areas.

Table 1. Notional Critical Measures, Scores, and Justifications for the Launch Subtask
Integration & Interoperability | 105

Table of Contents

Warfare Capability Baseline: Assessing Operational Gaps

Reporting Assessment Findings

The primary purpose of a single scored kill chain is 

to highlight broken and weak links or failed interoperability and integration that would prevent success 

in delivering a particular weapon against a particular 

target with a specific mission employment in a given 

environment. WCB findings are the delivered product 

for highlighting such capability gaps in individual kill 

chains. As the number of scored kill chains grows 

in a particular mission area, including functional, 

degraded, and broken kill chains, a broader picture 

of warfighting capability becomes clear, and major 

warfighting capability gaps identified.

In WCB Increments 1 and 2, a formal report documented the major assumptions, scores, justifications 

and findings for every assessed kill chain. In Increment 3, annotated electronic presentations served as 

the final report. In all three increments, scoring and 

justifications were placed into the WCB database 

maintained by COMOPTEVFOR.

Interpreting Results

The current WCB assessment process is not perfect, 

and care must be taken when interpreting the results. 

In particular:

• Because overall platform scores are based 

on a rollup of subsystem performance as 

well as tactics, doctrine, etc., a score of RED 

does not necessarily indicate the platform 

vice the doctrine or tactic is at fault. PIs will 

document the cause of the failure.

• Each AW kill chain to be assessed considers 

only one weapon and one shooter. The 

cumulative effect of multiple shooters or 

additional weapons is not included in the 

assessment. One implication of this decision 

is that a “broken/failed” kill chain does not 

necessarily imply that the targeted platform 

is at risk.

• Similarly, since the kill chain is assessed 

against a single-target raid, GREEN scores 

do not necessarily indicate the kill chain will 

perform well against a multi-target raid.

• Finally, in order to score a complete kill 

chain, it is likely that data will be pieced 

together from separate test events that 

occurred under different conditions. 

Consequently, the availability, applicability, 

and consistency of test data must be 

considered during the scoring process, when 

interpreting the results, and when drawing 

any conclusions.

Conclusion

The WCB process provides a valid method to assess 

current warfare capability based on actual system performance data from live-fire tests. The integration of 

warfighters’ experience with analysts’ technical expertise ensures that tactical needs will be addressed at 

the technical level. Findings from previous increments 

have established areas of focus for the acquisition and 

science and technology communities that align with 

the fleet’s highest priorities and concerns. 

Acknowledgements

The authors express their gratitude for the significant assistance provided by the WCB team, specifically: 

Charles Tatum, NSWCDD; CAPT Scott Guimond, 

COMOPTEVFOR; James Rogers, COMOPTEVFOR; 

and CDR Mark Carlton, COMOPTEVFOR. Their 

guidance and expertise greatly contributed to the 

development of this article.

Endnotes

a. It is important to note the weapons and threats identified in the 

WTPs are currently deployed systems. WCB does not attempt to assess 

capability of non-deployed weapon systems or of current weapon 

systems against future threats.

b. “Platform” is used generically to refer to the entity performing the 

kill chain function. This entity may be an aircraft, ship, vehicle, person 

or weapon, and includes all of the sensors, systems, equipment, and 

software used by that entity to carry out the given kill chain function.

References

1. “Warfare Centers of Excellence,” Rhumb Line, 28 May 2009.
Table of Contents

106 | LEADING EDGE January 2015

Closing the Gaps...

Capability Solutions Management

By Linda Learn
Table of Contents

Integration & Interoperability | 107

Closing the Gaps...Capability Solutions Management

With ever-changing political climates and rapidly 

evolving threats, our military is faced with the constant 

need to evaluate capabilities to ensure that they meet 

or outpace those of our adversaries. So how should 

the Navy’s leaders go about identifying and closing 

gaps in capability? This has been a long-standing 

question with far reaching strategic, budgetary, and 

organizational implications.

On 9 December 2010, the Chief of Naval Operations 

(CNO) hosted a summit on Integration and Interoperability (I&I) that tackled this question. At the summit, 

the Naval Air Warfare Center (NAVAIR) proposed 

a sequential, proactive approach to identify shortfalls in current capabilities, develop comprehensive 

solution recommendations to identify shortfalls and 

process the results within the Department of the Navy 

(DON) for approval, execution, and implementation 

in the Fleet. The objective of the process was to begin 

with input from the Fleet and look across systems to 

identify capability gaps and solutions in effects/kill 

chains associated with specific weapon-target pairs. 

A generic example of such a kill chain would be a 

scenario in which a United States Carrier Strike Group 

(CSG) is located in one of the world’s “hot spots” and 

the adversary in that region launches “Red Missile 

X” at the carrier; the CSG responds by engaging that 

threat with “Blue Weapon Y” (i.e., Blue Weapon Y 

vs. Red Missile X). The question becomes “How well 

does that weapon perform against that threat?” It’s a 

question that assesses the System of Systems (SoS) in 

a kill chain that spans search, detect, track, identify, 

engage, assess, etc. 

When it comes to fixing problems or inserting new 

technologies into the Fleet, funding lines have historically dictated that maturation and deployment of new 

capabilities focus on individual systems rather than 

SoSs. As voiced by NAVAIR’s Vice Admiral David 

Dunaway in a recent issue of Proceedings Magazine, 

“That important alignment is necessary and must 

continue to be executed vigorously. However, we 

must also implement a new process that includes a 

horizontally integrated view of how that system will 

work in the System of Systems (SoS).”
Warfare Area Effect Chain Unit/Type

Display Platform

Display Scoring

Primary Tactic

Alternative

Declutter

Reset

Effect Chain Selection

Mission Area: Weapon/System:

Surveil

Platform 1

Platform 2

Platform 3

Platform 4

Platform 5

Platform 6

Platform 7

Platform 8

Platform 9

Platform 10

Platform 11

Platform 12

Detect Track Launch Assess

Identification

(Commit/

Engage)

Weapon

(Kinematics

/Lethality)

Target:

Location: Environment: Conditions:

Primary Alternative

Table of Contents

108 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

Since the summit, representatives from NAVAIR; 

the Naval Surface Warfare Center, Dahlgren Division 

(NSWCDD); the Naval Undersea Warfare Center, 

Newport (NUWC Newport); Space and Naval Warfare Systems Command (SPAWAR); Chief of Naval 

Operations (OPNAV); Commander, Operational Test 

and Evaluation Forces (COTF); Fleet Forces Command (FFC); the Naval Air and Missile Defense Command (NAMDC); the Naval Mine and Anti-Submarine Warfare Command (NMAWC); and the Naval 

Strike and Air Warfare Center (NSAWC) have worked 

together to refine and implement the process initially 

proposed at the 2010 I&I Summit. In December of 

2012, these efforts culminated in the signing of the 

I&I Charter by the Vice Chief of Naval Operations, 

ADM Mark Ferguson. From that time, the process 

has been through several iterations. 

The first step in the I&I process is the Warfare 

Capability Baseline (WCB) assessment. The WCB 

team takes specific weapon-target pairs designated 

Figure 1. WCB Effects/Kill Chain Assessment Example
Integration & Interoperability | 109

Table of Contents

Closing the Gaps...Capability Solutions Management

by the Fleet as high priority and develops the mission threads and Tactical Situations (TACSITS) that 

provide context. They develop scoring criteria for 

such areas as search, detect, track, identify, engage, 

and assess (and others as necessary); then they score 

each task in technical detail. The findings are housed 

in a master database at COTF. A mock example of 

possible results is depicted graphically in Figure 1.

The next step in the process is to look for potential 

solutions across the entire spectrum that includes 

Doctrine, Organization, Training, Materiel, Leadership and Education, Personnel and Facilities (DOTMLPF). That’s where the Capability Solutions Management (CSM) Team comes in. Led by the Warfare 

Development Command (WDC) for the mission area 

under consideration (i.e., NAMDC is the WDC for 

the Integrated Air and Missile Defense (IAMD)), the 

team also includes experts from across the DOTMLFP 

spectrum. Their goal is to find solutions that can be 

acquired within the five years spanning the Fiscal 

Year Development Plan (FYDP). 

As a leader in RDT&E, NSWCDD’s primary role 

on the CSM Team has been to assemble and lead 

the Materiel Solutions Development Team (MSDT). 

A subset of the larger CSM team, the MSDT delves 

into potential materiel solutions and applies a rigorous analytical assessment. The team is comprised of 

experts from each of the following disciplines: Systems 

Engineering, Warfighter Integration, Architecture, 

and Warfare Analysis. 

This task is a challenge in communications as much 

as technical ability. It requires reaching across the 

internal NSWCDD departments as well as to other 

System Commands (SYSCOMs), WDCs, program 

offices, research facilities and others with expertise 

that can aid in the analysis and evaluation task. 

Each potential solution is considered in the context of the kill chain and TACSIT in which the WCB 

team evaluated it. The final output is presented in 

the Integrated Capability Package (ICP), a document 

written in a prescribed format that provides DOTMLPF recommendations to close gaps identified by the 

WCB. The final recommendations are grouped into 

three categories:

Immediate: Solutions that are currently available

Near-term: Solutions that can be delivered in 1-2 years

Mid-term: Solutions that can be delivered in 3-5 years

NAMDC presents the final document to FFC, who 

reviews it and makes any necessary changes. It is then 

delivered to OPNAV for consideration in the Program 

Objective Memorandum (POM) budgeting cycle.

The I&I process is not alone in the Navy’s efforts 

to identify and close gaps. NAMDC follows a Warfare Improvement Program (WIP) process. The 

Program Executive Office Integrated Warfare Systems 

(PEO IWS) employs a Capability Phasing Plan (CPP) 

process. SPAWAR makes evaluations and recommendations based on a Portfolio Health Assessment 

(PHA). Although seemingly disparate processes, the 

object of all of them is to identify gaps and submit 

proposed solutions to the POM cycle. Recent efforts 

have been driving toward a more cohesive approach 

that capitalizes on the “best of all worlds.” The I&I 

process is now merging with existing processes to 

feed the POM cycles at OPNAV.

To return to the original question... “How should 

our leaders go about identifying and closing capability gaps?” The answer will never be easy. We live 

in a world of growing complexity with increasingly 

complicated problems that require increasingly innovative solutions. The I&I process is a step in the right 

direction to ensure that the systems delivered have 

been evaluated across the DOTMLPF spectrum in 

the relevant mission contexts to ensure that the Navy 

brings the best possible solutions to the problems that 

are the highest priority to the Fleet. 
Table of Contents

110 | LEADING EDGE January 2015

Technical-to-Tactical Interfaces:

Importance of Consistent Interface between Warfighters 

and Scientists/Engineers

By Greg McHone and Dr. James D. Moreland, Jr.
Integration & Interoperability | 111

Table of Contents

Technical-to-Tactical Interfaces:

Importance of Consistent Interface between Warfighters and Scientist/Engineers

Extensive Fleet interaction establishes the necessary 

stakeholder relationships to understand operations, 

capture capability needs, and consider the art of the 

possible in scientific/engineering solutions through 

architectures, derived from mission area requirements, 

which are the vehicle for technical-to-tactical mission 

engineering principles. In July 2013, scientists and 

engineers from the Naval Surface Warfare Center, 

Dahlgren Division (NSWCDD), participated in an 

operational experiment and exercise known as Trident Warrior. The exercise teamed systems and software scientists/engineers with warfighters aboard a 

destroyer as well as in the Combat Direction Systems 

Activity (CDSA) laboratory to study surface warfare 

functionality and evaluate effects/kill chains to determine warfare effectiveness and technical execution of 

a specific operational mission thread. These subject 

matter experts witnessed and evaluated a Concept of 

Employment (CONEMP) for Surface Warfare (SuW); 

SuW Tactics, Doctrine, and Rules of Engagement 

(RoE); SuW effects/kill chain for target dissemination 

using tactical sensors; a prototype Common Weapons Control System (CWCS) that introduced SuW 

functionality; and technology for in-flight weapons 

communication. The CWCS was manned by operational warfighters to gain feedback on the systems’ 

design, functionality, and Human Computer Interface (HCI). Shipboard scientists/engineers provided 

expertise on the SuW CONEMP, in-flight weapon 

communications technology, and existing combat 

system capabilities and functions that were applied 

to the experiment. From this information, SuW operational requirements, system designs, system architectures, and other related technologies were then 

developed. Unquestionably, the technical-to-tactical 

insights gained from collaborating with warfighting 

operators provided an accurate view of capability gaps 

that would lead to better system designs and system 

architectures.

Definition

The Vice Chief of Naval Operation’s Integration and Interoperability (I&I) Activity has evolved 

NSWCDD’s coordination with the Fleet/Joint 

warfighters, other Systems Commands (SYSCOM) 

and Warfare Centers to one that is centered on 

mission engineering and the technical-to-tactical 

excellence principle. This principle is centered on the 

warfighter’s desired mission effects. The tactical 

capabilities required to achieve those effects are understood by the acquisition community and included 

in technical system designs by tight coordination 

and interaction between warfighters and the warfare centers’ scientists/engineers. The collaborative 

Fleet and scientific/engineering team create mission 

architectures based on a snapshot of Fleet exercises 

and experiments to serve as technical reference documents, Mission Technical Baseline (MTB), to inform 

the naval community on the validated means of 

executing particular mission threads. Mission architectures of deployed force operations are developed to 

support Fleet analysis, assessment, and requirements 

A carrier strike group sustainment exercise 

illustrates the complexity existing from 

multiple platforms and systems that are 

necessary for defense of a high value unit.
PEO SYSCOM

Collaboration

TECHNICAL-TO-TACTICAL

NAVAIR NAVSEA SPAWAR MCSC

I&I CHENG

(NAWC-WD)

I&I CHENG

(NUWC)

I&I CHENG

(NSWC-PC)

I&I CHENG

(NSWC-DD)

I&I CHENG

(SPAWAR HQ)

I&I DIRECTOR

Military Director

I&I DIRECTOR

Military Director I&I DIRECTOR I&I DIRECTOR

Fleet

Requirements

N9 & N2/N6

Technical Director

ROC/POE

Capability

Area(s)

Collaboration

• Lead Acquisition Management

• Programmatic Oversight

• Exercise Programmatic Authority

• Work with SYSCOM to ensure 

 TA is integral part of Program

• Operational/Mission Need

• Readiness

• Resource Programs

• Validate “CDD”

SYSCOMs*

• Analysis of Mission Areas,

 Systems, and Requirements

• Exercise Tech. Authority

• Certification of Weapons 

and IT Systems

* Per SECNAVINST 5400 15C CH-1

PEO

PMPEO

PMPEO

PMPEO

PM

Collaboration

US Fleet Forces

Command

WCOE

I&I CHENG

(MCSC)

ASN RDA* OPNAV

Table of Contents

112 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

generation of existing warfighting capabilities. MTB 

architectures are derived from defined mission objectives and effects, and deployed Joint and Navy assets 

operating under theater-defined doctrine to rehearse 

our blue force ability to effectively execute operational and contingency plans. Theater-specific MTB 

architectures that support the assessment of integrated warfighting capabilities are made available 

to the development and validation of system and 

system-of-systems specific architectures. The extensive Fleet interaction required to transition technical 

developments to tactical capabilities has been termed 

“warfighter integration.” 

Benefits of the technical-to-tactical principle are 

still being realized, as the interaction of warfighting 

and acquisition communities continues to mature. 

Across the naval enterprise, however, warfighting 

capability has lacked clear definition between the 

various acquisition and operational stakeholders. 

Efforts to standardize processes and employ consistent procedures to improve warfighting readiness 

continue to evolve.

Strategic Implementation

NSWCDD began to formalize the technical-to-tactical relationships by establishing Mission Focused 

Capabilities (MFC). Mission Focused Capabilities are 

represented by mission thrust areas in the context of 

the Joint Capability Areas (JCA) to solve warfighter 

challenges of today and tomorrow through an iterative 

operational and engineering process. Heavy emphasis 

continues to be focused on relationship building with 

key warfighter and sponsor stakeholders. The culmination of this effort is to understand the stakeholders’ 

operational needs and to develop a comprehensive 

understanding of the stimuli that drive them (e.g., 

threats, challenges, and opportunities) for current-, 

near-, mid-, and far-term capabilities. These operational needs and stimuli are used to drive the future 

direction of research and development activities to 

focus the naval enterprise on the right requirements 

and investments with the objective of increasing the 

transition of capabilities to the Fleet.

I&I activities institutionalize the processes and 

products across the naval enterprise that are necessary 

Figure 1. Naval Enterprise Governance Structure
Sponsors

NSWCDD CO

NSWCDD TD

People,

Processes, Tools, 

Tech Authority

Experimentation

Technical

Advisory Board

Warfighter Integration

I&I Surface Warfare 

CHENG

Security

Support

Warfare

Analysis

Cost

Estimating

SoS

Engineering

Mission

Architecture

OPNAV

Technical Departments

Provides:

• Security

Structure

• Security

Policy

• System

Commands

• Programs

• Warfare

Center

• Fleet Staffs

• Operational

Test Forces

• Sponsors

Integrated

Team

CO CIC/

NSW ExW STK FP SuW EW/DE

AAW

IAMD

Integration & Interoperability | 113

Table of Contents

Technical-to-Tactical Interfaces:

Importance of Consistent Interface between Warfighters and Scientist/Engineers

to effectively and efficiently improve warfighting 

readiness. The governance structure across the naval 

enterprise is illustrated in Figure 1 demonstrating a 

tight linkage between Fleet operational needs and 

readiness, OPNAV requirements and resourcing, and 

Assistant Secretary of the Navy for Research, Development, and Acquisition (ASN RD&A) programmatic 

oversight and acquisition direction.

Technical-to-tactical efforts provide a better understanding of mission-level gaps and deficiencies across 

the entire acquisition cycle and facilitate a decomposition of requirements from technology to the warfare 

level.

To align with Vice Chief of Naval Operation’s I&I 

Activity and achieve organizational strategic goals 

for technical-to-tactical relationships, NSWCDD created an organization that serves as the unifying focal 

point for mission engineering initiatives, known as 

the Mission Engineering Cell, to enforce the development of integrated warfighting capabilities. This 

organization, working in conjunction with the Fleet 

and similar organizations at other Warfare Centers 

and SYSCOMs, draws on the technical depth of the 

scientific/engineering community and tight Fleet 

relationships to produce affordable and integrated 

capabilities for the Fleet. These solutions consider 

the Doctrine, Organization, Training, Materiel, 

Leadership and Education, Personnel and Facilities 

(DOTMLPF) spectrum to make sure any technical 

inputs are tactically aligned to provide realistic recommendations for acquisition investment decisions 

and modifications to existing programs of record. By 

working with the Fleet components, Operational Test 

and Evaluation Force (COTF), Warfare Centers of 

Excellence (WCOE), and the Navy Warfare Development Command (NWDC), better Fleet experiments 

and data collection efforts are occurring to result in 

better requirements generation based on proven warfighting operational needs. The team also works with 

technology efforts to determine the future impact on 

mission capability with proper technology integration plans thus promoting the successful transition 

of promising new technologies. Figure 2 represents 

NSWCDD’s strategic organization alignment for mission engineering.

Figure 2. Naval Surface Warfare Center Dahlgren Mission Engineering Alignment
Table of Contents

114 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

The Deputy Assistant Secretary of the Navy 

(DASN) Research, Development, Testing and Evaluation (RDT&E) provides resources for I&I initiatives 

to support the future development of tools, processes, 

and workforce development for the new mission engineering discipline. Mission Level Assessment and 

Evaluation (MLA&E) is one of those initiatives that 

embeds mission engineering principles with Joint/

Naval exercises and experiments to capture desired 

warfighting capabilities. This initiative serves as the 

primary mechanism to institutionalize the technical-to-tactical principles. The MLA&E methodology 

closely aligns Fleet assessment processes by building 

stakeholder relationships and requirements identification through intensive warfighter integration efforts.

Finding Solutions to Warfighter Needs

Past technical-to-tactical relationships and developments have suffered from lack of collaboration 

between the major stakeholders leading to tactical 

implementation and weapon system introductions 

with inaccurate warfighter requirements, integration 

and interoperability problems, and uninformed user 

community. Fleet-produced Tactics, Techniques, and 

Procedures (TTPs) development driven by identified 

system shortfalls or responding to emerging threats 

have not always been specifically identified to the technical or acquisition community. Technical developments and 

acquisition programs have not routinely 

benefitted from direct observations 

and objective analysis obtained during 

Fleet exercises or been informed by TTP 

developments. Technical engineering 

and tactical solutions are different in 

their nature, but these efforts impact 

each other and must be informed by the 

other to achieve holistic, cost effective 

solutions. 

To address past technical-to-tactical 

deficiencies, NSWCDD and the Navy’s 

Surface Tactical Development Group 

(STDG) are developing a template for 

long-term collaboration and alignment 

between the WCOEs and the Naval Sea Systems Command’s Warfare Centers under the Surface Warfare 

Enterprise (SWE). The objective is to align the surface 

domain operational and engineering forces to ensure 

coordination of both technical and tactical developmental efforts; provide timely, pertinent information 

to both parties; establish recurring reporting procedures; ensure analyses are consistent; and address 

critical performance issues.

During recent Fleet engagements, NSWCDD 

aligned technical experts with Fleet tacticians in a 

collaborative Surface Warfare Improvement Program 

(SuWIP) tactical analysis process with the STDG and 

other operational and tactical Fleet organizations. The 

SuWIP produces an Integrated Prioritized Capabilities List (IPCL) for the Surface Warfare community 

to drive the investment areas within the Program 

of Memorandum (POM). This is accomplished by 

conducting in-depth analysis on operational and 

systems test data to define technical improvements 

that are aligned with Fleet priorities to determine 

possible improvement areas for warfare effectiveness 

through performance characteristics for combat/

weapon systems. The SuWIP Working Group consists of Anti-Surface Warfare (ASuW) tactical and 

technical subject matter experts and stakeholders 

Figure 3. Griffin Missile Firing from Coastal Patrol Craft
Integration & Interoperability | 115

Table of Contents

Technical-to-Tactical Interfaces:

Importance of Consistent Interface between Warfighters and Scientist/Engineers

with an overall objective to assess current tactical procedures and identify operational gaps in multi-platform combat/weapon systems and their employment 

against adversary threats. NSWCDD has incorporated 

I&I activities with SuWIP to ensure tactical/technical 

alignment of warfighting gap analysis, requirements 

generation, and proposed holistic solutions to accomplish mission wholeness. This technical-to-tactical 

collaboration supported implementation of new 

warfighting capabilities demonstrated in a forward 

deployed test firing as shown in Figure 3.

Fleet Exercise Thrust

National policy for realignment of forces to the 

Pacific Fleet area of operations provides strategic 

guidance for technical-to-tactical alignment resourcing requirements to achieve warfighting capabilities. 

To align technical community with key operational 

stakeholders and increased collaborative efforts with 

Pacific Theater combatant, component and tactical 

commanders are necessary to understand operational 

needs that drive technical and tactical solutions. 

Through DASN RDT&E-sponsored MLA&E initiatives, the mission engineering team embedded with 

key Pacific Theater Commanders (Pacific Command 

(PACOM), Pacific Fleet (PACFLT), Commander, 

Seventh Fleet (C7F)) and other theater component 

and tactical commanders to assess full warfighting 

capabilities. Using the Trident Warrior and Valiant 

Shield Fleet exercise venues and processes as a test 

bed, the team conducted engineering analysis of 

deployed combat system operational and test data to 

define performance characteristics and deficiencies in 

combat and weapon systems to enable full effects/kill 

chain capabilities. By articulating Fleet requirements, 

expectations were better managed across technical 

and operational stakeholders. Figure 4 shows the Fleet 

forces rehearsal during Valiant Shield exercise with 

NSWCDD scientists/engineers aboard to achieve a 

desired warfighting capability.

MLA&E warfighter integration application during 

Joint/Fleet exercise Valiant Shield surface warfare 

events demonstrated the utility of mission engineering 

principles as the underpinning for assessing existing 

mission performance and proposed solutions. This 

collaborative operational and technical community 

approach to major Fleet exercises laid the foundation of utilizing a Fleet user perspective to establish 

the guiding principles necessary to capture mission 

area requirements through capabilities-based architectures. The outcome has driven the development 

of combat system design specifications that enable 

full combat employment of weapon systems capabilities. A better understanding of the gaps and deficiencies is now 

achieved at the mission level across the 

entire acquisition cycle, which facilitates a decomposition of requirements 

from the development of technology to 

the execution of mission threads as an 

integrated warfighting capability.

Commander, U.S. Fleet Forces (Training) sponsored mission engineering 

support of Commander, Strike Forces 

Training Atlantic (CSFTL) assessors 

to align technical experts with tactical 

experts in order to identify improvements in data collection and analysis 

during fleet certification events, specifiFigure 4. Valiant Shield Exercise Carrier Strike Group Operations cally Composite Unit Training Exercises 
Table of Contents

116 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

(COMPTUEX). Fleet Training and Readiness processes employ a Navy Mission Essential Task List 

(NMETL) mapped to specific Navy mission areas 

(capabilities) in a continuous improvement process 

called the Navy Warfare Training System. The owners 

of Navy Mission Essential Tasks use feedback from 

exercises, operations, and other events to improve 

how the Fleet articulates requirements, measures 

performance, certifies readiness, and implements 

improvements. Efforts to date have focused on Fast 

In-Shore Attack Craft (FIAC) defense during Harry 

S. Truman and George H.W. Bush Strike Groups’ 

Sustainment Exercises (SUSTEX) and Composite Unit 

Training Exercises. It is expected that the MLA&E 

application of the ICF/OCD data model will lead to 

better defined measures of effectiveness that lead to 

advanced levels of Fleet readiness. Figure 5 shows a 

5-inch gun engagement of High Speed Maneuvering Targets representing Fast In-Shore Attack Craft 

during Harry S. Truman Strike Group’s Sustainment 

Exercise.

Summary

Intensive warfighter integration technical-to-tactical alignment enables achievement of surface domain 

excellence through I&I by reducing costs, preventing 

disjointed efforts, ensuring a linear approach, and 

reducing time required to develop and implement 

effective materiel and non-materiel solutions for the 

surface warfare community. Warfighter integration 

efforts to transition technical solutions to tactical 

capabilities through MLA&E create conditions for 

a philosophical change in evaluation, experimentation and assessment that enable mission engineers to 

participate in the planning, execution and analysis of 

Fleet events. This tight technical-to-tactical linkage 

results in higher confidence of producing integrated 

warfighting capabilities for our warfighters through 

validated MTB architectures from the Fleet. 

Figure 5. Gun Engagement of High Speed Maneuvering Targets During Sustainment Exercise
Integration & Interoperability | 117

Table of Contents

Exploring Solutions Across Doctrine, 

Organization, Training, Materiel, Leadership

and Education, Personnel and Facilities 

(DOTMLPF) Using Orchestrated 

Simulation through Modeling (OSM)

By Sara Wallace and Mary Ann Cummings

Historically, the Naval Surface Warfare Center, Dahlgren Division (NSWCDD) has 

solved many systems engineering challenges using deterministic, physics based Models 

and Simulations (M&S) of the systems. For example, NSWCDD has developed high 

fidelity M&S such as radar propagation models, six degrees of freedom models for 

missile trajectories and blast fragmentation lethality models to support various 

projects and sponsors. However, in order to perform analysis at the mission level, 

it is necessary to model the multiple systems involved in achieving the mission 

objective, especially the interactions between systems. New programs, such as the 

Integration and Interoperability (I&I) efforts, require NSWCDD to begin to analyze the interactions between the weapons, sensors, and weapon control systems.

Many of the current models used by the naval community analyze the interactions by adding the other systems to their own system model, especially for 

common systems such as threats, communications, etc. This introduces error 

and redundancy by having each modeling entity create its own models, which are 

neither shared nor reviewed between organizations. This dilemma can be solved 
SE HIERARCHY

MISSION

Force Focus

PLATFORM

Capability Focus

SYSTEM

Functional Focus

COMPONENT

End Item Focus

Table of Contents

118 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

by having a government-owned, modular, and scalable 

software framework capable of allowing the models 

to interact in a common environment. NSWCDD has 

invented and submitted a patent request for a software 

framework called Orchestrated Simulation through 

Modeling (OSM), which is truly a model agnostic and 

capable of modeling the interactions between systems 

by “plugging in” system models from government, 

industry or academia. Plug-in is a term for a software 

extension that adds capability or new features. Adding 

system models via plug-ins allows software customization for each project without making changes to the 

framework. In addition, OSM enables a mission-level, 

fast-running, qualitative assessment of the system’s 

interactions. This assessment permits more focused 

follow-on analytical deep dives with specialized tools 

currently used for systems engineering. This article 

describes the OSM framework; provides an example 

of how OSM was used to support the Commander, 

Operational Test and Evaluation Force (COMOPTEVFOR) (Navy); and discusses the future of M&S to 

support mission engineering.

Background—The Need for OSM and 

Mission Level M&S

Across NSWCDD programs, M&S is typically 

developed to support systems engineering for acquisition. Currently, the system commands are typically 

organized to develop and acquire systems (i.e., weapons, sensors, weapon control systems, etc.) and platforms (i.e., submarines, ships, etc.). Mission engineering accounts for the interactions and performance of 

the components, systems, and platforms that support 

the mission(s) being performed. This relationship is 

depicted in Figure 1.

As a rule, mission engineering studies aggregate 

the performance of systems in order to perform studies in a reasonable amount of time and effort. If the 

detailed performance M&S for each platform, system 

and component were used and each design parameter 

analyzed, the time required to perform the study 

would be astronomical and would not support timely 

acquisition decisions. The approach detailed in this 

article proposes use of behavioral models to represent 

the systems. Behavioral models represent the systems 

as independent entities that follow a defined state 

diagram where the system moves from state to state 

by stimulus from other systems or the environment. 

This approach allows for emergent behavior, for example, system performance, to be observed as systems 

interact throughout a scenario. The behavior is not 

predetermined or scripted. Each system’s behavior 

can be unique for each run since it is dependent on 

its interaction with other entities within the model. 

One interaction may lead to another interaction and 

so on. For example, a simple gun state machine could 

have two states: fire and stand-by. To move between 

the states, the gun could be stimulated by another 

system, for example, a human, who has a state of 

pulling the trigger. By using behavioral models, the 

mission analysis can focus on the performance effects 

of interactions of the systems. 

Traditional modeling uses a set of equations to 

dictate how the system operates and interacts, both 

Figure 1. The Systems Engineering Hierarchy
Integration & Interoperability | 119

Table of Contents

Exploring Solutions Across Doctrine, Organization, Training, Materiel, Leadership and Education, 

Personnel and Facilities (DOTMLPF) Using Orchestrated Simulation through Modeling (OSM)

independently and with other systems. This equation-based approach is used with many of the current System of Systems (SoS) M&S tools used in the 

Navy. SoS M&S tools often rely on the systems being 

described by varying degrees of complexity. Further, 

enormous energy is spent modeling system behaviors down to the minutest detail and attempting to 

fully exploit all possible outcomes of SoS behavior. 

While the high fidelity M&S approach is quite feasible, as stated earlier, it requires an abundance of time 

and funding. This approach does not eliminate the 

need for M&S at the lower levels since independent, 

detailed characterization of system performance is 

still required, especially when an issue is revealed at 

the mission level. However, using behavioral models 

allows program managers to be aware of a larger set 

of solutions across the DOTMLPF spectrum, vice 

the current paradigm of primarily looking at system 

materiel solutions. This approach will be extremely 

useful to explore “what-if” scenarios for mission engineering, which is focused on how changes to tactics 

and doctrine may close gaps in kill chains.

Analysis Definition—Integration & 

Interoperability (I&I) Process

As discussed throughout this edition of Leading 

Edge, the I&I process is different from most existing acquisition processes in the Navy. First, the fleet 

defines warfighting threads from operational plans 

and/or needs. Second, COMOPTEVFOR and the warfare centers characterize technical descriptions of the 

warfighting thread. Third, the Integrated Capability 

Framework (ICF) is used to identify possible solutions 

to gaps. These mission thread assessments are performed using M&S tools as well as operational test 

data. Finally, the warfare analysis products (results) 

become the basis for materiel and non-materiel solution decision making in support of Office of the Chief 

of Naval Operation’s (OPNAV’s) Capability Phasing 

Plan (CPP).

OSM Overview

OSM is an evolutionary SoS framework that was 

developed under NSWCDD in-house research funding 

to investigate and solve computer science challenges 

facing SoS modeling and mission engineering. OSM 

follows the Discrete Event System (DEVS) specification formalism which is a modular and hierarchical 

formalism for modeling and analyzing timed event 

systems. The OSM framework allows output visualizations and DEVS M&S frames to be developed 

separately as plug-ins and combined to form a mission 

level model of many systems. OSM was designed to 

use behavioral or agent-based models of the systems, 

since this allows the analysts to interrogate the interactions between the systems. There are five goals for 

the OSM development effort as listed below.

1. Create a government-owned, DEVS 

based M&S framework using Object 

Oriented Design (OOD), Agent Oriented 

Programming (AOP), and plug-in computer 

science design techniques to provide a 

model agnostic, scalable tool that enables 

integration and execution of multiple, 

independently developed models that can 

be used by government, industry, and 

academia.

2. Provide mission analysts with another (but 

not the only) M&S tool in their tool-box 

that supports M&S-based system analysis 

and validation from the component level up 

through the system, system-of-systems, and 

mission-level.

3. Minimize the effort required to integrate 

independently developed models and 

simulations. 

4. Facilitate the development of a rich set of 

re-usable plug-ins.

5. Facilitate the use and reuse of M&S 

components to advance the discipline of 

M&S based analysis.

OSM Framework Description

First and foremost, the OSM framework was 

designed to be as simple as feasible. OSM provides 

the minimum core set of services and functions that 

will enable multiple plug-ins (models or simulations) 

to exchange data and perform as an integrated program. The OSM framework includes only the core 
Sensor Threat Ship Weapon

Metrics Visualization Graphs

Comms Weather Discrete

Event

Continuous

OSM CORE

MODEL PLUG-INS

OUTPUT PLUG-INS

EXPERIMENTAL PLUG-INS

SIMULATOR PLUG-INS

Table of Contents

120 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

functionality and a common set of basic plug-ins that 

users can select when creating a specific instantiation. 

The OSM framework does not include user-specific 

(system) plug-ins. Figure 2 depicts the framework 

using several plug-in types.

The OSM framework is a core structure, or skeleton, onto which users attach software and/or hardware 

configuration items that are required to communicate 

with each other. The core structure is a well-defined 

set of classes, libraries, application program interfaces 

(APIs), and development rules that provides a large 

amount of reusable code and a systematic development and integration approach. To investigate an 

issue or solve analysis questions, users must formulate their analytic approach and either build new or 

re-use system models and output models on top of 

the framework. Four main types of plug-ins are used 

with the prototype OSM framework. They are the 

system, experimental, simulator, and output plug-ins.

The first type of plug-in used in the OSM framework, is the model plug-in. The model plug-ins 

depicted in Figure 2 are: sensor, threat, ship, and 

weapon. OSM uses behavior-based system models 

called agents to observe and analyze the interactions 

between the systems modeled. Prior to agent-based 

modeling techniques, many system of systems simulations relied upon systems dynamics modeling, which 

is based on ordinary differential equations (ODEs) to 

represent the interactions between systems. Using 

agents allows the models to interact so the analysts can 

explore the interrelationships and interdependencies 

between the systems.

The second type of plug-in used is an experimental 

plug-in, which follows the DEVS formalism. Experimental plug-ins define pieces 

of the scenario being modeled 

that are not part of the system 

behaviors. For example, experimental plug-ins could include 

weather and communications 

models. While weather is not 

part of a system model, changes 

in weather will affect the system, 

thus the interactions between 

the weather plug-in and system 

plug-in are important to model. 

The third type of plug-in is 

the simulator plug-in, which 

also follows the DEVS formalism. The kill chain analysis prototypes have typically used a 

discrete event plug-in, although 

other simulator plug-in types, 

including continuous and realtime simulator plug-ins, have 

been demonstrated in OSM. 

The real-time simulation plug-in allows the user to 

interact with the model to dictate its behavior. To 

describe the difference between the simulator plug-ins, 

an example could be a model which defines how a ship 

travels from point A to point B. For the discrete event 

plug-in, the ship transits by its defined behaviors in 

the model. These behaviors could include making the 

transit in the shortest distance, escorting another ship, 

and/or following a pre-defined pathway of points. For 

the real-time simulation, an analyst can override the 

Figure 2. Depiction of Prototype OSM Framework with Plug-ins
Integration & Interoperability | 121

Table of Contents

Exploring Solutions Across Doctrine, Organization, Training, Materiel, Leadership and Education, 

Personnel and Facilities (DOTMLPF) Using Orchestrated Simulation through Modeling (OSM)

behavior by manual input of directions for the ship. 

This shows potential to reduce model development 

time to evaluate the effects of changes in behavior. 

Instead of rewriting the model, the analyst can input 

new directions for the ship while the simulation is 

running. 

The last type of plug-in is the output plug-in. 

Output plug-ins are tools to aid in analysis. Many of 

the output plug-ins are developed to visualize the simulation. With so many systems interacting with each 

other, visualization tools aid the analyst in identifying 

issues. The prototype OSM development efforts have 

used NASA’s Worldwind (a 3-D geospatial tool), and 

OpenMap (2-D open source mapping tool) extensively 

for visualization. Using externally developed, open 

source tools allow analysts a plethora of benefits, 

including the ability to leverage existing tools within 

the open source community, to customize desired outputs, and use other familiar software. Other common 

output plug-ins include a metrics plug-in and a graph 

plug-in. The metrics plug-in is populated by the analyst with measures of performance, measures of effectiveness, and/or other numerical measures that are 

important to the specific analysis. The graph plug-in 

displays these metrics in charts for easy viewing. 

Since each analysis is unique, many output plug-ins 

have been customized to support specific projects. In 

addition, the OSM team developed an output plug-in 

that extracts all data into a generic text file, allowing 

the file to be imported and used to perform analysis 

in external analysis tools.

Behavioral Models using Agents

This section provides a high level description of 

agents. Agent-based modeling is important to the 

OSM framework, since agents were used to create the 

behavioral models for OSM. Agents, as used here, are 

defined as autonomous decision-making entities with 

diverse characteristics. In OSM, the rules for each 

agent are defined through state diagrams and vary by 

agent. The decision rules define models specific for the 

agent of the external world, and carry a level of sophistication. The rules employ memory of the agent’s 

experiences, and are limited by the cognitive ability 

or load of each agent. Each agent varies by its specific 

attributes and available accumulated resources. This is 

of particular significance because multiple, identically 

coded agents can exist even if their behaviors during 

the simulation are not identical.

The simulation is based on the interactions among 

agents and each agent will have its own, unique set of 

interaction experience. For example, for two, identically coded, unmanned aerial vehicle (UAV) agents, 

each may fly on a different flight path based on the 

rule to hold a stand-off distance from other UAVs. In 

another instance, the UAVs may have the same flight 

path, but when launched at different times, the view 

of the world will be different. Each UAV agent detects 

moving threats at different points in time and makes 

the decision to engage at a different time perhaps 

based on being fired at or receiving a command to 

fire. When faced with a decision, each agent makes an 

independent decision that may or may not be the same 

decision similar agents make. The decisions are based 

on each agent’s interactions with the world and other 

agents. The interactions can be simple or complex 

depending on what questions the analyst is trying to 

answer. Agents are beneficial to model interactions 

since, unlike traditional equation-based models, there 

is not a central authority or controller dictating how 

the system operates, how the system is modeled, or 

how the system moves from state to state in the state 

diagrams. Using agents and behavioral models has 

allowed interrogation into mission engineering challenges, especially with system interactions that were 

difficult to model with other M&S approaches.

OSM Instantiation

An instantiation of OSM is defined as a mission 

scenario that is run to answer specific analysis questions based on a defined analytical process for a given 

mission thread. An OSM instantiation includes the 

user-specific plug-ins (for systems, experimental, and 

output) plus selected common and shared plug-ins 

(output, experimental, and simulator) built on the 

OSM framework. The plug-ins are selected based on 

the analysis to be performed. During an instantiation, 

the OSM framework provides the core services that 
ANALYTICAL

QUESTIONS

ANALYSIS

DEFINITION

MODELING

FRAMEWORK

PLUG-INS ANALYTICAL

PROCESS

COMMON

UNIQUE

UNIQUE

SHARED

SHARED

RESULTS

Technical

Description

Operational

Warfighting

Need

COMMON

UNIQUE

UNIQUE

SHARED

SHARED Mission

Thread

Enables

Plug-ins Specifies

Mission

Thread

OSM

Table of Contents

122 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

enable multiple user-provided plug-ins to run as an 

integrated product. This is depicted in Figure 3.

An OSM instantiation, as depicted in Figure 3, 

consists of five main parts. The technical description 

of the problem to be assessed and the operational 

warfighting need are used to form the analytical questions. The analysis definition is created by the mission 

thread along with a description of the results to be 

collected. The OSM modeling framework allows the 

plug-ins to execute together and is populated with a 

combination of common, unique, and shared plug-ins. 

Common plug-ins include output plug-ins, such as 

visualization tools, which are useful to most applications and generic enough to be suitable for a variety of 

programs. A shared plug-in could be a missile model 

created for one program, but the model is shared 

with another program. A unique plug-in is a model 

developed and used by only one program. Finally, the 

analytical process is how the user chooses to execute 

the instantiation. The analytical process may include 

analysis assumptions, plans for technical reviews, and 

other tasks deemed important by the user. 

An example instantiation of OSM is a fleet exercise planning and assessment model developed for 

COMOPTEVFOR. COMOPTEVFOR’s vision was for 

the model to be used before a test event for planning, 

and after a test event for analyzing the event outcome. 

Agents were created for naval systems playing a critical 

role during the test event. This included guns (i.e., Phalanx Close-In Weapons System (CIWS), 5-inch, etc.), 

sensors, ship movement, helicopter movement, small 

boat threats, and ship tactics. The agents were based 

on the system behaviors including system interaction 

and reaction to other systems. The system behaviors 

were created by ship commanders and helicopter 

pilots from COMOPTEVFOR and by NSWCDD’s 

weapon experts and warfare analysts. The team documented the system behaviors and relationships in 

Systems Modeling Language, which was used to code 

the system plug-ins in OSM. The behavioral models 

also used data from high-fidelity, physics-based lethality models, including probability of kill and timeof-flight tables. Coupling high-fidelity algorithms 

and look up tables to the OSM instantiation adds 

the benefit of the high fidelity, lower system models, 

without significant increases in run time. In addition, 

COMOPTEVFOR wanted the interactions displayed 

on a map and metrics captured. Both types of output 

plug-ins were already created for other instantiations 

of OSM and were available for reuse. Plug-ins that 

can be shared across programs are called “common 

services” plug-ins. Typically, these include methods 

Figure 3. OSM Breakdown Structure
User Provided

Plug-in

CIWS Model

User Provided

Plug-in

Small Boat Threat

Model

User Provided

Plug-in

5” Gun Model

INSTANTIATION = OSM Framework + User Plug-Ins

Common Services

Plug-in

Display: Map

Common Services

Plug-in

Display: Run Metrics

Common Services

Plug-in

UNUSED

OSM Framework

RUN RESULTS

Gun

Effectiveness

Study

CIWS

THREAT

5” GUN

MAP METRICS

OSM 

Framework

OSM Software Foundation enables Common Services and user Specific plug-ins to execute together.

Integration & Interoperability | 123

Table of Contents

Exploring Solutions Across Doctrine, Organization, Training, Materiel, Leadership and Education, 

Personnel and Facilities (DOTMLPF) Using Orchestrated Simulation through Modeling (OSM)

for displaying the data output from the model. The 

COMOPTEVFOR instantiation of OSM is depicted 

in Figure 4.

The OSM framework allowed COMOPTEVFOR 

to build behavioral (agent-based) models of the 

systems of interest and use common services plugins from prior instantiations of OSM. In addition, 

COMOPTEVFOR wanted a way to display output 

metrics on the map plug-in. This included metrics 

measuring which weapons were engaging a particular 

threat, how many rounds had been fired, and which 

weapon killed the threat. 

Collaboration with NAVAIR Warfare Center 

China Lake

During the creation of the OSM instantiation for 

COMOPTEVFOR, the OSM team began a formal 

collaboration with the Naval Air Systems Command 

(NAVAIR) Warfare Center (NAWC) at China Lake 

as a research effort between two systems commands 

and warfare centers. The goal was to determine if an 

outside organization could effectively use and develop 

plug-ins, also called models, for OSM. The joint team 

selected a common threat and kill chain with Navy 

surface and air assets. NSWCDD built the surface 

agents (ships, ship sensors, ship weapons, etc.) while 

NAVAIR built the air agents (unmanned air vehicles, 

air sensors, air weapons, etc.). NAVAIR was given the 

OSM software specification and several examples 

of prior OSM instantiations. The team held weekly 

phone conferences, but, for the most part, NAVAIR 

developed their behavioral models independently. 

Since all the models were developed and complied 

with the specification, each had a common interface. 

At the end of the project, the two software development teams met face to face and were able to bring 

the models together in the OSM framework within 

a matter of hours. After integration, the software 

developers ran multiple scenarios to test how the 

models reacted to various conditions to ensure the 

integration was successful. A screen shot of the effort 

is shown in Figure 5.

Figure 5 depicts the scenario containing models 

from both NSWC Dahlgren and NAWC China Lake. 

The round, colored icons represent various agents 

depicting notional assets such as ships, unmanned 

vehicles, and threats. The wedges represent notional 

engagement areas for the weapons or coverage areas 

for the sensors. This effort proved that software developers external to the OSM were able to use the specification and create behavioral models that would work 

in the OSM framework with the models developed by 

Figure 4. COMOPTEVFOR Instantiation of OSM
Table of Contents

124 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

the OSM team. The effort also fostered collaboration 

between the two warfare centers and provided the 

OSM team with feedback on the performance of the 

framework.

OSM to Support Future Mission Engineering 

Projects

OSM enables the simulation of integrated 

warfighting capabilities to assess multiple weapon/

target pairs and tactics. The current assessment is 

a static assessment (on paper) of a single weapon, 

single target pair. The future of mission engineering 

will continue to include the assessment of gaps in 

warfare ability (currently performed during the 

Warfare Capability Baseline (WCB) studies), and 

an assessment of possible solutions as part of the 

Integrated Capability Plan (ICP). One concept for 

increasing the capability of mission engineering is 

to augment the WCB and ICP studies with cloud 

charting, the Integrated Capability Framework 

(ICF) and OSM. This futuristic, conceptual mission 

engineering process is depicted in Figure 6.

The first step in the proposed I&I process is identical to the existing I&I process. First, warfare gaps in 

kill chains will be identified during the WCB studies. Next, the kill chains assessed by the WCB will 

be “brought to life” by developing system behavioral models within the OSM framework by using 

data from test events, models, as well as from system 

architectures and other products from the naval community and the ICF. Technology 

insertions, as identified in the 

cloud charts, will be injected 

into the OSM simulation to 

determine the mission effect 

of the technology insertion on 

the kill chain. This capability 

will be a valuable new tool, 

allowing acquisition managers to assess which technologies produce the most significant impact to the mission. 

Further, the ICF of the future 

will contain test data, system models, architectures 

and acquisition data, all of which could be used to 

answer a diverse set of analysis questions using OSM. 

In addition, system and mission architectures could 

be used as a basis for the behavioral-based models. 

Architectures define the relationships and interactions 

between systems. The future assessments and analysis 

using OSM will allow for a more thorough assessment 

of the design space for solution sets to support the ICP. 

Another promising solution for future work is 

for OSM to tie into other Department of Defense 

(DoD) frameworks. OSM is designed for the analysis of SoS using behavioral-based models. While 

legacy models can be wrapped into OSM, many 

high-fidelity, physics-based models were not designed 

to work with other models. OSM has used look-up 

tables from high-fidelity models in lieu of wrapping 

the model, but other DoD frameworks may be able 

to offer additional functionality by quickly wrapping 

legacy, standalone models and hardware-in-the-loop 

test articles. One DoD framework that shows promise 

for linking legacy software is NAVAIR’s Architecture Management Integration Environment (AMIE). 

Funded by the Office of the Secretary of Defense 

(OSD), AMIE is a proven framework to wrap existing 

models and simulations of all types, including test 

articles. The OSM team is currently collaborating with 

the NAVAIR AMIE team. The OSM team proposes 

to use AMIE to wrap legacy software into OSM and 

Figure 5. Screen Shot of OSM Collaboration with NAWC China Lake
Kill Chains

OSM

Mission

Engineering Models Data

Capability

Evolution

Document

MISSION CAPABILITY 

Assessments

MISSION CAPABILITY 

Package

Integrated Capability 

Framework

Measure Progress

Verify

Dynamically 

Visualize

Inform

Align

Detail Improve

Generate

Generates

Inform

Improves

Utilize Mission-Based Scenarios

Cloud Chart

Integration & Interoperability | 125

Table of Contents

Exploring Solutions Across Doctrine, Organization, Training, Materiel, Leadership and Education, 

Personnel and Facilities (DOTMLPF) Using Orchestrated Simulation through Modeling (OSM)

is developing an OSM to AMIE interface. This would 

allow other models and simulations to interact with 

OSM through AMIE instead of being converted to 

a behavioral-based model and wrapped into OSM. 

Pairing multiple frameworks together may allow new 

and legacy code to be integrated seamlessly into a 

common simulation and achieve an ideal state for a 

mission engineering analysis process.

Conclusions/Recommendations

Using behavioral models in addition to the typical system performance models is advancing mission-level assessment capability. Building the behavioral models from the system architecture products 

created by the programs of record is recommended. 

Using architectures, such as DoD Architecture Framework (DODAF) products, to define the behavioral 

models ensures traceability and aids in validation 

of the models. If a change is made in a system architecture used in a behavioral model, the model will 

also reflect this change. This is important for predicting how the system behaviors will affect system 

performance in the context of a mission. Of particular interest would be to predict behavior patterns or 

element/system states that lead to a degradation in 

system performance. Once system behaviors can be 

predicted, system M&S efforts can focus the use of 

high-fidelity models to fully interrogate and solve the 

system interoperability issues unveiled by the high 

level model. The ability to characterize the system 

behavior by predicting the interactions of the system 

in order to predict system performance would allow 

the acquisition community to optimize investment 

strategies for future system development. 

In conclusion, although the OSM framework is 

still in the research stage, it shows great potential to 

be used to support future mission engineering efforts. 

The OSM framework is currently undergoing formal 

software validation and design utilizing NSWCDD 

internal investment funding. Once the validation is 

complete, the framework can be freely distributed 

within the Navy and across government agencies. 

Using a common software framework to allow the 

interaction of behavioral models created by separate 

entities, while maintaining the intellectual property 

of each model, will go a long way toward full collaboration on future mission engineering programs. 

Figure 6. OSM’s Use in the Future I&I Process
Table of Contents

126 | LEADING EDGE January 2015

Mission Level Test and Experimentation:

Drive Mission Level Performance During Early Development

By Neil Baron, Darren Barnes, and Dr. James D. Moreland, Jr.
Integration & Interoperability | 127

Table of Contents

Mission Level Test and Experimentation:

Drive Mission Level Performance During Early Development

Since the first test shot was fired over the Potomac 

River Test Range (PRTR) in 1918, the Naval Surface Warfare Center, Dahlgren Division (NSWCDD), has answered 

the nation’s call to support the warfighter. In today’s 

environment, naval systems are more complex than 

ever with the need to integrate many systems in order 

to provide full mission capability. How these systems 

work and communicate with each other is critical to 

understanding warfare systems performance. Instead 

of waiting for shipboard integration and testing to 

assess integrated capability, NSWCDD has developed 

the USS Dahlgren capability.

USS Dahlgren connects the PRTR, which provides real target data in a littoral environment to 

our plan-detect-control-engage-assess labs and systems for end-to-end testing at NSWCDD. Using USS 

Dahlgren, scientists and engineers are able to plan, 

detect, identify, track, engage and assess sensorweapons capabilities in a real world environment prior 

to shipboard integration and delivery to the Fleet. USS 

Dahlgren also allows us to evaluate new capabilities 

and assess the benefit to the warfighting system prior 

to full scale acquisition and development.

Background

Mission Level Test and Experimentation (MLT&E) 

is a system-of-systems (SoS) approach for validating 

the development of individual systems by evaluating 

these systems in a representative operational environment with the other systems that they must operate 

with to achieve an integrated warfighting capability. 

The intent of this approach is to gain technical insights 

on integration and interoperability (I&I) challenges 

as well as determine the derived requirements for 

individual systems based on mission success criteria 

during early development. Individual systems do not 

produce warfighting capabilities unto themselves but 

rely on the interaction of multiple systems to execute 

an effects/kill chain effectively and safely. By addressing I&I issues earlier in the acquisition cycle, we have 

a better chance at reducing expensive rework and 

expediting fielding capabilities to the warfighter without delay. MLT&E plays a critical role in determining 

the right requirements, assessing I&I wholeness for 
Kill Chains

Search/

Detect

Detect Control Engage Assess

Track Classify Launch Kinematics Lethality Assess

Platform A

Platform B

Primary Kill Chain

Alternate Kill Chain

Assessment of System Ability

to Satisfy Requirement

Capability Package 

Assessment Process

Capability Engineering Process

NWPS

NARS

ICD

CONOPS

OV4

OV3

OV2

OV1

NMETLS

OV5

MISSION

LEVEL

INPUTS

BUILD MISSION

LEVEL OVs BUILD NMETLS FORNEW CAPABILITIES

JCAS

UNTLS

NMETLS

Build a Mission Model Mission Model Output

Desired MOE’s

and MOPs

Identify, Parameters

Critical to Mission

Performance

Prioritized

Technical

Parameters

Refine

Mission

Model

Measurable

and testable

requirements Identify Critical

Interface

Interface

Exchange

Deterrent

Supports

Program

TSP

Supports

CDD

and 

SDS

Mission

Performance

Requirement

Build a Mission Thread

OV6

1

2 3

4

5

6

Table of Contents

128 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

integrated capabilities, and solving issues early, on 

the left side of the Systems Engineering “V” model, to 

reduce cost, thus making capabilities more affordable. 

The technical insights gained are also used to develop 

the Mission Technical Baseline (MTB) and Integrated 

Capability Technical Baseline (ICTB) architecture 

products, which serve as the technical reference documents to drive acquisition development of systems.

In today’s environment, the majority of test and 

experimentation activities occur for single systems 

to exercise that system’s ability to meet requirements 

irrespective of the mission success criteria. This focus 

on single system testing has driven us down a path of 

sub-optimizing individual systems and losing focus 

on the operational context in which the systems will 

operate when transitioned to the Fleet. As individual 

systems are matured independently, the integration 

of these systems within the required SoS has fallen 

on the Fleet at delivery. In addition, the assessment 

of Fleet-prioritized effects/kill chains indicates a 

compelling need to address I&I problems from an 

end-to-end mission thread perspective to consider 

all interfaces and relationships across many systems. 

Figure 1 illustrates a notional effects/kill chain with 

linkages to architectural products used in the acquisition community for developmental purposes. 

NSWCDD leadership developed a forum, Integrated Lab Council (ILC), to provide technical 

guidance across all departments in order to link 

the right labs together to make the USS Dahlgren 

concept a reality. The ILC provided a cross-departmental vision and a strategic plan for integrating 

NSWCDD Research, Development, Test and Evaluation (RDT&E) resources into a cohesive, net-centric engineering environment. A key aspect of this 

capability includes connectivity to the PRTR assets 

Figure 1. Interoperability & Integration/Warfare Capability Baseline
Integration & Interoperability | 129

Table of Contents

Mission Level Test and Experimentation:

Drive Mission Level Performance During Early Development

to leverage and incorporate real-time, live data from 

outdoor experiments. This end-to-end integrated 

capability will foster collaborative RDT&E demonstrations, experiments, integration events, and engineering development tests to accelerate and improve 

NSWCDD core technical capabilities and products 

for warfighter use.

USS Dahlgren Definition

In 2011, the Naval Surface Warfare Center, Dahlgren Division (NSWCDD), embarked on establishing the USS Dahlgren construct. In response to the 

driving need to evaluate the Naval fire-control-loop 

through RDT&E that spans all aspects of the kill 

chain in a land-based test environment, NSWCDD 

has established a virtual ship, the USS Dahlgren. As 

our naval systems continue to increase in complexity 

and the fire-control-loop continues to expand within 

and between naval platforms, the need to integrate 

and conduct MLT&E in a laboratory environment is 

fast becoming a necessity. Additionally, an emphasis 

is shifting Test and Evaluation (T&E) to the left in 

the development cycle to identify design flaws and 

preferred design alternatives prior to full-scale development or shipboard integration. This shift results 

in cost effectively meeting performance goals and 

eliminating corrections after fielding. Although not 

an uncommon approach in some development cycles, 

it has proven to be a less common approach when 

looking across the kill chains and mission threads 

of larger scale distributed weapon system platforms. 

The USS Dahlgren envelopes not just one ship combat 

system variant but all ship combat/weapon system 

variants that can be assembled utilizing the current 

distributed laboratory infrastructure. The entire firecontrol-loop must be developmentally exercised in the 

distributed laboratory environment and not for the 

first time onboard the intended platform(s). USS Dahlgren is the integration of NSWCDD RDT&E resources 

into a cohesive, real-time, deterministic, net-centric 

engineering environment, capable of replicating many 

of the fire-control (FC) systems including, sensor, 

command and control (C2), and weapons systems 

found onboard naval platforms. The unique aspect of 

USS Dahlgren is the ability to integrate and test new 

systems or their modification to existing systems early 

in the development cycle interoperating side-by-side 

with their already deployed counterparts. USS Dahlgren provides the infrastructure for early verification 

that fire-control systems perform in standalone and 

distributed environments, supporting identification 

of integration issues such as timing, data latency, 

and throughput early in the development cycle, in a 

high-fidelity environment. This environment provides 

for continuous testing across the acquisition life cycle, 

off-site connectivity to investigate ship, battle force, 

and joint force integration via Joint Mission Environment Test Capability (JMETC); and the ability 

to replicate the fleet environment to address issues 

observed at sea. 

Recent focus has been on the interconnectivity 

of the existing labs as a means to exercise an overall federated warfighting capability capable of 

supporting multiple mission areas of interest. The USS 

Dahlgren capability spans the six technical departments at NSWCDD as well as corporate networks, 

and includes remote connections via the JMETC 

to include other warfare/system center laboratories 

as well as our sister service laboratories for joint 

experimentation and testing.

USS Dahlgren Components

The USS Dahlgren is composed of the many laboratories that house the developmental and in-service 

systems utilized in the effects/kill chain as found 

on deployed naval platforms (see Figure 2). Integration of the laboratories is facilitated through local 

and global secure network connectivity, modeling 

and simulation environments to generate and exercise the synchronized warfighting scenarios across 

the disparate locations, and representative architectural lay-downs of the effects/kill chain within (and 

between where necessary) platforms that house the 

warfighting capability.

Results from exercising the USS Dahlgren on a 

specific mission thread scenario can then be evaluated 

with other model-driven, experimental, and at-sea 

test data to evaluate the overall performance of the 
 USS DAHLGREN Integrated

Network

Labs

Integrated

Warfare

• Local Connectivity

 - Secure

 - Classified

• M&S

 - System Level

• Architecture

 - Platform and System

 Representative

• Architecture

 - SoS Representative

 - OV - and SV - Specific

• M&S

 - Scenario Generation

• Global Connectivity

 - DREN

 - JMETC

Assessing Warfighter

Capability

• Evaluate Mission Threads

• Identify Capability Gaps

• Assess Engineering

 Design/Improvements

 - “Shift Left” T&E

• Replicate Fleet Environment

Building Kill Chains

DETECT

 CONTROL

 ENGAGE

 ASSESS

WARFARE

ELEMENT

LABORATORIES

(HWIL & M&S)

INTEGRATED

COMBAT SYSTEM

and

JOINT/COALITION

SYSTEM of SYSTEMS

ENGINEERING

ENVIRONMENT

S&T LABS

MSI LABS

GATOR LABS

SPY LABS

COMMAND & CONTROL

GUN RANGE

WEAPONS CONTROL

SEWP / SLQ-32

TOMAHAWK

CEC

RAILGUN

ELECTRIC WEAPONS

UNMANNED VEHICLES

SMP INTEGRATION 

ASSESSMENT

ADVANCED CAPABILITY 

BUILD (ACB) ASSESSMENTS

S&T TRANSITION

MISSION THREAD

EVALUATIONS

JOINT EXPERIMENTATION

INTEGRATION

Table of Contents

130 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

integrated mission capability. In this way, warfighter 

capability can be assessed earlier in the development 

cycle, and warfighter expectations can be demonstrated with high precision while still in development 

(where the cost of modification is minimized) and 

before taking final products to sea.

Mission Level T&E 

Utilizing The USS Dahlgren

There is an urgent need in the naval warfare centers 

and acquisition communities to develop techniques 

and procedures to map mission operational demands 

with system and SoS hardware and software capabilities. This has become increasingly relevant as emerging SoS are required to create, consume, and fuse a 

vast amount of data with the intent of informing a 

future decision maker about an operational scenario 

that has not been fully defined. The MLT&E environment through USS Dahlgren is providing that 

mechanism to postulate how to map an operational 

architecture to the system architectures to investigate 

I&I problems.

MLT&E provides the scientists/engineers a venue 

to obtain technical insights on I&I issues while 

Figure 2. USS Dahlgren - An Integrated Lab Environment
Integration & Interoperability | 131

Table of Contents

Mission Level Test and Experimentation:

Drive Mission Level Performance During Early Development

executing representative operational mission threads 

in a system-of-systems environment. This critical 

information is used to drive system requirements, 

specifications, and architectures in the acquisition 

community which thereby increases confidence in 

early development decisions. In today’s environment, 

systems are tested individually to determine Technology Readiness Levels (TRLs) that focus on the 

maturity of individual systems. This level of evaluation is necessary but not sufficient to field integrated 

warfighting capabilities involving dependencies across 

many systems. USS Dahlgren provides the ability 

to link all effects/kill chain systems to evaluate and 

demonstrate performance across an end-to-end mission thread.

The operational test community evolved the test 

environment from an individual system focus to 

mission-based test. To prepare the 

newly developed systems for operational tests in this mission-focused 

domain, the systems need to be thoroughly tested while being developed 

under those complex mission conditions. This requires good use of 

experimentation resources on the left 

side of the Systems Engineering “V” 

model. Rather than waiting until late 

in the acquisition life cycle to perform 

operational test for the integration 

of SoS, many I&I issues can be eliminated during system development 

to make mission-based test a much 

easier milestone for the transition of 

capabilities to the Fleet.

Lastly, the operational community can leverage the USS Dahlgren environment to 

investigate the performance of an SoS under different 

situations for the development of Tactics, Techniques, 

and Procedures (TTPs) as well as doctrine. These 

non-material drivers provide the details on how to 

operate systems within an operational context while 

working across legacy and new systems. At the same 

time, the operational warfighters get a chance to 

train on new systems under development from the 

beginning so that human systems integration can 

be handled as human systems engineering thereby 

ensuring that the operators’ needs are part of the 

overall design rather than an added piece.

USS Dahlgren Applications

Utilizing a crawl, walk, run strategy, NSWCDD 

began the USS Dahlgren initiative in September of 

2012 with an initial experiment utilizing the secure 

connection of an Aegis Combat System baseline 

located in one building with the Mk 160 Gun Fire 

Control System located in another building and the 

Mk 45 Mod 4 Naval gun on the gun line. The combat 

system generated a firing order, which was passed to 

the fire-control system, and the gun was fired at a test 

target at a range of over 8,000 yards on the Potomac 

River Test Range. As seen in Figure 3, this test was a 

success yielding multiple hits on the target. This first 

experiment exercised command and control, weapons 

control, gun control, river range control and multilevel security through a Cross Domain Solution (CDS) 

between the laboratories.

In 2013, the USS Dahlgren experimentation campaign continued to advance with the inclusion of an 

Unmanned Air Vehicle (UAV) in the mission thread 

providing target information messages and video 

Figure 3. Target Barge, USS Dahlgren Testing
Detect

Assess

Control

Engagement

LAND ATTACK

Cursor on Target / Video /

Battle Damage Assessment

Simulated Battle

Damage Assessment

VMS Messages / Video

Fire Order

Lab 1194 - UAV

B194

Lab 245 - TEAMS

B1000

Lab 196 - NFCS/MK160

B194

Lab 249 - RAILGUN

B1410

Table of Contents

132 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

utilizing cursor-on-target technology. The information passed a target object to the Naval Fire Control 

System in one laboratory, which then, through voice 

command, sent a firing order to the electromagnetic 

Railgun in another facility to engage the target. The 

UAV, still on station, was then utilized to stream video 

for battle damage assessment to a third assessment 

laboratory. A high-level operational view of this land 

attack mission thread is shown in Figure 4.

Currently, NSWCDD, sponsored by the Office of 

Naval Research, is investigating the Railgun as one 

of the next generation electric weapon system development programs. The Railgun uses electromagnetic 

power to achieve kinetic kill capabilities with a significant increase in speed and range and reduced 

cost per shot. The U.S. Navy has committed to testing the Railgun aboard a Joint Homeland Security 

Vessel in 2016; however, before testing commences, 

a significant amount of RDT&E will be performed at 

NSWCDD, to include testing of the Railgun on the 

PRTR. Advancement of the USS Dahlgren laboratory 

integration construct will include early development 

of systems such as the Railgun to assure other critical 

system elements of the mission thread are integrated. 

In addition to the Railgun, the technological 

advancement of lasers as another electric weapon 

system capability is also continuing. The Laser 

Weapon System Quick Reaction Capability (LaWS 

QRC) was tested onboard the USS Ponce (AFSB (I) 

15) in late 2014. This testing serves as the first of what 

is likely to be the eventual wide spread introduction 

of a High Energy Laser system into the U.S. Navy. 

Combat System integration of the laser capabilities 

into warfighting threads will be key in developing 

Figure 4. USS Dahlgren Land Attack Mission Operational View
Integration & Interoperability | 133

Table of Contents

Mission Level Test and Experimentation:

Drive Mission Level Performance During Early Development

and fielding a true directed energy capability at 

sea. In addition to tackling laser integration issues 

during the early phases of development through 

the USS Dahlgren capability, other aspects of the 

effects/kill chain will be exercised. A laser weapon 

system can be utilized as both a weapon and as a very 

high quality sensor for the ship thus able to satisfy 

multiple elements of many mission threads. LaWS 

and its follow-on laser weapon system variants are 

being designed for use against asymmetric threats 

to include small boats and UAVs, both of which 

have been tested in the PRTR environment with 

the USS Dahlgren. 

Railgun and LaWS serve as two prime examples 

of systems early in their development cycle that will 

benefit from being exercised via MLT&E. Mission 

testing via the USS Dahlgren does not simply focus 

on combat system integration, but fully exercises the 

systems end-to-end capability where other non-material aspects of overall performance can be investigated. Non-material aspects of tactics, training and 

human performance can also be evaluated in the USS 

Dahlgren capability.

Challenges and Way Ahead

Under the working capital fund model, Warfare 

Centers receive and are authorized to execute specific 

task orders for particular sponsors. The resources 

required to modify/build infrastructure and laboratories to perform MLT&E are not currently considered 

a necessity in task statements to get the job done. This 

is amplified by a fiscally constrained environment 

where infrastructure investments for extended capabilities are taking a back seat to satisfying immediate 

tactical needs. In addition, it is extremely difficult to 

coordinate schedules across multiple labs which all 

have their individual demands.

It is also a tremendous challenge to fully replicate a 

shipboard environment at sea in a laboratory environment. This operational context can only be simulated 

with the major focus of the MLT&E environment 

devoted to the technical execution of systems across 

an end-to-end mission thread. The involvement of 

warfighters in this environment provides a great deal 

of operational insight while they receive beneficial 

training on the new systems under development.

NSWCDD continues to develop the infrastructure architecture products to accurately represent 

the physical laydown of all assets. This allows leadership to develop a prioritized plan on infrastructure 

upgrades to make the best decisions on impacting 

operational developments with the largest return on 

investment from a military worth and affordability 

perspective. This includes the necessary trenching 

for logical wiring and physical connections required 

across individual labs to represent the functional 

areas of effects/kill chains. In addition, NSWCDD is 

defining the critical mission threads through effects/

kill chain products to provide the logical sequencing 

and connection points of individual systems for endto-end mission thread execution.

Conclusion

MLT&E in a laboratory environment is fast becoming a necessity, and even more so during the early 

phases of development to identify potential I&I issues 

as early as possible. The ability to resolve these issues 

early in the Systems Engineering acquisition cycle provides a better chance of getting the right requirements 

under affordable conditions resulting in integrated 

warfighting capabilities. 
134 | LEADING EDGE January 2015

Table of Contents

Strike Force 

Interoperability Officer (SFIO) Program

By Commander Matthew Carroll, USN

Preparing a Carrier Strike Group (CSG) or an 

Expeditionary Strike Group (ESG) for deployment is a 

complex undertaking from a program level perspective.

Regardless of complexity, successful preparation is 

traditionally bound by three elements: time, cost, and 

quality. The ship’s deployment cycle defines the time 

period of maintenance and modernization to meet the 

ship’s objective as delineated by the government. The cost 

is the component represented not only by the fiscal cost of 

modernization but also by the personnel costs associated 

with both those onboard and those in supporting 

functions ashore to train and certify the group. With 

regard to the quality of the group, it is more than 

the sum of the individual ship’s capabilities; it is 

the ability of those ships and aircraft to operate 

as a cohesive team, i.e., to be interoperable.
Integration & Interoperability | 135

Table of Contents

Strike Force Interoperability Officer (SFIO) Program

Dating back to the late 1990s, technology advances 

in Command and Control, Communications, Computers, Combat Systems, and Intelligence (C5I) capability, coupled with the need to become more interoperable, significantly impacted the surface force C5I 

modernization process. Although several capability 

gains were realized, a number of crippling deficiencies 

emerged as well at both the Strike Group and individual unit level. These increasingly complex and interdependent systems were developed and installed quickly 

as the Navy focused primarily on delivering enhanced 

capability to support commander’s missions and 

objectives. Complicating this process was the decision 

to transition from proprietary, government-designed 

equipment to commercial off-the-shelf (COTS) hardware and operating systems. Consequently, developmental shore-based testing of these new commercially procured capabilities did not afford adequate 

time nor did it represent the shipboard environment 

in which they were to operate. Some of the most 

noteworthy consequences of rapid fielding included 

an Operational Evaluation (OPEVAL) failure of the 

Advanced Combat Direction System (ACDS) Block 

1, the loss of an entire deployment cycle for a number 

of Aegis Cruisers due to a new Cooperative Engagement Capability (CEC) installation, and an additional 

Aegis Cruiser failing an 

operational test of the 

Advanced Tomahawk 

Weapon Control System 

(ATWCS). These failures 

resulted in system reliability falling well below 

design specifications for 

proper operation and 

employment. 

Although the U.S. 

Navy enjoyed the reputation of having the most 

modern and capable fleet 

of ships in the world, they 

were unable to effectively 

work together. The introduction of more complex 

combat systems that now relied on the networking of 

systems to achieve greater capability exposed a flaw. 

The realization was that ships working together in 

a battle group may not be able to operate as a team. 

Shortly after these deficiencies received national 

media attention in 19981 the Chief of Naval Operations (CNO) charged Naval Sea Systems Command 

(NAVSEA) with the central responsibility to address 

battle management and C5I systems interoperability 

problems within the Systems Commands (SYSCOMs) 

and Program Executive Offices (PEOs). Specific tenets 

of this responsibility included the implementation of 

the following: a common warfare system engineering and certification process; a codified process for 

defining, controlling, and certifying C5I configurations; interoperability milestones; and earlier testing 

of future systems with a more capable shore-based 

testing network. The goal was to deliver deploying 

assets capable of the highest warfighting readiness 

absent the distractions resulting from interoperability 

failures.

Striving for continuous improvement in warfighting readiness makes system modernization necessary for a number of reasons including being able 

to respond to a newly developed threat such as the 

development of the Close-In Weapons System (CIWS) 

Phalanx Close-In Weapons System (CIWS) 
Table of Contents

136 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

Block 1B surface mode to defeat small boat attacks, 

upgrading a weapon seeker head upgrade to counter 

an adversary’s new anti-ship missile, maintaining 

technological superiority over an adversary and 

thereby enhancing national sea power, and overcoming obsolescence and expensive in service costssuch as the COTS refresh of the Aegis hardware. 

Additionally, modernization processes are sometimes 

required to overcome deficiencies of fielded systems 

that were not apparent during initial test and evaluation activities. These changes vary greatly in their 

complexity and urgency and are more prevalent in 

software; but hardware can be impacted as well. The 

result is a variety of requirements managed by multiple organizations–each with its own asynchronous 

timeline, resource constraints, and quality standards. 

Few of these requirements align easily to the Fleet 

Readiness and Training Plan. Each misalignment 

results in risk to the quality of the deploying group 

and ultimately mission success. 

Although good policy, tight regulation, and robust 

certification applied to interoperability using sound 

systems engineering processes have been in place over 

the past 16 years to rectify the discrepancies, problems 

still exist. A lack of organizational ownership over 

the entire kill chain and the slow pace of rectification 

efforts for interoperability issues identified in test 

and evaluation activities were areas that demanded 

immediate focus. Through the observations of NAVSEA’s SFIO team, broad communication between the 

Fleet and the technical community can significantly 

improve modernization and interoperability issues 

described above. It is in this arena that the SFIO team 

efforts help to coordinate modernization actions and 

provide a valuable service to the Fleet. 

The SFIO team, with officers on each coast and 

overseas, is the primary interface for the warfighter. 

They assist with early identification of interoperability 

issues and advocate for prompt resolution within the 

technical community. The SFIO team helps warfighters identify and track their issues (or risks) throughout the cycle and provides ongoing support while 

deployed. 

Consisting of uniformed officers and project engineers located in major fleet concentration areas, the 

SFIO team is a small, but effective resource for the 

warfighter. The team is managed under Combat Systems Direction Activity (CDSA), an Echelon V command under Naval Surface Warfare Center, Dahlgren 

Division (NSWCDD). Since 2000, this fleet support 

effort has crossed SYSCOM boundaries and involved 

regular coordination with PEOs, Type Commanders 

(TYCOMs), Numbered Fleet Commanders (NFCs), 

SYSCOM program offices, In-Service Engineering 

Agents (ISEAs), Software Support Activities (SSAs), 

Alteration Installation Teams (AITs), Regional Maintenance Centers (RMCs), Class Squadrons (CLASSRONs), Program Managers’ Representatives (PMRs), 

and others. The SFIO’s primary customer is the CSG, 

ARG/MEU staff N6 or appropriate leadership on 

independent deployers.

Figure 1 illustrates the cycle of interaction with the 

Fleet at key periods in the Fleet Response Plan (FRP). 

Commencing with a C5I status brief delivered prior 

to planned modernization availabilities, the SFIO 

team highlights planned capability improvements and 

identifies potential interoperability and modernization issues for all ships and warfare areas.

The SFIO executes update briefings with ship representatives throughout the FRP and maintains contact 

with key staff to facilitate emergent issue resolution. 

Sometimes the fleet interaction role is as simple as 

assisting a ship with a troublesome equipment casualty by providing the support network contacts, or as 

complex as influencing the fielding plan for a carrier 

availability. The latter was achieved recently for the 

one of the CSGs where the carrier was scheduled to 

deploy without the Accelerated Midterm Interoperability Improvement Program (AMIIP) upgrades. The 

upgrade consisted of a series of software updates to 

the host combat system and other interdependent 

systems that allow for improved coordinated tactical 

picture compilation. The improvements are often best 

described as an “all or nothing” upgrade. 

Synchronization of program test, certification, and 

fielding plans allowed completion of this important 
Integration & Interoperability | 137

Table of Contents

Strike Force Interoperability Officer (SFIO) Program

upgrade on the cruiser and all of the destroyers in 

the deploying group. Each responsible organization 

worked its individual program’s resources, testing, 

and fielding plans with good systems engineering to 

meet the policy guidelines for each platform of system 

within their purview. When viewed holistically, it 

became apparent that raising the priority on the carrier would result in a vastly improved capability to 

the warfighter during the upcoming deployment. The 

SFIO team successfully advocated for the change to 

go ahead on behalf of the strike group and thereby 

improved the probability of mission success. 

Interoperability issues pose a potential risk to the 

quality of the deploying group. By acknowledging 

the risks, the Navy can assess, track, and manage 

interoperability using traditional treatment methods. Treating risk through elimination, i.e., including AMIIP on the carrier, removes the possibility 

of mission failure by eliminating the risk of a poor 

tactical picture. 

Management of interoperability risks across C5I 

systems requires a range of traditional risk control 

measures. Examples of these controls include: “substitution” as an appropriate control for a software 

installation that is rolled back to a previous version 

following discovery of significant issues post roll-out, 

and “engineering” as an appropriate control which 

might entail a minor software change to a host combat 

system software that prevents certain Variable Action 

Button action from placing the combat system into 

Figure 1. Strike Force Interoperability Officer Fleet Response Plan Interaction
Table of Contents

138 | LEADING EDGE January 2015

OPERATIONAL INTEGRATION & INTEROPERABILITY

an unsafe mode. This is not an ideal treatment, but 

suitable, most times, as an interim measure. When 

changes to hardware or software cannot be implemented immediately, due to schedule or budget constraints, “administrative” and “behavioral” controls 

are put in place in the form of Tactics, Techniques, 

and Procedures (TTPs). 

TTPs or workarounds for a system limitation are 

at best, temporarily effective; the ultimate goal should 

be to design out the problem. Unfortunately, in the 

real world we are constrained by resources and must 

determine if the investment to achieve such a solution 

is warranted. Often, training people in the techniques 

to limit the impact of the interoperability issue and 

provide an awareness and education to the operational 

commander and the team is the most cost effective 

solution.

Another significant factor in the treatment of 

interoperability risks is an understanding of the 

context in which the risk may present itself. The 

Strike Group Interoperability Capabilities and Limitations (SGI C&L) team, managed out of NSWC Port 

Hueneme, maintains a database of the known issues 

for each unit and potential strike group combination. 

This data is a valuable resource to the warfare commander in managing the risk that interoperability 

poses on deployment. Being at sea on an operational 

deployment creates a dynamic environment, and the 

commander continually evaluates the mission risk 

based on changing operational circumstances. Consequently, the interoperability risk must constantly 

be re-evaluated and the strike group optimized to 

manage that risk. This is particularly relevant when 

the composition of the strike group changes during 

the deployment as a result of the incorporation of an 

independent deployer or loss of a unit due to a significant defect. In addition to the SGI C&L database, 

for timely analysis, the ability to reach back to the 

shore support organizations is an important service 

provided by the SFIO team. With links and a wide 

network of relationships across SYSCOMs and PEOs, 

the team can help the Fleet “connect the dots” on 

interoperability or support issues that might otherwise distract the deployed Sailor from the mission. 
Integration & Interoperability | 139

Table of Contents

Strike Force Interoperability Officer (SFIO) Program

The ultimate goal of course, is to mitigate the impact 

of interoperability issues and risks to shorten the 

feedback loop between the fleet and the technical 

communities. This dynamic knowledge gathering and 

analysis becomes even more important as the continual constrained fiscal environment forces heavier 

reliance on integrated joint and coalition groups.

So where do we stand today? The message from 

the CNO in 1998 charged NAVSEA with the central 

responsibility for coordinating the resolution of C5I 

interoperability problems within the Fleet. Despite 

the establishment of robust policy and regulation 

that was imposed on the acquisition community to 

consider the interoperability of C5I systems during 

system design and integration, problems still persist. 

Perhaps the greatest achievement since the 1990s is 

the acknowledgment of the complex integrated fleet 

and the need to manage rather than solve interoperability. The dynamic operational environment in 

which the Navy is required to raise, train, and sustain 

surface strike groups around the world will always 

result in capability gaps and incompatibilities. The 

goal of the SFIO team is to educate, mitigate, and 

advocate for the best possible outcome that reduces 

the interoperability risk for the warfighter in a way 

that provides a value-added service to the Fleet and 

respects the fiscal and programmatic challenges the 

technical community faces. 

References

1. http://www.highbeam.com/doc/1G1-68454737.html
140 | LEADING EDGE January 2015

Throughout these articles, you have read how we have parlayed our traditional systems engineering 

disciplines that develop highly integrated and interoperable shipboard combat and weapon systems with 

an expanded view to the overall warfighting mission at the Naval and Joint Force levels. A much greater 

focus on “Technical-to-Tactical” is evolving across the Navy Enterprise, and we at NSWCDD are quickly 

ramping up to answer the call for Surface Warfare.

“We turn ships into warships.”

Captain Brian Durant

Commanding Officer

Naval Surface Warfare Center

Dahlgren Division

Table of Contents
January 2015
WARFARE CENTERS 

DAM NECK