Air University

Stephen R. Lorenz, Lt Gen, Commander

Air War College

Robert J. Elder, Jr., Maj Gen, Commandant

Stefan Eisen, Col, Dean

Lawrence E. Grinter, PhD, Series Editor

Dennis Rea, Col, Essay Advisor

Air University Press

Shirley B. Laseter, DPA, Director

Jeanne K. Shamburger, Content Editor

Sherry C. Terrell, Copy Editor

Ann Bailey, Prepress Production

Daniel Armstrong, Cover Design

L. Susan Fair, Illustrations

Please send inquiries or comments to

Editor

The Maxwell Papers

Air War College

325 Chennault Circle, Bldg. 1401

Maxwell AFB AL 36112-6427

Tel: (334) 953-7074

Fax: (334) 953-1988

http://www.au.af.mil/au/awc/awcgate/awc-mxwl.htm
AIR WAR COLLEGE

AIR UNIVERSITY

F-15A versus F/A-22 Initia

Operational Capability

A Case for Transformation

WILLIAM H. MOTT V

Lieutenant Colonel, USAF

Air War College

Maxwell Paper No. 36

Air University Press

Maxwell Air Force Base, Alabama

November 2005
ii

Disclaimer

Opinions, conclusions, and recommendations expressed or implied within 

solely those of the author and do not necessarily represent the views o

University, the United States Air Force, the Department of Defense, or any o

US government agency. Cleared for public release: distribution unlimited.

This Maxwell Paper and others in the series are avail

able electronically at the Air University Research Web

site http://research.maxwell.af.mil and the AU Pres

Web site http://aupress.maxwell.af.mil.
Foreword

The F/A-22 Raptor is a new weapons system replac

the F-15C Eagle. Its operational debut in 2005 comes a

time of constrained budgets, a changing global threat en

ronment, and the ongoing global war on terrorism (GWO

Because of the current security environment, the aircra

more than 15 years of development, and the close scrut

of the F/A-22’s test and evaluation (T&E) program, the 

Air Force needs the Raptor’s initial operational capabi

(IOC) status to be successful. One means of achieving t

is to recognize and implement the lessons learned fr

America’s current air superiority fighter, the F-15 Ea

And just perhaps the overall effect might be to challen

the US Air Force’s approach to major weapons-syst

development.

The F/A-22’s development, testing, and IOC declarat

at Langley AFB, Virginia, in December 2005 closely paral

the F-15A’s experience of 29 years ago. This paper provi

background information on both aircraft, their T

processes, and their first operational assignments 

Langley AFB. Comparisons are made, differences hi

lighted, and recommendations offered. While it may app

that everything about the F/A-22 is new, the path to 

IOC is well worn.

A fresh look at F-15A development yields valua

insights for emerging twenty-first-century USAF weapo

systems. Surprisingly, the overall lessons of the F-15A 

1976 still apply to the F/A-22 of 2005.

The 1st Fighter Wing is again tasked with birthin

major weapons system, and the difficulties of 29 years 

will again be met and overcome with the proven success

combination of leadership and teamwork. However, t

paper not only focuses on the 1st Fighter Wing’s path 

IOC but also considers the USAF’s T&E process.

Comparison of the two T&E processes has identif

that the F-15A and F/A-22 used the same model for T

(developmental testing and then operational testing). T

programs faced milestones that were used to keep the p

grams on track and aligned with the contracts. Both h

to react to changing oversight direction, both need

iii
greater operational involvement in the early phases of te

ing, and both completed their initial T&E processes imm

diately before IOC at Langley AFB, with further T&E to foll

Both programs introduced significant changes to the T

process, but striking similarities after nearly three deca

could point to a need for major transformational change 

the USAF’s T&E process.

In addition, the author identifies specific recommen

tions that could improve the IOC of new weapons system

Some are relatively intuitive: units declaring IOC sho

gather data from other units undergoing the same proce

capture the experience in the form of histories, and ma

an effort to get the aircraft “on the road” to better introdu

it to the Combat Air Forces (CAF). A suggestion to imp

ment an IOC advisory board to keep senior lead

informed of actions and benefit from senior officer exp

ence also may have merit. 

Recommendations such as improving the handoff betw

testers and operators or redefining the meaning and proc

of IOC are far-reaching suggestions, and their implem

tation would have an impact on organizations, process

and funding that could transform the way the US

acquires new weapons systems. Finally, Colonel Mott 

clearly on target when he concludes that the initial Rap

units, whether test or operational, must capture, challen

and doctrinally address the operational mind-set for empl

ing the F/A-22. F/A-22s should not simply use F-15 t

tics; the F/A-22’s transformational nature must be n

tured and exploited to provide maximum combat effect 

the war fighter.

Surprisingly, 29 years have not completely changed 

nature of bringing a new weapons system to IOC 

Langley AFB. The repetition of similar challenges such 

T&E timeliness and completion, reaction to chan

directed by oversight, assumptions about sortie product

rates, maintenance factors, and technology choke points 

combine to call for transformation of the USAF path 

weapons-systems IOC. This study calls for transformatio

change to be considered within either the T&E process 

the buildup to IOC such that IOC declarations of fut

weapons systems might be improved. By doing so, future w

fighters will get a better weapons system at IOC than th

iv
have either in the past or currently. This study sho

encourage a critical look at the F/A-22’s road to IO

prompting thinking on changes that might be made 

other new weapons systems like the Joint Strike Fighter 

CV-22.

As with all Maxwell Papers, this research is provided 

the spirit of academic freedom, open debate, and serio

consideration of the issues. We encourage your respons

v

ROBERT J. ELDER, JR.

Major General, USAF 

Commandant, Air War Coll
About the Author

Lieutenant Colonel Mott is assigned to Headquarters 

Education and Training Command, Randolph AFB, Tex

He is a command pilot with more than 3,000 hours in 

F-15C. An F-15C instructor pilot, Colonel Mott is a US

Weapons School graduate and former squadron command

His assignment prior to Maxwell AFB, Alabama, w

deputy group commander, 33d Fighter Wing, Eglin A

Florida. He has had operational tours at Nellis A

Nevada; Langley AFB, Virginia; and Bitburg Air Base, W

Germany. In addition, he served as a J-3 action offic

Headquarters North American Aerospace Defen

Command, Peterson AFB, Colorado. Colonel Mott is a d

tinguished graduate of the US Air Force Academ

Colorado. He holds master’s degrees from Embry-Rid

Aeronautical University, the US Army Command a

General Staff College, and the Air War College (class 

2005).

vii
F-15A versus F/A-22 Initial

Operational Capability

A Case for Transformation

You ask me a question about . . . air-to-air tactics. All I can say

is that the cardinal rules of air tactics haven’t changed one bit

since World War I . . . formation integrity, good eyesight, aggressiveness, training, of course, a plan of action helps, integrity of flight. . . . All these things are well known to fighter

pilots. Unfortunately, they are not so well understood by other

members of the Air Force or by people who sometimes design

airplanes and buy them.

—Col Robin Olds, Commander

—8th Tactical Fighter Wing

—12 July 1967

Although air-combat tactics have not changed dur

the last 100 years of manned flight, aircraft certainly ha

The purpose of air-superiority fighters remains the sam

to maximize the warrior’s ability to kill the enemy in the 

and achieve the critical war-fighting enabler of air do

nance. However, aerospace design, testing, and engineer

development have combined to make the modern jet figh

a more complex and costly machine than ever before.

In 2005 we see the dawn of a new fighter—one tha

specifically designed to dominate in every air battle, to p

vide the United States an undeniable ability to operate o

enemy territory, to exploit an enemy’s airspace, and 

hold his entire nation at risk. That aircraft is the Lockh

F/A-22 Raptor. At a program cost of $72 billion in fis

year 2004 dollars, an F/A-22 costs twice what an F-1

costs in constant dollars during the later stages of its p

duction and at a production rate half that of the F-15.1

The F/A-22 is a multibillion-dollar acquisition progr

that is the center of controversy ranging from within 

Department of Defense (DOD) to the United States Hou

and Senate. The F/A-22 has the technology to dominate 

air combat but at what cost? Is the aircraft an overpri

Cold War relic? Is it worth the investment in light of ev

increasing costs associated with the current global war 

terrorism (GWOT)? Even as late as December 2004 

total buy is uncertain, according to a defense consulta

“If the Pentagon’s reported plan to reduce F/A-22 orders 

1
about 160 planes is approved, production cuts would st

taking effect after the 2007 budget year and product

could end by 2010.”2 The military and political debate s

rounding the F/A-22 has produced tremendous interes

its acquisition, oversight, and testing programs. Yet, h

different will the first operational squadron of F/A-22s 

2005 be from the last grand opening of a squadron of a

superiority fighters at Langley AFB, Virginia?

In 1976 the 1st Tactical Fighter Wing (TFW) declared 

tial operational capability (IOC) in the 27th Tactical Figh

Squadron (TFS) with the McDonnell Douglas F-15A Ea

Like the F/A-22, the F-15A was—and still is—an o

standing air-superiority fighter. It was a generational le

in technology (beyond the F-4 Phantom it replaced) and 

center of a new, if not controversial, acquisition process. T

similarities are too great to evade the question: has 

USAF learned anything from history? Or, are the F/A

pilots of 2005/2006 going to relearn the same lessons 

1976?

A complete comparison of the F-15A’s and the F/A-2

acquisition programs is beyond the scope of this pap

This paper’s focus is first on the background events t

F/A-22 Raptor USAF P
led to the development of the F-15A air-superiority figh

It then considers the F-15A’s test and evaluation (T&

process and buildup to IOC at Langley AFB in 1976 a

repeats this evaluation for the F/A-22. Finally, it reco

mends adjustments to the F/A-22’s path to full ope

tional capability (FOC) and for the Joint Strike Fight

(JSF) IOC.

The declaration of IOC for both aircraft heralded a n

era in American fighter technology and capability and i

critical point for review. By distilling the development proc

of the F-15A to its culmination as a weapons system rea

for combat and comparing it to that of the F/A-22, i

possible to assess the F/A-22’s efficacy at IOC. Obviou

weapons systems introduced in 1976 have matured w

time in the field, and the same is true in 2005 with 

F/A-22. But should not this information age we live in p

duce a more complete, more capable system at IOC th

we had 29 years ago?

F-15A Eagle Acquisition Process

Procurement of the F-15A weapons system was a d

matic change for the USAF. Behind lay the tragedy of 

Vietnam War, the abortive attempt in the F-111 to bu

one fighter for both the US Navy and the USAF, and 

costly (and controversial) C-5A Galaxy acquisition proce

Ahead lay the F-16 low-cost, multirole fighter; the A

F-15A Eagle USAF P
tank-busting attack aircraft; and the on-again, off-ag

B-1 strategic bomber. The lessons learned with the F-1

would serve as a guidepost as the Cold War matured in 

1980s. The F-15A’s format for development, engineeri

and even testing would leave its mark on the future F/A

program.

The first step to understanding the F-15A’s acquisit

process is to examine the decision to build a fighter wit

single mission. What drove the USAF to write a requ

ment for and Congress to fund the development of an a

superiority fighter? The answer is found in the history 

the USAF and its combat experiences of World War 

Korea, and Vietnam. During World War II, the Army 

Corps started with a doctrine of ground attack: “Uni

States War Department Field Manual 31-35, ‘Aviation 

Support of Ground Forces,’ 9 April 1942, allocated aircr

resources to ground units and placed pursuit aviat

under Army control. It specifically prescribed that a

power’s mission was to attack ‘the most serious threat 

the operations of the supported ground force.’”4

This apportionment of fighters to ground command

contributed to defeat at Kasserine Pass and resulted i

change in doctrinal organization in January 1943 that w

modeled after the Royal Air Force.5 Suddenly, new doctr

ranked “the gaining of air superiority . . . the first requ

ment for the success of any major land operation, plac

this function ahead of tactical air’s interdiction and cl

air support missions.”6 From North Africa, the role of 

fighter continued to be focused on air superiority by su

porting the USAF’s daylight precision-bombing campa

against Germany. The influence of the P-47 Thunderb

and P-51 Mustang on the air war in Europe is well kno

and ultimately led to the establishment of air superio

and the declaration of the supreme commander, G

Dwight D. Eisenhower, that “if you see fighter aircraft o

you, they will be ours.”7

The importance of air superiority continued in the Kore

War as the advent of the MiG-15 upped the ante for te

nological supremacy over the Korean peninsula. A harbin

of air superiority’s importance came when B-29 atta

were operationally restricted, even to the point of fly

only at night due to their being shot down by MiG-15
The air war over “MiG Alley” and the Yalu River produ

fierce fighting between the F-86 Sabre and the MiG-

The USAF’s F-86 Sabre proved equal to the task, u

mately achieving a kill ratio of 10 to one.9

The war in Vietnam saw the critical move away from a

superiority fighters like the F-86. The 1960s saw a rise 

the “bomber generals” that focused Tactical Air Comma

(TAC) on nuclear delivery and a gradual dulling of 

skills of fighter air combat, commonly known as “dogfig

ing.” Hanson Baldwin writes in the introduction to Th

Ridge that “there had been too much dependence on n

clear weapons and ‘the bomber generals’ in the Air Fo

had long down-graded the tactical air arm.”10 Worse, 

procurement of aircraft resulted in weapons systems 

suited for the conventional war over North Vietnam. 

Walter Boyne, USAF, retired, observes that “out of the 8

production aircraft, some 350 F-105’s were lost to com

or other operational causes.”11 Obviously, all F-105 co

bat losses cannot be attributed directly to its design a

suitability for combat over North Vietnam, but the sh

number of F-105 losses is indicative of problems with 

aircraft’s operational record.

While the F-4 Phantom did achieve limited air superio

over North Vietnam, it was still not perfectly designed 

air combat in that it was missing a gun, a fact that was 

lost on Colonel Olds in the skies over Hanoi: “I gnash 

teeth in rage to think how much better this wing co

have done had we acquired a gun-carrying capability e

lier.”12 Fighter pilots in Vietnam found themselves in 

shooting war with a motivated foe, and they didn’t have 

right aircraft for the mission. That dissatisfaction with 

air war in Vietnam led to a great many changes in 

USAF in the 1970s and 1980s. The desire to have a su

rior dogfighting aircraft led to the design of the F-15A:

Although air superiority remained the “prerequisite” for conductin

any air operation, General Bruce K. Holloway, the Air Force Vic

Chief, wrote in 1968 that plans to develop a new day fighter wer

continually sacrificed in favor of interceptor and fighter-bomber de

signs: “Penetration was more important than maneuverability, ord

nance load-carrying capability more important than armament, aler

status more important than sustained sortie rates. The tactica

fighter became less and less an air superiority system and mor

and more what was once called an attack aircraft.”13
By the late 1960s, the stage was set for the resurgen

of single-mission aircraft, notably the F-15 and the A-

While the debate between “not a pound for air to grou

versus a multirole fighter continues today, the USAF p

sued a single-mission air-superiority fighter with the F-1

and again with the F-22. This was to last until 2002 wh

the F-22 Raptor was redesignated the F/A-22 by the ch

of staff of the Air Force (CSAF).14

Armed with an understanding of why the F-15A w

built, it is appropriate to look at how it was built. The bi

of the F-15A came at a time when there was great conc

for the ability of the government to successfully run a la

aircraft-acquisition program. According to Lt Col Edgar 

Lewis, USAF, retired, “The Air Force was being sever

criticized in the Research and Development/Program M

agement area because of the F-111 and C-5 problems. T

F-111 cost over-runs, schedule delays, technical difficul

and operational introduction problems have been w

documented in various Air Force and TAC . . . documen

The cost and schedule problems with the C-5 and in p

ticular the wing fatigue problem were in the headlines 

1969.”15 In addition, the F-15 was the technological le

agent for a family of systems developed during the la

decades of the Cold War. In 1973 Col Thomas G. McIner

noted that “the F-15 engine, radar, avionics (tactical el

tronic warfare system, internal navigation system), a

gun development programs will provide a family of syste

that will be available for many other defense requireme

throughout the 1970’s and early 1980’s, but for which 

F-15 program has absorbed the brunt of developm

costs.”16

Concerns for the F-15A program and the technical ch

lenges faced by the engineers can be grouped into th

categories: “blueline” (or streamlined) management, s

tem program office (SPO) and TAC relationships, and c

versus contract. Analysis within these areas not only sh

light on the F-15A’s history but also provides parallels 

the F/A-22. Prior to 1969, DOD systems-acquisition po

consisted of highly centralized management and to

package procurement.17 The change in policy implemen

by the DOD’s senior leaders resulted in decentralized 

thority for programs and a use of prototyping rather th
paper studies to reduce risk.18 This command relationsh

when coupled with blueline management, meant that p

gram monitors could rapidly gain access to senior US

decision makers: “The F-15 was put under the ‘Streamlin

management’ approach, wherein the System Program 

rector (SPD) reported directly to the Commander, Air Fo

Systems Command [AFSC], who in turn, reported direc

to the Air Force Chief of Staff and Secretary and the 

sistant Secretary of Defense (Mr. Packard).”19

Next the F-15A SPO (the developer) and TAC (the c

tomer/user) established formal relationships that allow

TAC visibility into the development process. The efficacy 

these relationships remains unclear. Research indicate

variety of comments. According to Colonel McInerney

1973 “the TAC/F-15 SPO relationship . . . [was] good,” wh

he concluded from an interview with Brig Gen Wa

Paluch, deputy chief of staff for requirements, Headquart

TAC.20 Yet, by 1975 Headquarters TAC had initia

monthly meetings/briefings to improve F-15 SPO/T

communication.21 That same year Gen David C. Jones, 

chief of staff, became involved in the detailed process of 

tablishing “Super-PARs,” or program assessment revie

to assess major weapons systems.22

Further, the History of the F-15 Eagle report that He

quarters TAC generated addressed organizational h

drances to full coordination between TAC and the F

SPO. At Wright Patterson AFB, Ohio, “the senior manage

directors were products of the Green Door/Blackb

(secret/closed access) society. The natural tendency was 

maintain the same closed society, i.e.: don’t discu

progress/problems with anyone outside of a very close k

group.”23 This philosophy prevailed at Headquarters T

as well:

The secretiveness even pervaded the TAC Staff. Problems wer

known and discussed only at the Director/DCS and Command sec

tion levels. The responsible Staff Officer often found out about 

problem by accident, deduction or after it was resolved. The close

loop was insisted on by the F-15 SPO, or the flow of information

would stop. In this manner “bad press” was avoided, but conversel

“good press” was not available to engender support for the F-1

program throughout the Air Force.24
Clearly, coordination was an issue in 1975–76. If t

problem continues today, consequences could be sign

cant. Aircraft are not developed in a short period of tim

technology, management, congressional oversight, lead

ship, and even roles and missions can change from 

time that the Operational Requirements Document (OR

is signed until the aircraft’s first flight. There should b

means to balance program changes versus engineer

commitments during the development process. With

proper controls, costs can increase; even worse, the co

mitment to purchase the entire airframe can erode. W

ness the 2004 US Army decision to cancel the Coman

attack helicopter and then redirect the funds to other 

pects of US Army aviation requirements.25 The F/A-22 h

also been crippled by cost overruns—“cost growth h

been a problem for the Raptor program. Most cost 

creases have been due to government decisions t

changed requirements, delayed development, reduced p

duction rates or cut the size of the planned fleet. Costs a

have been increased by technological challenges that w

not fully anticipated.”26

After streamlined management and TAC-SPO relatio

another important factor in ensuring the F-15A’s succ

was in its funding and contract development. The F-1

program’s contract is full of buzzwords that today are 

miliar to acquisition officers. At the time, they were in

vative and a result of lessons learned from the F-111’s a

the C-5’s program shortcomings. The F-15 would be a c

plus incentive fee (CPIF) for development, and critical su

systems were developed competitively and selected on 

basis of performance in a “fly-off.”27 Milestones—or measu

of performance—were included at significant developm

points to allow for decisions before further development

Cost/Schedule Control System (C/SCS) was implemen

that “provided meaningful performance measurement s

tus by comparing an established budget baseline plan w

actual resources expended.”28 The F-15 program was 

free from difficulties as its F-100 engine had numerous 

velopmental problems early in its history, yet McInern

says in his 1973 research that “this is probably one of 

best contracts ever written on such a complex developm

program despite the cost growth on the engine.”29 It wo
not be an understatement to say that the lessons learn

from the F-15’s acquisition form a foundation that endu

today. As discussed in the next section, perhaps the bigg

aspect of the F-15A’s development program that affec

the IOC was the requirement to “fly before you buy.”

F-15A Test and Evaluation Process

Fly before you buy acquisition means to develop p

gram milestones that demonstrate an appropriate leve

technical and flight-test confidence before decisions 

made to further obligate US government funds.30 Not o

does this develop a flight-test program that supports 

aircraft’s engineering and production, it also develops 

erational insight before the weapons system reaches IO

The different T&E steps occurred at different bases, w

flown by different pilots, overlapped in content and sco

and yielded a four-year process from first flight to IOC 

Langley AFB (fig. 1).

Contractor Developmental Test and Evaluation

Contractor developmental test and evaluation (CDT&

was conducted from first flight on 27 July 1972 to the e

of 1974.31 This piece of the overall development was c

ducted by contractor pilots (McDonnell Douglas Corpo

tion) at Edwards AFB in California. As would be expect

this phase of testing focused on translating the F-15A’s 

signed performance into actual aircraft-handling qualit

While not the focus of this paper, some of the findings d

ing this period resulted in changes (called engineer

change proposals [ECP]) to the raked wingtips, the d

tooth horizontal stabilators, the enlarged speed brake, a

20 other engineering or manufacturing refinements.32

Air Force Developmental Test and Evaluation

Similar to the contractor tests, Air Force DT&E (AFDT&

ran from April 1974 to mid-1975.33 Conducted by USAF 

lots at Edwards AFB, it again focused on the airframe a

the basics of getting the components to work together 

prescribed by the engineers. In this phase of developme
the F-15 SPO took a step back and allowed the Air Fo

Test and Evaluation Center (AFTEC) to take the lead 

testing the F-15A. Contractor and Air Force DT&E p

duced a common goal but with a different focus. The c

tractor focused on meeting the USAF’s requirements wh

at the same time refining and developing production me

ods at the factory. Although the USAF was also concern

with qualifying the design versus contractual requ

ments, it was more concerned with operating the airc

with minimum contractor support.34

A key point to F-15A testing is that CDT&E a

AFDT&E occurred hand-in-hand. They both flew at E

wards AFB, and both had a clear arrangement for test 

jectives and processes. But on 11 February 1971, 

deputy secretary of defense established a new test po

for the Air Force that required testing by the operator

First Flight

Edwards AFB, CA

TAC 1a

Luke AFB, AZ

Eagle 1b

Langley AFB, VA

Contractor Developmental Test

and Evaluation (CDT&E)

Air Force Developmental Test

and Evaluation (AFDT&E)

Initial Operational Test

and Evaluation (IOT&E)

Follow-on Operational Test

and Evaluation (FOT&E)

Air Intercept Missile Evaluation/

Air Combat Evaluation (AIMVAL/ACEVAL)

Langley Initial Operational Capability (IOC)

Langley Operational Readiness Inspection (ORI)

Ready Eagle c

71 72 73 74 75 76 77 78

a TAC 1 was the first F-15 to be delivered to Luke AFB.

b Eagle 1 was the first F-15 delivered to Langley AFB.

c Ready Eagle prepared pilots and jets to go to Bitburg AB, Germany.

Figure 1. F-15A developmental test (DT)/operational test a

evaluation (OT&E) schedule and milestones. (Data comp

from sources cited in notes 3, 4, 33, 49, and 53.)
addition to that performed by the contractor and develo

of the airframe.35

Initial Operational Test and Evaluation

Initial operational T&E (IOT&E) was a critical and 

unique phase in the development of the F-15A. It wa

vital complement to contractor and Air Force testing t

let the operator—TAC—have a say in the evaluation proce

A 1977 AFSC history explains that the “F-15 IOT&E w

jointly conducted with the Air Force/contractor devel

ment, test, and evaluation. This IOT&E was Air Force 

rected, TAC conducted, and AFTEC monitored. The IOT

provided estimates of system operational effectiveness a

suitability and also identified the need for modificatio

early in the acquisition process.”36

Now this description of IOT&E sounds great, but a “pe

ing of the onion skin” reveals a few unmistakable facto

A briefing by Lieutenant Colonel Robbins to the TAC/CC 

1976 summarized the key points of IOT&E: “1) No dedica

IOT&E flights, 2) Fulfill IOT&E objectives to the extent p

sible on Air Force Preliminary Evaluation/Contractor T

missions, [and] 3) Emphasis on basic aircraft handl

qualities, flying and operational qualities.”37

It is remarkable that the F-15A’s IOT&E was an add

to the developmental tests and that it was very limited 

scope. Today we expect IOT&E pilots to fly the jet in 

same fashion as operational pilots will fly the aircraft. T

same was true in 1976 but in a much more limited sen

The following test events for IOT&E demonstrate the s

plistic nature of this evaluation phase at that time:

1) Preflight: Start, taxi, ground operations

2) Takeoff: Flying qualities, gear retraction

3) Climb and en route: Cockpit visibility, formation

4) Maneuvering: Flying qualities, buffet, stall, tracki

performance, offensive and defensive maneuvers

5) Recovery: Approach, landing, rollout

6) Mission equipment: One-man operability38

The list of test items is suggestive. Did taxi and take

receive as much evaluation as offensive and defensive m

neuvers? How complex was this phase of the testing? 
almost seems like an afterthought to the F-15’s test proc

for two reasons. First, the USAF Historical Resea

Agency at Maxwell AFB, Alabama, contained no IOT&E 

port. Second are the comments of Lieutenant Gene

Starbird, the deputy director for T&E, Office of the Dir

tor of Defense Research and Engineering (DDR&E), to 

searcher Colonel McInerney in 1973 that “he would h

liked to see a ‘heavier’ IOT&E phase and the Tactical El

tronic Warfare System was not available for testing prio

the production decision.”39 The research here implies t

the F-15A’s IOT&E was of limited utility because of its a

on nature, the limited focus of the testing, the possi

nonavailability of the results, and the comments of lead

ship at the time. The scope and nature of the next ph

of testing also tend to confirm this hypothesis.

Follow-On Operational Test and Evaluation

Follow-on OT&E (FOT&E) was conducted at Luke A

Arizona, between March 1975 and July 1976. The test 

port’s executive summary notes that the “purpose of 

evaluation was to verify the operational effectiveness a

operational suitability of the production F-15A weapon s

tem.”40 It appears that FOT&E was the major evaluat

used to prepare the F-15A for operational service. Dur

the test, live weapons were employed; maintenance w

evaluated; operational deficiencies were found requir

immediate corrective action; and specific operational a

planning factors were determined, such as cost of own

ship.41 The specific test objectives to measure operatio

effectiveness were more detailed than in IOT&E:

1) Verify the operational effectiveness of the F-15A a

craft in its primary role as an air-superiority fight

2) Verify the operational effectiveness of the F-15A a

craft in its secondary role as an air-to-grou

weapons system.

3) Verify the operational effectiveness of the F-15A a

craft during routine phases of flight.

4) Evaluate the reliability and maintainability char

teristics of the F-15A and assess system availabi

and logistical supportability.
5) Estimate the probability that the F-15A will succe

fully complete representative missions.

6) Determine manpower requirements for operat

and support of the F-15A.

7) Evaluate the cost of ownership for the F-15A.42

As the last step in T&E for the F-15A, a number of 

pects come to light. This test was completed four mon

before IOC at Langley and was concurrent with the pi

training programs for the Langley IOC pilots. It also w

into considerable depth of evaluation versus the IOT&

FOT&E utilized 1,111 F-15A sorties and approximat

900 support sorties.43 Also, over 1,000 bombs were dropp

demonstrating the secondary capability of the F-15A

However, the weapons systems were not fully tested. T

AIM-9L, the major weapons change from the F-4 Phanto

was not available for FOT&E. A recommendation from 

report was that “incorporation of the AIM-9L missile sho

be expedited,”45 and, indeed, this weapon would not 

fired from the F-15A until the end of 1977.46 While FOT

was more detailed than IOT&E, the research conclus

here is that the F-15A weapons system was not tested 

its entirety until nearly one year after IOC.

Illustrative conclusions can be drawn from this revi

The F-15A’s T&E process was remarkable and produce

superior flying fighter. But hindsight from 2005 yie

some interesting observations. IOT&E was added onto 

velopmental testing, and FOT&E was not completed u

four months prior to IOC. Given these developments, h

much information, in a completed form, was available 

Langley’s IOC pilots? The limited nature of operatio

testing and the short time from its completion to IOC m

have posed difficulty in transferring lessons learned to I

pilots. Certainly, this must have resulted in the F-15A 

IOC being less of a weapons system than it would be i

matter of one or two years. The question remains—sho

tactics, techniques, and procedures be developed after I

or before? With the F-15A, they seem to have been dev

oped after IOC. 

The F-15A’s T&E process foreshadows the current syst

but without the detail and time needed for a complete a

timely report. Further evidence of this lies in the execut
of an additional F-15A test called Air Intercept Mis

Evaluation (AIMVAL)/Air Combat Evaluation (ACEVA

AIMVAL/ACEVAL culminated in June–November 19

half a year after IOC.47 The testing was flown on the Ne

AFB, Nevada, test ranges with F-15s and F-14s fight

F-5Es in scenarios illustrative of the projected scenario 

visual air combat in the 1985 time frame.48 And the resu

of the AIMVAL/ACEVAL tests? Some would argue, as t

author and F-15C instructor pilot (IP) does, that the te

in 1977 were the first evaluations of the F-15A’s weapo

system in the configuration and with the weapons tha

was designed to use. However, many saw the actual resu

as problematic:

The AIMVAL/ACEVAL exercises were badly flawed and very expen

sive. The flaws . . . had to do with an unrealistic scenario, incorrec

assumptions on missile capabilities, and somewhat arbitrary rule

on equipment carriage. . . . Due to the aggressive and competitiv

nature of fighter pilots, this quickly changed from a good-nature

evaluation of hypothetical missile concepts to as close as one can

get to combat without actually firing weapons.49

Clearly, the AIMVAL/ACEVAL test was late and imperf

but it was the test that was perhaps needed to close F-1

development before IOC was declared. It serves to summa

the F-15A’s T&E program—magnificent but incompl

and overdue. The F-15A’s T&E needed earlier operatio

involvement (IOT&E) and integration of the AIM-9L into 

aircraft. Ultimately, developmental programs must achi

a balance between testing in the IOC configuration a

meeting production milestones. Unfortunately, when 

developmental program fails to meet milestones, the o

solution is either to slip the IOC date or to finish devel

mental testing after IOC (as AIMVAL/ACEVAL did). 

either case, the operational unit assumes the risk a

must rise to the task during preparation for IOC. The n

section evaluates the 1st TFW’s experience with 

F-15A’s IOC process.

F-15A IOC at Langley AFB,

October 1976

The F-15A’s IOC was the culmination of the Eagle’s 

velopment process that put the weapons system in the w
fighter’s hands. While testing at Nellis AFB would contin

1976 marked a shift in focus from operational testing 

combat/training operations in the field. A number of l

sons are noteworthy regarding this stage of developmen

The actual celebration of the F-15A’s arrival at Lang

AFB—Eagle Day on 9 January 1976—was attended 

members of Congress, generals, and of course, the m

and women of the 1st TFW.50 Comments on record fr

visiting dignitaries testify to the excitement associated w

the arrival of the F-15A. Virginia’s First District congre

man concluded by saying that “every American can brea

a little easier when he knows the Eagle is on the perch

The F-15’s revolutionary impact on TAC was expressed 

Gen Robert J. Dixon, TAC commander, as he spoke to 

1st TFW:

You will set the pace for all who will follow . . . you into the F-15

but, more importantly, you will set the pace for those—like you—

who will be first in the AWACS [Airborne Warning and Control Sys

tem], A-10 and the F-16. You have the . . . responsibility to test th

validity of our research and development of our maintenance concept

and procedures, . . . our tactics and training . . . and to set . . . highe

standards that . . . become the baseline for those who follow.52

During 1975 the 1st TFW’s focus on beddown issues 

volved around three areas—facilities development, w

organization, and initial training.53 The change from 

F-4 to the F-15A necessitated many facility changes, a

the shift to line replaceable unit (LRU) maintenance a

intermediate-level maintenance necessitated changes 

hangars and work centers at Langley.54 A critical aspec

facilities was the “need to coordinate . . . [their] devel

ment . . . with the flow of incoming aircraft, equipme

and personnel.”55 Transition in 1976 from the two-seat 

Phantom to the single-seat F-15A Eagle also required 

change in manning, resulting in organizational chang

Such changes faced by the 1st TFW in 1976 due to the l

of the second aircrew were not duplicated in the F/A-2

IOC and are not considered in this research.

Initial training for both maintenance crews and pil

seems to have been the most critical issue for the wing 

1976. Obviously, aircraft knowledge and training w

achieved via the testing process and the Replacem

Training Unit at Luke AFB, Arizona; they were transfer
to the 1st TFW by means of written guidance as well as 

likely transfer of experienced personnel to Langley A

Pilot training at Luke AFB included the following aspec

Conversion training, F-15, was conducted at Luke AFB by th

555th Tactical Fighter Training Squadron. The training was divide

into two phases, conversion and aerial attack. IPs received a thir

phase. . . . Completion of the phases is concurrent with an 88 hou

academic program comprising 8 classes. . . . Sorties for line pilot

were lowered to 21. . . . Cockpit Procedural Trainers were used t

complete 38 lessons to augment academic and flying training. . . 

For simulator training . . . aircrews had to go to St. Louis, MO.56

But the real training issue, which resided at Lang

AFB, was almost a catch-22 situation between operatio

and maintenance. Operations needed to fly, and fly a 

so as to gain and maintain proficiency obtained from 

Luke F-15 course. But that required jets on station an

steadily increasing sortie-production rate. Paradoxica

the first quarter of 1976 saw a limit placed on sortie p

duction: “The training of the 1 TFW pilots prior to the w

possessing aircraft and slower than programmed aircr

delivery resulted in fewer than required sorties being av

able this quarter. The wing was some 78 sorties short 

meeting the minimums for Mission Ready, Mission Capab

and Basic Proficiency Aircrews.”57 To mitigate the effec

the wing sent pilots to Holloman AFB, New Mexico, for T

sorties and also received loaner F-15s from Luke AFB.5

Looking at aircraft beddown would help in understa

ing the nature of the sortie-generation problem. The f

F-15 for maintenance training arrived in October 1975

the first aircraft for operational use arrived on 18 Decem

1975,60 Eagle Day was 9 January 1976, and IOC was 

clared in October 1976. As F-15s arrived, sortie product

increased from 26 in January, to 93 in February, and 

183 in March.61

But actual sortie production was still beneath plann

production. The reasons for this shortfall—planning 

sumptions, maintenance, and technology—provide insi

when one considers sortie generation with new aircr

Captains Tom Lennon and Jim Wray of the 1st TFW 

scribe the biggest misconception regarding sortie prod

tion as “overcommitment . . . result[ing] . . . from excess

optimism based on data from contractor, developer a

user test programs. Consequently, operations in TAC star
with an advertised capability of 1.13 sorties per aircraft 

day. . . . Initial tasking was based on what was though

be a conservative .9 sortie production rate (SPR). Howev

in practice, a .61 SPR was the best achieved, and even th

only with considerable difficulty.”62 If Langley AFB in 19

experienced a sortie rate 60 percent of that planned, th

many pilots and maintainers watched jets that sat on 

flight line as they fell further behind on the path to IOC

Maintenance was the reason for low sortie rates dur

the final push to IOC. It was further explained that “a

craft reliability in that first year measured by mean ti

between failures (MTBF) was less than one-half that p

dicted. It took considerably longer to repair each failu

and spare parts did not cover the needs.”63 In hindsigh

is logical to expect that repair time would be longer wit

new aircraft and that spare parts availability would li

sortie production. Like sortie rates, a greater percentage 

downtime for maintenance must be allocated when deal

with new aircraft.

Finally, new technology was a contributing factor for 

sortie rates. The F-15A was the lead aircraft under 

maintenance concept of LRU repair at home station:

One of the most painful lessons learned in introducing the F-1

was the avionics automated test station area. The concept include

a built-in (BIT) system in the aircraft to isolate failure to any one o

99 LRU’s. The failed LRU was to be removed and replaced—or if n

spares were available to base supply—removed, repaired, and re

placed. The faulty unit was then repaired in the base-level inter

mediate shop where computer-driven test stations were used t

isolate the problem to a single shop replaceable unit (SRU).64

This was a great maintenance technological idea that c

tinues today, but in the 1976 time frame the practical 

sult was that while Langley could test and repair 92 of 

99 F-15 LRUs, an average of only 48 percent were actua

repaired at base level—the remainder went back to 

contractor for repair because test stations were down 

extended periods.65 Clearly, the LRU maintenance id

captured efficiencies to decrease maintenance downtim

but the overall system then had an unforeseen Achil

heel—test-station failures—that negatively affected IOC

Assumptions, maintenance, and technology combin

to create challenges for sortie production and, ultimat

training limitations for IOC. In a surprising move, the 
TFW’s leadership selected a unique method for prepar

for IOC—deployments.66 The 1st TFW’s F-15As part

pated in four 30-day deployments during 1976.67 Sev

teen days after the first squadron had received 24 aircr

the 27th TFS deployed to Nellis AFB for Red Flag trainin

While assuming risk by taking a new weapons syst

away from home station, the decision to conduct operati

on the road was crafted around three focused reasons:

First, the F-15 represented a much larger advance in air superiorit

capability than had been previously experienced by those associ

ated with the F-15 program and it was obvious that tactics to ex

ploit these capabilities were lagging. . . . Secondly, since trainin

sorties were the most precious resource available, it made sense t

go where sorties were the most productive. The third reason i

somewhat more controversial but, in the opinion of 1st TFW leaders

the F-15 support system was geared in attitudes and pace to th

development and training cycle rather than to operational demands

Early deployments produced beneficial pressures to shift gears an

the support system responded.69

The 1st TFW’s deployment to Nellis AFB was the catal

that overcame the system’s early shortcomings. How 

ployed operations could correct problems with main

nance and training is a significant point. By replicat

combat conditions in a training environment, the Red F

deployment placed an artificial “need” for the F-1

weapons system to perform—the “beneficial pressur

that Lennon and Wray point out. This aspect of the F-15

IOC points to the power of leadership: the 1st TFW sho

have been able to perform better at home station, 

morale and unit pride combined with demanding and re

istic off-station training, became keys to Langley’s s

cessful IOC.

In summary, facilities, training (operations and main

nance), sortie production (assumptions and technolog

and deployability seem to be focus areas for the F-15

IOC. Training and sortie production were the limiting f

tors, while deployments were used as a solution. Was 

F-15A’s IOC a success in October 1976? The USAF h

one squadron IOC with a second to follow in three mont

Even more telling was the wing’s ability to train and g

erate a squadron of jets bound for Bitburg Air Base, W

Germany, by April 1977.70 Clearly, the F-15A’s devel

ment process put “iron on the ramp,” with an ability to 
ploy and train soon after delivery. The F-15A’s T&E was l

and incomplete, but with the exception of the AIM-9L miss

the combatant commander (COCOM) who would use 

F-15A at IOC would have a significant improvement o

the F-4 Phantom. That successful IOC was a result of 

challenges met and overcome by the 1st TFW. Would th

challenges be repeated in the F/A-22’s IOC 29 years lat

F/A-22 Test and Evaluation Process

The F/A-22 Raptor is a great leap in aircraft technolo

It is a Cold War–legacy weapons system that has und

gone careful scrutiny and emerged as a needed paradi

shift for the air and space expeditionary force (AEF) of 

twenty-first century. The Cold War’s end and the review 

the aircraft’s need have combined to forge the F/A-22 i

a complex and comprehensive developmental program:

The overall development goal for the F/A-22 is to achieve a balanc

between performance, survivability, reliability, maintainability, an

affordability. The F/A-22 must have proven lethality to ensure firs

look, first kill in all operational environments, and maneuverabilit

and acceleration to ensure superiority over any known or predicte

threat in the close-in fight. It must have the survivability to conduc

its air superiority missions over enemy territory. It will do this through

a balanced combination of supersonic cruise (without afterburner)

reduced signatures, tailored countermeasures, and maneuverability

The F/A-22 systems must provide the pilot significantly improve

beyond visual range situational awareness using its highly inte

grated offensive and defensive systems. The Air Force will field th

F/A-22 with current generation weapons, but it must be capable o

taking advantage of improved and follow-on weapons.71

The F/A-22’s road to IOC is very similar to the F-15

but with a few marked differences that demonstrate the 

finement of the USAF’s T&E process. The F/A-22’s p

gram is under careful DOD oversight but still attempts 

give the war fighter a capable fighter at the outset of IO

The milestones—decision points—of the F/A-22’s prod

tion and testing phases serve several vital functions (fig. 

First, they serve to keep the USAF, Office of the Secret

of Defense (OSD), and Congress informed as to progr

status. Second, they allow decision makers the means 

ensure that the weapons system is meeting expectatio

during development. This means that the aircraft p

chased not only meets the ORD but also keeps the c
Milestones

Demonstration/Validation (Dem/Val)

Engineering andManufacturingDevelopmentTest Vehicles8 Aircraft

Production DeliveriesSpiral Road Mapa

Beddown

Milestone I 750 aircraft (a/c)Dem/Val Start

Major Aircraft Review

Milestone II (648 a/c)EMD Start

Bottom-Up Review (442 a/c)

Quality Deficiency Report(339 a/c) LRIP Authorization

Operational Test and Evaluation (OT&E)Start (Oct 03)

IOT &E Start (Apr 04)

Milestone III (Sep 04)(High-Rate-Production Authorization)

IOC (Dec 05)

YF-22 First Flight 9/90

YF-23 First Flight 8/90 First Flight EMD

268 Aircraft (381 Objective)

9 Flight Aircraft

(1 Static and 1 Fatigue Aircraft)

T&E/

Dedicated Initial

Operational

Test and Evaluation

(DIOT&E)

PRTV I

Assembly 2

PRTV II 6

Lot Delivery Quantities

10 13 20 22 24 25 26 32 32 32 32

LRIP HRP

2 3A 3B

Jan 03 Sep 03 Dec 04 a Spiral road map is used in acquisition contracts and here refers

to allowing for development of software as the process evolves.

FY 86 87 88 89 90 91 92 93 94 95 96 97 98 99 00 02 03 01 04 05 06 07 08 09 10 11 12 13

Nellis Tyndall Langley TBD

♦

♦

♦

♦

♦

♦

♦

♦

♦

♦ ♦ ♦

♦

▼

▼ ▼

Figure 2

F/A

22 milestones

(From Lt Col Scott Ruflin

HQ ACC DR

F/A

22

Langley AFB

VA

Power
tractor from having to react to changes to contracts or 

pectations by the military. Figure 2 demonstrates how 

F/A-22 proceeded from a first-flight aircraft to a prototy

and then to a high-rate production aircraft off the assem

line. This process yields the following terms: validation 

craft (YF-22A), engineering and manufacturing developm

(EMD) vehicles, production representative test vehic

(PRTV), low-rate initial production (LRIP) vehicles, and, 

nally, full-rate production (FRP) vehicles. These aircr

combine to yield a T&E fleet used for developmental a

operational testing.

Like the F-15A, the F/A-22’s test program was divid

between developmental and operational testing. Key agenc

are the F/A-22 SPO (developer), Air Combat Command (AC

(operators, replaced TAC in early 1990s), and Air Force O

erational Test and Evaluation Center (AFOTEC) (evaluato

replaced the AFTEC). Interestingly, the ties between th

agencies occur early in the process. The F/A-22’s f

flight was in 1990, AFOTEC was involved with developm

from 1996, and operational testing did not begin u

2003. AFOTEC’s final report on the F-22 states that 

“AFOTEC team has been fully integrated with the devel

ment effort from early on to ensure an in-depth und

standing and comprehensive insight in preparing for 

OA (operational assessment). Since 1996, AFOTEC h

supported virtually all F-22 IPT (integrated process team

and working groups.”72

Another “F-22” test program necessity was to determ

how best to evaluate the fighter and attack missions of 

F/A-22: “The [F/A-22’s] . . . IOT&E will focus on the ‘F

air superiority fighter capability of the F/A-22. . . . FOT&

currently scheduled to start in June 2005, will focus 

closing out unresolved and deferred IOT&E issues and 

the ‘A’ or attack capability.”73 Just as the F-15A “morph

into an attack version—the F-15E Strike Eagle—so did 

F/A-22 shed its single-mission façade and became an a

and-space dominance vehicle.74

Developmental Test and Evaluation

Like the F-15A, F/A-22 DT&E remained a contractor a

USAF Combined Test Center mission. The test empha
was on airframe certification and EMD process certifi

tion and modification. The basics of ORD validation w

accomplished in this phase. The F/A-22’s DT&E wa

critical step in the testing of a new weapons system, bu

did have limitations for the operational pilot. AFOTE

final report on the F-22 weapons system also indica

that “it is important to note that this stage of DT&E 

cludes virtually no mission avionics testing because th

capabilities have yet to be included in the test fleet. A

result, data for Operational Assessments includes resu

of F-22 testing from a variety of sources including vario

laboratories, models and simulations, and the Flying T

Bed (FTB).”75

While DT&E had a distinctly contractor/test-pilot flav

the F/A-22’s DT&E did attempt to look forward to ope

tional testing by incorporating an operational pilot. Lt 

David M. “Doc” Nelson, a graduate of the USAF Weapo

School and Test Pilot School, flew the F/A-22 during DT

to assess its Military Utility Testing (MUT),76 designed 

provide “an unprecedented degree of operational insi

into developmental testing with corresponding benefits 

both phases of testing.”77 Colonel Nelson’s evaluation 

the F/A-22’s ability to be ready for operational testing 

viewed such areas as signature, speed, flying qualities, a

refueling handling qualities, agility, and so forth.78 Ho

ever, “MUT was not completed until one month prior 

dedicated operational test. This left no time to implem

any changes based on MUT findings.”79

This brief look at the F/A-22’s DT&E yields three obs

vations. First, as for the F-15A, DT&E remained focu

on airframe and manufacturing development. Second, 

need to “look ahead” to operational testing is recogni

(via MUT) yet remained difficult to accomplish. Finally, 

need to have cooperative and overlapping T&E (develo

to operator) is a necessary part of the acquisition proce

Operational Assessment

The Defense Acquisition Board (DAB) established the 

quirement for an operational assessment to support 

decision to proceed with F/A-22 LRIP in August 200

The F/A-22’s OA ran from January 1998 to Decem
2000 and, using the results of the DT&E missions, u

mately supported the certification of system readiness 

operational testing.81 In supporting the LRIP decision, 

OA evaluated the F/A-22 against “the user’s concept of 

erations (CONOPS), the ORD, and joint employment c

cepts to define F-22 system requirements and the envir

ment in which it is expected to operate.”82 The OA a

looked at the challenges or issues with further operatio

testing and considered the operational environment t

the F/A-22 would operate in. Specifically, the OA conta

the critical operational issues (COI) and key performan

parameters (KPP) (table 1):

F-22 Critical Operational Issues

1. Is the F-22 support package required to meet sortie surge rates in

a global environment deployable with the authorized airlift?

2. Can the F-22 execute the counterair mission in the intended operating environment?

3. Is the F-22 more survivable operating in the intended surface-toair and air-to-air environment?

4. Will the F-22 meet the required sortie surge rate?

5. Can the F-22 deliver air-to-surface munitions (JDAM [Joint Direct

Attack Munition] 1000) during day, night, and all weather operations?

F-22 Key Performance Parameters

Radar Cross Section 

Supercruise 

Maneuver 

Acceleration 

Radar Detection Range Data 

Combat Radius 

Payload 

Mean Time between Maintenance 

Sortie Generation Rate 

Independent Airlift

Reprinted from AFOTEC Final Report 00-27, F-22 Weapons System Operational Assess

for LRIP (Kirtland AFB, NM: Air Force Operational Test and Evaluation Center, 2000), 5

Table 1. F/A-22 critical operational issues and key performance parameters

The COI/KPP product frames the entire OT&E proc

to produce meaningful results. Note that COIs one throu

four were tested in IOT&E, while COI five is held 
FOT&E. (“As directed by SAF/AQ in the Program Mana

ment Document, AFOTEC will defer this COI until Jo

Direct Attack Munition development has been completed

Combined Developmental Test/Operational Test

Like the F/A-22’s OA for LRIP, another aspect of F/A

testing that is distinct from F-15A testing is the combin

developmental test (DT) and operational test (OT) relati

ship. In the F-15A, IOT&E was an addition to the dev

oper’s/contractor’s testing. With the F/A-22 the relati

ship is formal, fulfills a specific purpose, and meets T

10 US Code: “The term ‘combined testing’ means test

conducted by the developmental and operational test

when, because of cost, schedule, or test item availabil

they must share test facilities, data or resources. Co

bined DT/OT serves as a bridge between the DT&E a

IOT&E test programs.”84

The major activities of combined DT/OT testing are s

sor integration, offensive-weapons integration, survivabil

suitability, climatic testing, and MUT.85 Combined DT/

testing meets the Title 10 legal requirements to evalu

separately but still achieve economies. The beauty of 

combined DT/OT testing, when matched with the OA

that it better prepares the evaluators to provide a super

product (in the form of reports and the actual weapo

system) to the operator at IOC. Formal DT/OT integrat

is necessary to reduce cost, minimize duplication of eff

and push operational considerations further forward i

the EMD process; it also clearly defines necessary progr

separations demanded by Title 10: “DT and OT may sh

data and resources; however, OT evaluators must evalu

and report data independently.”86 Finally, it is useful 

have some familiarity with the timelines for developmen

and operational testing (fig. 3).

F/A-22 to F-15C Comparison Testing

An interesting aspect of US fighter development is t

the T&E process is not set in stone. The DOD can inj

changes to the process to meet political, budgetary, 

oversight requirements. An example of this occurred 

1977 with the F-15A’s program when the AIMVAL/ACEV
test was guided by the director of defense research and 

gineering as a joint Navy/Air Force program with the N

designated as executive service.87 AIMVAL/ACEVAL look

at air combat in 1985, and “the results identified adv

tages and deficiencies of current air superiority aircraft 

a fighter sweep scenario and suggested changes in 

search and development efforts, procurement, train

and tactics to meet a forecast highly lethal, all-asp

threat in the visual arena.”88

In the case of the F/A-22, a comparison between 

F-15C and its replacement, the F/A-22, was directed 

an acquisition decision memorandum (ADM) in 1992 

compare effectiveness as a prerequisite for passing 

FRP decision.89 AFOTEC was directed “to compare the m

sion effectiveness of the F/A-22 to the F-15C at the Ne

Test and Training Range (NTTR) with a criterion of ‘t

times better’ under identical open-air conditions.”90

To meet the ADM’s requirements, F-15C and F/A

aircraft flew identical missions with specific methods 

evaluation and performance to prove the F/A-22’s tw

times-better superiority. These missions are representat

of current air-to-air doctrine and included force protecti

high-value airborne-asset attack; and first-look, first-

(FLFK) events.91 This final event is unique in that it focu

on the prime reason for the F/A-22’s superiority and a

enables the evaluators to focus on weapons-system avion

“ACC’s top-level ORD requirement for the F/A-22 is tha

achieve a first look, first shot, and first kill. Therefore, 

▼ ▼ ▼ ▼

1st

Flight LRIP

IOT&E

Certification Milestone

III

Operational Assessment

Dedicated

IOT&E

FOT&E Combined DT/OT

Calendar Year

1997 1998 1999 2000 2001 2002 2003 200

Figure 3. Timelines for developmental testing/operatio

testing. Air Force Operational Test and Evaluation Center, F

Combined DT/OT Requirements Document (Kirtland AFB, N

2000), 1–3.
purpose of the FLFK is to determine the extent to wh

the F/A-22 and F-15C have an opportunity to detect, 

gage, and launch a missile that reaches endgame bef

the Red AI [airborne interceptor] has an opportunity 

launch a missile.”92

Phase One Operational Test and Evaluation

As with comparison testing (CT), Phase One OT

(OT&E[1]) is an ADM-directed adjustment to the F/A-2

test program that demonstrated flexibility and a commit

focus to operational evaluation. While OT&E(1) baselin

the F/A-22’s performance, it also incorporated open

missions and simulator sorties at the Marietta Air Com

Simulator (ACS) facility.93 The point to OT&E(1), howev

is not the specific tests or number of missions flown, b

the attitude that it demonstrates. OT&E(1) was the int

duction, or warm-up, for IOT&E. As such, it demonstra

a commitment to make IOT&E as real, challenging, a

“evaluate-able” as possible: “The IOT&E pilots will fly op

air OT&E(1) and IOT&E trials at the Edwards AFB Ran

and NTTR and then ‘fly’ 240 IOT&E trials in the ACS. I

critical that they stabilize at a high level of F/A-22 fly

proficiency and also be highly proficient in F/A-22 tacti

employment before starting OT&E(1) and IOT&E.”94

As a warm-up for IOT&E, OT&E(1) emphasized that 

F/A-22’s IOT&E was more complex than the F-15A’s (

“add-on” to DT). It was more detailed in scope—more ope

tional and tactical measures of effectiveness than the F-15

evaluation of offensive and defensive maneuvers—a

demonstrated a desire to train the evaluators to a hig

standard than the F-15’s IOT&E or even its FOT&E. T

F/A-22’s pilot, maintenance, and test-support person

were all trained via OT&E(1) before IOT&E.95 OT&E(1) a

established a commitment to enter testing when rea

AFOTEC calls this an active ready-to-test effort, which 

cludes “quarterly reviews of all issues by AFOTEC/CC, 

Force Flight Test Center Commander, ACC Director of 

quirements, AFOTEC Director of Operational Test a

Evaluation, the F/A-22 SPO Director, and others as 

propriate.”96 The mission of OA, CT, combined DT/OT, a

OT&E(1) was to pave the way towards a smooth and eff
tive initial operational test of the F/A-22. This represe

a major change, even improvement, over the F-15A’s T&

Initial Operational Test and Evaluation

IOT&E was conducted between April and August 2004

The test report was not available at the time this resea

paper was written. Missions were similar to OT&E(1), w

some as large as four F/A-22s against eight threat a

craft.98 During the test, efforts were made to ensure t

the test aircraft were configured similarly to product

aircraft: “This includes efforts to eliminate all product

representative differences with respect to effectivene

such as mission planning, systems integration, empl

ment, and interoperability, as well as with respect to su

ability such as those that affect reliability, maintainabil

and availability.”99

However, the aircraft available for IOT&E were not “f

up.” That is, they had limitations to their flight envelop

the program imposed aircraft operating limitations that w

active through IOT&E.100 But again, these restrictions w

not expected to complicate or invalidate IOT&E results 

the impact was accountable by the pilots, and the prod

tion aircraft will have improved G and airspeed capabilit

over the IOT&E aircraft.101

Follow-on Operational Test and Evaluation

FOT&E is the final step in T&E for the F/A-22 en ro

to IOC at Langley AFB in December 2005. FOT&E beg

on 29 August 2005: “FOT&E will focus on air-to-grou

attack and re-testing selected areas and capabilities fou

either before or during IOT&E to be deficient. Also, AFOT

will test and evaluate those combat capabilities that 

not released until FRP, yet still required for full operatio

capability. Deferred testing includes: JDAM, ferry miss

demonstration, and pilot training system.”102

It is notable that FOT&E begins four months before La

ley declares IOC; the inferable impact is that any criti

lessons learned from FOT&E could be simultaneou

passed to the operators as they build up to IOC. But wh

lessons learned might be passed, changes to the aircraf

its avionics advised by FOT&E will not be possible. In 
dition, FOT&E will compete with IOC buildup for so

generation and maintenance priorities. Unfortunately, 

was the F-15A’s FOT&E, the F/A-22’s is late, relative 

IOC. However, it will be “complete” in that the weapons a

avionics in FOT&E match those of the IOC aircraft.

Summary

The various phases of T&E for the F/A-22 combine 

form an interesting picture of the overall process tha

easily compared to the F-15A’s experience. The F-15A’s T

proceeded from contractor, to USAF, to operational test

(add-on initial and then follow-on during the buildup to I

at Langley AFB), and finally to AIMVAL/ACEVAL test

after IOC to satisfy questions from oversight organizatio

The process fielded a capable weapons system at IOC, b

this researcher’s overall impression of the testing was t

it was incomplete and late.

By the time the F/A-22 was developed, the process h

changed. The program was milestone-dominated to supp

production decisions, and in the case of the OA for L

decision, data from developmental testing was used for 

OA. DT&E began the process, economies were achieved 

allow combined DT/OT testing, and operational testing 

curred in the familiar forms of initial and follow-on. B

IOT&E(1) was added as a warm-up to initial operatio

testing, and oversight guidance was honored by CT of 

F/A-22 versus the F-15C. As was the case with the F-1

FOT&E will be accomplished during the final months 

IOC buildup at Langley AFB.

Is the F/A-22’s T&E incomplete and late, as it was w

the F-15? The comparison is striking, yet the answer is 

definite—clearly the F/A-22’s process has yielded a weap

system that has been tested to an operational standard 

exceeding that of the F-15A. The testers and evaluat

have “done more” with the F/A-22 than was done with 

F-15A. The current test program shows the unmistaka

signs of the computer and the information age—more de

in the test plan, more organizational involvement, an

greater depth of evaluation than was possible in the 197

While the F/A-22’s COIs/KPPs do not address every asp
of operational employment, they certainly go beyond 

scope of assessment of the F-15A’s IOT&E and FOT&E

The best conclusion regarding the F/A-22’s T&E is t

it used the same basic model as the F-15A, but with po

tive changes that attempted to push operational test

further forward in the process. However, the same ch

lenges were met, and the same timing/delivery issues w

repeated. But the USAF’s experience of the past 29 year

along with the concurrent march of technology—has p

duced an aircraft with more capability at IOC than 

predecessor. Like the F-15A, the brunt of the work for I

still falls to the 1st Fighter Wing (FW) at Langley AFB. P

haps the real question after comparing the F-15A with 

F/A-22 is whether or not the T&E process has transform

during the last 29 years. That question will be conside

after reviewing the F/A-22’s IOC.

F/A-22 IOC at Langley AFB, 

December 2005

The first F-15s arrived at Langley AFB in Decem

1975, and so too the F/A-22 Raptor is destined to beco

IOC at Langley in December 2005. The buildup to IOC, 

well as issues and concerns, is similar for both aircr

Ironically, the comparison between IOC activities at Lang

F-15C and F/A-22 in formation over Langley AFB, Virginia USAF P
AFB for each aircraft bespeaks a 30-year technologi

gap; unfortunately, similarity remains at the core of b

programs. The critical issues for the F/A-22’s IOC in 20

are virtually identical to those in 1976—facilities, train

(operators and maintainers), sortie production (assum

tions and technology), and deployability.

F/A-22 Facilities Issues

Beddown of the F/A-22 at Langley AFB is closely mo

tored and planned at the wing and major command (MA

COM) levels by means of Site Activation Task Force (SAT

analysis, program management review (PMR) coordinati

and briefings to the ACC commander (COMACC). The P

briefings codify issues as either red, yellow, or green w

respect to their impact on IOC (table 2). The chart in

cates that planners have addressed construction issu

but that they have not identified potential limitations 

IOC. However, a problem that was not experienced with 

F-15A at Langley is the intense classification requireme

associated with the F/A-22 and the squadrons, hanga

and other maintenance facilities needed. This problem 

similar to the security issues associated with the F-1

and the B-2. While not significant, innovative solutions 

part and parcel of the 1st FW’s leadership at Langley A

during the F/A-22 beddown.103

F/A-22 IOC Training Issues

IOC calls for an aircraft, weapons, a support system

CONOPS for combat employment, and the people train

to operate the weapons system. This presents a probl

since the aircraft has to fly in order for both pilots a

maintainers to get training. Training is accomplish

through academic courses, transfer of lessons learn

from test and fighter training unit (FTU) operations, ma

tenance trainers, and simulators. All of these train

methods were in place for the F-15A, but the march of ti

has definitely left its mark—technology has changed 

means of learning and the degree of expertise required. 

way of analogy, replace the F/A-22 with a 2005 Cadill

and compare it to the technology, repair, and train

needed to operate a 1976 Cadillac. The computer is 
largest single technological change, and the simulator i

good example of the differences between the F-15A’s a

the F/A-22’s IOC.

Unlike the F-15A’s pilots, who did not have a simula

to use at Langley AFB until late 1976, the F/A-22’s pil

have two simulators available. The simulator at Lang

AFB was open for use in May 2005, and the ACS facility 

Lockheed in Marietta, Georgia, is also under contract 

pilot training.104 IOC pilots have had access to simula

training at their home station for six months prior to I

and to the ACS facility for nearly a full year. But simulat

Item Current Status Assessm

Low Observable and 

Composite Repair 

Facility (LO/CRF)

Configuration Review Board

(CRB) has some control

problems being corrected

Yellow

Flight Simulator Boeing working electrical

concerns w/their equipment.

Estimated completion date

(ECD)–Dec 04

Yl/Gr

Hush House Repairs Hush House #2 

ECD–30 Nov 04;

Hush House #1 

ECD–24 Dec 04

Green

Weapons Load Trainer 

(WLT) Repair

Awaiting Noise/Exhaust 

Data Report from SPO 

to begin design

Yellow

Missile Storage 

(Munitions Storage

Area [MSA])

Preconstruction meetings 

complete by 1 Dec 04

Green

Aerospace Ground

Equipment (AGE)/

Maintenance (MX)

Storage (MSA)

Preconstruction meetings 

complete by 1 Dec 04

Green

Munitions Assembly 

Conveyor (MSA)

Preconstruction meetings 

complete by 1 Dec 04

Green

Munitions Storage 

Igloo (MSA)

Preconstruction meetings 

complete by 1 Dec 04

Green

Squadron Ops 2 & 3 On Track Green

Table 2. 1st Fighter Wing facilities status

Adapted from Thomas Tinsley, commander, 1st Operations Group, Langley AFB, VA, Power

briefing and notes, subject: 1st Fighter Wing F/A-22 Status of Operations, 1 Nov. 2004, slid
do not replace real flying. In terms of fighter-pilot pr

ciency, simulators augment sorties, and it is in this a

that IOC meets its greatest challenge.

F/A-22 Sortie Production

The key to IOC from the operations perspective is 

have proficient pilots at IOC, and that necessitates 

need to generate sorties.105 IOC requires a combination 

aircraft, trained pilots from Tyndall AFB, Florida (the FT

and proficient pilots. As it did for the F-15A’s IOC, Lang

AFB will receive loaner jets from Tyndall AFB (Luke 

1976) for maintenance training and operations flying

The 1st FW built a plan that used the best data for aircr

delivery from the F/A-22 SPO, determined the proper le

of flying for pilot proficiency (modeled on current fight

training standards), and made sortie-generation assum

tions based on the F/A-22’s performance at Tyndall AF

The 1st FW made some key assumptions that show 

understanding of the lessons of the F-15A. First, the s

ties required for pilot proficiency are adjusted from th

used by a fully operational aircraft. Headquarters AC

Ready Aircrew Program (RAP) is designed to codify 

training-sortie rates needed for proficiency and ther

create planning factors for sortie rates and the ability 

justify flying budgets.107 What Langley has done is al

for a monthly increase in RAP requirements leading to IO

Pilots will receive increasing numbers of sorties per mon

approaching December 2005 (fig. 4). The second key 

sumption is expected sortie rate. In 2004 F/A-22s at Tynd

AFB flew on average 13 times per month—or an aircr

utilization rate (UTE) of 13 sorties per month per aircraft

Currently, the 1st FW’s leadership is programming fo

UTE that starts at five sorties per month per aircraft a

increases to 12 as IOC is approached (fig. 4).109 This allo

for maintenance limitations (training, proficiency, av

ability of tech-order data, etc.) and spare-parts issu

(availability, priority, etc.) with the F/A-22 that are sim

to those of the F-15A. The 1st FW’s leadership is count

on a learning curve that will allow near parity with Tynda

performance by IOC. Failure to reach these planned U

rates provides significant issues for pilot proficiency 
IOC. If sortie-production rates are less than anticipat

then IOC will either be declared with pilots with lower 

ing proficiency or with fewer pilots meeting RAP-manda

levels of proficiency. 

Sortie production is critical to the F/A-22’s IOC. An is

for all phases of T&E, it will likely remain the frustrat

aspect of IOC and the area requiring planning flexibil

Sortie production may even be a constant that cannot 

overcome amongst all new major weapons systems.

Just as planning assumptions translate sortie product

into pilot/maintainer proficiency at IOC, so will the F/A-2

new technology affect IOC. The F/A-22 represents a te

nological leap in combat performance. Its technological 

vances permeate the entire program and create ch

points for sortie production. Two specific areas that aff

IOC at Langley AFB are modifications to the aircraft a

they leave the factory and a specific system called In

grated Maintenance Information System (IMIS). Wh

these issues are not “showstoppers” to IOC, they could 

Buildup ▼

Sorties per Month

Preparation

Minimum Sorties Required

Sorties Available Based on UTE

1st FW FHP

RAP Sorties Required

IO

300

200

100

0

Feb-05Mar-05Apr-05May-05Jun-05Jul-05Aug-05Sep-05Oct-05Nov-05Dec-05

• Minimum sorties required based upon phased plan:

 – 5 sorties per month through May

 – 8 sorties per month Jun–Sep

 – 10 sorties per month (RAP) Oct–Dec for 18 pilots

• Sorties available based upon gradual climb in UTE:

 – Feb through Apr-5, May/Jun-7, Jul/Aug-10, Sep through Dec-12

Figure 4. Langley AFB RAP sortie requirements. (From 

Thomas Tinsley, commander, 1st Operations Group, Lang

AFB, VA, PowerPoint briefing and notes, subject: 1st Fighter W

F/A-22 Status of Operations, 1 Nov. 2004, slide 6.)
come the source for lower sortie rates that, in turn, redu

pilot proficiency at IOC.

The F/A-22’s production line is maturing as the test

proceeds. EMD aircraft serve to refine the production-l

process, and LRIP aircraft allow for incorporation of mo

fications identified as needed during DT&E and OT&E. T

actual incorporation of the needed modifications into 

fleet is, however, problematic. Aircraft are of different l

or production periods and have slight differences requir

careful program management. This problem is eradica

as the aircraft move into FRP and as changes to the a

frame or avionics are minimized, eliminated, or made c

sistent between all fielded aircraft. When modifications 

needed, changes can slow aircraft production and deliv

or require aircraft downtime in the field for modificatio

to be made.

Three such F/A-22 modifications affect the aircraft 

Langley AFB: Lot three Operational Flight Program (O

software changes, infrared countermeasures (IRC

shroud installation, and night air-to-air refueling (NAA

lighting changes.110 The details of these modifications 

beyond the scope of this paper, but the solutions and 

pact on IOC are noteworthy.

The NAAR lighting modification is a solution offered 

response to testing that identified a difficulty with nig

time refueling of the F/A-22 by the KC-135. The solut

is to change the existing lighting and paint scheme arou

the air-refueling receptacle.111 The installation plan is 

accelerate the retrofit kit production and divide the mo

fication work between the factory and Langley AFB.112 T

solution is common in fighter-aircraft production a

modification and uses the terms Group A and Group 

modifications. Group A is the work that can be done 

mediately during production or with parts on hand, wh

Group B is the portion of the modification that can 

completed after retrofit kits are available from the supp

and are usually installed at the operational location. T

Group A/B solution maximizes the factory’s product

time while minimizing the downtime that the operatio

site has to use for the modification. A response to fli

testing, IRCM shroud modification, was also solved vi

Group A/B solution with factory and operational-site 
stallations.113 While this method of fixing a deficiency id

tified in flight testing is commendable and done most e

ciently, it is not without cost.

The addition of these two modifications alone requi

running a modification line at Langley AFB for seven d

a week, using two shifts each, and working 10 hours 

shift.114 This kind of aircraft downtime translates into l

RAP sorties contributing to IOC, greater demand on ma

tenance at Langley AFB, and greater potential for delays 

aircraft are “cracked open” to allow internal modificatio

Realistically, the assumptions graphed in figure 4 co

change in real time as the modifications are made.

The final modification to the F/A-22 at Langley AFB

the avionics OFP change—is estimated to require m

modification downtime and is significant. Estimated ad

tional downtime per jet is 10 days beyond the other mo

fications, bringing the downtime for delivered aircraft 

Langley AFB to 51 days.115 This is a consequential issue 

the sortie-production assumptions contained in the pre

ous figures reflect “only a one month down time for 

OFP mod plan of two jets per month.”116 Assuming t

aircraft delivery significantly increased beginning in M

2005, a 51-day downtime on each aircraft represents 

percent of the available time to use the aircraft for sort

before IOC. Clearly, performing aircraft modifications 

the operational site could prove to be a significant hur

to achieving IOC with the training and proficiency desir

While aircraft modifications during IOC buildup can 

fect schedules, the F/A-22’s technology itself can cre

friction. One such case is the IMIS, which

provides the F-22 with a paperless maintenance environment an

includes areas such as aircraft forms, scheduling, report genera

tion, [and] technical order data. . . . By providing diagnostic and In

teractive Electronic Technical Manual (IETM) data, F-22 IMIS re

duces man-hours needed to service, troubleshoot and repair aircraf

systems. IMIS performance is critical to the achievement of high

sortie rates, minimized aircraft downtime, and maintenance with

minimum support resources.117

Yet the 1st FW identifies this system as undergoing mult

software versions with incompatibility issues with differ

lots of aircraft.118 Perhaps, similar to the avionics test-stat

failures of 1976, the IMIS (or another new technology 
the F/A-22) could be a technology choke point for 

F/A-22 in 2005.

At this point, a direct comparison with the F-15A’s I

in 1976 versus the F/A-22 is instructive. Both progra

faced issues with facilities, sortie production, and te

nology. With both aircraft the 1st FW had to overcome re

tively the same issues. There are, however, some immedi

differences. 

For instance, the F/A-22’s IOC has greater access 

simulators for training but must also deal with aircr

modifications at Langley AFB that produce aircraft dow

time and affect sortie rates. Another area of difference 

the meaning of IOC declaration. Langley AFB expects to 

ceive its 17th F/A-22 by 31 December 2005 and decl

IOC.119 In contrast, in 1976 Langley AFB received its 2

F-15A and 17 days later deployed to Nellis for a Red F

exercise.120 This was in July 1976, and IOC was not decla

until October, with one squadron IOC and another IOC 

December.121 Clearly, troops in 1976 had more exposure 

the F-15A prior to IOC than the men and women at Lang

AFB will have with the F/A-22 in December 2005. Wh

that may seem noteworthy, the key is the definition of IO

According to Headquarters ACC, “IOC decision is based 

providing an operationally credible capability to the co

batant commanders rather than providing a specified nu

ber of aircraft.”122

In short, the 1st TFW in 1976 had more aircraft for m

time before IOC than the 1st FW will with F/A-22s in 20

Did this translate into a “better” IOC than the F/A-22 w

have? It is certainly different, but the greater detail of 

T&E process coupled with the availability of the A

should make for better F/A-22 IOC than was available 

the F-15A. Perhaps the cause for concern lies in the sl

to the T&E program schedule that the 1st FW must u

mately accommodate to avoid a delay of IOC declaratio

Conclusions and Recommendations

The F/A-22’s IOC at Langley AFB in 2005 is a momento

time for the Combat Air Forces (CAF). It represents a pa

digm shift in USAF airpower and, like the F-15A’s arri
in 1976, an opportunity for change that may not be 

peated in a generation. Lessons from the F-15A’s IOC c

be used as a comparison and measure of merit against 

current buildup for F/A-22 IOC. If developmental or I

issues that occurred with the F-15A continue with 

F/A-22, then observations can be readily made—either 

issues are inherent with a new aircraft’s IOC or the T

system has not adapted—or perhaps even failed—to lea

from past lessons. Langley’s lessons of 1976 can eit

confirm or condemn the process now under way in 20

Even beyond the direct comparison between the two a

craft, the Raptor’s IOC could be used as a springboard 

further needed actions. It is conceivable that this compari

might call for transformation in the entire T&E process.

Comparison of the two T&E processes reveals that b

the F-15A and the F/A-22 used the same model for T

(DT and then OT). The programs faced milestones t

were used to keep them on track and aligned with the c

tracts. Both had to react to oversight direction (AIMVA

ACEVAL and F-15C versus F/A-22 CT), needed greater 

erational involvement in the early phases of the test

(add-on IOT&E in the F-15A and a call for greater MUT 

developmental testing for the F/A-22), and completed th

T&E processes right before IOC at Langley AFB. Testing w

more extensive and technology more prominent in fac

tating proficiency at IOC for the F/A-22 than for the F-1

But could more have been done?

An argument could be made that “better” IOC is neede

not an IOC that meets a rigid timeline but one that tr

represents an initial combat capability to the COCOM. P

haps IOC declaration needs to be adjusted to the del

that the T&E process (or the manufacturer) causes. P

haps IOC should be a more flexible target date, one t

moves as the program shifts. Instead of asking the I

base’s leadership to “do more with less,” perhaps the I

declaration date should be allowed to move. Unfortunat

it can be equally well argued that flexibility in the IOC d

laration date is unrealistic because it is the key milesto

that Congress uses to ensure that a program it funds 

implemented as it directs and is completed on time. U

mately, the point to debate is whether or not a hard I
date negatively affects organizational behavior during 

velopment or at the IOC base.

Another idea is to consider transforming the three-s

T&E process to update the model that the F-15A a

F/A-22 used. Surely the record of the F/A-22 does not 

dicate a T&E program that is beyond review. In 2002 

Thomas P. Christie, the director for OT&E in the OSD sa

“I took part in the DAB review in the fall of 1986 that 

proved its [the advanced tactical fighter] entry into Dem

stration/Validation (Dem/Val). More than 15 years and $2

later, we’re still at least a year away from IOT&E and s

eral years from Full Operational Capability for the F-22.

Transformation, defined by the USAF as “a process 

which the Air Force achieves and maintains advant

through changes in operational concepts, organizatio

and/or technologies that significantly improve its w

fighting capabilities or ability to meet the demands o

changing security environment,” is the watchword for 

DOD in the first part of the twenty-first century.124 Co

the USAF’s T&E system also be transformed? Perhaps 

relationship between Air Force Materiel Command (AFM

and ACC is antiquated and needs to be rethought. T

F/A-22’s combined DT/OT to achieve economies hints a

means of getting more out of T&E data. Could not the 

for greater MUT in DT also be an earmark for a need o

new model or construct for T&E? Was the F/A-2

OT&E(1) a fix or a look to the future? 

But just as delaying IOC declaration may be unrealis

so too may changes to the T&E process be impractical 

cause of the enormous bureaucracy and needed rev

that is associated with a modern weapons system. It co

be that acquisition reform or transformation is not poss

because of Title 10 legal requirements mandated by C

gress. However that may be, findings of this paper indic

that ideas for improvement on a lesser scale could be ma

for fighter IOC that can be accomplished at the MAJCO

level—ideas that are more practical than the aforemention

grand ones:

1. Visit the last base to declare IOC. Wings that are 

pecting to transition to a new weapons syst

should avail themselves of the experience of the m
recent base to go through the same process. For 

ample, the 1st FW should observe B-2 operations 

Whiteman AFB, Missouri, and learn as much a

can about the B-2’s IOC process. In all likeliho

certain B-2 lessons learned will be applicable to 

F/A-22 at Langley AFB.

2. Formally hand off F/A-22 from AFMC to ACC. As 

T&E process reaches milestones, decisions are ma

based on available information. In the case of 

F/A-22, AFOTEC produced an operational asse

ment for LRIP with the purpose of giving decis

makers every bit of information and a recommen

tion for the LRIP decision. A document of this fla

could be created and declared due to ACC at so

time before IOC. Certainly, the current coordinat

between the involved commands and agencies 

great. But no document could be located that said

effect, “Langley, these are the problems, issues, a

concerns that you will likely have with the F/A-22. 

The developer and evaluator recommend the followin

SATAFs, PMRs, T&E reports, and transfer of train

personnel are all great steps—however, a formal do

ment/procedure might improve the process.

3. Form an IOC advisory board. Numerous organi

tions (e.g., the US Air Force Academy) make use of 

advisory board that has the ability to look at an or

nization and its issues with the perspective of ex

rience and distance. Such a board might be usefu

helping the IOC wing think “outside of the box” a

offer positive ideas. Granted, people don’t want any

looking over their shoulder, neither do they want cl

oversight of the “in my day we did it this way” vari

by senior officers or extra work for an already ov

tasked unit trying to meet a deadline. But a posit

aspect of an advisory-board model might allow issu

to be rapidly elevated or provide senior officers w

the information they need to remain comfortable w

a project with political oversight.

4. Conduct F/A-22 road shows. Part of IOC needs to 

a campaign to get the new weapons system into 

CAF. Other weapons systems that are going to oper

with it need to know what to expect, and operatio
and strategic planners need to know the capabilit

and differences of the new aircraft. The 1st TFW 

1976 took the F-15A on the road to Nellis AFB f

times before IOC in order to affect maintenance a

training. A secondary effect was to get the CAF 

miliar with the new aircraft. The same could be t

now about the F/A-22. It represents a signific

change in capability, and the tactics of the F/A

will likely revolutionize how the rest of the CAF fig

with it. Certainly, public affairs releases can “sell 

Raptor,” but getting the F/A-22 visible in the fi

and at bases across the United States will exped

the normalization of the aircraft in the USAF. 

5. Capture the data from IOC. Besides histories, the sin

source for the F-15A’s IOC at Langley AFB was writ

by two captains, Lennon and Wray, in the 1st TF

Without their publication, a study of the focused I

issues would be extremely difficult and research 

tensive. As with Lieutenant Colonel Lewis’s History 

the F-15 Eagle, agencies connected with the F/A

and Langley AFB need to take the time now to cre

a document that records the lessons learned fr

development, testing, and IOC of the F/A-22. It is 

teresting to note that Colonel Lewis wrote his hist

in May of 1976 (five months before IOC), and C

tains Wray and Lennon wrote their study in Ju

1977 (eight months after IOC). If anything, this poi

to the need to record the events while the memor

are still recent.

6. Change the path to IOC. While changes to acquisit

and reform for T&E may be unrealistic due to the 

trenched nature of these processes, perhaps chan

to IOC and its definition might be possible. From AC

perspective, “ACC has developed tiered IOC rea

ness requirements to guide and monitor progress 

ward IOC. The requirements are divided into f

main categories: Aircraft Status, Operations, Main

nance & Logistics, and Base Support.”125 The I

definition could be tightened to create specific me

ures of effectiveness. If IOC dates cannot be chan

once determined, then a balance must be struck 

tween flexibility and combat capability provided 
the COCOM. The idea behind tighter IOC requ

ments might serve to focus energies on areas t

truly matter—for example, a checklist of achie

ments that relates to the ability of the new weapo

system to perform in combat:

a. Aircraft status: number of jets on ramp, comp

tion of OFP changes, or hardware modification

b. Operations: training production from the FTU, 

completed deployments (Red Flag and Weapo

System Evaluation Program [WSEP]).

c. Maintenance: training production from AETC, U

rate goals met, maintenance indicator goals m

d. Base support: facilities construction complete, f

operational readiness inspection (ORI) complet

7. Address transformational change. Finally, the last s

of the IOC process might be to step back from the 

tails of replacing an old weapons system with a n

one and determine if transformational change 

needed. If the F/A-22 brings radical change in w

fighting capabilities to the COCOM, then perha

far-reaching change should be implemented. For 

ample, the technology of the F/A-22 may need or

nizational changes in the 1st FW because of n

maintenance-repair ideas. Do maintenance squadr

need to change functions, names, or relationshi

Do the operational concepts of the F/A-22 call fo

change to the F-15’s fighter- squadron organizatio

Does the director of operations’ scheduling shop 

weapons shop need to be changed or augment

Could the Global Strike Task Force concept that 

F/A-22 is part of call for a different wing organi

tion? Maybe air-refueling tankers need to be par

the 1st FW to allow the F/A-22 to execute its m

sion. Does the air-to-ground mission of the F/A

demand that the airspace and ranges around Lang

AFB be changed to better support training for 

F/A-22 mission? What about adversaries to prov

training for F/A-22 pilots? If the F/A-22 is “he

and-shoulders” above other US fighters, then 

current method of conducting dissimilar air com

training might be inadequate for the F/A-22. 

short, if the F/A-22 is truly transformational, th
IOC will not be complete without out-of-the-b

thinking and analysis of how the USAF will use 

F/A-22.

In summary, the approach of the F/A-22’s IOC is no

time to lean back and smile like the parent of a newb

child. The process has been long, yet now is the time 

capitalize, to push out the old ideas of the F-15C, and 

lay the foundation for rapid growth of the F/A-22 a

weapons system that will mature in the next 20 or 30 yea

The changes can be radical—changing the T&E mo

used for the F-15A and the F/A-22, for instance. Or th

can be simple—perhaps merely capturing the history of 

process for future fighters to use. Either way, IOC needs 

go beyond “checking the squares” and determine a mea

of pushing for the transformation that the F/A-22 provi

the CAF. If nothing else, these recommendations indic

that there are still more questions than answers for C

leaders to consider as they plan ahead for the Raptors.

Fighter IOC is a significant event: the 29-year gap betw

the F-15A and the F/A-22 indicates that USAF lead

may face it only once in their careers. The cost and inve

ment alone calls for singular emphasis and insight. Co

parison of the F-15A’s and the F/A-22’s road to IOC 

veals fascinating similarities and differences. In retrosp

too much was similar, right down to the same IOC issu

of training, sortie-rate assumptions, and technology ch

points. In both cases the wing at Langley AFB was the fi

step in the process, and the leadership—separated by 

years—had to make the difference in order for IOC to 

successful. Surely this comparison calls for the USAF’s 

sion of transformation to be applied to how a new weapo

system reaches IOC.

Notes

1. Loren B. Thompson, PhD, Why Raptor? The Logic of Buying 

World’s Best Fighter, White Paper (Arlington, VA: Lexington Institute

July 2004), 14, http://www. lexingtoninstitute.org/docs.

2. Bob Cox, “Production of F/A-22 Could Stop Soon, but F-35 

Far Off,” Star-Telegram, 30 Dec. 2004, http://www.dfw.com/mld/d

news/10529582.htm?1c.
3. Lt Col Edgar M. Lewis, History of the F-15 Eagle (A Corporate Mem

vol. 1, (Tactical Air Command, 18 May 1976), 113. Document is now 

classified.

4. Ibid., vol. 2, 1.

5. Ibid.

6. Ibid.

7. Ibid., 2.

8. William T. Y’Blood, MiG Alley: The Fight for Air Superiority (Wa

ington, DC: Air Force History and Museums Program, 2000), 20, 28.

9. National Museum of the United States Air Force, “North Amer

F-86A ‘Sabre,’ ” https://www.asc.wpafb.af.mil/museum/ac/pg000106.

(accessed 22 Jan. 2005).

10. Col Jack Broughton, USAF, retired, Thud Ridge (Friendswood, 

Baxter Press, 1996), 17.

11. Col Walter J. Boyne, USAF, retired, Beyond the Wild Blue (

York: St. Martin’s Griffin, 1997), 165.

12. Col (later Brig Gen) Robin Olds, USAF oral history interview t

script, 12 July 1967, Corona Harvest no. 0001360, USAF Historical 

search Agency, Maxwell AFB, AL, 43. Document is now declassified. 

13. Lewis, History of the F-15 Eagle, vol. 2, 4.

14. USAF news release, “F-22 Redesignated F/A-22 Raptor to S

Air Dominance Role” (Washington, DC: Secretary of the Air Force, Pu

Affairs, Pentagon, 17 Sept. 2002), http://www.af.mil/mediacenter/p

release_print.asp? prID=123006703.

15. Lewis, History of the F-15 Eagle, vol. 1, 113.

16. Col Thomas G. McInerney, USAF, retired, “The USAF’s Man

ment System for the F-15 Air Superiority Fighter” (master’s thesis, Ge

Washington University, 24 Apr. 1973), 9.

17. Ibid., 10.

18. Ibid.

19. Lewis, History of the F-15 Eagle, vol. 1, 113.

20. McInerney, “USAF’s Management System,” 22.

21. Lewis, History of the F-15 Eagle, vol. 1, 111.

22. Ibid.

23. Ibid., 114.

24. Ibid.

25. “Army Ends 20-Year Helicopter Program,” CNN.com, 23 Feb. 2

http://www.cnn.com/2004/US/02/23/helicopter.cancel.ap.index.ht

26. Thompson, Why Raptor? 14.

27. McInerney, “USAF’s Management System,” 29.

28. Ibid., 34.

29. Ibid., 51.

30. Ibid., 54.

31. Michael J. Gething, Modern Fighting Aircraft: F-15 (New York: A

Publishing, 1983), 13.

32. Ibid., 12.

33. Herman H. Cole, “Design, Integration, and Testing of the F-

attachment 151 to Lewis, History of the F-15 Eagle, vol. 2, secs. 4, 5. 

cret) Information extracted is unclassified.

34. Ibid., 5.
35. Lieutenant Colonel Robbins, USAF, retired, “F-15 IOT&E Fli

for First Wing Release (TAC/DR briefing for TAC/CC),” vol. 2, sec. 4

tachment 149 to Lewis, History of the F-15 Eagle, slides 3 and 18. (Sec

Information extracted is unclassified.

36. History, Air Force Systems Command, 1 Jan.–31 Dec. 1977, 

1, 420. (Secret) Information extracted is unclassified.

37. Robbins, “F-15 IOT&E Flights,” briefing, slide 16.

38. Ibid., slide 17.

39. McInerney, “USAF’s Management System,” 56–57.

40. History, Air Force Test and Evaluation Center, 1 Jan.–31 D

1976, vol. 12, p. xi. (Secret) Information extracted is unclassified.

41. Ibid.

42. Ibid., 5.

43. History, Air Force Systems Command, vol. 1, 421.

44. History, Air Force Test and Evaluation Center, vol. 12, p. xi. 

45. Ibid., xiii.

46. History, Air Force Systems Command, vol. 1, 423.

47. Major Newell, PRPFT, “Point Paper on the ACEVAL Flight Tri

unpublished, 3 Feb. 1981, 1.

48. History, Air Force Systems Command, vol. 17, xv.

49. Dr. Thomas S. Amlie, A Non-Statistical Look at AIMVAL/ACEV

staff study for Dr. Walter B. LaBerge, principal deputy undersecretar

defense, 3 Feb. 1981. (Secret) Information extracted is unclassified. 

50. History, 1st Tactical Fighter Wing, 1 Jan.–31 Mar. 1976, vo

n.p., caption from picture of F-15 landing. (Confidential) Information 

tracted is unclassified. 

51. Ibid., 48.

52. Ibid., 49.

53. Tom Lennon and Jim Wray, Bringing the F-15 to Operati

Readiness (Langley AFB, VA: 1st Tactical Fighter Wing, June 1977), 

54. Ibid., 32, 40.

55. Ibid., 3.

56. History, 1st Tactical Fighter Wing, 1 Oct.–31 Dec. 1975, vol. 1

Document is now declassified.

57. Ibid., 1 Jan.–31 Mar. 1976, 18.

58. Ibid., 1 Oct.–31 Dec. 1975, 11.

59. Ibid., 10.

60. Ibid., 9.

61. Ibid., 1 Jan.–31 Mar. 1976, 18.

62. Lennon and Wray, Bringing the F-15, 4.

63. Ibid.

64. Ibid., 8.

65. Ibid., 9.

66. Ibid., 7.

67. Ibid.

68. Ibid.

69. Ibid.

70. Ibid., 52.
71. AFOTEC Det. 6 Test Report 03-001, F/A-22 Weapons System 

vised Initial Operational Test and Evaluation Plan (Kirtland AFB, 

AFOTEC, 2003), 4.

72. AFOTEC Final Report 00-27, F-22 Weapons System Operati

Assessment for LRIP (Kirtland AFB, NM: AFOTEC, 2000), 12.

73. AFOTEC Det. 6 Test Report 03-001, F/A-22 Weapons System

74. USAF news release, “F-22 Redesignated F/A-22.”

75. AFOTEC Final Report 00-27, F-22 Weapons System, 1.

76. Ibid., 62.

77. Ibid., 61.

78. Ibid., 62.

79. Ibid.

80. AFOTEC Det. 6 Test Report 03-001, F/A-22 Weapons System

81. AFOTEC Final Report 00-27, F-22 Weapons System, 1–2.

82. Ibid., 5.

83. Ibid., 17.

84. AFOTEC, F-22 Combined DT/OT Requirements Document (Kirtl

AFB, NM: AFOTEC, 2000), viii.

85. Ibid., 1–7.

86. Ibid., viii.

87. History, Air Force Test and Evaluation Center, 1 Jan.–31 D

1977, I-1.

88. Ibid., xv.

89. AFOTEC Det. 6 Test Report 03-001, F/A-22 Weapons System

90. Ibid., F-1.

91. Ibid.

92. Ibid., F-5.

93. Ibid., 13. Note that a telephonic interview with and e-mail f

Col Matthew Black, AFOTEC Det. 6/TM, 22 Feb. 2005, indicated that 

240 ACS missions were not flown as planned in IOT&E(1).

94. AFOTEC Det. 6 Test Report 03-001, F/A-22 Weapons System, 

95. Ibid., attachment B.

96. Ibid., 4.

97. Lt Col Scott Ruflin, HQ ACC DR-F/A-22, Langley AFB, VA, “F/A

Initial Operational Test and Evaluation,” bullet background paper, 2 J

2004. 

98. AFOTEC Det. 6 Test Report 03-001, F/A-22 Weapons System, 

99. Ibid., 5.

100. Ibid., 6.

101. Ibid.

102. Ruflin, “F/A-22 Raptor,” briefing and notes, 1; telecom w

Colonel Black, 26 Aug. 2005; and AFOTEC Det. 6 Test Report 03-0

F/A-22 Weapons System, 2.

103. Maj Kevin Dolata, 1st Tactical Fighter Wing, Langley AFB, VA

Lt Col Scott Ruflin, HQ ACC DR-F/A-22, Langley AFB, VA, e-mail, 20 D

2004, with attachment: F/A-22 Initial Operational Capability, bu

background paper, 18 June 2004.

104. Ibid.; and telecom with Colonel Black, 26 Aug. 2005.
105. Col Thomas Tinsley, commander, 1st Operations Group, Lan

AFB, VA, “1st Fighter Wing F/A-22 Status of Operations,” PowerP

briefing and notes, 1 Nov. 2004, slide 2.

106. Ibid., slide 5.

107. The best source for RAP information is the Air Force Instruc

(AFI) 11-2 series of documents. The RAP document for the F-15C is 

11-2F-15V1, F-15–Aircrew Training, 19 July 2004, http://ww

publishing.af.mil/pubfiles/af/11/afi11-2f-15v1/afi11-2f-15v1.pdf.

108. Tinsley, “1st Fighter Wing,” slide 6.

109. Ibid.

110. Ralph Andrews, Mark Parsons, and Charlie Jones, “Lan

SATAF Mods,” PowerPoint briefing and notes, Nov. 2004, slide 3.

111. Ibid., slide 7.

112. Ibid., slide 14.

113. Ibid., slide 20.

114. Ibid., slide 21.

115. Ibid., slide 33.

116. Tinsley, “1st Fighter Wing,” slide 6.

117. AFOTEC Final Report 00-27, F-22 Weapons System, 41.

118. Dolata, e-mail, 20 Dec. 2004.

119. Tinsley, “1st Fighter Wing,” slide 5.

120. Lennon and Wray, Bringing the F-15, 7.

121. Ibid., 6.

122. Lt Col Scott Ruflin, HQ ACC DR-F/A-22, Langley AFB, VA, F/A

Initial Operational Capability, bullet background paper, 18 June 200

123. Thomas P. Christie, director, Operational Test and Evaluat

“Test and Evaluation’s Role in Experimentation” (speech, Southern M

land International Test and Evaluation Association [ITEA] Chapter

Apr. 2002), 4, http://www.dote.osd.mil/presentations/Christie200204

paxspeech.html.

124. The Edge: Air Force Transformation (Washington, DC: Headq

ters US Air Force, Deputy Chief of Staff for Plans and Programs, 30 

2003), 2, http://www.xp.hq.af.mil/xpx/.

125. Ruflin, F/A-22 Initial Operational Capability, bullet backgro

paper.