U.S. GOVERNMENT PUBLISHING OFFICE

WASHINGTON : 

For sale by the Superintendent of Documents, U.S. Government Publishing Office

Internet: bookstore.gpo.gov Phone: toll free (866) 512–1800; DC area (202) 512–1800

Fax: (202) 512–2104 Mail: Stop IDCC, Washington, DC 20402–0001

i 

25–823 2017

[H.A.S.C. No. 115–37] 

CREATING A FLEXIBLE AND EFFECTIVE 

INFORMATION TECHNOLOGY 

MANAGEMENT AND ACQUISITION 

SYSTEM: ELEMENTS FOR SUCCESS IN 

A RAPIDLY CHANGING LANDSCAPE 

HEARING 

BEFORE THE

SUBCOMMITTEE ON EMERGING THREATS 

AND CAPABILITIES 

OF THE

COMMITTEE ON ARMED SERVICES 

HOUSE OF REPRESENTATIVES 

ONE HUNDRED FIFTEENTH CONGRESS 

FIRST SESSION 

HEARING HELD 

APRIL 26, 2017 
(II) 

SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES 

ELISE M. STEFANIK, New York, Chairwoman 

BILL SHUSTER, Pennsylvania 

BRAD R. WENSTRUP, Ohio 

RALPH LEE ABRAHAM, Louisiana 

LIZ CHENEY, Wyoming, Vice Chair 

JOE WILSON, South Carolina 

FRANK A. LOBIONDO, New Jersey 

TRENT FRANKS, Arizona 

DOUG LAMBORN, Colorado 

AUSTIN SCOTT, Georgia 

JAMES R. LANGEVIN, Rhode Island 

RICK LARSEN, Washington 

JIM COOPER, Tennessee 

JACKIE SPEIER, California 

MARC A. VEASEY, Texas 

TULSI GABBARD, Hawaii 

BETO O’ROURKE, Texas 

STEPHANIE N. MURPHY, Florida 

KEVIN GATES, Professional Staff Member 

LINDSAY KAVANAUGH, Professional Staff Member 

NEVE SCHADLER, Clerk 
(III) 

C O N T E N T S 

Page 

STATEMENTS PRESENTED BY MEMBERS OF CONGRESS 

Langevin, Hon. James R., a Representative from Rhode Island, Ranking 

Member, Subcommittee on Emerging Threats and Capabilities ..................... 2 

Stefanik, Hon. Elise M., a Representative from New York, Chairwoman, 

Subcommittee on Emerging Threats and Capabilities ..................................... 1 

WITNESSES 

Greer, Edward, Former Deputy Assistant Secretary of Defense for Developmental Test and Evaluation ................................................................................ 7 

Halvorsen, Hon. Terry, Former Department of Defense Chief Information 

Officer and Department of the Navy Chief Information Officer ...................... 5 

Levine, Hon. Peter, Former Department of Defense Deputy Chief Management Officer and Under Secretary of Defense for Personnel and Readiness .. 3 

APPENDIX 

PREPARED STATEMENTS: 

Greer, Edward .................................................................................................. 46 

Halvorsen, Hon. Terry ...................................................................................... 36 

Levine, Hon. Peter ............................................................................................ 27 

Stefanik, Hon. Elise M. .................................................................................... 25 

DOCUMENTS SUBMITTED FOR THE RECORD: 

[There were no Documents submitted.] 

WITNESS RESPONSES TO QUESTIONS ASKED DURING THE HEARING: 

Mr. Scott ............................................................................................................ 65 

QUESTIONS SUBMITTED BY MEMBERS POST HEARING: 

Mr. Franks ........................................................................................................ 73 

Mr. Shuster ....................................................................................................... 73 

Ms. Stefanik ...................................................................................................... 71 

(1) 

CREATING A FLEXIBLE AND EFFECTIVE INFORMATION 

TECHNOLOGY MANAGEMENT AND ACQUISITION SYSTEM: ELEMENTS FOR SUCCESS IN A RAPIDLY CHANGING LANDSCAPE 

HOUSE OF REPRESENTATIVES, COMMITTEE ON ARMED SERVICES, 

SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES, 

Washington, DC, Wednesday, April 26, 2017. 

The subcommittee met, pursuant to call, at 2:03 p.m., in room 

2118, Rayburn House Office Building, Hon. Elise M. Stefanik 

(chairwoman of the subcommittee) presiding. 

OPENING STATEMENT OF HON. ELISE M. STEFANIK, A REPRESENTATIVE FROM NEW YORK, CHAIRWOMAN, SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES 

Ms. STEFANIK. The subcommittee will come to order. I would like 

to welcome everyone to this hearing of the Emerging Threats and 

Capabilities Subcommittee of the House Armed Services Committee 

on the important topic of information technology [IT]. 

Several of us from the committee recently had a discussion with 

Eric Schmidt, who is the executive chairman of Alphabet, as well 

as the chairman of the Department of Defense [DOD] new Defense 

Innovation Board. That discussion was very helpful in provoking us 

to think about how the Department could do a better job incorporating best practices from industry. And while it is unrealistic to 

think that DOD will ever operate exactly like industry, it is nonetheless imperative that we achieve every ounce of efficiency out of 

our back office and IT operations in order to fully invest in combat 

capability. The challenge is finding that correct balance, and today’s hearing brings us one step closer. Information technology represents over $30 billion of the Department’s total budget; however, too often it is treated as a support 

tool secondary to platforms, weapons, training, and the operations 

of the Department. But anyone who has seen U.S. forces operate 

over the past 25 years understands that our military advantage 

comes from those network systems providing the intelligence, precision strike, information fusion and warning capabilities our warfighters have come to rely on. 

This committee has been focused on reforming the operations of 

the Department for the past 2 years, from streamlining acquisition 

regulations to streamlining an overly cumbersome bureaucracy. 

The end goal of these efforts has been to enable us to buy and develop systems with greater agility and flexibility so that state-ofthe-art tools get into the hands of our warfighters faster. 
2 

I know our witnesses today will help provide us with a better 

framework for understanding how to think about defense management and acquisition practices for information technology as we explore the critical questions before us, such as: What are the characteristics of well-performing programs that we should focus on? 

What leading indicators should we be monitoring to determine if 

programs are going off the rails and before it is too late? And how 

can we better identify, encourage, and reward those program managers that are executing information technology programs well? 

Let me now turn to the ranking member, Jim Langevin of Rhode 

Island, for any opening comments he would like to make. 

Jim. 

[The prepared statement of Ms. Stefanik can be found in the Appendix on page 25.] 

STATEMENT OF HON. JAMES R. LANGEVIN, A REPRESENTATIVE FROM RHODE ISLAND, RANKING MEMBER, SUBCOMMITTEE ON EMERGING THREATS AND CAPABILITIES 

Mr. LANGEVIN. Thank you, Chairwoman Stefanik, and thank you 

to our witnesses for being here today. I definitely look forward to 

hearing what you have to say, given your wealth of knowledge in 

this area. 

So management and acquisition of information technology at the 

Department of Defense is one of the most challenging and pressing 

DOD organizational and administrative issues facing us today. For 

years, Congress, the executive branch, and industry have attempted to bring DOD’s IT programs and processes into the 21st 

century. Despite attempts like the Joint Information Environment 

and streamlining acquisition processes, DOD’s pace to improve its 

IT posture is not progressing with the desired speed to achieve serious efficiencies, increase security, and take advantage of enhanced capabilities that are readily available. For example, the intelligence community has developed and is using commercial cloud 

services, yet widespread use of cloud computing has not yet taken 

place at the Department. 

All that said, I will be the first to acknowledge that the Department of Defense IT management, modernization, and cyber are extraordinarily complex. I will also acknowledge that Congress, in an 

attempt to be helpful, has taken action that has seemingly had the 

opposite effect. 

That is why today’s hearing is so important. IT management and 

modernization experts who served at the DOD sit before us. I look 

forward to their concrete recommendations for improving DOD’s IT 

posture in an expeditious manner, and I hope they provide granularity on legislative, regulatory, and cultural barriers, like risk 

aversity, that are inhibiting the rapid change required. And I am 

certainly glad that they help us understand what is working as 

well as highlight the success that the Department has already had 

in this arena. 

So with that, Madam Chair, I thank you for organizing this hearing. I thank our witnesses for being here today. And I yield back. 

Ms. STEFANIK. Thank you, Jim. 

Today, we welcome three distinguished witnesses, all of whom 

have served as senior officials within the Department of Defense, 
3 

and so they not only understand the government challenges with 

information technology management and acquisition, but also how 

private sector practices can be applied or need to be avoided. 

First, the Honorable Peter Levine, former Department of Defense 

Deputy Chief Management Officer [DCMO] and Under Secretary of 

Defense for Personnel and Readiness [P&R]. Next, the Honorable 

Terry Halvorsen, former Department of Defense Chief Information 

Officer [CIO] and Department of the Navy Chief Information Officer. And Mr. Ed Greer, former Deputy Assistant Secretary of Defense for Developmental Test and Evaluation [DT&E]. 

Welcome to all of our witnesses. I would like to remind you that 

your testimony will be included in the record, and we ask that you 

summarize key points from that testimony in 5 minutes or less for 

your opening statements. 

And, Mr. Levine, I will start with you. The floor is yours. 

STATEMENT OF HON. PETER LEVINE, FORMER DEPARTMENT 

OF DEFENSE DEPUTY CHIEF MANAGEMENT OFFICER AND 

UNDER SECRETARY OF DEFENSE FOR PERSONNEL AND 

READINESS 

Mr. LEVINE. Thank you. Chairwoman Stefanik, Ranking Member 

Langevin, and members of the subcommittee, I thank you for the 

opportunity to appear here this afternoon to discuss DOD’s information technology management and acquisition processes. What I 

would like to focus on, with your permission, is DOD business systems acquisition. 

For years, DOD has been trying to buy new business systems the 

same way that it buys weapon systems and other big ticket items, 

and the result has been a series of sometimes spectacular failures. 

The fact is that there is no such thing as a stand-alone business 

system and no such thing as a stand-alone business system acquisition. The DOD’s business systems are interconnected not only with 

each other, but also with the business processes that they run. 

When DOD wants to acquire a new financial management system or logistic system or HR [human resources] system, it inevitably finds links to dozens if not hundreds of other IT systems 

which either feed it data or rely on it for data. Each of these systems, no matter how outdated, redundant, or ineffective they may 

seem, has a core of devoted users who are likely to resist change. 

DOD tries to ensure that change management and business process reengineering take place concurrently with new business systems acquisitions, but it has not been easy. Acquisition officials 

don’t have the authority or expertise to require their customers to 

do business process reengineering when they buy new systems, and 

management officials responsible for those business processes have 

proven to be incapable of running large acquisitions. 

In the last couple of years, Congress has helped this issue considerably by streamlining section 2222 of title 10, which provides the 

statutory requirements for DOD business systems acquisitions, and 

by repealing requirements for major automated information systems, or MAIS programs, that mimic the acquisition process for 

major weapon systems. 

When I testified with Terry here last year, I promised that the 

Department would take advantage of the new flexibility granted by 
4 

Congress to better coordinate the roles of the DCMO, the CIO, and 

AT&L [Acquisition, Technology, and Logistics] in the acquisition of 

business systems. 

Two weeks after that hearing, I left the position of DCMO to become Acting Under Secretary for P&R, but I am pleased to say that 

streamlining effort went on in my absence and that just a couple 

of months ago the Department issued a new DOD instruction addressing business systems acquisitions. This new instruction appropriately sequences for the first time decision points regarding business solutions, IT solutions, and acquisition solutions so that we 

don’t have these redundant processes going on side by side without 

referencing each other, but we have a coordinated process. 

This is also the first step toward building an IT-specific acquisition process as the Defense Science Board [DSB] recommended almost a decade ago. Consistent with the DSB’s recommendations, 

the new instruction establishes a unique set of decision points for 

business systems acquisitions and requires prototyping and incremental delivery of capability. 

I would have to say, though, that the DSB report, which is a 

2009 report, recommended a unique acquisition system not just for 

the acquisition of business systems, but for the acquisition of all IT. 

It has been a disappointment for those who were here with the 

Congress—I was on the SASC [Senate Armed Services Committee] 

staff at the time that that report came out—that that still hasn’t 

been fully implemented for IT. 

Congress felt strongly enough about this that you enacted provisions on this twice, first in 2010 and then in 2015, telling the Department: Please go look at this DSB report and figure out a way 

to implement the kind of IT-unique acquisition system that the 

DSB is recommending, having looked at commercial practices, and 

figure out how it would best apply to the Department. I commend 

you to look at that report. It is still the right thing to do. It is still 

what the Department needs to do. 

So while I think the Department has made good steps in the direction of change, there is still a long way to go. And one thing I 

would add before I wrap up and hand over to Terry is that the 

issuance of a new DOD instruction like this one I just mentioned 

is the beginning of a process of change, not the end of a process 

of change. 

One thing that I have seen in the Department is it takes every 

bit as much effort to make sure that the instruction, the legislation, the regulation is really implemented, implemented in the spirit it was intended, as it does to enact it in the first place. So it will 

be up to the new team at DOD when it arrives to ensure that the 

new processes are appropriately implemented in practice, and in 

particular with regard to business systems, that the CMO [Chief 

Management Officer] will have to remain engaged throughout the 

acquisition life cycle to ensure that the Department adopts new 

and efficient business solutions rather than tailoring new business 

systems to old and inefficient processes. 

With that, I thank you again for inviting me here today, and I 

look forward to your questions. 

[The prepared statement of Mr. Levine can be found in the Appendix on page 27.] 
5 

Ms. STEFANIK. Thank you, Mr. Levine. 

Mr. Halvorsen, you are recognized for 5 minutes. 

STATEMENT OF HON. TERRY HALVORSEN, FORMER DEPARTMENT OF DEFENSE CHIEF INFORMATION OFFICER AND DEPARTMENT OF THE NAVY CHIEF INFORMATION OFFICER 

Mr. HALVORSEN. Madam Chairman, Ranking Member, distinguished members, thank you for the opportunity to testify today 

before this committee. This is an important topic, and I would like 

to echo much of what Peter just said. Obviously, we did testify together on this topic last year, and I did testify the last 2 years on 

this topic. 

DOD has made progress. They continue to face critical global 

challenges. I do think they will meet those challenges. But it is 

faced with an added and unprecedented dimension: This is arguably the period in history with the fastest developing and most 

complex technology. 

Unlike previous times, the vast majority of this technology 

growth is occurring in the private sector, not originating with the 

government. This means in addition to identifying the right capabilities to meet DOD requirements, DOD must be able to acquire 

and integrate this technology with greater agility. Today’s environment demands more broadly defining capability and not providing 

detailed requirements that dictate solutions. 

At times, the government process today, because of the current 

thinking, delivers legacy solutions. We really do need to think 

about how we talk about capabilities and not dictating a series of 

technical requirements that will be out [of] date the day we publish 

them. 

DOD needs a better understanding of the commercial environment to become more effective and efficient working with industry 

and determining how solutions should be implemented. 

With respect to business systems, DOD really needs to ask itself: 

Should it implement whole commercial solutions or some degree of 

hybrid solutions retaining some government capability? 

I will echo what Peter said, that this also needs to focus on the 

process, and I strongly recommend that the going-in position for 

business solutions until proven wrong through business case analysis is that the government and DOD should be adopting commercial solutions off the shelf. The real question really is what businesses should DOD be directly in and where should it offload those 

businesses to the commercial sector. 

Regarding systems that are more aligned with the primary mission of DOD, such as national security systems, DOD must more 

carefully weigh the mission risk, the mission security requirements, and since these systems are more likely to be operated by 

military and civilian members, DOD must really look at the workforce implications of training and sustainment. 

The new, changing environment also means DOD will be requiring more services from industry as opposed to just buying products. 

To successfully buy services in this exploding technical environment will require DOD to form better partnerships with industry 

and for industry to be more open to sharing technical data with 

DOD. 
6 

To facilitate the building of these critical partnerships, I believe 

this committee and others will need to look at the laws governing 

relationships and contact between DOD officials and industry 

members and expand those programs. The laws we have today 

have been written with great intention, but they are outdated. 

Many of them from a financial perspective still use numbers from 

the 1970s. They don’t make any sense, and they limit the ability 

for DOD and other government agencies to function correctly. 

We also need to look at the way we test commercial products for 

acceptance and security perspective today, and I elaborate this in 

my written testimony. The security accreditation process is costing 

both the government and industry lots of money and doing a disservice to, in the case of DOD, our service members for how long 

it takes to get those products certified. 

We insist on testing stuff that has commercially been accepted 

and tested in industry. We need some legislation that says: Why 

can’t I accept that testing as the official testing? Test can the government operate it correctly, 100 percent, that testing should be included, but we really need to look at how we do the testing. That 

has become one of the long poles in the tent. 

You have to improve efficiency. I will talk a little bit about the 

McKinsey report, about the 125 million. DOD definitely took that 

report seriously, but some of those numbers in there were not well 

thought out in the end. But they should be continuing to refer to 

that report. It does give the right topic areas that DOD should be 

looking at to gain more efficiencies in. 

As Peter, I am happy with the DOD CIO, and the DCMO relationship. I think it is strong today, and I think it will continue to 

get better. That is important, because it is a combination of the 

process and the technology. 

In the end, this is a little bit about a cultural solution more than 

it is laws and anything else. We need to think differently. I think 

we have unintentionally been building for a long time a culture of 

distrust and one that was based on overregulation and a foundational belief that all the players, industry, government, and academia, all needed to be protected from itself, and we have done a 

good job at doing that. 

Somewhere we have lost what was good in the systems we had 

prior. I was quoted at the DOD as saying that our secret weapon 

was our commercial capability and our relationships with industry. 

I would amend that to read our secret weapon is our commercial 

capability, our relationship with industry, and the combined efforts 

of the military, civilian, contractor, and commercial workforce to 

make it all work and deliver the results. 

Thank you, and I look forward to your questions. 

[The prepared statement of Mr. Halvorsen can be found in the 

Appendix on page 36.] 

Ms. STEFANIK. Thank you. 

Mr. Greer, you are recognized for 5 minutes. 
7 

STATEMENT OF EDWARD GREER, FORMER DEPUTY ASSISTANT SECRETARY OF DEFENSE FOR DEVELOPMENTAL TEST 

AND EVALUATION 

Mr. GREER. Chairwoman Stefanik, Ranking Member Langevin, 

and members of the subcommittee, thank you for the opportunity 

to express my views this afternoon. 

In 2013, this same subcommittee hosted a roundtable discussion 

on IT acquisition reform, and it seems like the same challenges and 

the same issues were discussed 3 or 4 years ago. So we do need 

to be mindful of what kind of progress we have made. 

Today, I am going to talk about MAIS [major automated information systems] challenges, best practices, the role of developmental 

test and evaluation, and business systems versus tactical weapon 

systems. 

MAIS challenges. The challenging nature of MAIS acquisition 

can be attributed to many factors. I will briefly discuss three. 

Program complexity. Typical MAIS programs have to be integrated into multiple existing and emerging enterprises that contain 

a large number of government and commercial entities. 

Another challenge: unstable requirements. In many cases, the 

changes are driven by advancements in technology, vendors updating hardware, operating system upgrades, and so on, or by changing world events. 

And the third challenge: build versus buy. While many IT companies regret building enterprise software solutions because it was 

much more expensive than expected, there are times when custom 

software solutions is the right answer. A rational build-versus-buy 

decision starts with well-defined requirements and a quantifiable 

outcome. 

Second, the best practices. There are many best practices within 

the commercial sector and within DOD. I would like to highlight 

just a few. 

Executive leadership participation. Robust and continued seniorlevel attention and participation contributed significantly to the 

success of agile MAIS acquisition programs as documented in the 

DOT&E [Director for Operational Test and Evaluation] 2016 annual report. When senior leaders are on-site, they can add resources as necessary and they can also shorten the decision cycle 

time. 

Another best practice, continuous developmental test and evaluation. The program office should have a coherent DT&E strategy to 

find and fix problems as each software component is developed and 

delivered. Deficiencies are much easier to fix before integrating into 

modules and components. 

And the best practice of evolutionary acquisition, which is a 

method intended to reduce cycle time and speed the delivery of capabilities to users. Often, in an evolutionary model, the development of increments must occur in parallel to deliver capability on 

time. 

The third topic, the role of developmental test and evaluation. 

Conducting DT&E in an agile environment must be done early and 

often. During a major weapon system development cycle, 80 percent of the test and evaluation is developmental test and evaluation. 
8 

There are DT&E organizations within the services and at OSD 

[Office of the Secretary of Defense]. I have worked in both. They 

serve a different purpose. The DT&E professionals within the services are typically funded by the program managers [PMs], whether 

it is acquisition programs, MAIS, or whatnot, and they are potentially subject to biased reporting due to the pressure from the PMs. 

I have seen it firsthand. The OSD DT&E organization is not funded by the program managers. They are mission funded and therefore independent. 

And the last area to talk about is business systems versus tactical weapons systems. It is important to make a distinction between cyber and IT/IA [information assurance] policies for warfighting systems and those pertaining to business systems or corporate enablers like email, common business systems, and cloud 

applications. Technical laboratories and communities should be 

held accountable for making recommendations for IT consolidation 

and savings, but should control their own destiny in determining 

the best solutions. There is not a cookie-cutter approach. 

To ensure there is an appropriate focus on the cyber systems engineering, it is now time to pair a traditional CIO function with an 

RDT&E [research, development, test and evaluation] warfighting 

system cyber assessment function, which is focused on the tactical 

weapon systems and RDT&E infrastructure. 

That concludes my remarks, and I look forward to answering 

your questions. 

[The prepared statement of Mr. Greer can be found in the Appendix on page 46.] 

Ms. STEFANIK. Thank you, Mr. Greer. 

We now turn to members’ questions. My question is for all three 

participants in the panel today. 

Industry best practice calls for the use of leading indicators as 

a way to measure the health and performance of a sector; however, 

DOD, as you know, tends to focus more on lagging indicators because they are easier to measure, but they don’t do a very good job 

of helping decision makers decide if a program is performing well 

or not, at least not until it is too late. 

What kinds of leading indicators should we be using to measure 

IT programs as well as performance of our acquisition systems? 

And what are the obstacles to being able to collect that sort of information currently? 

Mr. HALVORSEN. One of the first things we ought to do is concentrate on money. That is what industry does. We do look at how 

is the money being spent, and if you find that you are starting to 

have undocumented rising cost, that is generally a pretty good indicator that something is wrong. 

Performance is the next best thing if you look at industry, they 

will look at performance. And it is not the performance of how the 

acquisition process worked. You ask what are one of the problems. 

We grade the acquisition process. 

Now, I am going to say everybody in this room has fault in that. 

I did, and so does the legislation, because it requires that I had to 

report on how I was doing in the process. Most acquisitions, the 

day they were finally declared unfit, still had green in checking the 

acquisition process. That doesn’t make a lot of sense to me. 
9 

So why aren’t we looking at performance in the field? I mean, industry, if I am going to grade it, I grade it: Did it meet the customer requirements? That ought to be the number one, and track 

the money. If we did those two things, we would absolutely have 

a much better system today. And then get out of the rules and laws 

that require that we have to track and produce: Is the acquisition 

process itself being followed? The acquisition process is part of the 

problem. 

So I really think if we followed—you could take these templates 

right off industry guidelines, we are not different in that respect, 

and track the actual outcome performance results, not the process. 

Mr. LEVINE. I think that when we look at the DOD acquisition 

system not just for IT but for anything, we need to recognize that 

we rely on our contractors for almost all the data on their performance. So we need to be careful about what kind of data we prescribe, because if we say we want more data, then we are adding 

a contract requirement and a new regulation to a system where we 

don’t need a whole lot of new regulation. 

I would add to what Terry said about tracking dollars, though. 

It is important not just to be able to track dollars, but to be able 

to link dollars to performance. And certainly with the acquisition 

system in general, I have always felt that earned value management systems are one of the most important things that we have, 

because they don’t just track how much money you are spending, 

but track it against measurable milestones so you can see whether 

you are progressing in the way you thought that you should be progressing. 

Now, some of our contractors have earned value management 

systems, some of them don’t. It depends on whether—you know, if 

they are government contractors, if you are a Lockheed or a Boeing 

or somebody like that, you will have an earned value management 

system. If you are a commercial contractor, you may not have one 

that matches government standards. So we may need to figure out 

how we can work within data that is otherwise already provided 

and collected by commercial industry to determine these things 

without imposing new requirements. 

Mr. GREER. Industry does a very good job at quantifying their requirements. They stay rigid with their requirements. They developed a product based on quantifiable outcomes. They also make it 

a priority to make sure their senior leaders are proactively engaged. They are on-site where the decision time is much shorter. 

Proprietary data is an area that, as Peter mentioned, is a concern. In some instances, it is difficult to get our hands on the data, 

and when we do, it costs us dearly. So that is an issue. 

The last one is a partnership, perhaps, an integrated product 

team with industry so we don’t have to worry about lagging indicators. We will be right there working with the industry developing 

the leading indicators and monitoring the leading indicators. 

Ms. STEFANIK. Thank you. 

I now yield to Mr. Langevin. 

Mr. LANGEVIN. Thank you, Madam Chair. 

And I want to thank all of the witnesses for your testimony. 

So to carry on on this point, in your opinion, what do you think 

is the single most important action both Congress and the new ad-
10 

ministration can take to enable a rapid modernization of IT? I 

know it is a broad question, but kind of following on with our discussion. Please give us your insights. 

And then the other one, I will say, in your opinion, what cultural 

barriers, such as risk aversity or service preferences, exist within 

the Department and what recommendations do you have to overcome them? 

Mr. HALVORSEN. So the first thing I would say is we need to 

push the authority to acquire down, and we can do that in small 

numbers. So, I mean, here is an interesting thing. In my previous 

job, I was the CIO. I had a $37 billion budget. Approved that. It 

is the only place in the DOD where as a single entity, I as CIO, 

I had to approve that budget, write that budget, yet I couldn’t authorize directly a million dollars if I saw great technology to put 

right on the table. That doesn’t seem to make a lot of sense to me. 

I think that is the first thing I would do. Smaller numbers, control. Give it a limit of maybe $10 million 

total that the CIO could say, ‘‘Yes, that is great technology, I want 

it,’’ and waive some of the requirements that an institution as big 

as DOD has to review every piece of that against some competition 

criteria. That does not make sense. 

Part of the time, if I can see best practices in industry, and that 

is where industry is going, I ought to be able to say as the CIO, 

‘‘Industry CIOs do that, they say this is where industry is going, 

I am making the decision,’’ and we go. We could do that, try that, 

small amounts of money. I think that would really drive some 

rapid acquisition quickly. 

The second thing, your question about what else do we need to 

do, I would really recommend that this committee establish a 

group, and I think in 90 days they could get some very good results. But stack it with both industry and military business systems owners, not just the technical guys, because in the end, back 

to our original question, I never had my customers, DOD, soldiers, 

sailors, marines, and civilians are not real shallow, and they are 

not real—they will tell you what the problems are really quickly in 

language you will get to understand quickly if you go ask them. We 

don’t ask them enough. 

And so I think that is the other problem. Let’s go out and ask 

the customers. Bring some customers on this panel, bring some industry experts, bring some leadership, 90 days I think you could 

get a really good result on here are some of the specific things you 

could do to really get at the cultural issue, because this is cultural. 

Even when it is fixed and even when we have the authority, the 

whole system, which includes the DOD, the way we inspect and review, and, frankly, includes the way GAO [Government Accountability Office] inspects and reviews, we are using the mindset that 

was 20 years old. 

We have got to change that. That will take time. But I think this 

committee, because it has done it in the past, could establish that 

a committee take a review and put some things in legislation and 

then follow up on it with your personal attention, and that would 

get some good results. 

Mr. LANGEVIN. Thank you. 

Mr. Levine. 
11 

Mr. LEVINE. I agree with Terry, that I think that the real focus 

ought to be on planning and prioritization so that you are spending 

the money in the right place. We tend to focus in a hearing like 

this on what is wrong with DOD’s policies and practices and how 

can we make things better through that and we can make things 

better. 

But if you are really looking for how are you going to improve 

the IT environment at the Department of Defense, whether it is in 

business systems or weapon systems, the issue is, we have a lot of 

low-hanging fruit out there, things that need to be improved. We 

don’t necessarily spend our money in the right places for the right 

priorities, and that prioritization, I think, is the number one task 

that we have. 

Mr. GREER. In response to your first question, contracting is, in 

my mind, a major impediment, whether it is acquisition weapon 

systems or IT systems or whatnot. If you look at the entire food 

chain from A to Z in terms of delivering an IT agile system, it 

starts with the statement of work, then it goes into a do-loop in the 

contracting arena because of policies. It is not the executors, it is 

the policymakers, whether it is an OSD or the echelon command 

within the services on more pointed here at Congress, legislation 

that has been burdened on the contracting community. Conducting 

a zero-based review of all of the contracting policies certainly might 

be one approach. 

I mentioned earlier about the CIO. My personal view is—and I 

am an engineer, I have the technical background more so than the 

IT side of the house—but certainly the CIO has such a broad responsibility. That is another long lead time approval cycle. It has 

been from my experience. 

In terms of cultural barriers, a lot of the government organizations do have in-house software development teams. And sometimes they make less than informed decisions regarding whether or 

not they should develop in-house software solutions or go out to the 

commercial sector. And I believe that is a cultural issue that ought 

to be addressed. 

Thank you. 

Mr. LANGEVIN. Thank you. 

I yield back. I will have additional questions if we go a second 

round. 

Ms. STEFANIK. Mr. Scott. 

Mr. SCOTT. Thank you, Madam Chair. 

Gentlemen, thank you for your service. 

And, Mr. Greer, you mentioned the proprietary data and how 

much—you said it costs us dearly, I believe is the term that you 

used. And regardless of what weapon system it is or IT system it 

is, I mean, we have seen examples in the past where we as the government, we paid for the research, we paid for the development, 

and then in an error in the contracting, even though we paid for 

all of it, some private company ends up owning that data. 

I assume that we are not making that mistake anymore? Is that 

something that we have corrected in the contracting? 

Mr. LEVINE. Congressman, I would be happy to respond to that. 

Congress did change the rules on that several years ago, about 

3 or 4 years ago. I was with the Senate Armed Services Committee 
12 

then. But you changed the rules. In particular, there was a rule 

that said the way that rights and technical data work, if it is 100 

percent developed through private expenditure, the private company owns it; if 100 percent developed at government expenditure, 

the government owns it; and if it is mixed expenditure, we get just 

government purpose rights. 

But those rules also said you have to identify in the contract up 

front which data you are going to have ownership of as the government. So even if it was developed exclusively at government expense, if we failed to identify it up front, we didn’t own it, and the 

contractor would come back and tell us, ‘‘Well, we own it now because you didn’t say in the contract, so you are going to have to 

pay through the nose to use what you paid for in the future.’’ 

We changed the law so that it said, given that we were supposed 

to own it in the first place, if we failed to identify it in the contract, 

then the amount that we will owe you to provide us that data is 

the amount that it takes to produce the data. We are not going to 

have to pay you monopoly rights to buy it from you again. So we 

did make that change—or you did make that change—in the Congress a couple of years ago 

Mr. SCOTT. Okay. I just know that as somebody who represents 

a depot, this is a problem for us in supply chain management in 

some of the older systems out there where we paid for the development of a product and we are having to then pay for the schematic 

to manufacture the part because they don’t have it anymore. 

Mr. LEVINE. It has been a huge problem. It should be better 

going forward, but of course that doesn’t get us all the technical 

data from contracts in the past that we wrote under this old system. 

Mr. SCOTT. Absolutely. 

Mr. GREER. And I might add for just a couple of seconds here, 

even if it is just one piece of small source code within a weapon 

system, that will hold us hostage, and that is what is happening 

today. So you can have all of the lawyers get together and come 

up with compromises, generate legislation, but at the end of the 

day, the major prime contractors are still holding the government 

hostage. 

Mr. LEVINE. The change that we made a few years ago, the 

change that Congress made a few years ago also attempted to address that by saying that you have to segregate the smallest piece 

of technical data that you actually spent your money on and isolate 

that and tell us how we can work around it so that you can’t tell 

us that we have to pay for all of the technical data or rely on you 

for all the technical data. So there is an attempt to address that 

as well. 

There is also currently, I believe, Congress—— 

Mr. SCOTT. Is that attempt effective, or do we need different language to make it more effective? 

Mr. LEVINE. I believe it is effective, but it needs to be implemented and practiced over a period of time to see where the problems are. Congress has established a technical data panel to look 

at these issues. And I suspect that, with all due regard to my 

friends in the contractor community, they are going to try to reverse some of these changes. So I would urge you to be very alert 
13 

to the recommendations of that panel and make sure that there 

isn’t backsliding on those issues that you are concerned about. 

Mr. SCOTT. If you would provide additional information to us on 

that, I would be very appreciative of it. 

[The information referred to can be found in the Appendix on 

page 65.] 

Mr. SCOTT. Gentlemen, it took the Army 10 years to decide on 

which new pistol they were going to use. I don’t know if it was because of the law, the bureaucracy. But the iPhone came about 

about 10 years ago, the original iPhone. Flip phones were still popular back then. With technology there is certainly a timeline that 

we cannot operate in. Is it the law or is it the bureaucracy that 

needs to be changed? 

Mr. HALVORSEN. Yes. Some of the laws still need to be restructured on that. There are some things that still treat off-the-shelf 

technology like a weapon system. We have got to stop doing it. We 

really do need to say off-the-shelf technology, commoditized technology, does not need to be treated like a weapon system, but then 

we have got to get through the bureaucracy. 

Mr. SCOTT. Do you have any specific language that you could 

suggest, provide for our committee? 

Mr. HALVORSEN. I will shoot you—yes, sir, I will do that. 

[The information referred to can be found in the Appendix on 

page 65.] 

Mr. SCOTT. I would appreciate that. 

Madam Chair, I am down to about 20 seconds. I will yield the 

remainder of the time. 

Ms. STEFANIK. Thank you. 

Ms. Gabbard. 

Ms. GABBARD. Thank you. Thank you, gentlemen. 

I wonder if you can comment on what impact, if any, you think 

that Buy America provisions have on obtaining the best possible 

technology. 

Mr. LEVINE. It has always been a concern in the acquisition community that if you have Buy American provisions that become too 

restrictive, it will limit our ability to get the best weapon systems 

for our warfighters. 

The acquisition community always has a bias toward getting the 

best possible for the best possible price. As long as the Buy American provisions are in the area where they are now, the acquisition 

community has been able to work around them; but there is concern if you ramp it up too much, it might impede their ability to 

do what they need to do. 

Mr. HALVORSEN. I will echo what Peter said. There are some 

cases where, unfortunately, you can’t Buy American in much of the 

technical side. 

I think the other thing we want to talk about here is that part 

of the Buy American also addresses some security concerns. What 

we really ought to be looking at, and this is maybe a bad way to 

say it is, where don’t we want to buy? Very candidly, I don’t want 

to buy stuff, I didn’t want to in my last job, that was manufactured 

in certain nation-states. I think that needs to be more of the focus 

here. 
14 

And what I would say, the words here, we might in the technology area want to change that to Buy American and allied. The 

other problem we have in the technology communications business 

is we don’t go to war alone, the DOD does not, and we need to be 

able to communicate effectively with allies. So that is something we 

have really got to make sure that agency by agency we don’t put 

out Buy American rules that would preclude us from then communicating very well. 

I also think there is great benefit if we focus this around Buy 

American and allied and how we could do some better arrangements for everybody in terms of cost, delivery times, and production. 

Mr. GREER. Outsourcing integrated circuitry is a concern that I 

have had for quite a while from a cybersecurity and trusted systems point of view. We are second to none when it comes to manufacturing integrated circuitry and mass producing or producing 

small quantities. But the outsourcing of the large quantities is 

something that we all have to be concerned about only from malware perspective and other reasons. 

Mr. LEVINE. I would echo what Terry said, though, that there is 

a difference between worrying about your supply chain vulnerability, where you are worried about countries of concern, and what 

you do with Buy American, which is to say that all countries are 

off the table. 

Ms. GABBARD. Right. With regard to, you know, you mentioning 

Buy American and allied, what are some of the approaches that 

our allies are taking with regard to some of these challenges with 

workforce and improving their own speed of acquisition? Are there 

lessons we can learn? 

Mr. HALVORSEN. There are. So an example, like the U.K. [United 

Kingdom] and Australia, their CIOs are permitted up to I think it 

is 10 million pounds to look at existing technology and say: Hey, 

this would fit. Let’s buy a small amount of it. Let’s test it. Let’s 

put a performance objective around it and track the cost performance to Peter’s. 

They have to have those, but they can make the commitments 

right on spot to do that. That makes them a lot more attractive to 

new technology, because if you are a startup new technology, you 

are on a 6-month funding cycle. If you don’t get that 6 months, you 

are walking away. You have to. So that is something we could do. 

They also have changed their workforce construction. My counterpart in the CIO at the U.K. until just recently—he retired 

March 31—was what was they called a crown contractor. To get 

around the pay gap issues, they were able to negotiate not quite 

a civilian salary, but better than the government salary, and they 

were able to hire him, give him full civilian authority to make the 

decisions. But he came out of both industry and the military, he 

had been a retired two-star, he did 5 years with British Telecom. 

I think that is kind of what we want to be able to bring into the 

government, people who have both government and industry experience. You need both to succeed at the top level. 
15 

† ‘‘Five Eyes’’ (or FVEY), refers to an intelligence alliance involving Australia, Canada, New 

Zealand, the United Kingdom, and the United States. 

The Canadians are doing the same thing. And the way they are 

doing their supply chain also, those nations which are in Five Eye,† 

and there is only one nation that doesn’t have this authority, and 

it is probably the leading nation in the Five Eye community, which 

would be us, they are allowed to look at a computer, say, check a 

supply chain. 

So there are some U.S. companies that I wouldn’t want to buy 

from because when you look at their components, they are all from 

countries I don’t want. That is the most important thing. So they 

are allowed to say: Hey, if that is a U.K. company, but all of their 

parts are sourced out of a place we don’t like, well, then we are 

going to go buy an allied company. We ought to have that kind of 

flexibility restriction to make those decisions when it is in the best 

interest of defense. 

Ms. GABBARD. Thank you. 

Mr. LEVINE. One thing that the last administration did, if we 

want to look to a domestic example rather than overseas, is to have 

teams of IT experts who came in from Silicon Valley to help DOD 

and other agencies look at specific problems and identify solutions. 

So they would identify commercial solutions. Sometimes they would 

write software themselves to address it. 

I believe that the new administration is considering similar kinds 

of steps. But that gets you ahead of a cycle. You know, you don’t 

have to take 5 years—— 

Ms. STEFANIK. We are running out of time, Mr. Levine. 

Mr. LEVINE. Thank you. 

Ms. STEFANIK. Dr. Abraham. 

Dr. ABRAHAM. Thank you, Madam Chair. 

Gentlemen, thank you for your direct testimony. It is appreciated. There was some great solutions or possible solutions mentioned in you-all’s testimony. And as we know, industry has to 

make a profit. They have a board of directors that they report to. 

And, unfortunately, in the DOD or any bureaucracy, that word is 

not even in their dictionary. And, you know, that is a problem. 

Again, it all comes down to accountability. And when you have 

a budget, Mr. Halvorsen, of $37 billion or whatever and you can’t 

write a check for something that you see, it is just idiotcracy at its 

finest. 

So it is a national security issue. And I understand we have to 

maintain qualitative and quantitative advantage over our enemies. 

But for these programs that we could just literally pick off the shelf 

and save literally hundreds of millions of dollars, it just—it bodes 

poorly for us in the government for us to allow this to happen. 

Are there any examples, can you give me some examples of some 

successful industry partnerships? 

Mr. HALVORSEN. Well, you know, I hesitate, because some of 

those I was directly involved in. But I will tell you, the Microsoft 

partnership with DOD, the ability that Microsoft let us, looking at 

the license agreement, let me at that point make the decision that 

the entire Department could go to Windows 10, get to a single oper-
16 

ating system, without having to renegotiate the current contract, 

that was a partnership with Microsoft. 

That what they saw down the road, and I think this is going to 

happen if the government goes to Windows 10, to one of the Representative’s comments earlier, it will actually facilitate then the 

follow-on move, which I know is being planned, into the 365 and 

public cloud domain. 

So Microsoft saw that as investment. It was a good, open dialogue. That was very good. 

Again, I hate to say it, I will go on the record, I am now working 

for Samsung, but one of the things that happened early, we put out 

a capability statement that said we needed a secure smartphone. 

That is all we put out. Samsung came in. We worked with them. 

We actually co-developed what today they feel is their KNOX security system. It kept everybody’s prices low. We were able to field 

that at the secret and now top secret level in less than 24 months. 

The products, which are rare in my history, were delivered on time 

and at cost. And in the end, they actually cost less than we thought 

they would. 

Dr. ABRAHAM. That leads somewhat to the second question I 

have. Mr. Scott referenced the iPhone 10 years or less. And, again, 

these products are being developed by a different generation of entrepreneurs, engineers, IT people. What are we doing to attract 

some millennials into the fold? 

Mr. HALVORSEN. Not enough. We are going to have to address a 

couple things on that. One of them is the pay gap. I will tell you, 

I lost my director of security. You know, he went and he quadrupled his salary after spending, you know, 32 years serving the 

government. Good. 

That is one thing we are going to have to address. But we are 

also going to have to address some of the other things that the 

millennials want, which is they want their technology. 

One of the things that when I went in my old job and now in 

my new job, I talk to millennials, ‘‘Why don’t you look at DOD?’’ 

‘‘Well, because they will take my phones away. I won’t be able to 

do the things I want.’’ 

Now, that is not true as much anymore. We are using multifactor 

phones, those type of things. But there is the perception that the 

government lags greatly in the ability to be flexible in the workplace. 

We have got to get that message out. The rules are there. I 

mean, the laws are there. I mean, I could let my people telework. 

I could do those things. It just wasn’t the way we thought. That 

is a cultural issue. The laws are there to make that happen. 

That is something that maybe this committee and other committees ought to be asking some questions about what are we doing 

on the millennials, what are we doing to be more flexible in the 

workplace? There are some good answers to that, but that needs 

to be the holistic plan, not just individual efforts. 

Dr. ABRAHAM. A minute left. Mr. Levine. 

Mr. LEVINE. If I could just add one thing to that. We don’t take 

advantage enough of the things that we do have to offer. I think 

that the most attractive thing for the Department of Defense for 

people who work there is the mission and the feeling of commit-
17 

ment to the mission and that they can make a difference. And I 

think that we don’t take enough advantage of that. We don’t sell 

that well enough. When we treat civil servants as bureaucrats who 

are just too much, there are too many of them, we need to get them 

out of the way, we need to fire them and cut their pay, it doesn’t 

create the image that people want to see of themselves. And we 

need to do a better job of selling the mission to get the kind of people we want. 

Dr. ABRAHAM. Very good. 

Mr. Greer. 

Mr. GREER. And just one or two points here. In the IT community, I believe that we have to raise the level of our professional 

series. We have a lot of nonprofessional IT specialists, and they 

serve a very good purpose, but where we are today, what the cyber 

vulnerabilities and resiliency challenges, I think making sure that 

we hire enough professional series in the IT community is important. And it is not necessarily the pay that is attracting, like Peter 

said, it is the quality of work for these millennials. 

Dr. ABRAHAM. Thank you, gentlemen. 

I yield back. 

Ms. STEFANIK. Mr. Larsen. 

Mr. LARSEN. Thank you, Madam Chair. 

Mr. Greer, can I talk a little bit about your build-versus-buy conundrum in your testimony, in your written testimony? I am sorry 

for being late, you may have addressed this. But I wanted to ask 

you in your experience on this build-versus-buy tension generally 

are there some criteria that we should be thinking about where 

there is a leverage point within the DOD where they ought to build 

versus they ought to buy in general? 

Mr. GREER. That is a difficult question to answer in a short period of time, mainly because there is a hybrid solution. There are 

a lot of commercial software packages like SAP that we do purchase, but we use government people to program it and code the 

SAP system. And so that is always an issue. 

Mr. LARSEN. Sorry. Mr. Halvorsen looks like he is excited to answer too. But go ahead. 

Mr. GREER. Well, I am just expressing my views. 

Mr. LARSEN. That is great. 

Mr. GREER. Okay. And this is why—— 

Mr. LARSEN. So then continue. 

Mr. GREER. But in terms of having firm requirements and quantifiable outcomes and making that make-buy decision, it gets difficult within the government, mainly because of cultural issues. 

There are a lot of software programmers in the government, and 

they inherently lean in the in-house solution side. 

Mr. LARSEN. So let me ask you this then. Are there specific 

build-versus-buy decisions on cloud computing? Is that treated differently than, say, operational software like on the Microsoft decision? 

Mr. HALVORSEN. It should be, but it is generally not the culture. 

But I want to be very clear on this: If you could buy it, buy it. I 

did not find that to be as difficult for anything that wasn’t a weapon systems as [inaudible]. If you can buy it, we should buy that, 

and we should put—and we really did try to do this. 
18 

I mean, one of the plaques over at my desk was ‘‘customization 

bad.’’ You should fight every attempt we take to customize that 

software off the shelf, because every time we do it, it is a lifelong 

contract for contractors without much gain. Our business processing systems, back to Peter’s point, don’t differ that much from 

the best in industry. They shouldn’t, if they do. 

So one of the first criteria is, if it is a business system, you 

should have to come in with a different business case that says 

why I am not buying that solution, why am I building any specific 

software. 

It is getting that way for many communication systems, because 

we are using off-the-shelf communication. If it is off the shelf, really consider why you need to put something extra. And in the communications it might be for cybersecurity, but I have found if I go 

to industry, when I did that, if I told them I wanted that extra, 

most of the time they were happy to put it in, and most of the time 

they carried that in their commercial product because it was going 

to sell more as security becomes more important. 

Mr. LEVINE. The track record of the Department is, if we are 

buying a commercial-type business system, if we start tailoring it, 

we end up spending more money on the tailoring than we spent on 

the system in the first place. It just doesn’t make sense to do anything unique unless you absolutely have to. 

Mr. LARSEN. I am sure if we went back to the first hearing that 

I attended on this issue when I was first elected in 2001 a lot of 

the answers would be the same. 

On the build versus buy then does that principle, can that principle apply to the people? For instance, in Washington State, we 

have the example of the National Guard working with private sector and public sector folks on cybersecurity and doing red teaming 

and doing some mapping out of how to approach defense. And some 

of these folks are reservists and they are trying to attract folks in 

for 2 weeks a year, two weekends a month, in a sense buying people as opposed to building them internally. Does that model work 

in terms of the IT personnel acquisition side, if you are acquiring 

people? 

Mr. HALVORSEN. It certainly needs to be part of your plan. It 

can’t fill all the balance. And there are some security issues that 

you might not want to put everybody that way. But absolutely we 

need to be taking advantage of that. 

If you look at our last sets of operations, you look at the Reserves 

we put on board, many of them were with those technical skill sets. 

As the CIO when I was there, I would reach out and bring Reserves on who were working for Google, working for whatever big 

company. And it is actually part of the deal, I will give DOD great 

credit, they actually have that in their plan. 

Continuing that into other areas I think makes sense with caution. It is not as easy to do that with people as it is with software, 

but it is something that we certainly ought to be looking at. 

Mr. GREER. Contractor support is an area that we need to leverage even more. There is always a need for inherently governmental 

personnel to be able to represent the government. But I think we 

can leverage more on the contractor support side, especially as your 
19 

workload goes up and down. You are able to make adjustments 

much more quicker. 

Mr. LARSEN. Yeah. Thank you. 

Ms. STEFANIK. Thank you very much to each of our three panelists, Mr. Levine, Mr. Halvorsen, and Mr. Greer. And thank you to 

the members for their thoughtful questions. This is a critical issue 

that falls under this subcommittee’s jurisdiction that we will continue to focus on as we prepare and draft this year’s NDAA [National Defense Authorization Act], and we look forward to continuing to engage with you to get your suggestions. 

Thank you very much. This hearing is adjourned. 

[Whereupon, at 2:57 p.m., the subcommittee was adjourned.] 

A P P E N D I X 

APRIL 26, 2017 

PREPARED STATEMENTS SUBMITTED FOR THE RECORD 

APRIL 26, 2017 

(25) 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 
62 
WITNESS RESPONSES TO QUESTIONS ASKED DURING 

THE HEARING 

APRIL 26, 2017 

(65) 

RESPONSES TO QUESTIONS SUBMITTED BY MR. SCOTT 

Mr. LEVINE. The statutory provision to which I referred in my testimony is section 

815 of the National Defense Authorization Act for Fiscal Year 2012. As I indicated, 

this provision: • Ensures that DOD may require the delivery of data generated in the performance of a contract that is necessary for the purpose of reprocurement, sustainment, modification or upgrade, even if such delivery was not required by the 

original contract. In exchange for the delivery of the data (which was generated 

at government expense) the government is required to compensate the contractor only for reasonable costs incurred to covert and deliver the data in the appropriate form. 

• Requires the contractor to provide technical data ‘‘is necessary for the segregation of an item or process from, or the reintegration of that item’’ with the rest 

of the system. In other words, if a contractor wants to withhold data on a widget that was developed at contractor expense, it has to tell DOD how to work 

around the widget, so that the rest of the technical data, which DOD owns, 

won’t be useless. Since I am no longer with the Department, I am unable to 

give a report on the extent to which this provision has been successfully implemented. However, section 813(b) of the National Defense Authorization Act for 

Fiscal Year 2016 establishes a Government-Industry Advisory Panel to review 

statutory requirements regarding technical data rights. Some may seek to use 

this panel to unwind the 2012 reforms. [See page 13.] 

Mr. HALVORSEN. Language that stresses DOD should use commercial business 

case analysis to include review of DOD business process when procuring what is 

clearly a commercial solution to include hardware, software and IT service contracts. Something like the following: 

‘‘DOD will apply commercial best practices when conducting business process and 

detailed market analysis when procuring readily available IT hardware software 

and services in support of or in replacing business systems. If DOD can demonstrate 

through business intelligence and analysis a clear product/service leader(s) in a specific area DOD can pursue sole source or limited partnership procurements. These 

facts can include the ability to support a large enterprise like DOD, similar to how 

other large commercial companies develop procurement partnerships. 

An example of this would be using the MICROSOFT based Services Cloud supported by MICROSOFT 365 and Azure. This is the leading cloud solution for the 

fortune 500 and clearly dominates the market and would be considered a best of 

breed solution. If DOD can make this case it should be allowed to by-pass current 

acquisition rules and go direct to procuring a best of breed solution, negotiating and 

spending time on obtaining the best value (pricing and performance) instead of defending and deliberating on multiple of solutions. This doesn’t mean the MICRSOFT 

would be the guaranteed provider, it does mean that MICROSOFT products/services 

would be the used to form the service cloud architecture and operating system. 

I want to make it clear I am not a MICROSOFT employee and receive no compensation from MICROSOFT for these recommendations. My current employer 

Samsung receives no direct benefit as a result of these recommendations. These recommendations are aligned with statements and testimony I made as the DOD CIO. 

I use MICROSOFT as it is the most compelling example and in my opinion moving 

to the cloud is a necessary and critical step to improving DOD security, greatly improving productivity and saving dollars. 

With respect to all of the responses I have provided to the Questions for the 

record, they represent my personal opinions and beliefs and are in my own words. 

I am currently an employee of Samsung Electronics of America. The responses I 

have provided have not been endorsed, constrained of approved by Samsung Electronics of America or any other entity. [See page 13.] 

Mr. GREER. There are a few acquisition programs that have had difficulty obtaining data/software/interface rights by prime contractors. Despite language that stipulates the Government has purpose rights if Government funds are spent to develop 

a deliverable, industry still presses back when the Government attempts to exert 

its rights. Industry is interpreting that if a deliverable contains their proprietary 
66 

information or reveals the workings of what they claim as their intellectual property 

(IP), they may limit access to it. Additionally, for software intensive deliverables, 

industry has started ‘bundling’ IP with the deliverable to the government, and 

claiming the government has less than purpose rights to the overall product. This 

is leading to complex licensing issues for deliverables the Government should have 

purpose rights to. This has happened on F–35 as well as many other programs. Language should be clarified to state the Government has no less than purpose rights 

to the code segments (source and executable) and interfaces used in an overarching 

deliverable funded in any part by the Government. 

From an acquisition oversight perspective, with the KC–46A program, Boeing 

made it clear during contract negotiations that they would not use Boeing 767 reliability data to meet the government requirement. That sounded like the right approach at the time given the KC–46A differences in configuration, but in practice 

it meant that Boeing will not share any reliability test data from their Boeing 767– 

2C (the provisioned freighter) either from Boeing DT&E or FAA Amended Type Certificate (ATC) certification testing. Although we do get some anecdotal failure data 

from these flights in which the government participates, it is purely word of mouth 

from those attending the briefings or flying the tests. We do have reliability data 

from flights, which are used to meet contract specifications. The net result is that 

we have very incomplete reliability data. The DASD (DT&E)’s official MS C assessment concluded, ‘‘As a result, the Lead Developmental Test Organization (LDTO) 

used data only from flights in which the Air Force participated to estimate the system’s progress against the reliability growth curve, which presents an incomplete 

picture of the system’s reliability.’’ 

Also, the tension over Boeing 767 proprietary data makes it difficult and timeconsuming to obtain test data that the government feels is necessary to evaluate 

the system under test when that data comes from Boeing DT&E or ATC testing. 

When it was in Boeing’s interest to share more quickly and ‘‘freely’’ during the boom 

issues prior to MS C, the data flowed more freely. Most of the time, it can take 

weeks or longer to get the data the government believes that it needs. 

To the best of my knowledge, Boeing has provided (or will provide) the data required for gov’t spec verification, though it has been a painful process that runs 

through Boeing Commercial Aircraft (BCA) because they technically own the aircraft during EMD. As best as I can tell, we will never get the B–757–2C reliability 

data from Boeing internal and ATC testing. I sat in on a meeting between BCA, 

Boeing Defense, and the program office and can relate that BCA was surprised that 

we wanted the requested data. In the commercial world, BCA simply delivers aircraft that meet the customers’ specifications. The commercial customers don’t care 

how the aircraft was built and tested. Maybe part of this equation involves the gov’t 

rethinking what data we really need, which drives back to the acquisition process 

and how the gov’t buys stuff. 

The Department of Defense has tremendous capabilities to modify legacy weapon 

systems if full data rights were available. Government personnel could perform the 

Non Recurring Engineering development for small to medium upgrades in-house 

saving millions of taxpayer’s dollars. For example, The NAVAIRSYSCOM’s Naval 

Air Warfare Center, Aircraft Center has an Aircraft Prototyping Facility for modifying DOD weapon systems. The DOD should be taking advantage of facilities like 

this as opposed to sole sourcing modifications to the prime contractors. The Army 

has similar modification facilities. 

The lack of government access to design data related to items (platforms, systems, 

components, hardware, software, etc.) already purchased by the government has 

been and continues to be a major cost driver for any subsequent upgrade done to 

these items during their service lifetime. Although IT systems are certainly a critical element of this problem, the problem is pervasive. Asserted data rights involving organic government prototype development programs costs time and money at 

a minimum and, in the extreme, can simply stop an effort. 

This problem is further compounded on a broad scale by government laboratories 

and field activities that are reticent to reverse-engineer the data out of the government-owned items due to ultra-conservative legal advice predicting litigation as well 

as fear of Congressional inquiries. This is in stark contrast to the private sector 

where backing data out of owned items through reverse engineering is commonplace, protected by legal precedent and an essential element of one company modifying or upgrading another’s product. Reverse engineering is not always practical 

but it is often an effective work-around particularly in situations where the prime 

contractor no longer produces the item. 

The complexity in system acquisitions and understanding intellectual property 

rights in Modeling & Simulation (M&S) acquisitions, requires a clear understanding 

from Program Managers, System Engineers, and Procuring Contracting Officers to 
67 

M&S Integrated Product Teams of the Government’s license rights in intellectual 

property comprised of technical data (e.g. technical packages and other information 

relating to the developed product’s design, configuration, or to operation, maintenance, installation, or training data) and computer software (e.g. executable code, 

source code, code listings, algorithms, formulae, design details). The lack of clear understanding among various Program Offices highlights the need for specific guidelines for contracting for open M&S systems and the data rights required. Open System Architecture is a key pillar of the Department of Defense’s Better Buying Power 

(BBP) initiative to implement best practices to strengthen the Defense Department’s 

buying power, improve industry productivity, and provide an affordable, value-added 

military capability to the Warfighter. [See page 13.] 

QUESTIONS SUBMITTED BY MEMBERS POST HEARING 

APRIL 26, 2017 

(71) 

QUESTIONS SUBMITTED BY MS. STEFANIK 

Ms. STEFANIK. In addition to investments in IT to improve its use of data to inform decision-making, DOD also needs to address the cultural barriers that hinder 

effective information sharing throughout the Department. For example, RAND 

found in 2015 that ‘‘institutional structure and bureaucratic incentives to restrict 

data access are exacerbated by policy and guidance to protect information. The result is a strong conservative bias in labeling and a reluctance to share.’’ In your 

opinion, what are the initial steps that the Department needs to take to address 

these cultural barriers, and who should be responsible for leading this effort? 

Mr. LEVINE. There are both good and bad reasons for restricting data. Some restrictions on access to data are appropriate to safeguard classified and sensitive unclassified data from unauthorized disclosure. Widespread dissemination of such data 

to those without a need to know, even within the Department, could lead to compromise. 

Other restrictions appear to be imposed for more parochial reasons: program managers, PEOs, and component acquisition executives may seek to retain control over 

their programs by controlling the flow of data to Comptrollers, to CAPE, and even 

to others in their chain of command. 

Before I left the Department, DOD was seeking to address this issue through a 

coordinated ‘‘big data’’ project. The objective of the project was to identify data elements are needed by users at all levels of the acquisition system, as well as by outside stakeholders in the Department, and standardize that data to increase transparency. If successful, this project should help address the problem identified by the 

RAND review. 

Ms. STEFANIK. In addition to investments in IT to improve its use of data to inform decision-making, DOD also needs to address the cultural barriers that hinder 

effective information sharing throughout the Department. For example, RAND 

found in 2015 that ‘‘institutional structure and bureaucratic incentives to restrict 

data access are exacerbated by policy and guidance to protect information. The result is a strong conservative bias in labeling and a reluctance to share.’’ In your 

opinion, what are the initial steps that the Department needs to take to address 

these cultural barriers, and who should be responsible for leading this effort? 

Mr. HALVORSEN. Representative, when I was the DOD CIO and I believe it is still 

true, that DOD is making good strides in this area and this is currently being led 

by the Deputy Chief Management Officer and fully supported by the SD and DSD. 

I would encourage Congress and DOD to work with industry to look at the current 

laws and rules that restrict government employee direct interaction with industry 

and restrict the sharing of information with industry. Industry should also be challenged to share data more openly with DOD and the government in general. One 

specific recommendation I would make is for DOD supported by legislation to include industry more directly in developing policy/instructions with respect to business systems. While DOD CIO I explicitly involved industry in developing the cloud 

policy and security guidelines. This was challenged by many groups inside and out 

of DOD but the resultant document was well received and is allowing in this area 

much better communications between industry and government. 

Ms. STEFANIK. What is the DOD role as it relates to developmental test and evaluation for IT systems that are predominantly commercially based and where DOD 

is integrating, but not actually developing new capability? Is there a particular balance in developmental testing between what DOD is responsible for and what industry should be responsible for that can reduce duplication in effort, cost and schedule 

performance? 

Mr. GREER. Yes, there is a balance between what testing DOD is responsible for 

and what performance certifications can be provided by industry and accepted by 

the test community. DOD tests are conducted in mission relevant environments to 

ensure overall capability is effective and suitable and to assess resilience to CYBER 

threats. Component level testing of individual COTS hardware components may 

often be conducted at the supplier level and accepted by the Government. 

DOD is in the process of acquiring multiple defense business systems (DBS) based 

upon commercial enterprise resource planning (ERP) systems. The first priority is 
72 

to conduct business process re-engineering to try to change DOD processes to fit the 

commercially offered systems and minimize customization of the ERP. In most 

cases, the unique nature of the DOD identifies gaps between the DOD process and 

the ERP capabilities. In this case development of special report, interface, conversion, extension, form, and workflow (RICE–FW) software objects are required. 

In these cases, developmental testing of the DBS focuses on the RICE–FW and 

integration into the DOD facilities and networks rather than the basic capabilities 

that are proven by use in industry. Mission-oriented developmental testing using actual users performing mission tasks in a test environment allows integrated testing 

with the operational testers, early identification of human factors integration problems, and identification of software defects in the RICE–FW objects with sufficient 

time for correction prior to operational testing. 

Additionally, DOD’s roles in developmental test and evaluation of IT systems integration are in the realm of cybersecurity. DOD must ensure that the new IT system 

does not introduce a vulnerability to the DOD network and that the combined cyber 

defenses are integrated. Just because a commercially based IT capability meets industry, it does not mean that it is secure. The IT capability may meet standards 

of performance (certification) but it must be tested for effectiveness (penetration). 

DOD should be responsible for cybersecurity testing and the vendor must correct 

those deficiencies. This should be in a developmental test environment that allows 

time for the vendor to make corrections. After corrections are made, the IT capability is ready for operational cybersecurity testing. Cybersecurity is an area where 

the test-fix-test methodology is still relevant. 

Ms. STEFANIK. In addition to investments in IT to improve its use of data to inform decision-making, DOD also needs to address the cultural barriers that hinder 

effective information sharing throughout the Department. For example, RAND 

found in 2015 that ‘‘institutional structure and bureaucratic incentives to restrict 

data access are exacerbated by policy and guidance to protect information. The result is a strong conservative bias in labeling and a reluctance to share.’’ In your 

opinion, what are the initial steps that the Department needs to take to address 

these cultural barriers, and who should be responsible for leading this effort? 

Mr. GREER. Cultural barriers are often difficult to overcome. A good method of 

breaking down barriers is to encourage more transparency with regard to ongoing 

developmental efforts. Sharing visibility into IT related projects, investments, and 

infrastructure may help increase re-use of development that would otherwise be unknown across organizational boundaries, reducing duplication of effort. Establishing 

a community of interest (COI) for IT related investments may be beneficial in fostering this kind of exchange of ideas and information. This is a leadership challenge, 

as is all cultural change. At a minimum, the Director, Developmental Test and Evaluation should get back statutory authority for oversight of ACAT ID and MAIS programs and to ensure access to all data in DOD and have direct access to the Defense 

Acquisition Executive with regard to his independent assessments of system performance and issues regarding his ability to make those assessments in time to effectively inform decisions. 

I believe that an organization such as the Deputy Assistant Secretary of Defense 

for Developmental Test and Evaluation (DASD (DT&E)) helps break down these 

barriers, where they exist. For example, the staff specialists in the DASD (DT&E) 

engage in multiple programs across all the Services. Similarities exist in some programs and the staff specialists, by their close association with the program office 

testers, identify lessons learned and share this information among other staff specialists and other programs. Where appropriate, the staff specialists have encouraged and facilitated direct contact between programs to delve into the details of the 

lessons learned. 

Ms. STEFANIK. The 2016 Annual Report on Developmental Test & Evaluation stated that the Department’s current knowledge management capabilities and processes 

used to gain, collect, and analyze the information necessary to conduct acquisition 

assessments and evaluations are deficient and ineffective for today’s world. In your 

opinion, what investments in analytical tools and technologies should the Department prioritize in order to begin to remedy this situation? 

Mr. GREER. There is a need to transition to an Enterprise environment for Weapons Systems RDT&E and Cyber engineering to store, access, and share program 

RDT&E data. Establishment of a classified environment with the right controls in 

place as well as mechanisms to increase cross-program security collaboration is 

needed. With this, comes the need to pair an ‘‘RDT&E Warfighting System Cyber 

Assessment’’ CIO function that is focused on the tactical weapons systems impacts 

of Cyber to work alongside the ‘‘traditional’’ CIO function for business systems, 

email, common databases and promulgation of policy. Warfighting acquisition programs and Cyber technical work must be staffed by the appropriate mix of Govern-
73 

ment experts from the Systems Engineering and RDT&E community vice the traditional CIO community or corporate operations workforce. In the current complex 

Cyber threat environment, Defense Department needs have evolved far beyond traditional IT/IA and business systems best practices. Our ability to operate in the 

Cyber Warfare environment of the future hinges on agile changes to our policies, 

organizational structures, workforce, and infrastructure. Another Enterprise Solution that would benefit acquisition programs is a Shared Data Warehouse and Repository. Currently, data is stored and process on separate servers in a single program office. This data is isolated from many groups that can use it. A DOD Enterprise Solution to data storage and data warehouse tools would provide the necessary 

access for DOD acquisition programs to use and share data across various platforms. As mentioned in another question is an urgent need to gain access to industry acquisition data, which in most cases today is industry owned. The right Contract and Legal clauses to obtain access to this data freely and use it in government 

owned facilities and programs would provide the capabilities necessary to perform 

data analytics to conduct acquisition assessments and evaluations for all programs. 

QUESTIONS SUBMITTED BY MR. SHUSTER 

Mr. SHUSTER. Can you tell us how many DIUx projects have transitioned to programs of record or other major programs within the Department of Defense? What 

is the process for transitioning prototypes or pilots to the services or other agencies 

of the Department? Is the contractor required to lead these efforts? What outreach 

does DIUx do within the Department of Defense to alert them to the potential innovations available from commercial providers? 

Mr. HALVORSEN. Sir I cannot with any specifics and would recommend that this 

question be referred to DOD. I can say that I am a believer in the concept and programs that expand DOD presence in technology hubs like Silicon Valley are worth 

pursuing. I would also suggest that measuring the effectiveness is a little premature. DOD has not had this type of program in place much longer then a year 

and both DOD and the technology hubs are learning how to make this work. In addition to this program DOD has been pursuing employee exchange programs with 

industry. These programs though limited to date have been well received by DOD 

and participating companies. 

QUESTIONS SUBMITTED BY MR. FRANKS 

Mr. FRANKS. Can you give a specific example of a good, well-performing, or successful IT acquisition? What were the characteristics that made it successful? How 

long did it take? 

Mr. HALVORSEN. Sir, the Next Generation Department of Navy IT services contract is a good example of a successful IT procurement. This was the follow on to 

the Navy Marine Corps Internet (NMCI) contract. The contract yielded about $2B 

dollars in savings to the DON and as importantly continued commercial like services for the DON. Planning for this took over a year but was shorter then contracts 

of like size and scope. It involved close coordination with industry providers, a very 

detailed understanding of the capabilities desired, and a willingness to understand 

trade space as related to price and performance. I no longer have access to the details on this contract, but I am sure DOD could provide. There are still some issues 

with this contract but overall I give it very high marks. The negotiation of the Windows 10 upgrade license agreement is another good example of the DOD applying 

good practices and working well with an industry partner, in this case MICROSOFT. This negotiation was truly an enterprise effort in the department with DOD 

and the services working closely with MICROSOFT to interpret the action as an upgrade covered by existing agreements and not a major systems change. This enabled 

DOD to not have any bill for the actual license, while the business case for MICROSOFT was having DOD be better positioned to transfer to the cloud and elimination 

of costs in supporting older systems. When completed the transition to windows 10 

will improve security, position DOD for cloud transition and result in cost savings. 

The biggest cost associated with the transition to Windows 10 is equipment modernization, which had fallen behind in DOD, but would have to be done at some 

point. Doing the modernization in conjunction with the windows 10 upgrade, allows 

for some modernization delay while gaining 80% or more of the new windows 10 

capability. This is the type of DOD-industry partnership that is needed. 

Mr. FRANKS. When you consider the critical need of modernizing the network, particularly to harden against enemy attacks and infiltration, shouldn’t an immediate 

and complete overhaul of the IT procurement process be a priority? 
74 

Mr. HALVORSEN. Yes and this review needs to focus on both the requirements/capability process as well as the acquisition process. The current requirements process 

often results in very detailed and technically restrictive solutions being acquired 

that don’t get DOD the latest solutions or the best value. Many times because of 

the length of the process and over specification, DOD is buying legacy the day the 

contract awards. The review needs to look at successful commercial process and encourage partnership with industry. This will present some problems, especially with 

small business, but I believe creative solutions can be applied to offer small business 

an opportunity to partipate in major enterprise acquisitions. DOD also needs to increase the dialog between the mission/business owners, the acquisition workforce 

and the industry providers. We need to look at the current rules restraining DOD 

employees from direct discussions with industry, many of these rules have not been 

updated in 30 years. 

Mr. FRANKS. Can you give a specific example of a good, well-performing, or successful IT acquisition? What were the characteristics that made it successful? How 

long did it take? 

Mr. GREER. I can think of several IT acquisitions that were executed efficiently— 

the examples are described in more detail below: 

1) The Army Logistics Modernization Program (LMP) Increment 2 was a highly 

successful Defense Business System program that came to the Limited Deployment 

Decision milestone in 2015 with a 98% test case success rate during developmental 

testing. All failed test cases had work-arounds and corrective action plans acceptable to the user, and zero open severity 1 or 2 software problem reports. LMP success may be attributable to: 

The LMP program management office (PMO) was highly experienced, knowledgeable of the system, and effectively planned and executed the complex program consisting of three waves of deployment with multiple functional areas. The PMO also 

established a close working relationship with the functional user to perform business process re-engineering, clarify requirements, and resolve issues quickly and effectively. 

Increment 2 was a continuation of Increment 1; with the same system integrator 

using the well established and understood change management processes. Developmental testing (DT) included various events to specifically address performance, 

interoperability, and cybersecurity. Most importantly, the test strategy included 

mission-oriented developmental testing (known in the PMO as Business Operations 

Test (BOT)) that used actual users from the depots performing operational mission 

threads for their test cases. This identified human factors improvements that led 

to better user performance. The DT actual users became operational test (OT) trainers that resulted in a very high success rate for all the tasks and transactions that 

were tested during Initial Operational Test and Evaluation. The overall schedule 

from milestone B to milestone C was approximately 22 months. 

2) The Marine Corps’ ACAT IAC Common Aviation Command and Control System 

(CAC2S) Phase 2 was a very successful integrated acquisition. The operational test 

community worked well with the developmental test (DT) community, and accepted 

much of the DT data that satisfied OT requirements, reducing the overall acquisition time by 6-months. USD/AT&L nominated the CAC2S acquisition effort as a Defense Acquisition University integrated acquisition case study. Time between MS B 

and MS C was 34-months. 

3) Defense Agencies Initiative (DAI) has been a successful Defense Business System Program. It enables compliance with Congressional direction for all DOD organizations to be auditable. The Defense Agencies Initiative (DAI) transforms the 

budget, finance, and accounting operations of most DOD Defense Agencies in order 

to achieve accurate and reliable financial information in support of financial accountability and effective and efficient decision-making throughout the Defense 

Agencies in support of the missions of the warfighter. Defense Agencies Initiative 

(DAI) is a critical DOD effort to modernize the Defense Agencies’ financial management capabilities. 

Mr. FRANKS. We’ve all heard of Moore’s Law and know how rapidly technology 

advances and evolves. Do you know if there are any requirements for new systems 

we are building or purchasing to be capable of growing and evolving alongside technology—perhaps ‘‘upgradeable’’ systems which won’t be obsolete the moment they 

become operational or shortly thereafter? 

Mr. GREER. The rapid development pace of IT does mean that the Government 

will need to start driving requirements into new systems for Non-Proprietary 

deliverables, interfaces, and use of open systems architectures. Additionally, use of 

system virtualization and cloud technologies may mitigate IT obsolescence issues. 

Ensuring system designs, interfaces, and computing hardware are non-proprietary 

and open to the Government mitigates the possibility of ‘‘vendor lock in’’ and pro-
75 

vides multiple flexible upgrade paths. The DISA Defense Enterprise Office Solution 

(DEOS) is a program that provides e-mail, voice and data conferencing, office products (e.g., word processing, etc.) and other capabilities with a cloud-based solution, 

potentially across the entire DOD enterprise. The acquisition strategy for DEOS 

contains a provision, which DISA calls ‘‘Evergreen’’, that will require the system integrator to provide upgrades, at no charge, whenever DEOS capability upgrades are 

offered commercially. 

DODI 5000.02 supports growing and evolving alongside technology through the 

use of a modular open systems approach (MOSA). It requires programs to identify 

how a MOSA will or will not be used in their acquisition strategy. (Sec 801 PL 113– 

291). This approach integrates technical requirements with contracting mechanisms 

and legal considerations to support a more rapid evolution of capabilities and technologies throughout the product life cycle through the use of architecture modularity, open systems standards, and appropriate business practices. A modular design 

coupled with an appropriately open business model provides a valuable mechanism 

for continuing competition and incremental upgrades, and to facilitate reuse across 

the joint force. 

Æ