Request For White Papers:

Comprehensive Software Test & Evaluation

CAUTION NOTICE:........................................................................................................................................ 2

BACKGROUND: ............................................................................................................................................. 3

PROBLEM STATEMENT:.............................................................................................................................. 4

SUBMISSION CRITERIA & EVALUATION PROCESS.............................................................................. 6

PROJECT DURATION, ESTIMATED FUNDING & AWARD DATE:........................................................ 7
Project DLA-OTA-000-0004: Comprehensive Software Test & Evaluation

2 of 7

CAUTION NOTICE: 

1. Solicitation for application white papers, solution briefs or proposals does not guarantee 

that the government will make an award;

2. Offerors bear all costs to prepare and submit responses to this solicitation; 

3. By submitting a response, offerors agree that the government:

a. Shall reproduce the response, or any portions thereof, to the extent necessary to 

evaluate the offer;

b. Shall use information contained in the brief only for evaluation purposes. DoD shall 

not disclose, directly or indirectly, such information to any person including potential 

evaluators, unless that person has been authorized to receive such information.

4. For traditional defense contractors only: statute requires1 a cost sharing arrangement of at 

least one-third if a non-traditional defense contractor does not participate to a significant 

extent in this prototype project. A cost sharing arrangement is not a consideration for 

award; therefore, the government will give no evaluation preference to offerors that 

propose a cost sharing arrangement;

5. Any Prototype Other Transaction Agreement (“OTA”) awarded in response to this 

solicitation may result in the award of a follow-on production contract or transaction 

without the use of further competitive procedures. The follow-on production contract or 

transaction will be available for use by one or more organizations in the Department of 

Defense and, as a result, the magnitude of the follow-on production contract or agreement 

could be significantly larger than that of the Prototype OTA. As such, any Prototype 

Other Transaction Agreement will include the following statement relative to the potential 

for follow-on production: 

In accordance with 10 U.S.C. 2371b(f), and upon a determination that the prototype 

project for this transaction has been successfully completed, this competitively awarded 

prototype OTA may result in the award of a follow-on production contract or transaction 

without the use of competitive procedures.

1 10 U.S.C. § 2371b(d)(1)(c)
Project DLA-OTA-000-0004: Comprehensive Software Test & Evaluation

3 of 7

BACKGROUND:

The Defense Logistics Agency (“DLA”) is the Department of Defense’s largest logistics combat 

support activity. DLA provides worldwide support, primarily to the military services. Other 

customers include US civilian entities and foreign countries. DLA Information Operations (“J6”) is 

responsible to provide the IT infrastructure, hardware and software to accomplish that mission. The 

majority of the IT tools that we use are standard and well-established. Industry, however, develops 

new tools regularly. 

Many of these tools are easy for employees to use in their personal lives. Thus, a wide gulf is 

developing where end users can do certain things at home but cannot do those same things at work. 

The main reason DLA cannot simply purchase the commercial versions of tools is security, as current 

events show. DoD/DLA has, therefore, developed strong security controls to ensure government data 

is secure. 

These security controls create a feedback loop with the standard acquisition process to prevent rapid 

deployments. Slow deployments are somewhat appropriate when an organization is sure it wants a 

capability. To the contrary, when an organization is not sure about the capability, slow deployments 

ensure that we fail slowly.

The current paradigm: J6 buys new software but must also makes significant infrastructure and 

sustainment investments to stand up each new capability, often prior to a thorough evaluation of the 

software. Current evaluations satisfy general market research and acquisition standards but do not test 

for the user experience. Will users like the software? Will users adopt the software? Currently, we 

only discover this after large stand-up investments.

The prototype paradigm: The purpose of this project is to determine how might DLA J6 safely and 

rapidly test and evaluate popular software products without first making costly IT infrastructure 

investments. We seek to make the user experience the first step of a comprehensive business process 

prototype.
Project DLA-OTA-000-0004: Comprehensive Software Test & Evaluation

4 of 7

PROBLEM STATEMENT:

J6 Program Managers cannot safely and rapidly test and evaluate the user experience of software 

products in order to select which products become official DLA capabilities. 

Required Solution Features. Offerors have broad latitude to propose novel solutions. We 

encourage it. All solutions must, however, include the following components:

1. Cloud Management. This means setting up and configuring the test and evaluation 

environment in Amazon Web Services (“AWS”), Google Cloud and/or Microsoft Azure. 

Offerors may propose solutions that encompass one, two or all three platforms. Note: 

Each platform must conform to DLA’s cybersecurity standards, but part of the project is to 

work collaboratively to define the “minimum viable” standards—e.g. the awardee and DLA 

cybersecurity SMEs will develop this together. 

2. Technical Consulting. This means working with the government to conduct discovery 

during the test and evaluation period to identify technical requirements, create a roadmap, 

and to make a comprehensive cost estimate of deploying the software at scale in the 

production DLA environment. 

3. Technology Provider . This means obtaining, deploying and sustaining the requested 

technologies in the test and evaluation environment for the evaluation period. Additionally,

it means coordinating with other technology providers to get prototype solutions working 

in the test and evaluation space.

Likely Solution Features: These are not requirements. Rather, based on our research, this is a 

solution archetype available in the marketplace. Regardless of the proposed solution, offerors should 

take note of these bullets as they represent important considerations:

• Key personnel have expertise meeting and managing DoD applications in an IL2 (or greater) 

environment

• Offeror has access to or ability to rapidly stand up an IL2 environment (or greater)

• Offeror can obtain quotes from a variety of software providers, with the ability to compare 

competitors and get better-than-retail pricing (six-month to one-year trials)
Project DLA-OTA-000-0004: Comprehensive Software Test & Evaluation

5 of 7

Scaling Potential:

This project encompasses a series of prototype implementations as well as the DoD-like environment 

in which to deploy them. The prototype cloud environments will include a two-year period of 

performance to design a prototype process. DLA contemplates follow-on production for successful 

implementations as well as the overall approach. For awardees, this can mean OTA-for-production 

work on a rolling basis and in a variety shapes: software, cloud environments, implementation work. 

Since this is a need across the DoD, this effort might expand to other DoD stakeholders.

As with any iterative process, we cannot envision every possible scaling scenario. Based on the 

information the government currently has, here are some possible scenarios:

Archetype #1 – Deploy T&E software in DLA-managed environment (OTA awardee)

• OTA awardee provides implementation services—e.g., executes learnings from an individual 

software T&E effort

• This could occur with the first software pilot, as early as a few months post award

• This could occur on a rolling basis

Archetype #2 – Deploy T&E software in DLA-managed environment (software developer)

• Software developer provides implementation services—e.g., executes learnings from an 

individual software T&E effort

• This could occur with the first software pilot, as early as a few months post-award

• This could occur on a rolling basis

Archetype #3 – Deploy T&E software in DLA-managed environment (License Only)

• DLA autonomously implements software but acquires licenses through prime contractor or 

software subcontractor

• This could occur with the first software pilot, as early as a few months post-award

• This could occur on a rolling basis

Archetype #4 – Entire Process Moves from R&D to Production (Major Scaling)

• The entire business process proves cost-effective for the government; thus the refined version 

of T&E business process prototype becomes a sustained program

• DLA’s software “Front Door” evaluates approximately 150 new software products per year

• This would occur after the entire prototype period of performance

Archetype #5 – Novel Features from R&D to Production (Piecemeal Scaling)

• Cloud environments, technical implementation services, technical consulting`, etc.
Project DLA-OTA-000-0004: Comprehensive Software Test & Evaluation

6 of 7

SUBMISSION CRITERIA & EVALUATION PROCESS

The Government shall evaluate each offeror submission on the following five technical criteria, all of 

equal importance. The government shall also consider price. The first three technical criteria2, 

popularized by IDEO3, are a common method to drive enterprise innovation and build successful 

prototypes. These criteria help prevent common dead ends and drive prototypes that are actionable, 

which the government is likely to adopt and scale.

Feasibility – This criterion measures whether the technology exists or is likely to be developed in the 

scope of this prototype effort. Fundamentally, this criterion allows you to demonstrate technical 

mastery of cloud management and software implementation. Ways to prove this include: successful 

cloud deployments/similar efforts; a list of commercial customers with existing cloud deployments; 

relationships and certifications with Amazon, Google and Microsoft clouds.

 

Viability – This criterion measures whether DoD could easily adopt the prototype. Ways to prove this 

include: experience with DoD Secure Technical Implementation Guides (“STIG”s), Security

Requirements Guides (“SRG”s), and going through the authority to operate (“ATO”) process. The 

level of familiarity with these and other DoD technical requirements would dictate the overall viability 

score. 

Desirability – The output of the technical consulting work gives the government a clear 

understanding of the roadmap towards implementation that includes, at a minimum, the financial and 

technical requirements necessary to scale a prototype implementation to production. Offerors will 

NOT be evaluated on the desirability of the software being tested and evaluated in the environment. 

Rather, desirability goes to offer’s ability to create a novel, transparent and comprehensive process. 

Responsiveness - All submissions must be responsive to the solicitation’s problem statement. The 

government reserves the right to award any submission (as described in the Program Procedures). 

However, for the purposes of this effort, the government will ONLY consider CLEARLY responsive 

submissions. Non-responsive submissions may receive further consideration later, as dictated by 

available funding and priorities.

Time to Impact - Time-to-impact measures the time it will take to develop and deploy within the 

DoD against the demonstrable value it will provide DLA’s end users. Solutions that offer major 

impacts with quick development and deployment timelines are better than solutions with negligible 

impacts and long timelines. The best solutions will begin to quickly deliver major gains while still in the 

iterative design and development phase. 

Offerors must submit the application form for consideration; the government shall not consider any 

information other than the application itself and other explicitly requested documents. 

2 https://medium.com/innovation-sweet-spot/desirability-feasibility-viability-the-sweet-spot-for-innovationd7946de2183c

3 IDEO is a prominent Silicon Valley-based design firm. See IDEO.org
Project DLA-OTA-000-0004: Comprehensive Software Test & Evaluation

7 of 7

Within one month from closing date, the government shall respond to each application. At that time, 

the government shall inform each offeror that: 

1.) the government has not selected to move forward with the submitted application; or 

2.) the government requests that the offeror participate in an in-person (or virtual) pitch; or

3.) there has been a delay in evaluation and the new expected timeline.

The government shall use the same criteria to evaluate pitches as described for white paper

applications.

After pitches, the government shall further down-select potential awardees and issue a request for 

prototype proposal (“RPP”) to the remaining candidate firm(s). The RPP will have specific guidelines. 

Chiefly, offerors must submit a statement of work and a detailed price breakdown as it relates to 

payment milestones. The government shall use the same criteria to evaluate prototype proposals as 

described for the application and solution briefs.

PROJECT DURATION, ESTIMATED FUNDING & AWARD DATE:

We seek to develop a pricing model collaboratively. Price proposals might address items like:

• Cloud set-up, deployment and successful authority to operate (“ATO”)

• Configuration and sustainment (labor) costs for the environment and for each software 

implementation

• Required Software licenses 

• Implementation roadmap during test and evaluation period

Initial Funding: $300,000

The government will likely provide software licenses for the first pilot. (e.g., a test software product). 

For each additional software product, either DLA will provide the licenses or (more likely) DLA will 

add additional funds to this award prove out the comprehensive business process prototype. 

The Government reserves the right to award multiple prototypes. Depending on technical merit, the 

government may allocate this funding:

• Across multiple awards to various companies;

• To a single company; or 

• Any other configuration/allocation, including no award.

Please direct all questions and comments before the white paper submission deadline to 

accelerate@dla.mil . Electronic copies of white papers due: February 26, 2020 by 1:00 PM ET.