##

# Dod Instruction 5000.88

## Engineering Of Defense Systems



Originating Component:
Office of the Under Secretary of Defense for Research and Engineering

Effective:
November 18, 2020

Releasability:
Cleared for public release.  Available on the Directives Division Website
at https://www.esd.whs.mil/DD/.

Incorporates and Cancels: Enclosure 3, "Systems Engineering," of DoD Instruction 5000.02T,
"Operation of the Defense Acquisition System"

Deputy Secretary of Defense Memorandum, "Policy Memorandum for Independent Technical Risk Assessments for Major Defense Acquisitions Programs," December 3, 2019

Approved by:
Michael J. Kratsios, Acting Under Secretary of Defense for Research and
Engineering

Purpose: In accordance with the authority in DoD Directive (DoDD) 5137.02 and the guidance in Section 133a of Title 10, United States Code (U.S.C.), this issuance establishes policy, assigns responsibilities, and provides procedures to implement engineering of defense systems.



## Table Of Contents

 SECTION 1:  GENERAL ISSUANCE INFORMATION ........ 4
1.1.  Applicability. ........ 4 1.2.  Policy. ........ 4
SECTION 2:  RESPONSIBILITIES ........ 5
2.1.  Under Secretary of Defense for Research and Engineering (USD(R&E)). ........ 5 2.2.  Under Secretary of Defense for Acquisition and Sustainment (USD(A&S))........ 5 2.3.  Under Secretary of Defense for Intelligence and Security. ........ 5 2.4.  DoD Component Heads, Except the Chairman of the Joint Chiefs of Staff. ........ 6 2.5.  Chairman of the Joint Chiefs of Staff. ........ 7
SECTION 3:  ENGINEERING APPLICATION REQUIREMENTS ........ 8
3.1.  Introduction. ........ 8
a.  Engineering Overview. ........ 8 b.  Engineering Guidance. ........ 8
3.2.  Independent Review Teams (IRTs). ........ 9 3.3.  ME and Concept Development. ........ 9 3.4.  Program Technical Planning and Management. ........ 11
a.  SEP. ........ 11 b.  Technical Baseline Management. ........ 14 c.  Configuration and Change Management. ........ 14 d.  Program Schedules........ 15 e.  Test and Evaluation. ........ 15 f.  Risk, Issue, and Opportunity Management. ........ 15 g.  Program Protection. ........ 16
3.5.  Program Technical Reviews and Assessments. ........ 16
a.  Technical Reviews. ........ 16 b.  ITRA. ........ 16 c.  Additional Assessments. ........ 19
3.6. Specialty Engineering. ........ 19
a.  Software Engineering. ........ 19 b.  Reliability and Maintainability (R&M). ........ 21 c.  Quality and Manufacturing. ........ 22 d.  Human Systems Integration. ........ 23 e.  System Safety. ........ 23 f.  Parts Management. ........ 25
3.7.  Design and Architectural Factors........ 25
a.  MOSA. ........ 25 b.  Spectrum Supportability. ........ 26 c.  Corrosion Prevention and Control. ........ 27 d.  Item Unique Identification. ........ 27 e.  Supportability. ........ 27 f.  Standardization. ........ 27
GLOSSARY ........ 28
G.1.  Acronyms. ........ 28

G.2.  Definitions. ........ 29
REFERENCES ........ 30

## Section 1:  General Issuance Information 1.1.  Applicability.

This issuance applies to OSD, the Military Departments, the Office of the Chairman of the Joint Chiefs of Staff and the Joint Staff, the Combatant Commands, the Office of Inspector General of the Department of Defense, the Defense Agencies, the DoD Field Activities, and all other organizational entities within the DoD (referred to collectively in this issuance as the "DoD Components").

## 1.2.  Policy.

The DoD will conduct a comprehensive engineering program for defense systems, including the engineering management activities necessary to guide the development of defense systems.

a.  The engineering management activities include, but are not limited to:
(1)  Mission engineering (ME).
(2)  Systems engineering. (3)  Technical risk assessments.
b.  A systems engineering plan (SEP), which provides a foundational engineering approach for all technology based programs, is required for Major Defense Acquisition Programs (MDAP) and acquisition category (ACAT) II and III programs, unless waived by the SEP approval authority.

(1)  SEP content for MDAPs and ACAT II and III programs can be tailored with approval by the SEP approval authority.

(2)  SEPs are a recommended best practice for all other defense system development.

This issuance can be tailored, as necessary, for each acquisition pathway.

## Section 2:  Responsibilities

2.1.  UNDER SECRETARY OF DEFENSE FOR RESEARCH AND ENGINEERING (USD(R&E)).

The USD(R&E):
 a.  Establishes policies and strategic guidance and leads defense research; engineering;
developmental prototyping and experimentation; technology development, exploitation, transition, and transfer; developmental test and evaluation; and manufacturing technology activities, including operation of the DoD manufacturing innovation institutes; and microelectronics activities across the DoD Components.

b.  Establishes policy and guidance for the conduct of independent technical risk assessments
(ITRAs), consistent with Section 2448b of Title 10, U.S.C.

c.  Conducts and approves ITRAs for ACAT ID programs.
d.  Determines ITRA approval authority for ACAT IB/IC programs. e.  Establishes policy (with the exception of Middle Tier of Acquisition prototyping) and exercises oversight authority over all DoD uses of developmental prototyping.
f.  Establishes policies for development and approval of systems engineering plans and program protection plans.

## 2.2.  Under Secretary Of Defense For Acquisition And Sustainment (Usd(A&S)).

The USD(A&S):
 a.  Serves as milestone decision authority (MDA) for ACAT 1D programs and designates alternate MDAs, where appropriate.

b.  Reviews and approves, as appropriate, the acquisition strategy at all required decision points for ACAT 1D programs, consistent with Section 2431a (d) of Title 10, U.S.C.

c.  Approves the use of the Middle Tier of Acquisition pathway for programs that exceed the MDAP threshold.

## 2.3.  Under Secretary Of Defense For Intelligence And Security.

The Under Secretary of Defense for Intelligence and Security:
a.  Advises and assists the program in the integration of intelligence data, counterintelligence, and security requirements during defense systems engineering as part of the acquisition lifecycle.

b.  Oversees defense intelligence enterprise performance in meeting critical intelligence priorities for defense systems engineering.

c.  Advises and assists the DoD Component heads with identifying critical program and technology information in support of Program Protection Plan (PPP) preparation.

## 2.4.  Dod Component Heads, Except The Chairman Of The Joint Chiefs Of Staff.

Except for the Chairman of the Joint Chiefs of Staff, who is covered in Paragraph 2.5., the DoD
Component heads:

a.  Implement the procedures outlined in this issuance.
b.  Appoint program managers (PMs) who will embed the engineering disciplines, management, and technical focus described in this issuance into program planning and execution to support the entire system life-cycle.
c.  Conduct ITRAs for ACAT IB/IC programs.  Ensure that DoD Component MDAs take under advisement the results from ITRAs when making acquisition program decisions.

d.  Consider technical advice provided by the USD(R&E) during analysis of alternatives
(AoA) studies.

e.  For ACAT ID programs:
(1)  Provide engineering information necessary to make informed technical assessments to the USD(R&E).

(2)  Provide technical data, as identified in the SEP and requested by the USD(R&E). (3)  Ensure that SEPs are developed in accordance with this issuance and approved by the USD(R&E) or his or her delegated authority.

f.  Implement ME and mission integration management (MIM) procedures established by the USD(R&E) and develop DoD Component guidance, as appropriate.

g.  Transparently share data, to the greatest extent possible, in its native form and require minimal formatting and manipulation.  All DoD data will be shared as widely as possible across the Military Services and OSD.  Options to prevent data transparency should not be entertained.

h.  Implement engineering processes focused on a series of best practices to include concept exploration, ME, technical baseline management, engineering technical reviews, peer and independent reviews, risk and configuration management (CM), and technical decisions.

## 2.5.  Chairman Of The Joint Chiefs Of Staff.

The Chairman of the Joint Chiefs of Staff:
a.  Advises and assesses on joint military capability needs in accordance with DoDD 5000.01.

b  Prepares and coordinates military analysis, options, and plans related to the engineering of defense systems in accordance with DoDD 5000.01, to include providing advice and analysis upon request through validated and approved capabilities documents.

## Section 3:  Engineering Application Requirements 3.1.  Introduction. A.  Engineering Overview.

(1)  The application of this engineering instruction covers systems engineering, and other engineering disciplines, in the development of DoD systems which aligns with industry best practices.  Systems engineering comprises a methodical and disciplined approach for the specification, design, development, realization, technical management, operations, and retirement of a system.

(a)  The engineering processes apply critical thinking in a structured multi-disciplined approach to solve problems including the balance of cost, schedule, and performance while maintaining an acceptable level of risk.

(b)  Although this issuance employs some terminology mainly applicable to the major capability acquisition pathway, the principles and practices described herein should be applied, as appropriate, to all DoD systems.

(2)  ME and MIM activities will be performed as part of concept and system development to inform developmental decisions and ensure the department is systematically investing in the appropriate capabilities, in an integrated and cost effective manner, to meet mission needs.

(a)  ME and MIM activities start before conducting material solution analysis in order to inform development of the concept baseline, and continue through the acquisition life-cycle.

(b)  The Office of the USD(R&E) (OUSD(R&E)) will promulgate ME and integration management guidance, standards, and infrastructure to govern and digitally facilitate integration and data sharing across all DoD and OSD Components and life-cycle phases.

(3)  The Military Services, PMs, lead systems engineers (LSEs), and product support managers will implement engineering processes focused on a series of best practices to include concept exploration, ME, technical baseline management, engineering technical reviews, peer and independent reviews, test and evaluation, risk and CM, and technical decisions, while ensuring the security and integrity of capabilities and services.

(4)  The systems engineering and engineering management approach and processes that guide all technical activities of the program will be documented in an SEP.  The SEP describes key technical risks, processes, resources, metrics, engineering products, organizations, and design considerations.  The SEP is highly specific for each program and will be updated as needed to reflect the program's evolving systems engineering approach, plans, and current status.

## B.  Engineering Guidance.

Engineering application in the development of DoD systems is provided in the Defense Acquisition Guidebook and other guidance documents listed in the References section.

## 3.2.  Independent Review Teams (Irts).

a.  Periodic review by independent technical personnel is a core best practice fundamental to engineering development and managing risk.

(1)  The component acquisition executive (CAE) will implement a technical review process, in accordance with Paragraph 3.5., which incorporates participation and review by an IRT.  The CAE will approve IRT members to ensure all organizational, professional, and relational influences from the program management office are avoided.

(2)  Large acquisition programs, such as MDAPs, may require IRT composition from separate U.S. Government organizations, whereas smaller acquisition programs may be able to structure an independent team from within the organization.  Ideally, the IRT is consistent throughout the program life-cycle and serves as a trusted technical advisor to the CAE.

b.  The IRT will identify and document critical issues that jeopardize achieving program or mission objectives, to include recommended corrective action.  Results will be provided directly to the CAE, with coordination but not undue influence from the Program Managers Office.  The PM, with support from the LSE, will review, develop, and implement corrective action to the satisfaction of the CAE.

c.  OUSD(R&E) will monitor the implementation of the independent review process.

## 3.3.  Me And Concept Development.

a.  Engineering activities begin at the identification of a military need and continue throughout sustainment of the end item.  OUSD(R&E) and the DoD Components will conduct an intensive process of ME and engineering trades leading to a concept design review and the establishment of a concept baseline.  ME products will be developed as required to guide Planning, Programming, Budgeting, and Execution inputs, drive new technology development and experiments, provide mission-based inputs to the requirements process, and justify program or prototype initiation or continuation.

b.  OUSD(R&E) and the DoD Components will collaboratively perform ME and MIM in accordance with the USD(R&E) ME Guide and Section 855 of Public Law 114-328.  ME will consist of an evolving analysis, including gap analysis, of the mission-efficacy based on programs or prototypes contributing to a mission area.

(1)  These analyses and artifacts constitute the mission baseline and will contain mission definition and scenario(s), mission objectives, and interdependencies with other existing and future systems architectures, mission measures of effectiveness, security, threat quantification, analytical models, and data.

(2)  The content and order of artifact development to support the mission and concept baselines should be tailored into the program acquisition strategy.

c.  Before making a materiel development decision, DoD Components will conduct concept exploration and ME activities.  DoD Components will assess and leverage, as applicable, analyses, prototyping, experimentation, and test results and activities to assist in the formulation of the mission and concept baselines.  DoD Components will utilize all of these activities to establish the mission baseline to support the development of preliminary concept design(s), and to inform the AoA study guidance and plan.  While mission baselines are established before conduct of the AoA, concept design(s) and concept baseline activities are continued after completion of the AoA.

d.  Mission reviews will be conducted before the materiel development decision to establish a mission baseline and a preliminary concept design trade matrix.

(1)  A USD(R&E) representative will chair mission reviews for joint missions.  The applicable Service representative will chair mission reviews for Service-specific missions.

(2)  The mission review is used to formally review the initial ME assessment of the prioritized mission gaps, initial capability concept(s) and alternatives, and initial assessment of risks.

(3)  The Joint Staff and Military Services, in collaboration with OUSD(R&E), will review mission and capability gaps and concepts as part of the requirements process governed by Chairman of the Joint Chiefs of Staff Instruction 5123.01H and the Joint Capabilities Integration and Development System Manual.

(4)  The mission review products will include the approved concept design trade matrix and mission baseline.  These products will support a concept design review, a materiel development decision, and entry into the applicable acquisition pathway.

e.  A concept design review will be conducted before the materiel development decision where the initial concept baseline(s) will be established.  The concept design review will be chaired by a USD(R&E) representative for joint missions and by the applicable Service representative for Service-specific missions.  The concept baseline should include:

(1)  Framing assumptions. (2)  Capabilities-based assessment. (3)  Initial capabilities document.
(4)  Concept design trade matrix.
(5)  ME analysis.
(6)  A concept of operations (CONOPS) or Operational Mode Summary/Mission Profile.
(7)  Assessment of program risks along with technology development and other risk
mitigation activities, appropriate affordability targets, and initial schedule basis.
(8)  Cyber security assessment.
f.  After the materiel development decision and acquisition pathway initiation, the DoD
Component will refine the mission baseline and concept design trade matrix.  The component technical lead will:
(1)  Conduct ME and engineering trade-off analyses to inform an AoA, finalize the AoA
report, AoA checklist, and support the development of associated requirements or capabilities documentation.

(2)  Use data driven benchmarks from previous program developments, and any applicable prototyping and experimentation, to serve as a basis for identifying risks and opportunities, technical work breakdown, performance growth, schedule, and cost.
g.  For all MDAPs, the DoD Components or the PM will:
(1)  Make ME and MIM analysis results available to OUSD(R&E) and the Office of the USD(A&S) (OUSD(A&S)) to be included in executive-level and technical trades, and to support technical reviews, ITRAs, and milestones.
(2)  Make updated set of ME analysis and artifacts available to OUSD(R&E) and OUSD(A&S) as exhibits in support of change(s) to a requirements or capabilities document, developmental planning trade-offs, an ITRA, and a milestone decision (or equivalent).  DoD
Components or the PM are encouraged to share ME artifacts to foster synergistic solutions, creatively explore alternative solutions, and foster modular open system approaches.

(3)  Make digital models and computationally consumable data, created from engineering, analysis, test, modeling, and simulations, available to the USD(R&E) and the Joint Chiefs of Staff in an agreed upon format.

(4)  Make mission-based inputs available in a digital format to the USD(R&E) and the Joint Chiefs of Staff to support the requirements process, assessment of concepts, prototypes, design and test options, budgeting and resource allocation, and program and portfolio management.

(5)  Support coordination with the Combatant Commands on the development of CONOPS and operational plans.

(6)  Use existing ME constructs as a basis in performing ME activities for new and emerging capabilities to the maximum extent practicable.

## 3.4.  Program Technical Planning And Management. A.  Sep.

(1)  SEPs are highly program specific and an important tool in managing complex technology based system development.

(a)  The LSE will:
1.  Under the direction of the PM, develop a SEP in order to document and guide the program's specific systems engineering activities.

2.  Develop a SEP in accordance with the DoD SEP Outline and include the content described in Paragraph 3.4.a.(3).

(b)  SEPs are required for all MDAP programs unless waived by the approval authority.  SEPs are also required for all ACAT II and III programs unless waived by the DoD Component.  The USD(R&E), or designee, is the approval authority for ACAT ID program SEPs.  The MDA, or designee, is the approval authority for ACAT IB/IC SEPs.  The CAE will designate an approval authority for all other programs.

(2)  SEPs will be approved before release of requests for proposals (RFPs) supporting major program phases to include each major prototyping effort; technology maturation and risk reduction (TMRR); engineering and manufacturing development (EMD); low rate initial production; and full rate production.

(a)  The SEP will be included with the RFP.

(b)  As required, the LSE will update the SEP to address substantive changes resulting from contract award.  The updated SEP, if required, will be approved at least 120 days after contract award or 30 days before the next technical review, whichever comes first.

(c)  ACAT ID SEPs will be submitted to the USD(R&E) for review and approval at least 30 days before the required approval date.

(d)  For other MDAPs, SEPs should be submitted within 30 days of approval to the designated approval authority, with approved SEPs provided to the USD(R&E) for information purposes.

(3)  For MDAPs, ACAT II, and ACAT III programs, the SEP will contain these elements, unless waived by the SEP approval authority:
(a)  The overall technical approach for system design and development, which balances system performance, life-cycle cost, schedule, and risks in addressing mission needs.
For MDAPs, the technical approach will incorporate a modular open systems approach (MOSA) to the maximum extent practicable.  All other programs should consider implementing MOSA.

(b)  The engineering management approach to include technical baseline management; requirements traceability; CM; risk, issue, and opportunity management; and technical trades and evaluation criteria.

(c)  The software development approach to include architecture design considerations; software unique risks; software obsolescence; inclusion of software in technical reviews; identification, tracking, and reporting of metrics for software technical performance, process, progress, and quality; software system safety and security considerations; and software development resources.
(d)  Engineering trade-off analyses to be performed, including trade-offs to assess system affordability and technical feasibility to support requirements, investment, and acquisition decisions.

(e)  Planning assumptions, along with a description of methods and frequency for conducting formal and informal schedule risk assessments and health checks over the lifecycle.

(f)  A description of the program's integrated master plan (IMP) and integrated master schedule (IMS) process, to include definitions, updated schedules, audits, baseline control, and the integration between program-level and contractor detailed schedules.  The program-level IMP will be included as an attachment to the SEP, and the IMS will be made available in its native format to support ITRAs and other assessments.

(g)  Specific technical performance measures and metrics, and system engineering leading indicators to provide insight into the system technical maturation relative to a baseline plan.  Include the maturation strategy, assumptions, reporting methodology and maturation plans for each metric with traceability of each performance metric to system requirements and mission capability characteristics.

(h)  Specific technical data to be provided digitally, in an agreed upon format, and the frequency of the availability of the technical data.

(i)  Reliability growth curve(s) along with assumptions, planning factors, and planned assessment tools and methods.

(j)  The required contract deliverables, technical data, design artifacts, and the periodicity of reporting.

(k)  The timing, conduct, and entry and exit criteria for technical reviews.

(l)  A description of technical baselines (e.g., concept, functional, allocated, and product), baseline content, and the technical baseline management process.

(m)  The digital engineering implementation plan to include model elements, element relationship diagrams, activity diagrams, block definition diagrams, and use case diagrams.  The plan must include the evolution of a continuous end-to-end digital representation, or integrated set of digital representations, of the system being produced and the establishment of a digital authoritative source of truth (i.e., configuration controlled digital baseline).  The PM will make the relevant digital model(s) accessible to OSD, Joint Staff stakeholders, and interdependent programs, throughout the life of the program and will maintain CM.
(n)  A high level description of the CONOPS that includes mission scenarios, design reference missions, and operational functions of the system and the relation to the design approach.  Programs should provide the draft or approved CONOPS as an attachment.

(o)  Unless otherwise justified, a development and operations strategy enabling early and continuous integration and testing to validate mission effectiveness early and throughout the development life-cycle.

(p)  For MDAPs, the plan to assess and document the technology maturity of all potential critical technologies and plans to provide test results and artifacts demonstrating technology maturity to the ITRA team for independent assessment.
(q)  The program's major technical risks, issues, opportunities, and mitigations and planning activities.

(r)  The MOSA and program interdependencies with other programs and components, to include standardized interfaces and schedule dependencies.
(s)  The plan to manage intellectual property (IP) and data rights. (t)  Specialty engineering and architectural factors as described in Paragraphs 3.6. and
3.7., and any additional applicable design considerations as described in the Defense Acquisition Guidebook.

## B.  Technical Baseline Management.

The PM will implement and describe in the SEP a technical baseline management process as a mechanism to manage technical maturity, to include a mission, concept, functional, allocated, and product baseline.  If practicable, the PM will establish and manage the technical baseline as a digital authoritative source of truth.

(1)  The LSE, under the direction of the PM, will establish and maintain the functional, allocated, and product baselines via the appropriate systems engineering technical reviews as described in the Defense Acquisition Guidebook.

(2)  The PM will assume control of the initial product baseline Class I configuration changes, as defined in accordance with the program's CM plan, from the contractor at completion of the system-level critical design review (CDR).

## C.  Configuration And Change Management.

The LSE, under the direction of the PM, will implement a digital CM approach and automated tools to establish, control, and curate product attributes and technical baselines across the total system life-cycle.  The CM approach will:
(1)  Identify, document, audit, and control schedule, cost, functional, physical, and performance characteristics of the system design.

(2)  Specifically, track any changes (e.g., a dynamic change log for in and out of scope changes, formal engineering change proposals) and provide an audit trail of program design decisions and design modifications.
(3)  Provide for traceability of mission capability to system requirements to performance and execution metrics.

## D.  Program Schedules.

(1)  In accordance with the Department of Defense Earned Value Management System Interpretation Guide, the PM will ensure an IMP and IMS are developed and maintained throughout the life of the program.

(a)  The program IMP and IMS will account for program activities, review and assessment events, interdependencies with other programs, and contracted technical activities and tasks.

(b)  For programs where the program office is serving as the systems integrator, the PM will develop and maintain the system-level IMP and IMS.  For other programs, the PM may contract this task to the contractor(s).

(2)  The PM will provide (or make digitally accessible) an updated IMP and IMS and a schedule risk assessment in accordance with the Defense Contract Management Agency's EA Pamphlet 200.1 in support of technical reviews, ITRAs, major milestones, and significant unplanned program changes.

## E.  Test And Evaluation.

The PM will ensure test and evaluation planning and program activities are conducted in accordance with Enclosures 4 and 5 of DoD Instruction (DoDI) 5000.02T.  To the greatest extent possible, the test and evaluation plan will use and contribute to the information contained in the evolving digital system representation.

## F.  Risk, Issue, And Opportunity Management.

(1)  The LSE will advise the PM on major technical risks, issues, opportunities, and mitigation planning and implementation and document them in the SEP.  The PM will:
(a)  Integrate risk, issue, and opportunity management planning and execution in accordance with the Department of Defense Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs.

(b)  Establish a process that considers risks across the entire life-cycle and not be constrained to the current phase.

(2)  Risk management plans will address risk identification, analysis, mitigation planning, mitigation implementation, and tracking.  Technical risks and issues will be reflected in the program's IMP and IMS.

(3)  Opportunity management will identify potential opportunities to include technology development that could have a positive impact on providing better value in performance, improved mission capability, and reduced cost and schedule.

g.  Program Protection.
To maintain technology dominance, the PM will prepare a PPP.  The PPP will serve as a technical planning tool to guide system security engineering activities, which includes software assurance, for the program in accordance with DoDI 5000.83.

## 3.5.  Program Technical Reviews And Assessments. A.  Technical Reviews.

(1)  Systems engineering technical reviews provide a venue to establish the technical baselines, assess the system's technical maturity, and review and assess technical risks.  At each technical review, the PM will, to the extent practicable, use information from the digital authoritative source of truth to assess key risks, issues, opportunities, and mitigation plans in order to understand cost, schedule, and performance implications.

(2)  Unless waived through the SEP approval process, the PM will conduct these system level reviews, or equivalent:

(a)  System requirements review or system functional review.
(b)  Preliminary design review (PDR). (c)  CDR.
(d)  System verification review or functional configuration audit.
(e)  Production readiness review. (f)  Physical configuration audit.
(3)  The PM will include participation of OUSD(R&E) representatives for ACAT ID
programs.  Additionally, the PM will ensure a USD(R&E) representative is invited to all ACAT ID sub-system PDRs and CDRs.

(4)  In accordance with Section 2366b of Title 10, U.S.C., OUSD(R&E) will conduct a PDR assessment for ACAT 1D programs.  In addition, OUSD(R&E) will conduct a CDR assessment for ACAT ID programs.  The results of these assessments will be used to inform the MDA of any technical risks, maturation of the technical baseline, and the program's readiness to proceed.  For all other MDAPs, the DoD Component concerned will conduct PDR and CDR assessments.

## B.  Itra. (1)  Itras:

(a)  Provide a view of program technical risk, independent of the program and the chain of command leading to the MDA.
(b)  Are conducted on all MDAPs before approval of Milestone A, Milestone B, and any decision to enter into low-rate initial production or full-rate production.
(c)  Pursuant to Section 2448b of Title 10, U.S.C., are required for programs either initiated or having a Milestone A after October 1, 2017.  For programs initiated or having a Milestone A before October 1, 2017, ITRAs are regulatory and may be waived at the discretion of the USD(R&E).  Formal requests will provide appropriate justification and will be submitted through the MDA.

(d)  Consider the full spectrum of technology, engineering, and integration risk.
These areas could include mission capability, technology, system development, MOSA, software, security, manufacturing, sustainment, and their potential impacts to cost, schedule, and performance.  For ITRAs conducted before Milestone A, identifies critical technologies and manufacturing processes that need to be matured.  Subsequent ITRAs will re-assess technology and manufacturing process maturity, accounting for demonstrations in relevant environments.

(e)  Are conducted and approved by the USD(R&E) on all ACAT ID programs.  The USD(R&E) will determine ITRA approval authority for ACAT IB/IC programs, providing periodic written notification, based on the following criteria:

1.  Significant or strategic joint mission integration and interoperability
requirements.

2.  Significant contribution to one or multiple national defense strategy or
OUSD(R&E) modernization roadmaps.
3.  Demonstrated program poor performance such as a Nunn-McCurdy breach or
program restructure.
4.  Criticality to a major interagency requirement or technology development
effort, or having significant international partner involvement.
5.  Congressional or special interest due to scope, complexity, or other issues.
(f)  As determined by the USD(R&E), are conducted on special access programs that exceed MDAP dollar thresholds and programs designated by an MDA as ACAT I special interest programs.  ITRAs for special access programs will be coordinated through the DoD Special Access Program Central Office in accordance with DoDD 5135.02 and comply with DoDD 5205.07 and DoDI 5205.11.

(g)  Facilitate the MDA's establishment of program cost, schedule, and performance goals pursuant to Section 2448a of Title 10, U.S.C.
(h)  Support MDA determinations, certifications, and reporting to Congress.
Consistent with Sections 2448b, 2366a, and 2366b of Title 10, U.S.C., MDAs will consider the results of the ITRA before approving milestone or production decisions for an MDAP.

(2)  DoD Components will conduct ACAT IB/IC ITRAs.  Approval will be consistent with Paragraph 3.5.b.(1)(e).  The approval authority must be independent and may not be in the program's chain of command.  ITRAs are not required for non-MDAP programs, but if conducted, will follow the OUSD(R&E)-published ITRA policy and guidance.  ITRAs will be conducted in accordance with DoD ITRA guidance and the DoD ITRA Framework for Risk Categorization both developed and maintained by OUSD(R&E).
(3)  The organization conducting the ITRA designates a lead, who will form a team composed of technical experts with in-depth domain knowledge of technical considerations associated with the program under assessment.  Team members should be independent from the program office and the direct chain of command between the program office and MDA.
(a)  The ITRA team:
1.  Should engage as early as possible in the program lifecycle to maximize program understanding and facilitate engagement in ongoing program activities with the goal of minimizing program impact.
2.  Leverages existing program information, modeling, simulation results, analysis results, prototyping activities, test and evaluation reports, artifacts (digital and non-digital), and any other information, in native format, deemed appropriate.

(b)  The ITRA team lead:
1.  Reviews findings and risks with the PM as early as possible to allow for mitigation activities deemed appropriate by the PM.
2.  Should prepare a final assessment in time to support approval not later than 30
days before the Milestone or production decision.
(4)  Consistent with Sections 2366a(c)(2), 2366b(c)(3), and 2366c(b) of Title 10, U.S.C., organizations conducting and approving ITRAs will retain the underlying documentation and analysis supporting the assessment of risks, findings, and assertions for congressional committee inquiry.

(5)  For programs for which an ITRA is conducted, a technology readiness assessment report is not required.  Programs will continue to assess and document the technology maturity of all critical technologies consistent with the technology readiness assessment guidance.  ITRA
teams may leverage technology maturation activities and receive access to results in order to perform independent technical reviews and assessments.

(6)  The designated DoD Component, agency, or PM will:
(a)  Support ITRA execution, to include providing access to programmatic and technical information and facilitating ITRA team visits to the program office, product centers, test centers, and contractor(s).

(b)  Provide OUSD(R&E) advanced notice of milestone or production decision dates for programs requiring an ITRA, to include special interest or special access programs, to facilitate timely execution and determination of OUSD(R&E) roles.  Initial notification should occur as soon as practicable, with a goal of 18 months before the milestone or production decision or 9 months before RFP release, whichever is earlier.

(c)  Assess and document the technology maturity of all potential critical technologies and provide the results for independent review and assessment by the ITRA team.

(d) Provide OUSD(R&E) with copies of approved ITRA reports including substantiating documentation and analysis needed to support assessment of risks, findings, and assertions in accordance with the requirements of Sections 2366a(c)(2), 2366b(c)(3), and 2366c(b) of Title 10, U.S.C.

## C.  Additional Assessments.

(1)  OUSD(R&E) may also conduct non-advocate reviews, or focused technical assessments on any program, or may conduct an out of cycle assessment, at the request of senior leadership in OSD or the Military Services.

(2)  PMs and program executive officers may also request an assessment through OUSD(R&E).  These requests will be made to and approved by the USD(R&E).  While some of these assessments may be highly tailorable, the assessment team will work with the PMs and program executive officers in an attempt to utilize ITRA methods and practices when practicable.

## 3.6. Specialty Engineering.

The impact of specialty engineering activities on total system cost, schedule, and performance will determine the extent of their application during the system design process.  Execution of activities in specialty engineering will, to the largest extent practicable, use information from, and contribute to, the digital authoritative source of truth.

## A.  Software Engineering.

The development and sustainment of software can be a major portion of the total system cost and should be considered throughout the acquisition life-cycle.

(1)  The PM will select the appropriate software development approach based on scope, requirements, schedule, and risk, and should consider an iterative software development process using modern agile development and operations methods.  The PM should:
(a)  Assign a lead software engineer to manage the software acquisition team, software engineering processes, and delivery of code.

(b)  Consider establishing a software factory with multiple pipelines to deliver capability in a series of manageable, minimum viable products, to gain user acceptance and feedback for the next viable product.  The software factory includes the trained personnel, culture, architecture, processes, and tools that automate the activities in software development, build, test, and delivery cycles.

(2)  The PM and lead software engineer will implement a software development approach.

(a)  The approach will address:
1.  Software architecture design considerations. 2.  Software re-use and commercial off-the-shelf (COTS) integration.
3.  Software obsolescence.
4.  Inclusion of software configuration items in technical reviews. 5.  Software system safety and software security considerations.
6.  Metrics identification, tracking, and reporting to address software technical performance, development process, and quality.

7.  Software development resources. 8.  Software unique program risks.
(b)  The program may automate collection of metrics as much as possible.
(c)  For those metrics that cannot be automated initially, the program may develop a plan for moving toward automation.  Programs may consider providing an automated read only self-service metrics portal for the Program Office, PEO, CAE, Defense Acquisition Executive (DAE), OUSD(A&S), OUSD(R&E), and other approved stakeholders as deemed appropriate.

(d)  PMs will be cognizant of and comply with DoDI 4630.09 in their software engineering development approach.  The PM and lead software engineer will document the software development approach and minimum metrics in the SEP.

(3)  The PM and lead software engineer will estimate the overall size and cost of the software development project using multiple software estimation methods. Initial software sizing estimates should be provided for each computer software configuration item and for each major build.

(a)  Software sizing estimates should include:
1.  Newly developed code. 2.  Reuse of pre-existing code. 3.  Modified existing code.
4.  Auto-generated software.
(b)  The integration, test, and certification of COTS software should be estimated separately in the program work break down structure.  COTS software should not be included as part of the initial size estimate.  Systematic estimation methods should be used to scope the software development effort and to compute software size (e.g., source lines of code, story points, function points, sprints) and must be normalized to be used for program benchmarking, comparisons for future builds and analogous programs.

## B.  Reliability And Maintainability (R&M).

(1)  For all defense acquisition programs, the LSE, working for the PM, will integrate R&M engineering as an integral part of the overall engineering process and the digital representation of the system being developed.

(a)  The LSE will plan and execute a comprehensive R&M program using an appropriate strategy consisting of engineering activities, products, and digital artifacts, including:

1.  R&M allocations, block diagrams, and predictions. 2.  Failure definitions and scoring criteria. 3.  Failure mode, effects, and criticality analysis, 4.  Maintainability and built-in test demonstrations. 5.  Reliability testing at the system and subsystem level. 6.  A failure reporting, analysis, and corrective action system maintained through
design, development, test, production, and sustainment.

(b)  For ACAT I (MDAPs) and II (Major Systems) weapon systems designs, the PM
will include in the contract and in the process for source selection, clearly defined and measureable R&M requirements and engineering activities as required by Section 2443 of Title 10, U.S.C.  The PMs of MDAPs and Major Systems must provide justification in the acquisition strategy for not including R&M requirements and engineering activities in TMRR, EMD, or production solicitations or contracts.

(2)  For MDAPs, the PM will conduct a preliminary reliability, availability, maintainability, and cost rationale analysis in support of the Milestone A decision or program initiation decision in accordance with the Reliability, Availability, Maintainability, and Cost Rationale Report Outline Guidance.

(a)  The analysis provides a quantitative basis for R&M performance attributes during the development of capability requirements, including product support and operating and support cost rationale and its specific correlation with the system's R&M attributes, ensuring the requirements are valid (e.g., support warfighter needs) and technically feasible.

(b)  The analysis will be attached to the SEP at Milestone A, or program initiation decision, and updated at subsequent milestones.

(3)  Assessments of development test data provide measures of effectiveness for the R&M engineering program and are used to track progress on reliability growth planning curves.

(a)  The LSE, working for the PM, will develop planning curves for each reliability threshold and include them in the SEP and, beginning at Milestone B, in the Test and Evaluation Master Plan.

(b)  Planning curves will be stated in a series of intermediate goals and tracked through fully integrated system-level test and evaluation events.  If a curve is not adequate to describe overall system reliability, curves for critical subsystems should also be developed.
Reliability growth will be monitored and reported in quarterly DAE Summary reviews, throughout developmental testing until the reliability threshold(s) are achieved.

(4)  The PMs of MDAPs and major systems will ensure incentive fees and penalties (as appropriate) that incentivize achievement of design specification requirements for R&M in all EMD and production solicitations and contracts is encouraged, pursuant to Section 2443 of Title 10, U.S.C.

(a)  Data collection methods to measure R&M requirements and to base determinations of contractor performance during EMD and production will be described in the contract.  The collected R&M data will be shared with appropriate contractor and U.S. Government organizations to the maximum extent practicable.

(b)  MDAs will notify the congressional defense committees upon entering into an EMD or production contract that includes incentive fees or penalties to the contractor based on achievement of R&M design specifications.  The MDA will provide a copy of the notification letters to OUSD(A&S) and OUSD(R&E).

## C.  Quality And Manufacturing.

The production, quality, and manufacturing (PQM) lead, working for the PM, will ensure manufacturing, producibility, and quality risks are identified and managed throughout the program's lifecycle.

(1)  Beginning in the materiel solution analysis phase, manufacturing readiness and risk will be assessed and documented in the SEP.

(2)  By the end of the TMRR Phase, manufacturing and quality processes will be assessed and demonstrated to the extent needed to verify that risk has been reduced to an acceptable level.

(3)  During the EMD Phase, the PQM lead will advise the PM on the maturity of critical manufacturing and quality processes to ensure they are affordable and executable.

(4)  Before a production decision, the PQM lead, working for the PM, will ensure that:

(a)  Manufacturing, producibility, and quality risks are acceptable. (b)  Supplier qualifications are completed.
(c)  Any applicable manufacturing processes are or will be under statistical process control.

## D.  Human Systems Integration.

The LSE will:
(1)  Working for the PM, use a human-centered design approach for system definition, design, development, test, and evaluation to optimize human-system performance.

(2)  Conduct frequent and iterative end user validation of features and usability for identifying, communicating, and visualizing user needs under defined operational conditions and expected mission threads.

(3)  Working for the PM, ensure human systems integration risks are identified and managed throughout the program's life-cycle.  For more information, refer to Enclosure 7 of DoDI 5000.02T.

## E.  System Safety.

The system safety standard practice identifies the DoD Systems Engineering approach to eliminating hazards, where possible, and minimizing risks where those hazards cannot be eliminated.

(1)  System Safety Engineering.
The LSE, working for the PM, will:
(a)  Integrate system safety engineering into the overall systems engineering process.

The LSE will use the methodology in Military Standard (MIL-STD)-882E to address environment, safety, and occupational health (ESOH) risks associated with system-related hazards.  In addition to MIL-STD-822E, the LSE will use the guidance identified in the DoD
Joint Software Systems Safety Engineering Handbook to achieve an acceptable level of software system safety risk.
(b)  Identify, document, and analyze identified hazards and assess the ESOH risks where hazards cannot be eliminated.

1.  The user representative, as defined in MIL-STD-882E, must be part of this process throughout the life-cycle and will provide formal concurrence before serious and high risk acceptance decisions.  Before exposing people, equipment, or the environment to known system-related hazards, the LSE will document that the associated risks have been accepted by these acceptance authorities: the CAE (or DAE) for high risks, program executive officer-level for serious risks, and the PM for medium and low risks.

2.  For joint programs, risk acceptance authorities reside within the lead DoD
Component.  The PM will report the status of ESOH risks and acceptance decisions at technical reviews.  Acquisition program reviews and fielding decisions will address the status of all serious and high ESOH risks.  The PM will manage risks associated with ESOH statutory requirements using the program overall risk, issue, and opportunity management processes.

## (2) Programmatic Esoh Evaluation (Peshe).

The PM of a major capability acquisition program, regardless of ACAT level, will maintain a PESHE that documents the status, results, and conclusions of the ESOH analyses and statutory compliance activities conducted in support of program execution.

(a)  For all other acquisition pathway programs, the PESHE may be tailored based on program schedule and performance requirements.

(b)  For all systems containing energetics, the LSE, working with the PM, will comply with insensitive munitions requirements in accordance with the DoD and component policy requirements as required by Section 2389 of Title 10, U.S.C.

(c)  The PESHE will summarize, at a minimum:

1.  Identified ESOH risks and their current status. 2.  Required external safety reviews, approvals, and certifications.
3.  Section 4321 of Title 42, U.S.C., also known and referred to in this issuance as the "National Environmental Policy Act of 1969 (NEPA)," and Executive Order (E.O.) 12114 compliance schedule.

4.  Identified hazardous materials, wastes, and pollutants (e.g., discharges, emissions, and noise) associated with the system and its support as well as the plans for minimization and/or safe disposal.

5.  Additional system and ESOH information needed by users, training and test locations, and receiving activities to prepare arrival and sustainment support of the system.

## (3) Nepa And E.O. 12114.

The PM will maintain a NEPA and E.O. 12114 compliance schedule that covers all known or projected system-related activities through FOC that may trigger compliance requirements including testing, fielding, and support of the system.

(a)  The compliance schedule will provide timelines and locations for system-related activities to enable consideration of potential impacts to the environment and completion of appropriate documentation in accordance with DoD Component implementing procedures.

(b)  The PM will conduct and document the NEPA and E.O. 12114 analyses for which the PM is the action proponent.  The PM will provide system-specific analyses and data to support other organizations' NEPA and E.O. 12114 analyses when the PM is not the action proponent.

(c)  The CAE or designee is the approval authority for system-related NEPA and E.O. 12114 documentation for which the PM is the action proponent.  For joint programs, the CAE is the lead DoD Component.

## (4) Mishap Investigation Support.

The LSE, working for the PM, will support system-related Class A and B mishap investigations by providing analyses of hazards that contributed to the mishap and recommendations for materiel risk mitigation measures, especially those that minimize human errors.

## (5) System Safety In Sep.

The SEP will be used to document a strategy for the system safety engineering program in accordance with MIL-STD-882E.  In addition, the PM will document the ESOH risk and compliance requirements management planning in the SEP by attaching the PESHE and NEPA
and E.O. 12114 compliance schedule, in accordance with Section 4321 of Title 42, U.S.C.

## F.  Parts Management.

The PM will ensure that a parts management process is used for the selection of parts during design to consider the life cycle application stresses, standardization, technology (e.g., new and ageing), reliability, maintainability, supportability, life cycle cost, and diminishing manufacturing sources and material shortages.  As applicable, parts management requirements should be specified in the RFP's statement of work for the TMRR, EMD, and production acquisition phases.

## 3.7.  Design And Architectural Factors. A.  Mosa.

(1)  The LSE, under the direction of the PM, will use a modular, open systems approach in product designs to the maximum extent practicable in accordance with Sections 2446a, 2446b, and 2446c of Title 10, U.S.C.  The modular and open systems approach will be documented in the digital authoritative source of truth.  The PM will acquire the appropriate rights to the interface technical data to allow system evolution and interoperability in accordance with the program's IP strategy.

(2)  The PM will use an appropriate open business model and system architecture that allows major system components to be severable at the appropriate level for incremental addition, removal, or replacement over the system's life-cycle.  The selection of severable components will take into consideration:

(a)  Enhanced competition and innovation.
(b)  Cost savings or avoidance.
(c)  Obsolescence.
(d)  Incremental and evolutionary technical upgrades.
(e)  Schedule reduction.
(f)  Increased system-on-system interoperability, mission integration, and reuse across the joint force.

(g)  Availability of IP and government rights thereto.
(3)  In accordance with Sections 2446a, 2446b, and 2446c of Title 10, U.S.C., the LSE, working for the PM, will clearly define major system interfaces between the major system platform and major system components, between major system components, and between major system platforms.  Specifically consider the expected evolution of the platform, subsystem, and major component as well as interdependent systems dependencies.
(4)  The LSE, working for the PM, will use consensus-based standards for interfaces, unless unavailable or unsuitable, and provide open sharing of definitions to interdependent programs.  The PM will provide justification to the MDA if consensus-based standards are not used.

(5)  In support of Milestone B (or equivalent), the PM will provide to the MDA the program's modular open system approach. The MDA will review the approach to ensure standardized interfaces and appropriate arrangements for obtaining necessary IP rights have been addressed and implemented.  The PM will provide justification to the MDA if MOSA is not used.  The MDA will review and determine whether or not the justification to not use MOSA is appropriate.
(6)  The PM will ensure that the RFPs for development or production contracts include compliance with MOSA enabling interfaces, the modular open system approach, appropriate data rights requests, and identification of the minimum set of major system components to which the design and data sharing requirements apply.

## B.  Spectrum Supportability.

The PM will:
(1)  Ensure compliance with U.S. and host nation electromagnetic spectrum regulations in accordance with Section 305 of Title 47, U.S.C., and Sections 901 through 904 and Section 104
of Public Law 102-538.

(2)  Submit written determinations to the DoD Component chief information officer or equivalent that the electromagnetic spectrum necessary to support the operation of the system during its expected life-cycle is or will be available in accordance with DoDI 4650.01.  These determinations will be the basis for recommendations provided to the MDA by the DoD Component chief information officer, or equivalent.

## C.  Corrosion Prevention And Control.

The LSE will:
(1)  Working for the PM and in conjunction with the product support manager, evaluate corrosion considerations throughout the acquisition and sustainment phases that reduce, control, or mitigate corrosion in sustainment.

(2)  Perform corrosion prevention and control planning and include corrosion control management and design considerations for corrosion prevention and control in the SEP and lifecycle sustainment plan.

(3)  Ensure that corrosion control requirements are included in the design and verified as part of test and acceptance programs established pursuant to DoDI 5000.67.

## D.  Item Unique Identification.

The PM will plan for and implement item unique identification to identify and track applicable major end items, configuration-controlled items, and U.S. Government-furnished property to enhance life-cycle management of assets in systems acquisition and sustainment, and to provide more accurate asset valuation and property accountability.  Item unique identification planning and implementation will be documented in an item unique identification implementation plan linked to the program's SEP.  DoDI 8320.04 provides the standards for unique item identifiers.

## E.  Supportability.

The PM, in conjunction with the product support manager, will include supportability analyses (e.g., failure modes, effects and criticality analysis; level of repair, source of repair; maintenance task, provisioning) as an integral part of the systems engineering process at acquisition pathway initiation and continuing throughout the program life-cycle.

(1)  The supportability analysis results should be reflected in the evolution of the digital authoritative source of truth.

(2)  The LSE, working for the PM, will ensure that engineering analyses conducted by the specialty engineering disciplines inform the supportability analyses and sustainment risk mitigation strategies.

## F.  Standardization.

The PM will plan for the identification and implementation of specifications and standards that support interoperable, reliable, technologically superior, and affordable capabilities pursuant to DoDI 4120.24.

## Glossary G.1.  Acronyms.

ACRONYM
MEANING
ACAT
acquisition category
AoA
analysis of alternatives
CAE
component acquisition executive
CDR
critical design review
CM
configuration management
CONOPS
concept of operations
COTS
commercial off-the-shelf
DAE
Defense Acquisition Executive
DoDD
DoD directive
DoDI
DoD instruction
EMD
engineering and manufacturing development
E.O.
Executive order
ESOH
environment, safety, and occupational health
IMP
integrated master plan
IMS
integrated master schedule
IP
intellectual property
IRT
independent review team
ITRA
independent technical risk assessment
LSE
lead systems engineer
MDA
milestone decision authority
MDAP
Major Defense Acquisition Program
ME
mission engineering
MIL-STD
military standard
MIM
mission integration management
MOSA
modular open systems approach
NEPA
National Environmental Policy Act of 1969
OUSD(A&S)
Office of the Under Secretary of Defense for Acquisition and Sustainment
OUSD(R&E)
Office of the Under Secretary of Defense for Research and Engineering
PDR
preliminary design review
ACRONYM
MEANING
PESHE
Programmatic ESOH Evaluation
PM
program manager
PPP
program protection plan
PQM
production, quality, and manufacturing
R&M
reliability and maintainability
RFP
request for proposal
SEP
systems engineering plan
TMRR
technology maturation and risk reduction

U.S.C.
United States Code
USD(A&S)
Under Secretary of Defense for Acquisition and Sustainment
USD(R&E)
Under Secretary of Defense for Research and Engineering


## G.2.  Definitions.

A complete Glossary of acquisition terms is maintained on the Defense Acquisition University website.  The Defense Acquisition University Glossary can be found at https://www.dau.edu/tools/t/DAU-Glossary.

## References

Chairman of the Joint Chiefs of Staff Instruction 5123.01H, Charter of the Joint Requirements
Oversight Council (JROC) and Implementation of the Joint Capabilities Integration and Development System (JCIDS)," August 31, 2018
Chairman of the Joint Chiefs of Staff Joint Capabilities Integration and Development Systems
Manual, 31 August 20181
Defense Contract Management Agency-EA Pamphlet 200.1, "Earned Value Management
System (EVMS) Program Analysis Pamphlet (PAM)," October 2012
Defense Acquisition University, Defense Acquisition Guidebook, Chapter 3, "Systems
Engineering," current edition
Department of Defense Joint Software Systems Safety Engineering Handbook, August 27, 2010
DoD Directive 5000.01, "The Defense Acquisition System," May 12, 2003, as amended
DoD Directive 5135.02, "Under Secretary of Defense for Acquisition and Sustainment
(USD(A&S))", July 15, 2020
DoD Directive 5137.02, "Under Secretary of Defense for Research and Engineering
(USD(R&E))", July 15, 2020
DoD Directive 5205.07, "Special Access Program (SAP) Policy," February 4, 2020 DoD Instruction 4120.24, "Defense Standardization Program", October 15, 2018 DoD Instruction 4630.09, "Communications Waveform Management and Standardization,"
July 15, 2015, as amended
DoD Instruction 4650.01, "Policy and Procedures for Management and Use of the
Electromagnetic Spectrum," January 9, 2009, as amended
DoD Instruction 5000.02T, "Operation of the Defense Acquisition System," January 7, 2015, as
amended
DoD Instruction 5000.67, "Prevention and Mitigation of Corrosion on DoD Military Equipment
and Infrastructure," August 31, 2018, as amended
DoD Instruction 5205.11, "Management, Administration, and Oversight of DoD Special
Access Programs (SAPs)," February 6, 2013
DoD Instruction 8320.04, "Item Unique Identification (IUID) Standards for Tangible Personal
Property," September 3, 2015, as amended
Executive Order 12114, "Environmental Effects Abroad of Major Federal Actions," January 4,
1979
Military Standard 882E, "Standard Practice for System Safety," May 11, 2012 Office of Acquisition, Analytics, and Policy, "Department of Defense Earned Value
Management System Interpretation Guide (EVMSIG)," March 14, 2019


Office of the Assistant Secretary of Defense for Research and Engineering, "Technology
Readiness Assessment (TRA) Guidance," April 2011, as amended
Office of the Deputy Assistant Secretary of Defense for Systems Engineering, "Department of
Defense Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs," January 2017
Office of the Deputy Assistant Secretary of Defense for Systems Engineering, "Reliability,
Availability, Maintainability, and Cost (RAM-C) Rationale Report Outline Guidance,"
Version 1.0, February 28, 2017
Office of the Under Secretary of Defense for Research and Engineering, "Department of Defense
Independent Technical Risk Assessment Framework for Risk Categorization," June 2018
Public Law 102-538, "The National Telecommunications and Information Administration
Organization Act," October 27, 1992
Public Law 114-328, "National Defense Authorization Act for Fiscal Year 2017," December 23,
2016
United States Code, Title 10 United States Code, Title 42 United States Code, Title 47